Spatial-spectral hierarchical vision permutator for hyperspectral image classification

ABSTRACT

In recent years, the convolutional neural network (CNN) has been widely applied to hyperspectral image classification because of its powerful feature capture ability. Nevertheless, the performance of most convolutional operations is limited by the fixed shape and size of the convolutional kernel, which causes CNN cannot fully extract global features. To address this issue, this article proposes a novel classification architecture named spatial-spectral hierarchical Vision Permutator (S2HViP). It contains a spectral module and a spatial module. In the spectral module, we divide the data into groups along the spectral dimension and treat each pixel within the group as a spectral token. Spectral long-range dependencies are obtained by fusing intra- and inter-group spectral correlations captured by multi-layer perceptrons (MLPs). In the spatial module, we first model spatial information via morphological methods and divide the resulting spatial feature maps into spatial tokens of uniform size. Then, the global spatial information is extracted through MLPs. Finally, the extracted spectral and spatial features are combined for classification. Particularly, the proposed MLP structure is an improved Vision Permutator, which presents a hierarchical fusion strategy aiming at generating discriminative features. Experimental results show that S2HViP can provide competitive performance compared to several state-of-the-art methods.

KEYWORDS:

• Hyperspectral image classification
• vision permutator
• feature fusion
• long-range dependencies

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data that support the findings of this study are available at http://www.ehu.eus/ccwintco/index.php?%20title=Hyperspectral_Remote_Sensing_Scenes.

Additional information

Funding

This work was supported in part by the National Natural Science Foundation of China under Grant No. 62071204 and by the Natural Science Foundation of Jiangsu Province under Grant No. BK20201338.