1,432
Views
0
CrossRef citations to date
0
Altmetric
Research Article

Spatial-spectral hierarchical vision permutator for hyperspectral image classification

, , ORCID Icon &
Article: 2153747 | Received 30 May 2022, Accepted 28 Nov 2022, Published online: 08 Dec 2022

References

  • Audebert, N., Le Saux, B., & Lefèvre, S. (2019). Deep learning for classification of hyperspectral data: A comparative review. IEEE Geoscience and Remote Sensing Magazine, 7(2), 159–16. https://doi.org/10.1109/MGRS.2019.2912563
  • Bandos, T. V., Bruzzone, L., & Camps-Valls, G. (2009). Classification of hyperspectral images with regularized linear discriminant analysis. IEEE Transactions on Geoscience and Remote Sensing, 47(3), 862–873. https://doi.org/10.1109/TGRS.2008.2005729
  • Benediktsson, J. A., Palmason, J. A., & Sveinsson, J. R. (2005). Classification of hyperspectral data from urban areas based on extended morphological profiles. IEEE Transactions on Geoscience and Remote Sensing, 43(3), 480–491. https://doi.org/10.1109/TGRS.2004.842478
  • Bioucas-Dias, J. M., Plaza, A., Camps-Valls, G., Scheunders, P., Nasrabadi, N., & Chanussot, J. (2013). Hyperspectral remote sensing data analysis and future challenges. IEEE Geoscience and Remote Sensing Magazine, 1(2), 6–36. https://doi.org/10.1109/MGRS.2013.2244672
  • Camps-Valls, G., Gomez-Chova, L., Muñoz-Marí, J., Vila-Francés, J., & Calpe-Maravilla, J. (2006). Composite kernels for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 3(1), 93–97. https://doi.org/10.1109/LGRS.2005.857031
  • Camps-Valls, G., Tuia, D., Bruzzone, L., & Benediktsson, J. A. (2013). Advances in hyperspectral image classification: Earth monitoring with statistical learning methods. IEEE Signal Processing Magazine, 31(1), 45–54. https://doi.org/10.1109/MSP.2013.2279179
  • Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020). End-to-end object detection with transformers. In European conference on computer vision, 213–229. Springer.
  • Carrino, T. A., Crósta, A. P., Toledo, C. L. B., & Silva, A. M. (2018). Hyperspectral remote sensing applied to mineral exploration in southern Peru: A multiple data integration approach in the Chapi Chiara gold prospect. International Journal of Applied Earth Observation and Geoinformation, 64, 287–300. https://doi.org/10.1016/j.jag.2017.05.004
  • Chang, C. I. (Ed.). (2007). Hyperspectral data exploitation: Theory and applications. John Wiley & Sons.
  • Chen, Y., Jiang, H., Li, C., Jia, X., & Ghamisi, P. (2016). Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 54(10), 6232–6251. https://doi.org/10.1109/TGRS.2016.2584107
  • Chen, Y., Lin, Z., Zhao, X., Wang, G., & Gu, Y. (2014). Deep learning-based classification of hyperspectral data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(6), 2094–2107. https://doi.org/10.1109/JSTARS.2014.2329330
  • Dalla Mura, M., Atli Benediktsson, J., Waske, B., & Bruzzone, L. (2010). Extended profiles with morphological attribute filters for the analysis of hyperspectral data. International Journal of Remote Sensing, 31(22), 5975–5991. https://doi.org/10.1080/01431161.2010.512425
  • Dalla Mura, M., Benediktsson, J. A., Waske, B., & Bruzzone, L. (2010). Morphological attribute profiles for the analysis of very high resolution images. IEEE Transactions on Geoscience and Remote Sensing, 48(10), 3747–3762. https://doi.org/10.1109/TGRS.2010.2048116
  • Dang, L., Pang, P., Zuo, X., Liu, Y., & Lee, J. (2021). A dual-path small convolution network for hyperspectral image classification. Remote Sensing, 13(17), 3411. https://doi.org/10.3390/rs13173411
  • Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., … Houlsby, N. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929.
  • Duan, Y., Huang, H., & Tang, Y. (2021). Local constraint-based sparse manifold hypergraph learning for dimensionality reduction of hyperspectral image. IEEE Transactions on Geoscience and Remote Sensing, 59(1), 613–628. https://doi.org/10.1109/TGRS.2020.2995709
  • Duan, Y., Huang, H., & Wang, T. (2022). Semisupervised feature extraction of hyperspectral image using nonlinear geodesic sparse hypergraphs. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–15. https://doi.org/10.1109/TGRS.2021.3110855
  • Fauvel, M., Tarabalka, Y., Benediktsson, J. A., Chanussot, J., & Tilton, J. C. (2013). Advances in spectral-spatial classification of hyperspectral images. Proceedings of the IEEE, 101(3), 652–675.
  • Ghamisi, P., Benediktsson, J. A., & Ulfarsson, M. O. (2014). Spectral–spatial classification of hyperspectral images based on hidden Markov random fields. IEEE Transactions on Geoscience and Remote Sensing, 52(5), 2565–2574. https://doi.org/10.1109/TGRS.2013.2263282
  • Ghamisi, P., Maggiori, E., Li, S., Souza, R., Tarablaka, Y., Moser, G., De Giorgi, A., Fang, L., Chen, Y., Chi, M., Serpico, S. B., & Benediktsson, J. A. (2018). New frontiers in spectral-spatial hyperspectral image classification: The latest advances based on mathematical morphology, Markov random fields, segmentation, sparse representation, and deep learning. IEEE Geoscience and Remote Sensing Magazine, 6(3), 10–43. https://doi.org/10.1109/MGRS.2018.2854840
  • Ghamisi, P., Yokoya, N., Li, J., Liao, W., Liu, S., Plaza, J., Plaza, J., Rasti, B., & Plaza, A. (2017). Advances in hyperspectral image and signal processing: A comprehensive overview of the state of the art. IEEE Geoscience and Remote Sensing Magazine, 5(4), 37–78. https://doi.org/10.1109/MGRS.2017.2762087
  • Gong, N., Zhang, C., Zhou, H., Zhang, K., Wu, Z., & Zhang, X. (2022). Classification of hyperspectral images via improved cycle‐mlp. IET Computer Vision.
  • Guo, H., Liu, J., Yang, J., Xiao, Z., & Wu, Z. (2020). Deep collaborative attention network for hyperspectral image classification by combining 2-D CNN and 3-D CNN. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13, 4789–4802. https://doi.org/10.1109/JSTARS.2020.3016739
  • Ham, J., Chen, Y., Crawford, M. M., & Ghosh, J. (2005). Investigation of the random forest framework for classification of hyperspectral data. IEEE Transactions on Geoscience and Remote Sensing, 43(3), 492–501. https://doi.org/10.1109/TGRS.2004.842481
  • Hamida, A. B., Benoit, A., Lambert, P., & Amar, C. B. (2018). 3-D deep learning approach for remote sensing image classification. IEEE Transactions on Geoscience and Remote Sensing, 56(8), 4420–4434. https://doi.org/10.1109/TGRS.2018.2818945
  • He, X., & Chen, Y. (2021). Modifications of the multi-layer perceptron for hyperspectral image classification. Remote Sensing, 13(17), 3547. https://doi.org/10.3390/rs13173547
  • He, L., Li, J., Liu, C., & Li, S. (2018). Recent advances on spectral–spatial hyperspectral image classification: An overview and new guidelines. IEEE Transactions on Geoscience and Remote Sensing, 56(3), 1579–1597. https://doi.org/10.1109/TGRS.2017.2765364
  • Hendrycks, D., & Gimpel, K. (2016). Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415.
  • He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770–778).
  • He, J., Zhao, L., Yang, H., Zhang, M., & Li, W. (2020). HSI-BERT: Hyperspectral image classification using the bidirectional encoder representation from transformers. IEEE Transactions on Geoscience and Remote Sensing, 58(1), 165–178. https://doi.org/10.1109/TGRS.2019.2934760
  • Hong, D., Han, Z., Yao, J., Gao, L., Zhang, B., Plaza, A., & Chanussot, J. (2022). SpectralFormer: Rethinking hyperspectral image classification with transformers. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–15. https://doi.org/10.1109/TGRS.2022.3172371
  • Hou, Q., Jiang, Z., Yuan, L., Cheng, M. M., Yan, S., & Feng, J. (2022). Vision permutator: A permutable mlp-like architecture for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence.
  • Imani, M., & Ghassemian, H. (2020). An overview on spectral and spatial information fusion for hyperspectral image classification: Current trends and challenges. Information Fusion, 59, 59–83. https://doi.org/10.1016/j.inffus.2020.01.007
  • Khodadadzadeh, M., Li, J., Plaza, A., Ghassemian, H., Bioucas-Dias, J. M., & Li, X. (2014). Spectral–spatial classification of hyperspectral data using local and global probabilities for mixed pixel characterization. IEEE Transactions on Geoscience and Remote Sensing, 52(10), 6298–6314. https://doi.org/10.1109/TGRS.2013.2296031
  • Li, J., Bioucas-Dias, J. M., & Plaza, A. (2012). Spectral–spatial hyperspectral image segmentation using subspace multinomial logistic regression and Markov random fields. IEEE Transactions on Geoscience and Remote Sensing, 50(3), 809–823. https://doi.org/10.1109/TGRS.2011.2162649
  • Li, X., Ding, M., & Pižurica, A. (2019). Deep feature fusion via two-stream convolutional neural network for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 58(4), 2615–2629. https://doi.org/10.1109/TGRS.2019.2952758
  • Lin, M., Jing, W., Di, D., Chen, G., & Song, H. (2022). Multi-scale U-shape MLP for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters. https://doi.org/10.1109/LGRS.2022.3141547
  • Li, S., Song, W., Fang, L., Chen, Y., Ghamisi, P., & Benediktsson, J. A. (2019). Deep learning for hyperspectral image classification: An overview. IEEE Transactions on Geoscience and Remote Sensing, 57(9), 6690–6709. https://doi.org/10.1109/TGRS.2019.2907932
  • Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., … Guo, B. (2021). Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 10012–10022).
  • Liu, J., Wu, Z., Li, J., Plaza, A., & Yuan, Y. (2016). Probabilistic-kernel collaborative representation for spatial–spectral hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 54(4), 2371–2384. https://doi.org/10.1109/TGRS.2015.2500680
  • Liu, J., Wu, Z., Xiao, L., Sun, J., & Yan, H. (2020). Generalized tensor regression for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 58(2), 1244–1258. https://doi.org/10.1109/TGRS.2019.2944989
  • Liu, J., Wu, Z., Xiao, L., & Wu, X. J. (2022). Model inspired autoencoder for unsupervised hyperspectral image super-resolution. IEEE Transactions on Geoscience and Remote Sensing.
  • Liu, B., Yu, A., Gao, K., Tan, X., Sun, Y., & Yu, X. (2022). DSS-TRM: Deep spatial–spectral transformer for hyperspectral image classification. European Journal of Remote Sensing, 55(1), 103–114. https://doi.org/10.1080/22797254.2021.2023910
  • Li, T., Zhang, J., & Zhang, Y. (2014). Classification of hyperspectral image based on deep belief networks. In 2014 IEEE international conference on image processing (ICIP) (pp. 5132–5136). IEEE.
  • Makantasis, K., Karantzalos, K., Doulamis, A., & Doulamis, N. (2015, July). Deep supervised learning for hyperspectral data classification through convolutional neural networks. In 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS) (pp. 4959–4962). IEEE.
  • Ma, W., Yang, Q., Wu, Y., Zhao, W., & Zhang, X. (2019). Double-branch multi-attention mechanism network for hyperspectral image classification. Remote Sensing, 11(11), 1307. https://doi.org/10.3390/rs11111307
  • Melgani, F., & Bruzzone, L. (2004). Classification of hyperspectral remote sensing images with support vector machines. IEEE Transactions on Geoscience and Remote Sensing, 42(8), 1778–1790. https://doi.org/10.1109/TGRS.2004.831865
  • Meng, Z., Zhao, F., & Liang, M. (2021). SS-MLP: A novel spectral-spatial MLP architecture for hyperspectral image classification. Remote Sensing, 13(20), 4060. https://doi.org/10.3390/rs13204060
  • Minar, M. R., & Naher, J. (2018). Recent advances in deep learning: An overview. arXiv preprint arXiv:1807.08169.
  • Mou, L., Ghamisi, P., & Zhu, X. X. (2017). Deep recurrent neural networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 55(7), 3639–3655. https://doi.org/10.1109/TGRS.2016.2636241
  • Pal, M. (2012). Multinomial logistic regression-based feature selection for hyperspectral data. International Journal of Applied Earth Observation and Geoinformation, 14(1), 214–220. https://doi.org/10.1016/j.jag.2011.09.014
  • Paoletti, M. E., Haut, J. M., Fernandez-Beltran, R., Plaza, J., Plaza, A. J., & Pla, F. (2019). Deep pyramidal residual networks for spectral–spatial hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 57(2), 740–754. https://doi.org/10.1109/TGRS.2018.2860125
  • Pesaresi, M., & Benediktsson, J. A. (2001). A new approach for the morphological segmentation of high-resolution satellite imagery. IEEE Transactions on Geoscience and Remote Sensing, 39(2), 309–320. https://doi.org/10.1109/36.905239
  • Prasad, S., & Bruce, L. M. (2008). Limitations of principal components analysis for hyperspectral target recognition. IEEE Geoscience and Remote Sensing Letters, 5(4), 625–629. https://doi.org/10.1109/LGRS.2008.2001282
  • Qing, Y., & Liu, W. (2021). Hyperspectral image classification based on multi-scale residual network with attention mechanism. Remote Sensing, 13(3), 335. https://doi.org/10.3390/rs13030335
  • Roy, S. K., Krishna, G., Dubey, S. R., & Chaudhuri, B. B. (2020). HybridSN: Exploring 3D-2D CNN feature hierarchy for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 17(2), 277–281. https://doi.org/10.1109/LGRS.2019.2918719
  • Roy, S. K., Manna, S., Song, T., & Bruzzone, L. (2021). Attention-based adaptive spectral–spatial kernel ResNet for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 59(9), 7831–7843. https://doi.org/10.1109/TGRS.2020.3043267
  • Shen, D., Liu, J., Wu, Z., Yang, J., & Xiao, L. (2022). ADMM-HFNet: A matrix decomposition-based deep approach for hyperspectral image fusion. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–17. https://doi.org/10.1109/TGRS.2021.3112181
  • Song, W., Li, S., Fang, L., & Lu, T. (2018). Hyperspectral image classification with deep feature fusion network. IEEE Transactions on Geoscience and Remote Sensing, 56(6), 3173–3184. https://doi.org/10.1109/TGRS.2018.2794326
  • Sun, L., Zhao, G., Zheng, Y., & Wu, Z. (2022). Spectral-spatial feature tokenization transformer for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing.
  • Sun, S., Zhong, P., Xiao, H., & Wang, R. (2015). An MRF model-based active learning framework for the spectral-spatial classification of hyperspectral imagery. IEEE Journal of Selected Topics in Signal Processing, 9(6), 1074–1088. https://doi.org/10.1109/JSTSP.2015.2414401
  • Tang, X. J., Liu, X., Yan, P. F., Li, B. X., Qi, H. Y., & Huang, F. (2022). An MLP Network Based on Residual Learning for Rice Hyperspectral Data Classification. IEEE Geoscience and Remote Sensing Letters, 19, 1–5. https://doi.org/10.1109/LGRS.2022.3149185
  • Tang, Y. Y., Yuan, H., & Li, L. (2014). Manifold-based sparse representation for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 52(12), 7606–7618. https://doi.org/10.1109/TGRS.2014.2315209
  • Tolstikhin, I. O., Houlsby, N., Kolesnikov, A., Beyer, L., Zhai, X., Unterthiner, T., & Dosovitskiy, A. (2021). Mlp-mixer: An all-mlp architecture for vision. Advances in Neural Information Processing Systems, 34. https://proceedings.neurips.cc/paper/2021/file/cba0a4ee5ccd02fda0fe3f9a3e7b89fe-Paper.pdf
  • Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., & Jégou, H. (2021). Training data-efficient image transformers & distillation through attention. In International Conference on Machine Learning, 10347–10357. PMLR.
  • Tu, T. (2000). Unsupervised signature extraction and separation in hyperspectral images: A noise-adjusted fast independent component analysis. Optical Engineering, 39(4), 897–906. https://doi.org/10.1117/1.602461
  • Tu, B., Zhou, C., He, D., Huang, S., & Plaza, A. (2020). Hyperspectral classification with noisy label detection via superpixel-to-pixel weighting distance. IEEE Transactions on Geoscience and Remote Sensing, 58(6), 4116–4131. https://doi.org/10.1109/TGRS.2019.2961141
  • Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30. https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf
  • Wang, Q., Lin, J., & Yuan, Y. (2016). Salient band selection for hyperspectral image classification via manifold ranking. IEEE Transactions on Neural Networks and Learning Systems, 27(6), 1279–1289. https://doi.org/10.1109/TNNLS.2015.2477537
  • Wei, X., Yu, X., Liu, B., & Zhi, L. (2019). Convolutional neural networks and local binary patterns for hyperspectral image classification. European Journal of Remote Sensing, 52(1), 448–462. https://doi.org/10.1080/22797254.2019.1634980
  • Wu, Z., Liu, J., Yang, J., Xiao, Z., & Xiao, L. (2021). Composite kernel learning network for hyperspectral image classification. International Journal of Remote Sensing, 42(16), 6066–6089. https://doi.org/10.1080/01431161.2021.1934599
  • Xia, J., Chanussot, J., Du, P., & He, X. (2015). Spectral–spatial classification for hyperspectral data using rotation forests with local feature extraction and Markov random fields. IEEE Transactions on Geoscience and Remote Sensing, 53(5), 2532–2546. https://doi.org/10.1109/TGRS.2014.2361618
  • Yang, X., Ye, Y., Li, X., Lau, R. Y., Zhang, X., & Huang, X. (2018). Hyperspectral image classification with deep learning models. IEEE Transactions on Geoscience and Remote Sensing, 56(9), 5408–5423. https://doi.org/10.1109/TGRS.2018.2815613
  • Yu, H., Zhang, H., Liu, Y., Zheng, K., Xu, Z., & Xiao, C. (2022). Dual-channel convolution network with image-based global learning framework for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 19, 1–5. https://doi.org/10.1109/LGRS.2021.3139358
  • Zhang, H., Wu, C., Zhang, Z., Zhu, Y., Lin, H., Zhang, Z., … Smola, A. (2020). Resnest: Split-attention networks. arXiv preprint arXiv:2004.08955.
  • Zhang, L., Zhang, L., & Du, B. (2016). Deep learning for remote sensing data: A technical tutorial on the state of the art. IEEE Geoscience and Remote Sensing Magazine, 4(2), 22–40. https://doi.org/10.1109/MGRS.2016.2540798
  • Zhao, C., Wan, X., Zhao, G., Cui, B., Liu, W., & Qi, B. (2017). Spectral-spatial classification of hyperspectral imagery based on stacked sparse autoencoder and random forest. European Journal of Remote Sensing, 50(1), 47–63. https://doi.org/10.1080/22797254.2017.1274566
  • Zhao, J., Zhong, Y., Jia, T., Wang, X., Xu, Y., Shu, H., & Zhang, L. (2018). Spectral-spatial classification of hyperspectral imagery with cooperative game. ISPRS Journal of Photogrammetry and Remote Sensing, 135, 31–42. https://doi.org/10.1016/j.isprsjprs.2017.10.006
  • Zheng, Z., Zhong, Y., Ma, A., & Zhang, L. (2020). FPGA: Fast patch-free global learning framework for fully end-to-end hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 58(8), 5612–5626. https://doi.org/10.1109/TGRS.2020.2967821
  • Zhong, Z., Li, J., Luo, Z., & Chapman, M. (2018). Spectral–spatial residual network for hyperspectral image classification: A 3-D deep learning framework. IEEE Transactions on Geoscience and Remote Sensing, 56(2), 847–858. https://doi.org/10.1109/TGRS.2017.2755542
  • Zhong, Z., Li, Y., Ma, L., Li, J., & Zheng, W. S. (2022). Spectral-spatial transformer network for hyperspectral image classification: A factorized architecture search framework. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–15. https://doi.org/10.1109/TGRS.2022.3217887
  • Zhu, X. X., Tuia, D., Mou, L., Xia, G. S., Zhang, L., Xu, F., & Fraundorfer, F. (2017). Deep learning in remote sensing: A comprehensive review and list of resources. IEEE Geoscience and Remote Sensing Magazine, 5(4), 8–36. https://doi.org/10.1109/MGRS.2017.2762307