TrmGLU-Net: transformer-augmented global-local U-Net for hyperspectral image classification with limited training samples

References

• Agarwal, A., El-Ghazawi, T., El-Askary, H., & Le-Moigne, J. (2007, December 15-19). Efficient hierarchical-PCA dimension reduction for hyperspectral imagery. (Ed.),^(Eds.). 2007 IEEE International Symposium on Signal Processing and Information Technology. Giza, Egypt.
   Google Scholar
• Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017). SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(12), 2481–17. https://doi.org/10.1109/TPAMI.2016.2644615
   PubMed Web of Science ®Google Scholar
• Benediktsson, J. A., Palmason, J. A., & Sveinsson, J. R. (2005). Classification of hyperspectral data from urban areas based on extended morphological profiles. IEEE Transactions on Geoscience and Remote Sensing, 43(3), 480–491. https://doi.org/10.1109/TGRS.2004.842478
   Web of Science ®Google Scholar
• Bruzzone, L., Mingmin, C., & Marconcini, M. (2005, July 29). Transductive SVMs for semisupervised classification of hyperspectral data. Proceedings. 2005 IEEE International Geoscience and Remote Sensing Symposium, 2005. IGARSS ’05. Seoul, Korea.
   Google Scholar
• Cai, Y., Zhang, Z., Liu, X., & Cai, Z. (2020). Efficient graph convolutional self-representation for band selection of hyperspectral image. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13, 4869–4880. https://doi.org/10.1109/JSTARS.2020.3018229
   Web of Science ®Google Scholar
• Chen, Y., Jiang, H., Li, C., Jia, X., & Ghamisi, P. (2016). Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 54(10), 6232–6251. https://doi.org/10.1109/TGRS.2016.2584107
   Web of Science ®Google Scholar
• Chen, L. C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018). DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4), 834–848. https://doi.org/10.1109/TPAMI.2017.2699184
   PubMed Web of Science ®Google Scholar
• Chen, Y., Zhao, X., & Jia, X. (2015). Spectral–spatial classification of hyperspectral data based on deep belief network. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 8(6), 2381–2392. https://doi.org/10.1109/JSTARS.2015.2388577
   Web of Science ®Google Scholar
• Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., & Houlsby, N. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. CoRr, Abs/2010.11929. https://arxiv.org/abs/2010.11929.
   Google Scholar
• Fang, B., Li, Y., Zhang, H., & Chan, J. C.-W. (2020). Collaborative learning of lightweight convolutional neural network and deep clustering for hyperspectral image semi-supervised classification with limited training samples. ISPRS Journal of Photogrammetry and Remote Sensing, 161, 164–178. https://doi.org/10.1016/j.isprsjprs.2020.01.015
   Web of Science ®Google Scholar
• Gao, K., Liu, B., Yu, X., Qin, J., Zhang, P., & Tan, X. (2020). Deep relation network for hyperspectral image few-shot classification. Remote Sensing, 12(6), 923. https://doi.org/10.3390/rs12060923
   Web of Science ®Google Scholar
• Gao, K., Liu, B., Yu, X., & Yu, A. (2022). Unsupervised meta learning with multiview constraints for hyperspectral image small sample set classification. IEEE Transactions on Image Processing, 31, 3449–3462. https://doi.org/10.1109/TIP.2022.3169689
   PubMed Web of Science ®Google Scholar
• Gao, K., Liu, B., Yu, X., Zhang, P., Tan, X., & Sun, Y. (2021). Small sample classification of hyperspectral image using model-agnostic meta-learning algorithm and convolutional neural network. International Journal of Remote Sensing, 42(8), 3090–3122. https://doi.org/10.1080/01431161.2020.1864060
   Web of Science ®Google Scholar
• Gao, K., Yu, X., Tan, X., Liu, B., & Sun, Y. (2021). Small sample classification for hyperspectral imagery using temporal convolution and attention mechanism. Remote Sensing Letters, 12(5), 510–519. https://doi.org/10.1080/2150704X.2021.1903611
   Web of Science ®Google Scholar
• Haut, J. M., Paoletti, M. E., Plaza, J., Plaza, A., & Li, J. (2019). Hyperspectral image classification using random occlusion data augmentation. IEEE Geoscience and Remote Sensing Letters, 16(11), 1751–1755. https://doi.org/10.1109/LGRS.2019.2909495
   Web of Science ®Google Scholar
• He, X., Chen, Y., & Lin, Z. (2021). Spatial-Spectral transformer for hyperspectral image classification. Remote Sensing, 13(3), 498. https://doi.org/10.3390/rs13030498
   Web of Science ®Google Scholar
• He, K., Zhang, X., Ren, S., & Sun, J. (2016, June 27-30). Deep residual learning for image recognition. (Ed.),^(Eds.). 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA.
   Google Scholar
• He, J., Zhao, L., Yang, H., Zhang, M., & Li, W. (2020). HSI-BERT: Hyperspectral image classification using the bidirectional encoder representation from transformers. IEEE Transactions on Geoscience and Remote Sensing, 58(1), 165–178. https://doi.org/10.1109/TGRS.2019.2934760
   Web of Science ®Google Scholar
• Hong, D., Gao, L., Yao, J., Zhang, B., Plaza, A., & Chanussot, J. (2021). Graph convolutional networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 59(7), 5966–5978. https://doi.org/10.1109/TGRS.2020.3015157
   Web of Science ®Google Scholar
• Hong, D., Han, Z., Yao, J., Gao, L., Zhang, B., Plaza, A., & Chanussot, J. (2022). SpectralFormer: Rethinking hyperspectral image classification with transformers. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–15. https://doi.org/10.1109/TGRS.2022.3172371
   Web of Science ®Google Scholar
• Hou, S., Shi, H., Cao, X., Zhang, X., & Jiao, L. (2022). Hyperspectral imagery classification based on contrastive learning. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–13. https://doi.org/10.1109/TGRS.2022.3215431
   Web of Science ®Google Scholar
• Huang, G., Liu, Z., Maaten, L. V. D., & Weinberger, K. Q. (2017, July 21-26). Densely connected convolutional networks. (Ed.),^(Eds.). 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA.
   Google Scholar
• Hu, W., Huang, Y., Wei, L., Zhang, F., & Li, H. (2015). Deep convolutional neural networks for hyperspectral image classification. Journal of Sensors, 2015, 1–12. https://doi.org/10.1155/2015/258619
   Web of Science ®Google Scholar
• Jiang, G., Sun, Y., & Liu, B. (2021). A fully convolutional network with channel and spatial attention for hyperspectral image classification. Remote Sensing Letters, 12(12), 1238–1249. https://doi.org/10.1080/2150704X.2021.1978582
   Web of Science ®Google Scholar
• Jiao, L., Liang, M., Chen, H., Yang, S., Liu, H., & Cao, X. (2017). Deep fully convolutional network-based spatial distribution prediction for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 55(10), 5585–5599. https://doi.org/10.1109/TGRS.2017.2710079
   Web of Science ®Google Scholar
• Jia, S., Wu, K., Zhu, J., & Jia, X. (2019). Spectral–spatial gabor surface feature fusion approach for hyperspectral imagery classification. IEEE Transactions on Geoscience and Remote Sensing, 57(2), 1142–1154. https://doi.org/10.1109/TGRS.2018.2864983
   Web of Science ®Google Scholar
• Lee, H., Eum, S., & Kwon, H. (2022). Exploring cross-domain pretrained model for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–12. https://doi.org/10.1109/TGRS.2022.3165441
   Google Scholar
• Li, W., Chen, C., Su, H., & Du, Q. (2015). Local Binary Patterns and Extreme Learning Machine for Hyperspectral Imagery Classification. IEEE Transactions on Geoscience and Remote Sensing, 53(7), 3681–3693. https://doi.org/10.1109/TGRS.2014.2381602
   Web of Science ®Google Scholar
• Li, C., Chu, H., Kuo, B., & Lin, C. (2011, July 24-29). Hyperspectral image classification using spectral and spatial information based linear discriminant analysis. (Ed.),^(Eds.). 2011 IEEE International Geoscience and Remote Sensing Symposium. Vancouver, BC, Canada.
   Google Scholar
• Li, S., Song, W., Fang, L., Chen, Y., Ghamisi, P., & Benediktsson, J. A. (2019). Deep learning for hyperspectral image classification: An overview. IEEE Transactions on Geoscience and Remote Sensing, 57(9), 6690–6709. https://doi.org/10.1109/TGRS.2019.2907932
   Web of Science ®Google Scholar
• Liu, B., & Yu, X. (2021). Patch-free bilateral network for hyperspectral image classification using limited samples. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14, 10794–10807. https://doi.org/10.1109/JSTARS.2021.3121334
   Google Scholar
• Liu, B., Yu, A., Yu, X., Wang, R., Gao, K., & Guo, W. (2021). Deep multiview learning for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 59(9), 7758–7772. https://doi.org/10.1109/TGRS.2020.3034133
   Web of Science ®Google Scholar
• Liu, B., Yu, X., Yu, A., Zhang, P., Wan, G., & Wang, R. (2019). Deep few-shot learning for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 57(4), 2290–2304. https://doi.org/10.1109/TGRS.2018.2872830
   Web of Science ®Google Scholar
• Liu, B., Yu, X., Zhang, P., & Tan, X. (2021). Deep 3D convolutional network combined with spatial-spectral features for hyperspectral image classification. IEEE/CVF International Conference on Computer Vision (ICCV). Montreal, QC, Canada. 48, 9992–10002. https://doi.org/10.1109/ICCV48922.2021.00986
   Google Scholar
• Liu, B., Yu, X., Zhang, P., Yu, A., Fu, Q., & Wei, X. (2018). Supervised deep feature extraction for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 56(4), 1909–1921. https://doi.org/10.1109/TGRS.2017.2769673
   Web of Science ®Google Scholar
• Luo, F., Zhang, L., Du, B., & Zhang, L. (2020). Dimensionality reduction with enhanced hybrid-graph discriminant learning for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 58(8), 5336–5353. https://doi.org/10.1109/TGRS.2020.2963848
   Web of Science ®Google Scholar
• Ma, W., Yang, Q., Wu, Y., Zhao, W., & Zhang, X. (2019). Double-branch multi-attention mechanism network for hyperspectral image classification. Remote Sensing, 11(11), 1307. https://doi.org/10.3390/rs11111307
   Web of Science ®Google Scholar
• Mei, S., Ji, J., Geng, Y., Zhang, Z., Li, X., & Du, Q. (2019). Unsupervised spatial–spectral feature learning by 3d convolutional autoencoder for hyperspectral classification. IEEE Transactions on Geoscience and Remote Sensing, 57(9), 6808–6820. https://doi.org/10.1109/TGRS.2019.2908756
   Web of Science ®Google Scholar
• Melgani, F., & Bruzzone, L. (2004). Classification of hyperspectral remote sensing images with support vector machines. IEEE Transactions on Geoscience and Remote Sensing, 42(8), 1778–1790. https://doi.org/10.1109/TGRS.2004.831865
   Web of Science ®Google Scholar
• Mou, L., Ghamisi, P., & Zhu, X. X. (2017). Deep recurrent neural networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 55(7), 3639–3655. https://doi.org/10.1109/TGRS.2016.2636241
   Web of Science ®Google Scholar
• Nalepa, J., Myller, M., & Kawulok, M. (2020a). Training- and test-time data augmentation for hyperspectral image segmentation. IEEE Geoscience and Remote Sensing Letters, 17(2), 292–296. https://doi.org/10.1109/LGRS.2019.2921011
   Web of Science ®Google Scholar
• Nalepa, J., Myller, M., & Kawulok, M. (2020b). Transfer learning for segmenting dimensionally reduced hyperspectral images. IEEE Geoscience and Remote Sensing Letters, 17(7), 1228–1232. https://doi.org/10.1109/LGRS.2019.2942832
   Web of Science ®Google Scholar
• Pan, B., Xu, X., Shi, Z., Zhang, N., Luo, H., & Lan, X. (2020). Dssnet: A simple dilated semantic segmentation network for hyperspectral imagery classification. IEEE Geoscience and Remote Sensing Letters, 17(11), 1968–1972. https://doi.org/10.1109/LGRS.2019.2960528
   Web of Science ®Google Scholar
• Ren, S., He, K., Girshick, R., & Sun, J. (2017). Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031
   PubMed Web of Science ®Google Scholar
• Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention - {MICCAI} 2015 - 18th. International Conference Munich, Germany. 234–241. https://doi.org/10.1007/978-3-319-24574-4_28https://www.wikidata.org/entity/Q104451999.
   Google Scholar
• Shelhamer, E., Long, J., & Darrell, T. (2017). Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4), 640–651. https://doi.org/10.1109/TPAMI.2016.2572683
   PubMed Web of Science ®Google Scholar
• Sun Y, Liu B, Wang R, Zhang P and Dai M. (2023). Spectral–Spatial MLP-Like Network With Reciprocal Points Learning for Open-Set Hyperspectral Image Classification. IEEE Trans. Geosci. Remote Sensing. 61, 1–18. https://doi.org/10.1109/TGRS.2023.3280183
   Google Scholar
• Sun, Y., Liu, B., Yu, X., Yu, A., Gao, K., & Ding, L. (2022). From video to hyperspectral: Hyperspectral image-level feature extraction with transfer learning. Remote Sensing, 14(20), 5118. https://doi.org/10.3390/rs14205118
   Web of Science ®Google Scholar
• Sun, Y., Liu, B., Yu, X., Yu, A., Gao, K., Ding, L., Zhang, B., & Plaza, A. (2022). Perceiving spectral variation: Unsupervised spectrum motion feature learning for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–17. https://doi.org/10.1109/TGRS.2022.3221534
   Google Scholar
• Sun, Y., Liu, B., Yu, X., Yu, A., Xue, Z., & Gao, K. (2022). Resolution reconstruction classification: Fully octave convolution network with pyramid attention mechanism for hyperspectral image classification. International Journal of Remote Sensing, 43(6), 2076–2105. https://doi.org/10.1080/01431161.2022.2054299
   Web of Science ®Google Scholar
• Villa, A., Benediktsson, J. A., Chanussot, J., & Jutten, C. (2010, June 14-16). Independent component discriminant analysis for hyperspectral image classification. (Ed.),^(Eds.). 2010 2nd Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing. Reykjavik, Iceland.
   Google Scholar
• Wang, D., Du, B., & Zhang, L. (2022). Fully contextual network for hyperspectral scene parsing. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–16. https://doi.org/10.1109/TGRS.2021.3050491
   Web of Science ®Google Scholar
• Wang, Z., Du, B., Zhang, L., Zhang, L., & Jia, X. (2017). A novel semisupervised active-learning algorithm for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 55(6), 3071–3083. https://doi.org/10.1109/TGRS.2017.2650938
   Web of Science ®Google Scholar
• Wang, Q., Zhang, F., & Li, X. (2020). Hyperspectral band selection via optimal neighborhood reconstruction. IEEE Transactions on Geoscience and Remote Sensing, 58(12), 8465–8476. https://doi.org/10.1109/TGRS.2020.2987955
   Web of Science ®Google Scholar
• Wan, L., Tang, K., Li, M., Zhong, Y., & Qin, A. K. (2015). Collaborative active and semisupervised learning for hyperspectral remote sensing image classification. IEEE Transactions on Geoscience and Remote Sensing, 53(5), 2384–2396. https://doi.org/10.1109/TGRS.2014.2359933
   Web of Science ®Google Scholar
• Xing, C., Ma, L., & Yang, X. (2016). Stacked denoise autoencoder based feature extraction and classification for hyperspectral images. Journal of Sensors, 2016, 1–10. https://doi.org/10.1155/2016/3632943
   Web of Science ®Google Scholar
• Xu, Y., Du, B., & Zhang, L. (2020). Beyond the patchwise classification: Spectral-spatial fully convolutional networks for hyperspectral image classification. LEEE Transactions on Big Data, 6(3), 492–506. https://doi.org/10.1109/TBDATA.2019.2923243
   Web of Science ®Google Scholar
• Xue, Z., Yu, X., Liu, B., Tan, X., & Wei, X. (2021). HResNetAM: Hierarchical residual network with attention mechanism for hyperspectral image classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14, 3566–3580. https://doi.org/10.1109/JSTARS.2021.3065987
   Web of Science ®Google Scholar
• Xue, Z., Yu, X., Yu, A., Liu, B., Zhang, P., & Wu, S. (2022). Self-supervised feature learning for multimodal remote sensing image land cover classification. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–15. https://doi.org/10.1109/TGRS.2022.3190466
   Web of Science ®Google Scholar
• Yao, Y., Luo, Z., Li, S., Shen, T., Fang, T., & Quan, L. (2019, June 15-20). Recurrent MVSNet for high-resolution multi-view stereo depth inference. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA, USA.
   Google Scholar
• Yue, J., Mao, S., & Li, M. (2016). A deep learning framework for hyperspectral image classification using spatial pyramid pooling. Remote Sensing Letters, 7(9), 875–884. https://doi.org/10.1080/2150704X.2016.1193793
   Web of Science ®Google Scholar
• Zeng, M., Cai, Y., Cai, Z., Liu, X., Hu, P., & Ku, J. (2019). Unsupervised hyperspectral image band selection based on deep subspace clustering. IEEE Geoscience and Remote Sensing Letters, 16(12), 1889–1893. https://doi.org/10.1109/LGRS.2019.2912170
   Web of Science ®Google Scholar
• Zhang, S., Xu, M., Zhou, J., & Jia, S. (2022). Unsupervised spatial-spectral CNN-Based feature learning for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–17. https://doi.org/10.1109/TGRS.2022.3153673
   Google Scholar
• Zhang, L., Zhang, Q., Du, B., Huang, X., Tang, Y. Y., & Tao, D. (2018). Simultaneous spectral-spatial feature selection and extraction for hyperspectral images. IEEE Transactions on Cybernetics, 48(1), 16–28. https://doi.org/10.1109/TCYB.2016.2605044
   PubMed Web of Science ®Google Scholar
• Zhang, J., Zhang, P., Li, B., Jing, L., & Lv, T. (2020). Semisupervised feature extraction based on collaborative label propagation for hyperspectral images. IEEE Geoscience and Remote Sensing Letters, 17(11), 1958–1962. https://doi.org/10.1109/LGRS.2019.2958410
   Web of Science ®Google Scholar
• Zheng, Z., Zhong, Y., Ma, A., & Zhang, L. (2020). FPGA: Fast patch-free global learning framework for fully end-to-end hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 58(8), 5612–5626. https://doi.org/10.1109/TGRS.2020.2967821
   Web of Science ®Google Scholar
• Zhong, Z., Li, Y., Ma, L., Li, J., Zheng, W. S., Jiang, T., & Wu, S. (2022). Spectral–spatial transformer network for hyperspectral image classification: A factorized architecture search framework. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–15. https://doi.org/10.1109/TGRS.2021.3115699
   Web of Science ®Google Scholar
• Zhong, Y., Wang, X., Xu, Y., Wang, S., Jia, T., Hu, X., Zhao, J., Wei, L., & Zhang, L. (2018). Mini-UAV-Borne hyperspectral remote sensing: From observation and processing to applications. IEEE Geoscience and Remote Sensing Magazine, 6(4), 46–62. https://doi.org/10.1109/MGRS.2018.2867592
   Web of Science ®Google Scholar
• Zhu, L., Chen, Y., Ghamisi, P., & Benediktsson, J. A. (2018). Generative adversarial networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 56(9), 5046–5063. https://doi.org/10.1109/TGRS.2018.2805286
   Web of Science ®Google Scholar
• Zhu, M., Fan, J., Yang, Q., & Chen, T. (2022). SC-EADNet: A self-supervised contrastive efficient asymmetric dilated network for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–17. https://doi.org/10.1109/TGRS.2022.3230829
   Google Scholar