Advanced search
Publication Cover
European Journal of Remote Sensing Volume 56, 2023 - Issue 1
Submit an article Journal homepage
651
Views
0
CrossRef citations to date
0
Altmetric
Research Article

Deep convolutional transformer network for hyperspectral unmixing

Fazal HadiSchool of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, Jiangsu, ChinaView further author information
,
Jingxiang YangSchool of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, Jiangsu, ChinaView further author information
,
Ghulam FarooqueSchool of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, Jiangsu, ChinaView further author information
&
Liang XiaoSchool of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, Jiangsu, ChinaCorrespondence[email protected]
ORCID Iconhttps://orcid.org/0000-0003-0178-9384View further author information
ORCID Icon
Article: 2268820 | Received 09 Jun 2023, Accepted 05 Oct 2023, Published online: 30 Oct 2023

References

  • Ba, J. L., Kiros, J. R., & Hinton, G. E. (2016). Layer normalization. arXiv preprint arXiv:1607.06450.
  • Berman, M., Kiiveri, H., Lagerstrom, R., Ernst, A., Dunne, R., & Huntington, J. F. (2004). ICE: A statistical approach to identifying endmembers in hyperspectral images. IEEE Transactions on Geoscience and Remote Sensing, 42(10), 2085–18. https://doi.org/10.1109/TGRS.2004.835299
  • Bioucas-Dias, J. M. (2009). A variable splitting augmented Lagrangian approach to linear spectral unmixing. 2009 First workshop on hyperspectral image and signal processing: Evolution in remote sensing.
  • Bioucas-Dias, J. M., Plaza, A., Dobigeon, N., Parente, M., Du, Q., Gader, P., & Chanussot, J. (2012). Hyperspectral unmixing overview: Geometrical, statistical, and sparse regression-based approaches. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 5(2), 354–379. https://doi.org/10.1109/JSTARS.2012.2194696
  • Boardman, J. W. (1993). Automating spectral unmixing of AVIRIS data using convex geometry concepts. In JPL, Summaries of the 4th Annual JPL Airborne Geoscience Workshop, USA (Vol. 1). AVIRIS Workshop.
  • Borsoi, R. A., Imbiriba, T., & Bermudez, J. C. M. (2019). Deep generative endmember modeling: An application to unsupervised spectral unmixing. IEEE Transactions on Computational Imaging, 6, 374–384. https://doi.org/10.1109/TCI.2019.2948726
  • Clark, R. N., Swayze, G. A., Livo, K. E., Kokaly, R. F., Sutley, S. J., Dalton, J. B., McDougal, R. R., & Gent, C. A. (2003). Imaging spectroscopy: Earth and planetary remote sensing with the USGS tetracorder and expert systems. Journal of Geophysical Research: Planets, 108(E12). https://doi.org/10.1029/2002JE001847
  • Dobigeon, N., Moussaoui, S., Coulon, M., Tourneret, J.-Y., & Hero, A. O. (2009). Joint Bayesian endmember extraction and linear unmixing for hyperspectral imagery. IEEE Transactions on Signal Processing, 57(11), 4355–4368. https://doi.org/10.1109/TSP.2009.2025797
  • Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., & Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929.
  • Drumetz, L., Meyer, T. R., Chanussot, J., Bertozzi, A. L., & Jutten, C. (2019). Hyperspectral image unmixing with endmember bundles and group sparsity inducing mixed norms. IEEE Transactions on Image Processing, 28(7), 3435–3450. https://doi.org/10.1109/TIP.2019.2897254
  • Farooque, G., Xiao, L., Sargano, A. B., Abid, F., & Hadi, F. (2023). A dual attention driven multiscale-multilevel feature fusion approach for hyperspectral image classification. International Journal of Remote Sensing, 44(4), 1151–1178. https://doi.org/10.1080/01431161.2023.2176721
  • Farooque, G., Xiao, L., Yang, J., & Sargano, A. B. (2021). Hyperspectral image classification via a novel spectral–spatial 3D ConvLSTM-CNN. Remote Sensing, 13(21), 4348. https://doi.org/10.3390/rs13214348
  • Gao, L., Han, Z., Hong, D., Zhang, B., & Chanussot, J. (2021). CyCU-Net: Cycle-consistency unmixing network by learning cascaded autoencoders. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–14. https://doi.org/10.1109/TGRS.2021.3064958
  • Ghosh, P., Roy, S. K., Koirala, B., Rasti, B., & Scheunders, P. (2022). Deep hyperspectral unmixing using transformer network. IEEE Transactions on Geoscience and Remote Sensing 60, 1–16. arXiv preprint arXiv:2203.17076. https://doi.org/10.1109/TGRS.2022.3196057
  • Guo, J., Han, K., Wu, H., Tang, Y., Chen, X., Wang, Y., & Xu, C. (2022). Cmt: Convolutional neural networks meet vision transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.
  • Guo, R., Wang, W., & Qi, H. (2015). Hyperspectral image unmixing using autoencoder cascade. In 2015 7th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS), Tokyo, Japan.
  • Hadi, F., Yang, J., Ullah, M., Ahmad, I., Farooque, G., & Xiao, L. (2022). DHCAE: Deep hybrid convolutional autoencoder approach for robust supervised hyperspectral unmixing. Remote Sensing, 14(18), 4433. https://doi.org/10.3390/rs14184433
  • Heinz, D. C. (2001). Fully constrained least squares linear spectral mixture analysis method for material quantification in hyperspectral imagery. IEEE Transactions on Geoscience and Remote Sensing, 39(3), 529–545. https://doi.org/10.1109/36.911111
  • Hong, D., Gao, L., Yao, J., Zhang, B., Plaza, A., & Chanussot, J. (2020). Graph convolutional networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 59(7), 5966–5978. https://doi.org/10.1109/TGRS.2020.3015157
  • Hong, D., Gao, L., Yokoya, N., Yao, J., Chanussot, J., Du, Q., & Zhang, B. (2020). More diverse means better: Multimodal deep learning meets remote-sensing imagery classification. IEEE Transactions on Geoscience and Remote Sensing, 59(5), 4340–4354. https://doi.org/10.1109/TGRS.2020.3016820
  • Hong, D., Han, Z., Yao, J., Gao, L., Zhang, B., Plaza, A., & Chanussot, J. (2021). SpectralFormer: Rethinking hyperspectral image classification with transformers. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–15. https://doi.org/10.1109/TGRS.2022.3172371
  • Hong, D., Yokoya, N., Chanussot, J., Xu, J., & Zhu, X. X. (2020). Joint and progressive subspace analysis (JPSA) with spatial–spectral manifold alignment for semisupervised hyperspectral dimensionality reduction. IEEE Transactions on Cybernetics, 51(7), 3602–3615. https://doi.org/10.1109/TCYB.2020.3028931
  • Hong, D., Yokoya, N., Chanussot, J., & Zhu, X. X. (2018). An augmented linear mixing model to address spectral variability for hyperspectral unmixing. IEEE Transactions on Image Processing, 28(4), 1923–1938. https://doi.org/10.1109/TIP.2018.2878958
  • Hong, D., Yokoya, N., Chanussot, J., & Zhu, X. X. (2019). CoSpace: Common subspace learning from hyperspectral-multispectral correspondences. IEEE Transactions on Geoscience and Remote Sensing, 57(7), 4349–4359. https://doi.org/10.1109/TGRS.2018.2890705
  • Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, Lille, France.
  • Iordache, M.-D., Bioucas-Dias, J. M., & Plaza, A. (2011). Sparse unmixing of hyperspectral data. IEEE Transactions on Geoscience and Remote Sensing, 49(6), 2014–2039. https://doi.org/10.1109/TGRS.2010.2098413
  • Iordache, M.-D., Bioucas-Dias, J. M., & Plaza, A. (2012). Total variation spatial regularization for sparse hyperspectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 50(11), 4484–4502. https://doi.org/10.1109/TGRS.2012.2191590
  • Iordache, M.-D., Bioucas-Dias, J. M., & Plaza, A. (2013). Collaborative sparse regression for hyperspectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 52(1), 341–354. https://doi.org/10.1109/TGRS.2013.2240001
  • Jin, Q., Ma, Y., Fan, F., Huang, J., Mei, X., & Ma, J. (2021). Adversarial autoencoder network for hyperspectral unmixing. In IEEE Transactions on Neural Networks and Learning Systems, Piscataway, NJ, USA.
  • Keshava, N., & Mustard, J. F. (2002). Spectral unmixing. IEEE Signal Processing Magazine, 19(1), 44–57. https://doi.org/10.1109/79.974727
  • Khajehrayeni, F., & Ghassemian, H. (2020). Hyperspectral unmixing using deep convolutional autoencoders in a supervised scenario. IEEE Journal of Selected Topics in Applied Earth Observations & Remote Sensing, 13, 567–576. https://doi.org/10.1109/JSTARS.2020.2966512
  • Khan, M. J., Khan, H. S., Yousaf, A., Khurshid, K., & Abbas, A. (2018). Modern trends in hyperspectral image analysis: A review. IEEE Access, 6, 14118–14129. https://doi.org/10.1109/ACCESS.2018.2812999
  • Li, J., & Bioucas-Dias, J. M. (2008). Minimum volume simplex analysis: A fast algorithm to unmix hyperspectral data. In IGARSS 2008-2008 IEEE International Geoscience and Remote Sensing Symposium, Boston, Massachusetts, USA.
  • Li, J., Bioucas-Dias, J. M., Plaza, A., & Liu, L. (2016). Robust collaborative nonnegative matrix factorization for hyperspectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 54(10), 6076–6090. https://doi.org/10.1109/TGRS.2016.2580702
  • Li, J., Hong, D., Gao, L., Yao, J., Zheng, K., Zhang, B., & Chanussot, J. (2022). Deep learning in multimodal remote sensing data fusion: A comprehensive review. International Journal of Applied Earth Observation and Geoinformation, 112, 102926. https://doi.org/10.1016/j.jag.2022.102926
  • Nascimento, J. M., & Dias, J. M. (2005). Vertex component analysis: A fast algorithm to unmix hyperspectral data. IEEE Transactions on Geoscience and Remote Sensing, 43(4), 898–910. https://doi.org/10.1109/TGRS.2005.844293
  • Nasrabadi, N. M. (2013). Hyperspectral target detection: An overview of current and future challenges. IEEE Signal Processing Magazine, 31(1), 34–44. https://doi.org/10.1109/MSP.2013.2278992
  • Ozkan, S., Kaya, B., & Akar, G. B. (2018). Endnet: Sparse autoencoder network for endmember extraction and hyperspectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 57(1), 482–496. https://doi.org/10.1109/TGRS.2018.2856929
  • Palsson, B., Sigurdsson, J., Sveinsson, J. R., & Ulfarsson, M. O. (2018). Hyperspectral unmixing using a neural network autoencoder. IEEE Access, 6, 25646–25656. https://doi.org/10.1109/ACCESS.2018.2818280
  • Palsson, B., Sveinsson, J. R., & Ulfarsson, M. O. (2019). Spectral-spatial hyperspectral unmixing using multitask learning. IEEE Access, 7, 148861–148872. https://doi.org/10.1109/ACCESS.2019.2944072
  • Palsson, B., Ulfarsson, M. O., & Sveinsson, J. R. (2019). Convolutional autoencoder for spatial-spectral hyperspectral unmixing. In IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan.
  • Qi, L., Gao, F., Dong, J., Gao, X., & Du, Q. (2022). SSCU-Net: Spatial–spectral collaborative unmixing network for hyperspectral images. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–15. https://doi.org/10.1109/TGRS.2022.3150970
  • Qu, Y., & Qi, H. (2018). uDAS: An untied denoising autoencoder with sparsity for spectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 57(3), 1698–1712. https://doi.org/10.1109/TGRS.2018.2868690
  • Rasti, B., Koirala, B., Scheunders, P., & Chanussot, J. (2022). Misicnet: Minimum simplex convolutional network for deep hyperspectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–15. https://doi.org/10.1109/TGRS.2022.3146904
  • Rasti, B., Koirala, B., Scheunders, P., & Ghamisi, P. (2021). UnDIP: Hyperspectral unmixing using deep image prior. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–15. https://doi.org/10.1109/TGRS.2021.3067802
  • Roy, S. K., Deria, A., Hong, D., Rasti, B., Plaza, A., & Chanussot, J. (2022). Multimodal fusion transformer for remote sensing image classification. IEEE Transactions on Geoscience & Remote Sensing 61, 1–20. arXiv preprint arXiv:2203.16952. https://doi.org/10.1109/TGRS.2023.3286826
  • Roy, S. K., Krishna, G., Dubey, S. R., & Chaudhuri, B. B. (2019). HybridSN: Exploring 3-D–2-D CNN feature hierarchy for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 17(2), 277–281. https://doi.org/10.1109/LGRS.2019.2918719
  • Shao, Y., Liu, Q., & Xiao, L. (2023). IVIU-Net: Implicit variable iterative unrolling network for hyperspectral sparse unmixing. IEEE Journal of Selected Topics in Applied Earth Observations & Remote Sensing, 16, 1756–1770. https://doi.org/10.1109/JSTARS.2023.3241249
  • Shen, S., Yao, Z., Gholami, A., Mahoney, M., & Keutzer, K. (2020). Powernorm: Rethinking batch normalization in transformers. In International Conference on Machine Learning, Vienna, Austria.
  • Stuart, M. B., McGonigle, A. J., & Willmott, J. R. (2019). Hyperspectral imaging in environmental monitoring: A review of recent developments and technological advances in compact field deployable systems. Sensors, 19(14), 3071. https://doi.org/10.3390/s19143071
  • Su, Y., Li, J., Plaza, A., Marinoni, A., Gamba, P., & Chakravortty, S. (2019). DAEN: Deep autoencoder networks for hyperspectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 57(7), 4309–4321. https://doi.org/10.1109/TGRS.2018.2890633
  • Su, Y., Marinoni, A., Li, J., Plaza, A., & Gamba, P. (2017). Nonnegative sparse autoencoder for robust endmember extraction from remotely sensed hyperspectral images. In 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Fort Worth, USA.
  • Sun, L., Zhao, G., Zheng, Y., & Wu, Z. (2022). Spectral–spatial feature tokenization transformer for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–14. https://doi.org/10.1109/TGRS.2022.3231215
  • Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30.
  • Wang, X., Zhong, Y., Zhang, L., & Xu, Y. (2017). Spatial group sparsity regularized nonnegative matrix factorization for hyperspectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 55(11), 6287–6304. https://doi.org/10.1109/TGRS.2017.2724944
  • Winter, M. E. (1999). N-FINDR: An algorithm for fast autonomous spectral end-member determination in hyperspectral data. Imaging Spectrometry V.
  • Wu, X., Hong, D., & Chanussot, J. (2021). Convolutional neural networks for multimodal remote sensing data classification. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–10. https://doi.org/10.1109/TGRS.2020.3040277
  • Xie, Q., Zhou, M., Zhao, Q., Meng, D., Zuo, W., & Xu, Z. (2019). Multispectral and hyperspectral image fusion by MS/HS fusion net. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.
  • Yang, H., Yu, H., Hong, D., Xu, Z., Wang, Y., & Song, M. (2022). Hyperspectral image classification based on multi-level spectral-spatial transformer network. In 2022 12th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing (WHISPERS), Rome, Italy.
  • Yao, J., Zhang, B., Li, C., Hong, D., & Chanussot, J. (2023). Extended vision transformer (ExVit) for land use and land cover classification: A multimodal deep learning framework. IEEE Transactions on Geoscience & Remote Sensing, 61, 1–15. https://doi.org/10.1109/TGRS.2023.3284671
  • Yu, H., Xu, Z., Zheng, K., Hong, D., Yang, H., & Song, M. (2022). Mstnet: A multilevel spectral–spatial transformer network for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–13. https://doi.org/10.1109/TGRS.2022.3186400
  • Zhang, X., Sun, Y., Zhang, J., Wu, P., & Jiao, L. (2018). Hyperspectral unmixing via deep convolutional neural networks. IEEE Geoscience and Remote Sensing Letters, 15(11), 1755–1759. https://doi.org/10.1109/LGRS.2018.2857804
  • Zhang, X., Zhang, A., & Meng, X. (2015). Automatic fusion of hyperspectral images and laser scans using feature points. Journal of Sensors, 2015, 1–9. https://doi.org/10.1155/2015/415361
  • Zhao, G., Ye, Q., Sun, L., Wu, Z., Pan, C., & Jeon, B. (2022). Joint classification of hyperspectral and LiDAR data using a hierarchical CNN and transformer. IEEE Transactions on Geoscience & Remote Sensing, 61, 1–16. https://doi.org/10.1109/TGRS.2022.3232498
  • Zhong, Z., Li, Y., Ma, L., Li, J., & Zheng, W.-S. (2021). Spectral–spatial transformer network for hyperspectral image classification: A factorized architecture search framework. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–15. https://doi.org/10.1109/TGRS.2022.3225267
  • Zhuang, L., Lin, C.-H., Figueiredo, M. A., & Bioucas-Dias, J. M. (2019). Regularization parameter selection in minimum volume hyperspectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 57(12), 9858–9877. https://doi.org/10.1109/TGRS.2019.2929776

Related research

People also read lists articles that other readers of this article have read.

Recommended articles lists articles that we recommend and is powered by our AI driven recommendation engine.

Cited by lists all citing articles based on Crossref citations.
Articles with the Crossref icon will open in a new tab.