A spectral and spatial transformer for hyperspectral remote sensing image super-resolution

ABSTRACT

Due to the generally low spatial resolution of hyperspectral images (HSIs), early multispectral images lacked corresponding panchromatic bands, and as a result, fusion methods could not be used to enhance resolution. Many researchers have proposed various image super-resolution methods to address these limitations. However, these methods still suffered from issues, such as inadequate feature representation, lack of spectral feature representation, and high computational cost and inefficiency. To address these challenges, a spectral and spatial transformer (SST) algorithm for hyperspectral remote sensing image super-resolution is introduced. This algorithm uses a spatial transformer structure to extract the spatial features between the image pixels and a spectral transformer structure to extract the spectral features within the image pixels. The integration of these two components is applied to HSI super-resolution. After comparative experiments with currently advanced methods on three publicly available hyperspectral datasets, the results consistently show that our algorithm has better performance in both spectral fidelity and spatial restoration. Furthermore, our proposed algorithm was applied to real-world super-resolution experiments in the region of China's Ruoergai National Park, and subsequently, pixel-based classification was conducted on the super-resolution images, the results indicate that our algorithm could also be applied to future remote sensing interpretation tasks.

KEYWORDS:

• Transformer
• spatial features
• spectral features
• hyperspectral image super-resolution

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The datasets that have been utilized to support the research in this paper, namely, Pavia Centre, Houston, and Chikusei, are available for download from https://www.ehu.eus/ccwintco/index.php/Hyperspectral_Remote_Sensing_Scenes#Pavia_University_scene, http://www.grss-ieee.org/community/technical-committees/data-fusion/2013-ieee-grss-data-fusion-contest, and https://naotoyokoya.com/Download.html, respectively. Additionally, the self-constructed dataset for Ruoergai National Park has already been uploaded to https://gitee.com/a_small_tree_of_joy/sst.

Additional information

Funding

This study was supported by the Project of the Sichuan Science and Technology Program [grant number 2023YFS0499], the National Major Science and Technology Projects of China for the High-resolution Earth Observation System [grant number 87-Y50G28-9001-22/23], and the Aba Science and Technology Projects of Sichuan Province [grant number R23YYJSYJ0006].