Water index and Swin Transformer Ensemble (WISTE) for water body extraction from multispectral remote sensing images

ABSTRACT

Automatic surface water body mapping using remote sensing technology is greatly meaningful for studying inland water dynamics at regional to global scales. Convolutional neural networks (CNN) have become an efficient semantic segmentation technique for the interpretation of remote sensing images. However, the receptive field value of a CNN is restricted by the convolutional kernel size because the network only focuses on local features. The Swin Transformer has recently demonstrated its outstanding performance in computer vision tasks, and it could be useful for processing multispectral remote sensing images. In this article, a Water Index and Swin Transformer Ensemble (WISTE) method for automatic water body extraction is proposed. First, a dual-branch encoder architecture is designed for the Swin Transformer, aggregating the global semantic information and pixel neighbor relationships captured by fully convolutional networks (FCN) and multihead self-attention. Second, to prevent the Swin Transformer from ignoring multispectral information, we construct a prediction map ensemble module. The predictions of the Swin Transformer and the Normalized Difference Water Index (NDWI) are combined by a Bayesian averaging strategy. Finally, the experimental results obtained on two distinct datasets demonstrate that the WISTE has advantages over other segmentation methods and achieves the best results. The method proposed in this study can be used for improving regional to continental surface water mapping and related hydrological studies.

KEYWORDS:

• Deep learning
• water body extraction
• Swin Transformer
• water index
• dual-encoder

Acknowledgments

This work was partially supported by the Shenzhen Key Laboratory of Precision Measurement and Early Warning Technology for Urban Environmental Health Risks (ZDSYS20220606100604008), SUSTech research start-up grants (Y01296129; Y01296229), the CRSRI Open Research Program (Program SN: CKWV20221009/KY) and the Natural Science Foundation of China (no. 42174045; no. 41874012).

Disclosure statement

No potential conflict of interest was reported by the author(s).

Authorship contribution statement

Donghui Ma: Conceptualization, Methodology, Software, Writing-original draft preparation, Writing-review and editing. Liguang Jiang: Conceptualization, Methodology, Writing-review and editing, Supervision, Project administration. Jie Li: Analysis of the datasets and experimental results. Yun Shi: Writing-review and editing.

Code availability section

Name of the method: Water Index and Swin Transformer Ensemble (WISTE)

Contact: [email protected]

Hardware requirements: NVIDIA RTX A6000 × 2

Programming language: Python 3.8.13

Software needed: PyTorch 1.11.0, mmsegmentation 0.26.0, opencv-python 4.6.0.66

Program size: 31 M

The source codes are available for download at the following link: https://github.com/xxx7913586/WISTE

Data availability statement

The datasets used in this study are publicly available. The Gaofen Image Dataset (GID) is available at https://x-ytong.github.io/project/GID.html, and the DeepGlobe dataset is downloadable at https://competitions.codalab.org/competitions/18468

Supplementary Material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/15481603.2023.2251704

Additional information

Funding

The Shenzhen Key Laboratory of Precision Measurement and Early Warning Technology for Urban Environmental Health Risks [ZDSYS20220606100604008]; the CRSRI Open Research Program [Program SN: CKWV20221009/KY]; Natural Science Foundation of China [No. 42174045; No. 41874012]; and the SUSTech research start-up grants [Y01296129; Y01296229].