A dual-difference change detection network for detecting building changes on high-resolution remote sensing images

Abstract

Existing deep learning-based change detection networks encounter challenges related to the temporal dependency inherent in dual-temporal images. In this study, a weight-shared dual-difference change detection network(DDCDNet) model is proposed based on feature extraction networks. The model employs feature discrimination modules fused with spatial and channel attention mechanisms at different hierarchical levels of the backbone network. In the encoding phase, the tiny version of the Swin Transformer is utilized as the backbone network, with a weight-sharing strategy applied to extract feature information from bi-temporal remote sensing images. The proposed model in this paper is experimented on the LEVIR-CD + and DSIFN datasets, achieving F1-scores (F1) of 87.71% and 85.79%,recalls of 83.87% and 81.17%, and IoU (Intersection over Union) scores of 78.11% and 75.12%, respectively. These results indicate that the proposed model significantly outperforms other comparative models, demonstrating a better capability of identifying temporal changes in buildings, excellent generalization capability.

Keywords:

• Change detection of buildings
• swin transformer
• spatial attention
• channel attention
• remote sensing

1. Introduction

Change detection is the process of identifying differences in the state of objects or phenomena through observations at different times (Shi et al. 2020). Nowadays, it has been widely applied in various domains such as land resource planning, urban planning, construction land supervision, and natural disaster monitoring and assessment (Song et al. 2014; Hölbling et al. 2015; Feizizadeh et al. 2017). Timely and accurate understanding of surface cover changes plays a crucial role in national economic development, social progress, and ecological preservation. Buildings the primary location for human activities and represent artificial geographic target, and their accurate change detection holds profound significance for rational land resource utilization (Li et al. 2019), urban development (Chen et al. 2021), improving the well-being of residents (Chen et al. 2021). Therefore, how to detect changes of buildings has emerged as a prominent research topic in the fields of remote sensing, measurement (Wen et al. 2021), and artificial intelligence.

Over the past few decades, researchers have proposed numerous change detection algorithms, which have evolved and developed from traditional pixel-based methods to deep learning-based approaches (Zheng et al. 2022). Early methods primarily relied on image processing techniques such as threshold segmentation and morphological operation These methods were simple and effective, but they were sensitive to image noise and prone to false detections. Subsequently, feature-based approaches are proposed wherein such methods enhance the representation capability of buildings changes by extracting features such as gradients and textures from images, thereby further improving the accuracy of buildings change detection. Meanwhile, change detection methods based on multispectral features (Du et al. 2024; Wu et al. 2022; Eftekhari et al. 2023; Basavaraju et al. 2022) have been extensively utilized, demonstrating efficacy in alleviating the loss of image details (Zhou et al. 2022). Due to the presence of various complex change types and noise interference in images, as well as the inherent issue of uncertain edges in buildings where the boundaries are not well-defined, the task of building change detection remains a challenging problem.

In recent years, deep learning has achieved revolutionary breakthroughs in the field of computer vision. In particular, convolutional neural networks (CNNs)(Kattenborn et al. 2021) have demonstrated outstanding performance in tasks such as image classification and object detection (Yang et al. 2023). Therefore, many researchers have introduced deep learning to the change detection of buildings. For instance, in CosimNet (Guo et al. 2018), two fully convolutional siamese network structures, FC-Siam-Conc and FC-Siam-Diff (Daudt et al. 2018), with skip connections were proposed for the first time. Building upon DenseNet(Huang et al. 2016), SNUNet-CD (Fang et al. 2022) combined siamese networks with Unet++(Zhou et al. 2019) to propose the densely connected siamese network, SNUNet, for detecting building changes. In addition, methods based on multi-level feature fusion (Chen et al. 2021; Peng et al. 2021; Xu et al. 2023) have also become the focus of many researchers. For instance, the BCDetNet (Basavaraju et al. 2023) utilizes multiple feature extraction modules to detect small change regions. The RSCDNet (Barkur et al. 2023), which improves the context feature fusion approach, achieves pixel-level change detection in dual-temporal high-resolution remote sensing (HRRS) images and exhibits promising performance. Nevertheless, these methodologies often face challenges in effectively harnessing the abundant feature information present at different times to discriminate the underlying differences. Moreover, they are vulnerable to the impact of data imbalance issues (Morgan et al. 2022), ultimately leading to a degraded performance in the model’s generalization capability.

To address the aforementioned issues, a deep learning-based dual-difference architecture for building change detection is proposed. Firstly, we employ an adaptive weight allocation strategy to dynamically adjust the weights of the loss function based on the complexity of local image features, thereby mitigating the impact of data imbalance on the model. Secondly, different feature difference modules are utilized for the feature maps from different levels of the backbone network output, aiming to fully exploit the feature information within the maps. Furthermore, a multi-level feature fusion strategy is designed to extract multi-scale features of building changes effectively by introducing cross-scale contextual information. Through experiments conducted on multiple remote sensing image datasets, we demonstrate the superior performance of the proposed method in building change detection tasks. Therefore, this paper aims to propose a deep learning-based change detection algorithm to enhance the accuracy and applicability of change detection.

The main contributions of this paper are as follows:

1. A novel building change detection algorithm based on siamese networks is proposed for high-resolution remote sensing images of building. This method effectively addresses the issue of uncertain edge changes in buildings by fully leveraging the rich features inherent in remote sensing images and integrating low-level detailed information with high-level semantic information.

2. A feature processing module is designed during the encoding phase to address the differences in channel numbers and feature map sizes between low-level and high-level features. This module strengthens the network’s ability to extract local information features andenhances context modeling, thereby reducing noise interference and improving theoverall performance and efficiency of the model.

3. Detailed experiments and visual analysis are conducted on multiple datasets for building change detection, validating the superior performance and robustness of the proposed method in the task of detecting changes in building.

The remaining sections of this paper are organized as follows: Chapter 2 elaborates on the detailed structure of the proposed building change detection model, DDCDNet, including differential modules employing different strategies and a multi-level feature fusion strategy. Chapter 3 presents the experimental setup and analysis of the experimental results. Chapter 4 conducts an analysis and discussion of the results. Finally, Chapter 5 concludes the paper and outlines future research directions.

2. Methods

In existing remote sensing image-based methods for building change detection, challenges persist in fully leveraging the differential information between non-simultaneous features in dual-temporal images, as well as in discerning uncertain edge changes in buildings. Therefore, a change detection model utilizing a dual-difference module is proposed in this paper, for efficient extraction of feature information from remote sensing images of buildings.

To reduce overfitting and obtain an optimized model, data augmentation techniques such as random rotation, vertical flipping, and random scale cropping are applied to the training dataset. Then, the dual-temporal building remote sensing images at time points 1 and 2 are used as inputs for the model training to extract deep-level features, followed by a decoder to obtain the change detection results. Finally, comparative experiments are conducted on different datasets, and detailed visual analysis is performed to evaluate the superiority of the proposed model in this paper.

2.1. The overall architecture of the network

The pre-change and post-change dual-temporal remote sensing images in this paper are used as inputs to the DDCDNet model, which outputs a binary classification change detection map. The overall architecture of the DDCDNet change detection network model, as shown in Figure 1, consists of the following core components: a weight-shared Swin-tiny backbone network for feature extraction, spatial difference module, dilated convolution pooling difference module, and multi-level feature fusion module. The network structure follows an Encoder-Decoder (Cho et al. 2014) architecture.

Figure 1. The structure of DDCDNet.

The registered dual-temporal remote sensing images are first inputted into the weight-shared encoder. In this paper, the Swin Transformer (Liu et al. 2021) tiny version is employed to construct dual encoders that separately process the pre-change and post-change dual-temporal images, enabling multi-scale semantic feature extraction. Different discrepancy modules are used to compute the differences in multi-level features between the pre-change and post-change images from the Swin Transformer encoder. The discrepancy modules proposed in this paper go beyond a mere calculation of the absolute difference between $F_{\mathit{\text{pre}}}^i$ and $F_{\mathit{\text{post}}}^i.$ Instead, they dynamically learn the optimal distance measure for each scale during the training process, contributing to enhanced performance in building change detection. The change feature information of the shallow stages, namely stage 1 and stage 2, is processed by the difference1 module, while the change feature information of the deep stages, namely stage 3 and stage 4, is processed by the difference2 module. This differential feature extraction method effectively utilizes the feature information at different levels. By extracting features for each scale of the images, multi-scale feature representations are generated, and all levels of feature maps possess strong semantic information. The obtained features $F^1,$ $F^2,$ and $F^3$ are then convolved with a 3 × 3 kernel and combined with $F^4,$ which are jointly fed into the multi-level feature fusion module. This fusion process integrates both high-level features and low-level features.

The following sections provide a detailed introduction to the four core modules used for change detection in the encoder-decoder structure: the weight-shared Swin-tiny backbone network for feature extraction, spatial difference module, dilated convolution pooling difference module, and multi-level feature fusion module.

2.2. The backbone network for feature extraction is based on the weight-sharing Swin-transformer approach

Traditional change detection models often employ CNN as the backbone network, which primarily extracts features from images through convolutional operations. However, due to fixed convolutional kernels, CNNs can only capture local information and struggle to effectively process global information. Furthermore, CNNs suffer from issues such as gradient vanishing and overfitting. In contrast, Transformer (Strudel et al. 2021) utilize self-attention mechanisms to capture relationships within sequences, enabling simultaneous consideration of both global and local information. Additionally, Transformers possess advantages such as parallelizability and high computational efficiency. Notably, Swin Transformer, as a Transformer-based network structure, has achieved remarkable performance in the field of computer vision.

The Swin Transformer utilizes a hierarchical attention mechanism to capture features at different scales (Figure 2). Compared to traditional Transformers, the Swin Transformer introduces a local window mechanism that applies attention mechanisms to local regions, reducing computational complexity. Additionally, the Swin Transformer employs a multi-stage strategy to balance computational efficiency and accuracy. Furthermore, it adopts a weight-sharing strategy, reducing the number of parameters during network training, thereby improving the model’s generalization capability.

Figure 2. Two successive Swin Transformer blocks.

The entire model follows a hierarchical design, comprising a total of 4 stages. With the exception of the first stage, each stage undergoes a Patch Merging layer to reduce the resolution of the input feature map, thereby performing downsampling operations. This process is analogous to the progressive enlargement of receptive fields in CNNs, enabling the acquisition of global information.

The Patch Merging operation is employed as the final step in each stage, following the Swin Transformer Block operation and preceding the downsampling process. However, in the last stage, downsampling is not required. Instead, a subsequent fully connected layer is utilized to compute the loss in conjunction with the target label.

2.3. The spatial difference module (difference1 module)

In this module (Figure 3), the features $F^{(i+1)}$ are concatenated with F through a cascading operation. This process involves extracting features from multiple scales of feature maps and concatenating them to effectively handle objects of different scales. By incorporating these cascaded feature maps, the model can better capture and represent objects at various scales.

Figure 3. The structure of difference1 module.

By employing the Swin Transformer as the backbone network, the outputs of stage 1 and stage 2 exhibit larger feature map sizes and contain minimal semantic information. However, they are capable of capturing local features within the image.

In stage 1 and stage 2, the feature representation is enhanced through the utilization of a Spatial Attention Module (Woo et al. 2018; Yang et al. 2022). This module operates by first obtaining query, key, and value tensors through three convolutional layers applied to the input feature map. Next, the query and key tensors undergo a dot product operation, resulting in a similarity matrix that calculates the weights for each position. The similarity matrix is then subjected to a softmax operation to obtain the weights for each position. Finally, the weighted feature representation is obtained by multiplying the weights with the corresponding values. By computing the weights for each position in the feature map, the model can capture different feature information at various stages, thereby improving its performance. (1) $$F_i^{\prime }=\mathit{\text{Add}}(\mathit{\text{Concat}}(F_{\mathit{\text{pre}}}^i,F_{\mathit{\text{post}}}^i),\mathit{\text{ Upsample}}(¨\mathit{\text{bilinear}}¨,\mathit{\text{Concat}}(F_{\mathit{\text{pre}}}^{i+1},F_{\mathit{\text{post}}}^{i+1})))$$(1)

The fused $F_i^{\prime }$ serves as the input to the spatial attention module. The advantage of the Spatial Attention Module lies in its ability to learn the importance of each position, thereby effectively capturing key features within the image. In the context of change detection tasks, these key features play a crucial role in distinguishing different classes of pixels. By incorporating the Spatial Attention Module, the model can better discriminate between different classes of pixels. Consequently, in stage 1 and stage 2, the Spatial Attention Module is applied to local regions, with a particular emphasis on considering neighboring pixels. This approach enables the module to capture local feature information more effectively.

2.4. The dilated convolution pooling difference module (difference2 module)

To better capture global feature information, the difference2 module (Figure 4) is employed to process the stage 3 and stage 4 of the Swin Transformer backbone network. (2) $${\widehat{F}}^i=\mathit{\text{Concat}}(F_{\mathit{\text{pre}}}^i,F_{\mathit{\text{post}}}^i)$$(2) ${\widehat{F}}^i$ serves as the input to the ASPP (Atrous Spatial Pyramid Pooling) (Chen et al. 2018) module. The ASPP module is employed to extract features at different scales. It achieves this by utilizing multiple dilated convolutions (Li and Wang 2022), which involve inserting holes within the convolutional kernels to expand their receptive fields, thereby capturing a broader range of contextual information. By employing dilated convolutions of varying sizes, the ASPP module effectively handles feature information at different scales. This enables the model to better understand objects and scenes of different scales within the image, thus improving its performance without sacrificing resolution. By incorporating the ASPP module, the model can increase its receptive field and enhance its capability to capture diverse contextual information.

Figure 4. The structure of difference2 module.

During the feature extraction process, downsampling of the input image is often performed, leading to information loss. However, by incorporating the ASPP module, it becomes possible to extract features at different scales without reducing the resolution. This mitigates the loss of information and allows for the preservation of important details. The ASPP module effectively addresses this concern by enabling the extraction of multi-scale feature information, thereby minimizing information loss while maintaining the original resolution of the image.

The ASPP module is employed in conjunction with channel attention (Khotimah et al. 2022) to enhance the model’s adaptability to different scenes and lighting conditions, thereby improving its robustness. This approach increases the receptive field, enabling the acquisition of a broader range of contextual information. It facilitates the model’s understanding of objects and scenes at various scales within the image. By adaptively adjusting the weights of each channel based on their significance, the module effectively handles feature information at different scales, allowing the model to capture features of varying scales more effectively. The channel attention module aids in capturing relationships among different channels, thereby enhancing the model’s feature representation capabilities. It assists in better understanding the relationships among different regions in aerial images of buildings. Additionally, the adaptive adjustment of channel weights reduces the number of model parameters, enhancing efficiency and generalization ability.

2.5. The multi-level feature fusion (MLFF) module

The Multi-Level Feature Fusion (MLFF) module effectively integrates features from different levels of the feature hierarchy. It primarily employs a series of convolutional layers with skip connections to extract features at multiple scales. In this study, a design is proposed that utilizes a series of convolutional layers with skip connections to merge multi-scale feature maps with lower-level features. The skip connections are introduced to preserve the spatial information of the lower-level features. Specifically, the lower-level feature $F_{\mathit{\text{diff}}}^2$ is concatenated with the output of the first convolutional layer in the fusion module. Subsequently, a series of convolutional layers is applied to further fuse the features and reduce spatial resolution. This approach enables the combination of detailed information from lower levels with semantic information from higher levels.

The Multi-Level Feature Fusion (MLFF) module efficiently integrates information from different scales and levels, thereby enhancing the performance of feature extraction. By combining features from various scales and levels, the MLFF module effectively captures both local and global information, allowing for a more comprehensive representation of the input data. This integration of multi-scale and multi-level information contributes to improved feature extraction capabilities, leading to enhanced performance in various tasks.

3. Results

3.1. Experimental data

To thoroughly validate the effectiveness and robustness of the proposed model in this paper, experiments are conducted on two widely used remote sensing image change detection datasets, namely LEVIR-CD+(Chen and Shi 2020) and DSIFN (Zhang et al. 2020). These datasets are selected specifically to address various challenges encountered in change detection tasks. The following provides a detailed description of the data:

1. LEVIR-CD + dataset

   The LEVIR-CD + dataset is a large-scale dataset specifically designed for building change detection. It is constructed based on the existing publicly available dataset, LEVIR-CD. The LEVIR-CD + dataset consists of a total of 1,970 samples, with 985 samples designated for training and the remaining 985 samples reserved for testing.

   Each sample in the dataset includes a pair of remote sensing images: the pre-event image and the post-event image, along with their corresponding building change label map. The remote sensing images in this dataset have a size of 1024 x 1024 pixels, with a pixel resolution of 0.5 meters.

   The label map is of the same size as the images, with pixel values of either 0 or 255 (Figure 5). A pixel value of 0 indicates the absence of building changes, while a value of 255 represents the presence of building changes. The dataset contains approximately 80,000 instances of changed buildings.

• 2. DSIFN dataset

Figure 5. The examples of multi-temporal images from the LEVIR-CD + dataset.

The DSIFN dataset is manually collected from Google Earth. It consists of six large-scale pairs of high-resolution images captured at different time points, covering six cities in China, namely Beijing, Chengdu, Shenzhen, Chongqing, Wuhan, and Xi’an. Five major image pairs (i.e. Beijing, Chengdu, Shenzhen, Chongqing, and Wuhan) are cropped into 394 sub-image pairs, each with a size of 512 × 512 pixels. Through data augmentation, a collection of 3,940 pairs of dual-time images was obtained.

For model testing purposes, the Xi’an image pairs are further cropped into 48 sub-image pairs (Figure 6). The training dataset consists of 3,600 image pairs, the validation dataset contains 340 image pairs, and the test dataset comprises 48 image pairs.

Figure 6. The examples of multi-temporal images from the DSIFN datase.

3.2. Experimental environment

All experiments in this study are conducted on the Ubuntu 19.10 operating system. The PyTorch deep learning framework is employed for model development and training. The hardware configuration consistes of a NVIDIA GeForce GTX 1080 Ti GPU with 11GB of dedicated memory. The software configuration includes Python 3.9.7 and PyTorch 1.10.2, with CUDA version 10.2 for GPU acceleration.

3.3. Setting of training parameters

For the configuration of training parameters, we employed the MMSegmentation framework. We employed Cross-Entropy as the loss function and utilized AdamW as the optimizer, with a learning rate of 0.0006 and an AdamW weight decay of 0.01. The training schedule employs the poly learning rate decay strategy.

3.4. Data augmentation and evaluation metrics

To mitigate overfitting and enhance the training model’s performance, data augmentation techniques are employed on the original data. Random rotation, Gaussian blur, and random scale cropping are utilized as means of data augmentation. Subsequently, the dual-temporal remote sensing images are fed as inputs to the dual encoders to extract deep-level features. These features are then fused through a feature fusion module and decoded by the decoder to obtain the final results of change detection in this study.

To assess the performance of the model, this study employs F1-score, Recall, Intersection over Union(IoU), and Precision as evaluation metrics. The definitions of these metrics are as follows: (3) $$\mathit{\text{Precision}}=\frac{\mathit{\text{TP}}}{\mathit{\text{TP}}+\mathit{\text{FP}}}$$(3) (4) $$\mathrm{\text{Recall}}=\frac{\mathit{\text{TP}}}{\mathit{\text{TP}}+\mathit{\text{FN}}}$$(4) (5) $$\mathit{\text{IoU}}=\frac{\mathit{\text{TP}}}{\mathit{\text{TP}}+\mathit{\text{FP}}+\mathit{\text{FN}}}$$(5) (6) $$F1-\mathit{\text{score}}=\frac{2\mathit{\text{Precision Recall}}}{\mathit{\text{Precision}}+\text{Recall}}$$(6)

In the equations, TP(True Positive) represents the number of correctly detected changed pixels. FP(False Positive) represents the number of non-changed pixels that were incorrectly detected as changed. FN(False Negative) represents the number of changed pixels that were not detected. TN(True Negative) represents the number of correctly detected non-changed pixels. For the task of change detection, a higher Precision value indicates a lower number of false positives, while a higher Recall value indicates a lower number of false negatives. The values of F1-score and IoU reflect the overall performance and robustness of the model. These two metrics provide insights into the model’s ability to accurately detect changes and achieve spatial agreement with the ground truth.

3.5. Analysis of experimental results

The proposed method is applied to two datasets, namely LEVIR-CD + and DSIFN, for experimental evaluation. To validate the superiority of the proposed method, comparative experiments are conducted against several traditional change detection methods in remote sensing images (FC-EF, FC-Siam-Diff, and FC-Siam-Conc), as well as deep learning-based change detection methods (SNUNet, ChangerEx(Fang et al. 2023), ChangeFormer(Bandara and Patel 2022), and BIT (Chen et al. 2022). The experimental results are presented in Tables 1 and 2, with the bold numbers indicating the best values.

Table 1. Performance with the LEVIR-CD + dataset.

Methods	Backbone	F1	Precision	Recall	IoU	Params.(M)	Training time/h
FC-EF		72.39	68.55	76.69	56.73	1.35	4.6
FC-Siam-Diff		74.64	83.6	67.42	59.55	1.35	4.6
FC-Siam-Conc		75.14	73.12	77.27	60.18	1.54	4.6
BIT	R50	84.97	90.13	80.36	73.86	3.55	5.1
SNUNet	C16	83.11	86.72	79.79	71.11	3.01	5.1
ChangerEx	R18	86.93	91.76	82.59	76.89	11.39	9.1
ChangeFormer	Mit-b0	80.46	85.85	75.7	67.3	3.85	5.7
ours	Swin-t	87.64	91.93	83.87	78.0	35.27	8.3

Table 2. Performance with the DSIFN dataset.

Methods	Backbone	F1	Precision	Recall	IoU
FC-EF		58.63	79.68	46.38	41.47
FC-Siam-Diff		65.79	77.98	56.89	49.02
FC-Siam-Conc		69.24	68.78	69.7	52.95
BIT	R50	66.96	81.14	57.0	50.33
SNUNet	c16	76.45	79.44	73.67	61.87
ChangerEx	R18	73.41	88.0	62.97	57.99
ChangeFormer	Mit-b0	80.04	86.55	74.43	66.72
ours	Swin-t	85.79	89.36	81.17	75.12

The results demonstrate that the proposed method performs well on both datasets. Compared to the state-of-the-art change detection algorithm ChangerEx, the proposed method achieves an IoU improvement of 1.11% and 17.13% on the two datasets, respectively. Particularly, it exhibited significant advantages on the DSIFN dataset, with an F1-score increase of 12.38% and a Recall improvement of 1.36% compared to ChangerEX. Moreover, the proposed method demonstrates more stable performance across different datasets, whereas ChangerEx exhibits notable performance variations between the LEVIR-CD + and the DSIFN datasets. Notably, the algorithms employed in this study utilize lightweight backbone networks, highlighting their efficiency and effectiveness.

To visually demonstrate the advantages of the proposed method, the test images from both datasets are subjected to visualization process. The experimental results are shown in Figures 7 and 8, where a randomly selected set of test results from the LEVIR-CD + and DSIFN datasets respectively are visually presented. It can be observed that, compared to the state-of-the-art ChangerEx method, the proposed method exhibits lower false positive and false negative rates, and the detected change areas are more closely aligned with the ground truth, showcasing its superior performance.

Figure 7. Visualization of the change detection results for the LEVIR-CD + dataset. Different colors are used for a better view, i.e. black for true negative, green for true positive, red for false negative, and yellow for false positive.

Figure 8. Visualization of the change detection results for the DSIFN dataset. Different colors are used for a better view, i.e. black for true negative, green for true positive, red for false negative, and yellow for false positive.

3.6. Ablation experiment

To assess the effectiveness of each module in DDCDNet, ablation experiments were conducted on the LEVlR-CD + dataset, utilizing the Swin Transformer as the base model. The resulting experimental outcomes are detailed in Table 3. It is evident from the table that each module within the proposed DDCDNet exhibits superior performance. While the introduction of the difference1 module attains the highest precision, other metrics show suboptimal results. However, upon integrating thedifference2 and MLFF (Multi-Level Feature Fusion) modules, the overall metrics demonstrate a balanced and optimal performance, thereby confirming their correctness. Remarkably, DDCDNet achieves peak values for F1 score, recall, and loU.

Table 3. Ablation studies of different modules on LEVIR-CD + dataset al.l scores are expressed as percentages (%). the best scores are marked in bold.

Model	F1	Precision	Recall	IoU
base	84.77	88.84	81.05	73.56
base + difference1	86.47	93.0	80.8	76.17
base + difference1 + difference2	87.17	91.9	82.9	77.25
base + difference1 + difference2 + mlff(ours)	87.71	91.93	83.87	78.11

4. Discussion

In this section, the advantages and future directions for improvement of the proposed method are discussed. The method proposed in this paper is compared with several advanced deep learning-based change detection methods on the LEVIR-CD + and DSIFN building change datasets. The comparison indicates that the proposed method can achieve better performance than other methods, particularly in scenarios with a limited number of small change detection samples. This improvement is partially attributed to the construction of sample extraction for data augmentation. Distinct from other CNN-based backbone networks, the proposed network in this study employs the Swin Transformer, based on the Transformer architecture, as the backbone network for extracting features from the input dual-temporal building remote sensing images. This choice allows for a more comprehensive focus on the differences between ground features at different times. The adopted dual-difference module can better extract low-level and high-level features, effectively improving the model’s ability to capture both local and global features in the input images and reducing false positives and false negatives.

In order to further validate the performance of DDCDNet, experiments were conducted on the LEVIR-CD dataset, comparing it with the latest feature fusion-based building change detection methods, RSCDNet and BCDetNet. The experimental results are presented in Table 4. Despite a slightly lower F1 score compared to BCDetNet, DDCDNet still demonstrates outstanding performance.

Table 4. The comparative experimental results of DDCDNet with the latest state-of-the-art (SOTA) methods on the LEVIR-CD dataset.

Model	F1-score	Precision
RSCDNet	88.00	–
BCDetNet	93.52	93.00
ours	92.44	93.39

It must be acknowledged that DDCDNet is limited by its reliance on a large amount of manual pixel-by-pixel annotations as the training data, which increases both the labor and practical costs. Through comparison, it was found that DDCDNet suffers from the issue of excessive parameter count, which is primarily attributed to the incorporation of numerous fusion modules. Due to the low spectral resolution of the input images, a limited number of channels(bands), similarity in foreground and background colors of buildings, and occlusion caused by trees and other factors in high-resolution remote sensing images, there are cases of missed detections. Future work will focus on the following aspects:

1. Constructing weakly supervised or even unsupervised network models to reduce the time and labor costs associated with manual annotation.

2. Incorporating unmanned aerial vehicle (UAV) point cloud data to eliminate occluded regions of buildings.

3. Conducting further research on buildings with similar foreground and background colors to improve detection accuracy.

5. Conclusions

To enhance the accuracy of building change detection from remote sensing images, this study proposes different feature extraction modules and feature fusion modules for various hierarchical feature maps. The use of the difference1 modules allows for better handling of objects at multiple scales, enabling improved capture of local features in the images. The difference2 module is utilized to better utilize its extracted global features. Finally, the MLFF module is employed to fuse the extracted features. The proposed dual-difference module in this study achieves an F1-score improvement of 7.25% and 5.75% over Changeformer on the LEVIR-CD + and DSIFN datasets, respectively. In the future, each module of DDCDNet will be applied to more datasets, and existing modules will be adjusted or new modules will be added to adapt to multi-class change detection tasks, further improving the performance of change detection in remote sensing images. Improving the difference module and reducing the model size will be considered as future research endeavors. Additionally, the construction of weakly supervised models will be explored to further mitigate the artificial and temporal costs associated with manual annotations.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The datasets supporting this study’s findings available online. The LEVIR-CD + dataset is available for download at https://justchenhao.github.io/LEVIR. The DSIFN dataset is available for download at https://github.com/GeoZcx/A-deeply-supervised-image-fusion-network-for-change-detection-in-remote-sensing-images/tree/master/dataset.

Additional information

Funding

Science and Technology Project of Qinghai Province (No. 2023-QY-208).