Object-oriented polarimetric SAR image classification via the combination of a pixel-based classifier and a region growing technique

ABSTRACT

Land-cover type interpretation by the use of remote sensing image classification techniques is always a hot topic. In this paper, an object-oriented method is presented for fully polarimetric synthetic aperture radar (SAR) image classification. Differing from most of the traditional object-oriented classification algorithms, the proposed method employs an innovative classification strategy that combines a pixel-based classifier and a region growing technique. Firstly, taking each individual pixel as a seed pixel, the homogeneous areas are extracted by a region growing technique. Then, using the information of the pixel-based classification result, the pixels located in each homogeneous area are all assigned to a certain class. Finally, the majority voting strategy is deployed to determine the final class label of each pixel. The experiments conducted on two fully polarimetric SAR images reveal that the proposed classification scheme can obtain pleasing classification accuracy and can provide the classification maps with more homogeneous regions than pixel-based classification.

KEYWORDS:

• Synthetic aperture radar
• object-oriented
• image classification
• region growing
• polarimetric scattering mechanism
• majority voting

Introduction

In the last two decades, the development of polarimetric SAR (PolSAR) technique has drawn a wide attention in the area of remote sensing, especially after the launch of spaceborne missions which can work in several different modes. The PolSAR sensors have the capability to observe the land-objects of the earth in both day and night, and for almost all-weather conditions. They can obtain the fully polarimetric information of the observed land targets by emitting and receiving fully polarized radar waves. All of the aforementioned advantages make the land-cover type interpretation task by the use of PolSAR data become a hot topic (Denize et al., 2019; Wu et al., 2022; Yin et al., 2020).

Generally speaking, the remote sensing image classification methods can be categorized two main families, namely, the pixel-based methods and the object-oriented or the super-pixel-based methods. In the early years, most of the pixel-based PolSAR classification algorithms were developed by only utilizing the statistical traits or the polarimetric scattering mechanisms of PolSAR data (Cloude & Pottier, 1997; Lee et al., 1994). In 2001, Famil et al. proposed an innovative PolSAR classification method which took both the statistical traits and Cloude polarimetric decomposition parameters (Cloude & Pottier, 1996) into account, which opened an important branch of PolSAR classification algorithms by considering both the statistical traits and the polarimetric scattering mechanisms (Bi et al., 2017; Lee et al., 2004). In recent years, thanks to the rapid development of the theories of image processing, deep learning based algorithms have shown their good performances and potential in PolSAR image classification (Cheng et al., 2021; Hua et al., 2020; Ma et al., 2020). Although the results of the aforementioned algorithms are generally good, they still have some drawbacks: the traditional pixel-based classification methods often produce a characteristic and inconsistent salt-and-pepper classification map, and they are not capable of extracting the complete objects of interest. The deep learning based method can alleviate the above problem to a large degree by considering the semantic information, however, sufficient sample pixels are often needed for most of the existing deep learning based method, and the lack of labelled samples can lead to a poor classification result.

To solve the aforementioned problems, object-oriented or super-pixel-based classification algorithms have been investigated. The main idea behind this methodology is to take each land-object in the real world rather than each single pixel as the basic processing unit, and uses the spectral and semantic information of the whole object to identify its class. During the past few years, some PolSAR object-oriented classification methods have been presented in the literature (Hao et al., 2021; Mao et al., 2022; Zhang et al., 2021), and most of them have shown positive classification performances.

It should be pointed out that, to carry out the classification task in the traditional object-oriented techniques, the mean spectral value of all the pixels in each object is often simply utilized to characterize the spectral traits of the object. However, since the over-segmentation problem is often inevitable for these methods, the approach of taking the mean value of all the pixels in a segment as a representation of the object can introduce bias, which brings unsatisfactory classification results. The so-called “spectral-spatial” classification methods (Fu et al., 2022; Munishamaiaha et al., 2022; L. Zhu et al., 2021), which were originally designed for optical image classification, have provided a new idea to solve the problem. These methods combine an image segmentation approach with a pixel-based classification method, and assign all the pixels to the most frequent class inside an image object. However, the main problem of most spectral-spatial classification methods for PolSAR images is that, the image segmentation approach is often time-consuming, hence increases the computation burden of the classification process.

In this paper, inspired by the idea of spectral-spatial classification and considering the computational efficiency of classification, we propose an innovative PolSAR classification method by combining a pixel-based classifier with a region growing technique. In the proposed method, homogeneous areas are extracted by the region growing technique using each image pixel as a seed pixel, and the idea of spectral-spatial classification is employed to determine the land-cover-type of each homogeneous area. Since each pixel can be included in different areas, the majority voting strategy is deployed to determine the final class label of each pixel. The advantages of the proposed method is notable in both classification accuracy and computational efficiency; moreover, more homogeneous areas are obtained in the classification maps, which is more consistent with the actual ground scenario.

Proposed method

Statistical traits of fully PolSAR data

A fully PolSAR system measures the relation between the transmitted and received electromagnetic wave of a medium in two orthogonal polarizations, which can be characterized by the following scattering vector in the reciprocal backscattering case (Ma et al., 2022; Sun et al., 2020):

(1) $v={\left(S_{\mathrm{H}\mathrm{H}},\hspace{0.17em}\sqrt{2}{S}_{\mathrm{H}\mathrm{V}},\hspace{0.17em}{S}_{\mathrm{V}\mathrm{V}}\right)}^{\mathrm{T}}$(1)

where $S_{\mathrm{H}\mathrm{V}}$ represents the scattering information of the vertical transmitting and horizontal receiving polarizations, which is consist of the amplitude $\left|S_{\mathrm{H}\mathrm{V}}\right|$ and the phase ${\phi }_{\mathrm{H}\mathrm{V}}:S_{\mathrm{H}\mathrm{V}}=\left|{\mathrm{S}}_{\mathrm{H}\mathrm{V}}\right|e^{j{\phi }_{\mathrm{H}\mathrm{V}}}$. The other elements in the scattering matrix are similarly defined. To compress data and reduce speckle, the neighboring pixels in PolSAR data are often averaged in the imaging procedure, which is known as “multilook process”. The multilook processed PolSAR data can be represented by the following polarimetric covariance matrix:

(2) $\mathit{C}=\mathit{v}{\mathit{v}}^{\mathrm{H}}=\left[\begin{array}{ccc}|S_{\mathrm{H}\mathrm{H}}{|}^2& \sqrt{2}{S}_{\mathrm{H}\mathrm{H}}{S}_{\mathrm{H}\mathrm{V}}^{\ast }& S_{\mathrm{H}\mathrm{H}}{S}_{\mathrm{V}\mathrm{V}}^{\ast }\\ \sqrt{2}{S}_{\mathrm{H}\mathrm{V}}{S}_{\mathrm{H}\mathrm{H}}^{\ast }& 2|S_{\mathrm{H}\mathrm{V}}{|}^2& \sqrt{2}{S}_{\mathrm{H}\mathrm{V}}{S}_{\mathrm{V}\mathrm{V}}^{\ast }\\ S_{\mathrm{V}\mathrm{V}}{S}_{\mathrm{H}\mathrm{H}}^{\ast }& \sqrt{2}{S}_{\mathrm{V}\mathrm{V}}{S}_{\mathrm{H}\mathrm{V}}^{\ast }& |S_{\mathrm{V}\mathrm{V}}{|}^2\end{array}\right]$(2)

where the superscript $\ast$ and $\mathrm{H}$ are, respectively, the conjugate operator and the conjugate transpose operator.

It is well known that, given the number of looks L, the above $3\times 3$ matrix follows the following complex Wishart distribution (Goodman, 1963):

(3) $P({\mathit{C}}_{L\hspace{0.17em}}|\mathit{Z})=\frac{L^{3L}{\left|{\mathit{C}}_L\right|}^{L-3}exp\left\{-L\mathrm{T}\mathrm{r}\left({\mathit{Z}}^{-1}{\mathit{C}}_L\right)\right\}}{Q\left(L,\hspace{0.17em}q\right){\left|\mathit{Z}\right|}^L}$(3)

with

(4) $Q\left(L,q\right)={\pi }^3{\mathrm{\Pi }}_{i=1}^3\mathrm{\Gamma }\left(L-i+1\right)$(4)

where $\mathrm{T}\mathrm{r}\left(\cdot \right)$, $\left|\cdot \right|$, and $\mathrm{\Gamma }\left(\cdot \right)$ denote, respectively, the trace operator, the determinant operator, and the Gamma function. $\mathit{Z}$ is the population covariance matrix. The parameter $q=3$ is the dimension of the polarimetric covariance matrix.

PolSAR image classification scheme

Differing from most object-oriented methods that segment the image pixels into patches and take each patch as a whole to conduct the classification task, an innovative PolSAR classification method combining a supervised pixel-based classifier and a region growing technique is proposed here.

Proposed region growing technique

The basic idea of the proposed region growing technique (as shown in Figure 1) is to take each pixel as the center seed pixel, and search the similar pixels, starting from the center along the eight directions, until dissimilar pixels are found in all directions. We then regard the octagonal region constructed by the boundaries that connect the end points as a homogeneous area. To control the size of the region, the maximum length of a side of the octagonal region is set as 10. As each step of searching similar pixels along a certain direction proceeds, a new area will be integrated if the current pixel is considered a similar one. However, to alleviate the over-segmentation problem, the homogeneity of the new integrated area is calculated, and the current pixel is rejected if this area is heterogeneous.

Figure 1. Schematic of the proposed region growing technique.

The first question for the proposed region growing technique now comes down to the measure of the similarity between the seed pixel and a pixel on the side. In this study, to take into account the fully polarimetric information and the statistical traits of PolSAR data, we employ the Wishart likelihood-ratio test statistic proposed by Conradsen et al. (2003) to measure the similarity between pixels. Given two independent 3 × 3 covariance matrices $\mathbf{X}$ and $\mathbf{Y}$ that follow a complex Wishart distribution, the Wishart likelihood-ratio test value can be deduced as:

(5) $Q=\frac{2^{6L}{\left|\mathbf{X}\right|}^L{\left|\mathbf{Y}\right|}^L}{{\left|\mathbf{X}+\mathbf{Y}\right|}^{2L}}$(5)

Taking the logarithm of the likelihood-ratio test value and ignoring the constant terms, we have:

(6) $lnQ=6ln2+ln\left|\mathbf{X}\right|+ln\left|\mathbf{Y}\right|-2ln\left|\mathbf{X}+\mathbf{Y}\right|$(6)

It can be easily proved that $lnQ\le 0$, and the more dissimilar the two pixels are, the smaller the value is. The second question for the region growing technique is the determination of the threshold of the likelihood-ratio value to decide whether two pixels are similar. In this study, to solve the above problem, we utilize the information obtained in the training step of the supervised classifier. For each land-object type, we select one or more rectangular sample areas, obtain the likelihood-ratio value between any two neighboring pixels along a certain direction, and calculate the mean of all the values. We then take this mean value as the threshold to decide whether two pixels are similar.

Apart from the statistical property, the polarimetric scattering mechanism should also be considered to measure the similarity between pixels, since two pixels may have quite different polarimetric scattering mechanisms even if they are statistically similar. To characterize the scattering mechanism of a pixel, we choose the scattering model based decomposition technique developed by (Freeman & Durden, 1998), which is based on realistic scattering models, for its effectiveness in providing scattering powers for each scattering component. In this decomposition model, the scattering characteristics of pixels are decomposed into three basic scattering mechanisms: surface scattering, double-bounce scattering, and volume scattering. The dominant scattering category of a pixel is determined by its highest power among the three scattering components. In this research, two pixels are considered to be similar if they are statistically similar and have the same dominant scattering mechanism.

To measure the homogeneity of the new integrated area, we employ a rapid approach proposed by Lopes et al. (1990): a patch can be regarded as a homogeneous one if $\mathrm{S}\mathrm{T}\mathrm{M}<=\sqrt{\left(4/\pi -1\right)/L}$, where $\mathrm{S}\mathrm{T}\mathrm{M}$ denotes the standard deviation to the mean of the total intensity of the area. The current pixel is rejected if this area is heterogeneous.

Compared with the traditional segmentation methods, the proposed region growing technique has the following benefits: 1) the computational burden of this approach mainly comes from the similarity measure between pixels, and since only the pixels in the eight directions are compared with the center pixel, the segmentation procedure is speeded up to a large degree; and 2) this region growing technique can ensure many pixels to be included into different homogeneous areas, thus the misclassification issue brought by over-segmentation can be alleviated by employing the subsequent majority voting process.

Class label assignment

In the proposed method, we employ the supervised Wishart classification method (Lee et al., 1994) as the base classifier, since it is easy to operate and performs relatively well. In the Wishart classification method, a Wishart distance between a pixel to be classified and a class center is developed based on the theory of maximum likelihood, and a pixel is assigned to a certain class if they have the minimum distance. The Wishart distance is given by:

(7) $d(\mathbf{C}|{\mathbf{Z}}_i)=ln|{\mathbf{Z}}_i|+Tr({\mathbf{Z}}_i^{-1}\mathbf{C})$(7)

where $C$ is the polarimetric covariance matrix of the pixel, and ${\mathit{Z}}_i$ is the center covariance matrix of the $i$th cluster.

Once the region growing procedure is accomplished, the homogeneous areas centered at different pixels are constructed. Then, as in the spectral-spatial classification scheme, we assign all the pixels in each homogeneous area to the most frequent class identified by the supervised Wishart classification method inside this area. Clearly, since each pixel can be included in several different areas, and several class labels might be assigned to the same pixel, the last step of the proposed classification framework is to determine the final class label of each pixel. A simple but effective way is to use the majority voting strategy, which is a decision rule that selects one of the many alternatives, based on the predicted class with the most votes.

Experimental part

To illustrate the classification performance of the proposed method, the results obtained with two PolSAR datasets are reported here. Three typical PolSAR classification methods were also implemented for comparison purposes: the supervised Wishart classification method (Lee et al., 1994), the PolSAR support vector machine (SVM) classification method (Maghsoudi et al., 2013), and the PolSAR Object-Oriented Random Forest (OORF) based classification method (Zhang et al., 2021). Before classification, both images were processed by the PolSAR nonlocal means filter (Chen et al., 2011), since it is able to effectively reduce the speckle and preserve the polarimetric properties of the images. To evaluate the classification performances of the different method, we utilize the overall accuracy (OA) which reflects the percent of pixels classified correctly. However, adopting OA alone is not very objective, since the training pixel numbers of different classes are often different and, in the real world, the class distribution suffers from an imbalanced phenomenon. Therefore, class-specific accuracy should be also employed to illustrate the classification performances of a method on different classes. For each type of land object, we selected two groups of pixels: one for the training procedure of the supervised classification methods, and the other for the assessment of the classification accuracy. The hardware and software configurations for the experiments are listed in Table 1.

Table 1. The hardware and software configurations for the experiments.

Configuration	Details
CPU	5600X 6-Core Processor
RAM	32 GB
GPU	NVIDIA GeForce RTX 3070
Programming Language	Python 3.7

Classification of the Yunnan image

The first dataset, from Yunnan Province of China, was acquired by the Chinese Gaofen-3 (GF-3) system. The field investigation map is shown in Figure 2. This image contains four main classes of land-objects, namely, forest, bare soil, wheat, and rapeseed. The image size is 1147 × 1245 pixels, and the number of training sample pixels for all the supervised classification methods is 5408 (nearly 0.4% of the total image pixels). The class-specific accuracy and OA of the different method are listed in Table 2.

Figure 2. Classification results for the Yunnan dataset: (a) the ground-truth map; (b) the Wishart classification result; (c) the SVM classification result; (d) the OORF classification result; (e) the classification result of the proposed method.

Table 2. Classification accuracies (%) and computational time (s) of the different methods on the Yunnan image.

		Classification method
Class no.	Class name	Wishart	PolSAR SVM	OORF	The proposed method
1	Bare soil	87.1	86.7	90.1	90.2
2	Wheat	88.3	88.5	91.0	91.9
3	Forest	86.4	86.9	93.8	93.3
4	Rapeseed	90.5	90.3	94.8	94.7
OA		88.7	88.5	91.3	92.0
Computational time		17.4	28.3	42.2	29.1

It can be clearly observed from Figure 2 and Table 2 that, generally speaking, the Wishart classification method and the PolSAR SVM method obtained comparable results, for both the classification accuracy and the classification map. Although these two methods generally obtain good classification results, the misclassification problem in the forest areas is relatively serious. Many forest pixels are misclassified as rapeseed or wheat, which may be because the dominant scattering mechanisms of these three land-object types are all volume scattering.

Compared with the above two pixel-based methods, OORF and the proposed object-oriented method notably overcome the salt-and-pepper phenomenon and obtain more homogeneous areas in the classification maps, which is more consistent with the real state. Thanks to the aforementioned advantages, the overall classification accuracies of these two methods are both improved. However, it should be pointed out that, as marked by the black ellipses in the figures, some notable misclassification still occurs in the OORF classification map, due to the over-segmentation issue. In contrast, since the proposed method combines a pixel-based classifier and the region growing technique, the misclassification problem brought by over-segmentation is effectively alleviated, which directly improves its overall classification accuracy. The Wishart classifier is most time-saving, since its classification strategy, which directly compares the statistical similarity between a pixel and a cluster center, is relatively straightforward and easy-operating. The OORF method is most time-consuming because the segmentation approach is relatively complicated. The SVM method is more time-consuming than the Wishart classifier, due to the complicated process of finding the optimal boundaries to classify different land-cover types. The proposed method is much more time-saving than the OORF method and is quite comparable with the SVM classifier, thanks to the use of the region growing strategy. In fact, we also found that the processing speed of the proposed method depends on the complexity of the image scenes to some degree. Normally, the more complicated the image scenes are, the faster the region growing process stops.

Classification of the Flevoland image

The second PolSAR dataset used for validation was the widely used L-band AIRSAR image acquired in Flevoland, Netherlands. Compared with the Yunnan dataset, the scene of the Flevoland image is more complicated which contains 11 land-cover types. Specially, most of the land-cover types are plants, which increases the difficulty of interpretation. The image size is 750 × 1024 pixels, and the number of training sample pixels for all the supervised classification methods is 9702 (nearly 1.3% of the total image pixels). Figure 3 shows the classification results of the different methods, and Table 3 lists the classification accuracies.

Figure 3. Classification results for the Flevoland data set: (a) original image of the Flevoland dataset; (b) the ground-truth map; (c) the Wishart classification result; (d) the PolSAR SVM classification result; (e) the PolSAR SGP classification result; and (e) the classification result of the proposed method.

Table 3. Classification accuracies (%) and computational time (s) of the different methods on the Flevoland image.

		Classification method
Class no.	Class name	Wishart	PolSAR SVM	OORF	The proposed method
1	Stem beans	92.0	60.9	91.5	91.6
2	Forest	80.4	86.7	98.8	92.4
3	Water	55.0	90.8	95.6	61.9
4	Potatoes	88.2	74.4	96.0	96.4
5	Lucerne	86.0	86.3	98.6	98.0
6	Wheat	78.8	80.2	73.9	74.3
7	Bare soil	100	93.3	99.0	99.5
8	Beet	83.7	83.1	89.1	89.4
9	Rapeseed	62.4	62.1	56.5	82.2
10	Peas	83.3	83.7	90.5	90.7
11	Grass	79.2	80.0	85.4	88.6
OA		81.1	82.3	87.9	88.1
Computational time		15.5	25.3	37.7	26.0

As can be observed from Table 3, for the Flevoland image, the classification accuracies of all four methods are notably decreased compared with those for the Yunnan image, which is because the land-cover types are complicated and many of them have quite similar scattering traits. Compared with the PolSAR SVM method, the supervised Wishart method does not perform as well, which is mainly because some water pixels are misclassified as bare soil. The cause of this problem may be that the Wishart distance is based largely on the value of the Pauli decomposition elements (diagonal elements of the covariance matrix), or to say, highly depends on the intensity information (Atwood et al., 2012), and both the water pixels and bare soil have weak backscattering intensity. Besides, both the SVM and Wishart methods have low the class-specific accuracy for the rapeseed.

Once again, compared with the pixel-based classifiers, the two object-oriented methods effectively overcome the salt-and-pepper problem and obtain more homogeneous areas in the classification maps. The proposed method is much more time-saving than the OORF method and is quite comparable with the SVM classifier. At first sight, the OORF method and the proposed method obtain comparable classification results. However, for the classification of rapeseed, the proposed method has obtained notable improvement over the other three methods, which indicates the superiority of proposed classification scheme. It should be also noted that, some water pixels are misclassified as bare soil by the proposed method. Clearly, the cause of this problem lies in the fact that the proposed method assigns the class labels to the pixels based on the decision of the Wishart classifier. That is to say, the classification accuracy of the proposed method is related to the base classifier, to some degree. Therefore, in practice, one can use a more suitable base classifier when applying the proposed method, according to the specific demands.

Comparison with a deep learning based method

In recent years, deep learning based algorithms have shown their good performances and potential in PolSAR image classification. However, in most cases, the good classification results of a deep learning network highly dependent on a large number of labelled samples. To solve this problem, some improved deep learning based methods were studied by researchers, and the most typical kind of methods is the so-called “semi-supervised” methods. By using much less training samples, the semi-supervised methods can achieve comparable performance with the traditional deep learning based methods. In this part, we also compare the performance of the proposed method with that of the Semi-Supervised Convolutional Neural Network (SSCNN) based Polarimetric SAR classification method proposed by L. Zhu et al. (2021).

Figure 4 displays the classification results of the SSCNN method and the proposed method on the Yunnan image when the number of training samples was only set as 0.4% of the total image pixels. Clearly, as marked out by the black ellipses in the figures, the misclassification phenomenon of the SSCNN method on the bare soil can be observed in some places. This indicates that, in the case of very limited training samples, the performance of the SSCNN method is not favourable.

Figure 4. Classification results for the Yunnan dataset: (a) the ground-truth map; (b) the classification result of the SSSCNN method; (b) the classification result of the proposed method.

To further inspect the influence of the number of training samples on the classification performance, we also display the OA values of the SSCNN method and the proposed method with different number of training samples in Figure 5. As expected, with the increase of the training samples, the performance of the SSCNN method improved significantly, achieving comparable result with the proposed method when the percent of training samples is 0.6% and significantly outperforming the proposed method when the percent is 1.0%. On the contrary, the performances of the proposed method are stable and the improvements of the proposed method are not notable with the increase of the training samples.

Figure 5. Trends of the OA with different numbers of training samples for the Yunnan image.

Conclusion

In this paper, an object-oriented classification technique that combines pixel-based classification and a region growing technique has been presented. In the proposed method, homogeneous areas are extracted by the region growing technique using each image pixel as a seed pixel, and the idea of spectral-spatial classification is employed to determine the land-cover-type of each homogeneous area. Since each pixel can be included in different areas, the majority voting strategy is deployed to determine the final class label of each pixel. Two real PolSAR datasets which were, respectively, acquired from GF-3 and AIRSAR systems, were used to compare the classification performance of the proposed method with that of two pixel-based classifiers, one object-oriented method and one semi-supervised deep learning based method. The experiments showed that the proposed method can effectively overcome the salt-and-pepper problem and obtain more homogeneous areas in the classification maps. Furthermore, in some cases, the misclassification problem brought by over-segmentation in the traditional object-oriented methods can be alleviated by the proposed method, due to the use of the region growing technique and the spectral-spatial classification strategy. In addition, the proposed method is more computationally efficient than most object-oriented methods.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

Due to the nature of this research, participants of this study did not agree for their data to be shared publicly, so supporting data is not available.

Additional information

Funding

This work was supported by the Project of Engineering Technology Research Center of Science and Technology Department of Jiangxi Province (NO. 20192BCD40021) and by the Science and Technology Research and Development Project of Powerchina Electric Power Engineering Corporation (NO. KJ-2020-098).