Ship detection and classification based on cascaded detection of hull and wake from optical satellite remote sensing imagery

ABSTRACT

Satellite remote-sensing provides a cost- and time-effective tool for ship monitoring at sea. Most existing approaches focused on extraction of ship locations using either hull or wake. In this paper, a method of cascaded detection of ship hull and wake was proposed to locate and classify ships using high-resolution satellite imagery. Candidate hulls were fast located through phase spectrum of Fourier transform. A hull refining module was then executed to acquire accurate shapes of candidate hull. False alarms were removed through the shape features and textures of candidate hulls. The probability that a candidate hull is determined as a real one increased with the presence of wakes. After true ships were determined, ship classification was conducted using a fuzzy classifier combining both hull and wake information. The proposed method was implemented to Gaofen-1 panchromatic and multispectral (PMS) imagery and showed good performance for ship detection with recall, precision, overall accuracy, and specificity of 90.1%, 88.1%, 98.8%, and 99.3%, respectively, better than other state-of-the-art coarse-to-fine ship detection methods. Ship classification was successfully achieved for ships with detected wakes. The accuracy of correct classification was 83.8% while the proportion of false classification was 1.0%. Factors influencing the accuracy of the developed method, including texture features and classifiers combination and key parameters of the method, were also discussed.

KEYWORDS:

• Ship detection
• ship classification
• wake detection
• optical remote sensing
• machine learning

1. Introduction

Maritime transport is the backbone of international trade and global economy and involves various types of ships. Taking the world merchant fleet as an example, the number of their registered ships amounted to 116,857 in 2018 (EMSA 2018). As an important task for maritime security, fishery management, vessel salvage, and transportation surveillance, ship monitoring has drawn a lot of attention. Although automatic identification system (AIS) allows the identification and tracking of ships, quite a few ships may close AIS or offer false information. At present, there are no freely available AIS data, whereas commercial AIS data are very expensive. In view of its advantage of coverage over large spatial and temporal scales, satellite remote-sensing provides an effective and economical tool for ship monitoring. Synthetic aperture radar (SAR) imagery has widely been utilized to monitor ships for its capacity independent of weather and illumination conditions. Numerous methods were proposed for ship detection based on SAR imagery, including traditional methods (Xing et al. 2013; Leng et al. 2015; Xu, Zhang, and Zhang) and deep learning methods (Lin et al. 2019; Sun et al. 2021; Kang et al. 2017; Sun et al. 2021; Xiong et al. 2022; Xu et al. 2022; Zhang and Zhang 2022a, 2022c; Zhang et al. 2019; Zhang and Zhang 2019; Xu, Zhang, and Zhang 2022; Zhang et al. 2021). Ship classification by SAR imagery were also investigated (Zhang and Zhang 2022b, 2022d). However, the boundaries of ship hulls in SAR images are usually vague subject to imaging characteristics, such as the dihedral reflector composed of hull and water, hull deformation by virtue of motion. In addition, most high-resolution SAR imagery is not freely available and expensive. Despite being affected by clouds, high-resolution optical satellite sensors became under scrutiny for ship detection and monitoring in recent years since they can provide larger data quantity than SAR sensors. Furthermore, detailed information on ship hulls and wakes can be obtained from optical imagery. This makes them complementary for maritime ship monitoring.

Hull and wake represent two intrinsic features for a moving ship. In this scenario, two types of remote-sensing-based methods for ship detection were developed, i.e. hull detection and wake detection. Ship hull is usually a salient target surrounded by dark water in optical images. Hull detection is similar to other target detection in the field of computer vision that has been developed for decades. Most previous researches focused on hull detection, for which the traditional coarse-to-fine approach was usually exploited since it had low demand for hardware and can be applied to extensive devices with less time consumption than other approaches (Zhu et al. 2010; Kanjir, Greidanus, and Oštir 2018; Nie et al. 2020; Shi et al. 2014). Candidate hulls were first searched fast across imagery. Shape and texture features of candidate hulls were then extracted to distinguish true hulls from false alarms through classifiers.

Recently, hull detection based on deep learning becomes a hot topic. There are mainly two schemes: two-stage framework and one-stage framework. In the two-stage framework, typified by the Faster R-CNN, region of interests (ROI) that may contain ship hulls is first obtained through region proposal network (RPN) and then refined classification of ROIs and regression of hull boundaries are conducted by other network branches (Ren et al. 2015; Zhang, Guo et al 2020; Liu et al. 2021; Guo et al. 2020). In order to improve detection efficiency, the one-stage framework abandons RPN and directly outputs confidence scores and coordinate offsets of boundary boxes (Wei et al. 2016; Redmon et al. 2016; Zhang et al. 2020). In addition, anchor-free framework and semantic segmentation network were also proposed for hull detection (Chen et al. 2020; Ma et al. 2019; Wang et al. 2021; Cui et al. 2021).

Compared with ship hulls, ship wakes present more remarkable features in satellite images since they can reach tens of thousands of meters, which is much larger than the size of ship hulls. Ship wakes can be divided into two forms based on their structures in remote sensing images (Liu and Deng 2018). One is linear wake, such as turbulent wake, Kelvin arm, and internal wake. The other is striped wake with periodic structure, mainly including transverse and divergent waves of Kelvin wakes. Different approaches are required for the detection of these two types of wakes.

Striped wakes can be well captured in high-resolution optical images despite their infrequent appearance. A literature review identifies few studies on the detection of striped wakes (Kuo and Chen 2003; Tian et al. 2019) and that linear wake detection is aligned into the mainstream of ship wake detection. Among the approaches to achieve the goal, the Radon transform (Radon 1986), the Hough transform (Hough 1962), and the scan curve (Eldhuset 1996) are most commonly used. They transform line detection to extreme point detection. However, background noises on sea surface pose challenges, which cannot meet the requirement that ship wakes are ideal lines with uniform brightness. Different solutions were proposed to mitigate the influence of noises, such as enhancing linear features of wakes through image processing and optimizing point detection in the transform space (Rey et al. 1990; Courmontagne 2005; Aggarwal and Karl 2006; Ai et al. 2011; Li, Qu, and Peng 2016; Biondi 2018, 2019; Karakuş, Rizaev, and Achim 2020; Graziano, D’errico, and Rufino 2016). These approaches were implemented to SAR data for a few special cases and their applicability to optical satellite imagery has never been tested yet. Liu et al. (2021) proposed a novel approach to detect ship wakes from optical imagery, whose effectiveness was verified through application to multi-sensor satellite imagery.

Although different ship detection methods have been proposed to handle various scenarios, some problems still need solving. Firstly, ship hull detection and wake detection are independent of each other in existing methods, although they can be coupled for complementary purpose. Secondly, valuable and comprehensive information from combining hull and wake detection is inevitably missed if only one of them is employed, as done in most previous studies. Thirdly, classification of ships is scarcely conducted despite great desire to meet both civil and military needs, especially for small ships, since most existing methods focus on extracting ship locations.

In this study, we aim to (1) propose a method of cascaded ship hull and wake detection to provide a solution for the challenges mentioned above, (2) classify ships based on hull and wake information from optical imagery, and (3) increase ship detection accuracy and improve classification accuracy via combination of hull and wake. The rest of this paper is organized as follows. Section 2 presents the data and method in detail. The experiment results of our developed approach and comparison with the state-of-the-art detection methods are described in Section 3. Factors influencing the accuracy of the developed method are discussed in Section 4. Finally, conclusions are made in Section 5.

2. Data and method

2.1 Satellite data

Hull and wake detection involves different spatial resolution of satellite imagery. The shape and texture features of ship hulls can be more distinguishable in high-resolution satellite imagery than in low-resolution one. This situation holds true for striped wakes since the wavelength of striped wakes is usually short (Liu and Deng 2018). In contrast, a moderate resolution is needed for line wake detection to reduce the influence of noises and striped wakes based on the premise that as many wakes as possible can be identified (Liu, Zhao, and Qin 2021). In virtue of these reasons, satellite imagery collected by the Gaofen-1 (GF-1) instrument was used in this study, which is equipped with two 2-m resolution panchromatic and 8-m resolution multispectral cameras (PMS). The technical specifications of the GF-1 sensor are listed in Table 1.

Table 1. Technical specifications for the GF-1 sensor.

Band	Spectral range	Spatial resolution	Revisit time	Local time of descending node	Breadth
Panchromatic	450–900 nm	2 m	4 day at the soonest	10:30 AM	60 km
Blue	450–520 nm	8 m
Green	520–590 nm
Red	630–690 nm
NIR	770–890 nm

Examples with presence of ships in GF-1 imagery of different wavebands are depicted in Figure 1. The multispectral data were all rescaled to 0–1. Through the comparison of panchromatic and multispectral bands, it can be found that ship hulls and striped wakes are more discernible at the panchromatic band of 2-m resolution, and turbulent wakes are smoother with less noises and the Kelvin arm presents as a more easily detectable line at the multispectral band of 8-m resolution. In addition, the contrast between background water and ship wakes is stronger at the infrared band than at other bands due to the strong water absorption at the infrared band. Therefore, ship hulls and striped wakes were detected from 2-m panchromatic images while linear wakes were extracted from 8-m near-infrared (NIR) images, following the approach proposed by Liu et al. (2021). GF-1 data were downloaded from the Guangdong Data and Application Center for High-resolution Earth Observation System. Surface reflectance was produced using the ENVI software. The built-in Fast Line-of-Sight Atmospheric Analysis of Spectral Hypercubes (FLAASH) module was exploited to remove aerosol contributions to multispectral signals. Ocean-land separation was done using coastline vector data and ship detection was conducted only in ocean areas. The location and category of 312 ships from 11 GF-1 images were recorded manually to test the proposed method.

Figure 1. Some ship examples in different bands of the GF-1 imagery.

2.2 Cascaded detection of ship hull and wake

In this study, ship location and classification were accomplished through cascaded detection of ship hull and wake. The flowchart is shown in Figure 2. Candidate ship hulls were extracted ahead of wakes since the latter was delineated starting from ship hulls without azimuth shift in optical imagery. Isotropic descriptors were employed to recognize candidate ship hulls. Hull shapes, as a crucial parameter for wake detection, were directly extracted using a traditional coarse-to-fine method. Candidate hulls were fast located through a visual saliency detection method called phase spectrum of Fourier transform (PFT) (Guo, Ma, and Zhang 2008). Hull refining module was executed to generate accurate shapes of candidate hulls and false alarms were eliminated according to shape features. Texture features were then extracted for further distinguishing true hulls from false alarms by Gaussian processes (GP) classifier. Both low- and high-resolution subimages were clipped with each candidate hull at the center for the detection of striped and linear wakes, respectively. Striped wakes were detected in the Fourier transform space of the subimage of high resolution. Meanwhile, line wakes were detected in the filtered subimage using the method proposed by Liu et al. (2021). Finally, the categories of candidate ships were determined by expert decision-making rules according to hull and wake information. The details are described in the following subsections.

Figure 2. Flow chart of cascaded detection of ship hull and wake in this study.

2.2.1 Hull detection

Ship hulls present remarkable discrepancy from the adjacent water while occupy only a small proportion of whole images. In view of these characteristics, ship hulls can be readily captured through visual inspection and regarded as salient targets, which is suitable for preliminary hull identification. Although satellite imagery of high resolution contains tens to hundreds of millions of pixels, PFT requires low computational complexity with good accuracy and can generate the saliency map of the whole image with one-time calculation. Therefore, PFT was employed to achieve the saliency map given its high processing efficiency. Detailed steps are summarized below (Guo, Ma, and Zhang 2008).

(1) $F(x,y)=F[I(x,y)]$(1)

(2) $P(x,y)=P[F(x,y)]$(2)

(3) $S(x,y)=g(x,y)\ast {∥F^{-1}\left[exp\left(iP(x,y)\right)\right]∥}^2$(3)

where I(x, y) is the image intensity; F and F⁻¹ denote Fourier transform and inverse Fourier transform, respectively; P(F) represents the phase spectrum of the transformed image F; g(x, y) is a 2D Gaussian filter; $∥\hspace{0.17em}∥$ denotes the modulo operation. Salient targets were extracted as candidate hulls through threshold segmentation. The threshold (Th) was calculated from:

(4) $Th=m+k\sigma$(4)

where m and$\sigma$are the mean and standard deviation of S(x, y), respectively. k is a constant and empirically set to 1.5 to keep as many targets as possible.

Salient targets may contain adjacent bright noises since a global threshold was used for the entire image. To achieve actual shapes of salient targets, a hull refining module was designed based on the approach proposed by Liu et al. (2021). The schematic diagram for the process is shown in Figure 3 and described below.

1. A subimage was segmented from the whole image through dilating by 200 pixels from the bounding box of a candidate hull in four directions (left, right, upward, and downward), which ensured that the hull was totally inside the subimage with enough water pixels around.

2. A series of hulls were achieved through binarization as the threshold growing from the initial value calculated by the Otsu method (Otsu 1979) to the maximum value for each subimage.

3. The shape index (SI) of each hull was calculated as the summation of the convexness and the rectangularity based on the assumption that the hull shape in high-resolution images can be approximated as a rectangle. Each SI corresponds to a threshold T_i.

4. T_i corresponding to the first peak of SI that exceeded 1.4 was used to binarize the subimage to obtain the final refined hull in order to avoid that only the brightest parts of a hull were retained. The final refined hulls must intersect with the unprocessed candidate hulls.

5. If all candidate hulls were processed, the hull refining ended. Otherwise, the above steps would be repeated.

Figure 3. The process of hull refining module.

Finer shapes of candidate hulls were obtained by hull refining. False alarms were then eliminated based on shape features, including area, length, width, and length-width ratio. Their ranges for real ships were experimentally determined according to ground truth and listed in Table 2. The biggest ship in the world was about 400 m long. Given the turbulent regions beside and after ships, the largest length and width were set to 600 m and 100 m, respectively. Shapes of small ships tend to appear as circles rather than rectangles in satellite imagery due to the limitation of image resolution, which was inconsistent with reality. Therefore, the minimum length-width ratio was small enough to retain small candidate targets as many as possible.

Table 2. Minimum and maximum of hull shape features for real ships.

Shape feature	Minimum	Maximum
Area	60 m	60000 m^2
Length	20 m	600 m
Width	10 m	100 m
Length-width ratio	1.1	8

However, false alarms may still survive after shape filtering due to the complexity of rough sea surface. Texture features were then used to separate true hulls from false ones. Since ships may travel in all directions, resulting in the difficulty of obtaining accurate hull orientation, especially for small ships, rotation invariant features were then introduced. Local binary pattern (LBP) (Ojala, Pietikainen, and Maenpaa 2002), region covariance descriptor (RCD) (Tuzel, Porikli, and Meer 2006; Dong, Liu, and Fang 2018) and KAZE features (Alcantarilla, Bartoli, and Davison 2012) were exploited to generate rotation invariant features of candidate hulls.

Candidate hulls were labeled as true ships or false alarms according to texture features in typical coarse-to-fine methods. In this study, wake information was combined to improve the classification accuracy, which will be depicted in Section 2.2.3. First of all, the probability that each candidate hull can be determined as a real one was needed. The powerful machine learning-based GP was employed as the classifier since it can deliver clear probabilistic interpretation for label predictions (Rasmussen 2004). In addition, GP can automatically determine the hyperparameters and covariance from the training set without cross-validation and consider predictive variance during the decision procedure. Comparison between the performance of GP and other representative classifiers will be discussed in Section 4.

The derivation of the GP classifier is briefly described here. Please refer to Rasmussen (2004), Bazi and Melgani (2010) for more details. The training and test datasets from texture feature matrixes are denoted by $\mathbf{x}=\left[{\mathbf{x}}_1;{\mathbf{x}}_2;\dots ;{\mathbf{x}}_N\right]\in {\mathbb{R}}^{N\times M}$, ${\mathbf{x}}^{\ast }=\left[{\mathbf{x}}_{{ }_1}^{\ast };{\mathbf{x}}_{{ }_2}^{\ast };\dots ;{\mathbf{x}}_{N^{\ast }}^{\ast }\right]\text{break}\in {\mathbb{R}}^{N^{\ast }\times M}$, respectively. Their corresponding label vectors are ${\mathbf{y}}^{\ast }=\left[y_{{ }_1}^{\ast };y_2^{\ast };\dots ;y_{{ }_{N^{\ast }}}^{\ast }\right]\in {\mathbb{R}}^{N^{\ast }}$ and ${\mathbf{y}}^{\ast }=\left[y_{{ }_1}^{\ast };y_2^{\ast };\dots ;y_{{ }_{N^{\ast }}}^{\ast }\right]\in {\mathbb{R}}^{N^{\ast }}$ with $y_i,y_i^{\ast }\in \left\{-1,+1\right\}$.

Different from the regression scenario, labels in the classification do not satisfy the prior hypothesis for GP. Therefore, two latent variables f and f* associated with the posterior probability of y and y* are introduced to bridge the gap. In this paper, the numerical connection of the latent variables and the posterior probability was described by the Logistic function:

(5) $F\left(f\right)=\frac{1}{1+exp\left(-f\right)}$(5)

The probability of y* = + 1 can then be calculated from:

(6) $P\left({\mathbf{y}}^{\ast }=+1\left|{\mathbf{f}}^{\ast }\right.\right)=\int F\left({\mathbf{f}}^{\ast }\right)P\left({\mathbf{f}}^{\ast }\left|\mathbf{x},\mathbf{y},{\mathbf{x}}^{\ast }\right.\right)d{\mathbf{f}}^{\ast }$(6)

The posterior distribution $P\left({\mathbf{f}}^{\ast }\left|\mathbf{x},\mathbf{y},{\mathbf{x}}^{\ast }\right.\right)$ should be derived before estimating the posterior probability of y*. According to the GP principle, the latent variables are supposed to subject to the following joint normal distribution:

(7) $P\left(\mathbf{f}\mathbf{,}{\mathbf{f}}^{\ast }\left|\mathbf{x},\mathbf{y},{\mathbf{x}}^{\ast }\right.\right)=\mathrm{N}\left(\left[\begin{array}{c}\mathbf{0}\\ \mathbf{0}\end{array}\right],\left[\begin{array}{cc}K\left(\mathbf{x},\mathbf{x}\right)& K\left(\mathbf{x},{\mathbf{x}}^{\ast }\right)\\ K\left({\mathbf{x}}^{\ast },\mathbf{x}\right)& K\left({\mathbf{x}}^{\ast },{\mathbf{x}}^{\ast }\right)\end{array}\right]\right)$(7)

where K(·) is the kernel function. $P\left({\mathbf{f}}^{\ast }\left|\mathbf{x},\mathbf{y},{\mathbf{x}}^{\ast }\right.\right)$ can be achieved through marginalization over f:

(8) $P\left({\mathbf{f}}^{\ast }\left|\mathbf{x},\mathbf{y},{\mathbf{x}}^{\ast }\right.\right)=\int P\left({\mathbf{f}}^{\ast }\left|\mathbf{x},\mathbf{y},\mathbf{f}\right.\right)P\left(\mathbf{f}\left|\mathbf{x},\mathbf{y}\right.\right)d\mathbf{f}$(8)

$P\left({\mathbf{f}}^{\ast }\left|\mathbf{x},\mathbf{y},\mathbf{f}\right.\right)$ in Eq. 7(7) $P\left(\mathbf{f}\mathbf{,}{\mathbf{f}}^{\ast }\left|\mathbf{x},\mathbf{y},{\mathbf{x}}^{\ast }\right.\right)=\mathrm{N}\left(\left[\begin{array}{c}\mathbf{0}\\ \mathbf{0}\end{array}\right],\left[\begin{array}{cc}K\left(\mathbf{x},\mathbf{x}\right)& K\left(\mathbf{x},{\mathbf{x}}^{\ast }\right)\\ K\left({\mathbf{x}}^{\ast },\mathbf{x}\right)& K\left({\mathbf{x}}^{\ast },{\mathbf{x}}^{\ast }\right)\end{array}\right]\right)$(7) follows a normal distribution and can be calculated from the conditional distribution of Eq. 6(6) $P\left({\mathbf{y}}^{\ast }=+1\left|{\mathbf{f}}^{\ast }\right.\right)=\int F\left({\mathbf{f}}^{\ast }\right)P\left({\mathbf{f}}^{\ast }\left|\mathbf{x},\mathbf{y},{\mathbf{x}}^{\ast }\right.\right)d{\mathbf{f}}^{\ast }$(6) :

(9) $P\left({\mathbf{f}}^{\mathbf{\ast }}\left|\mathbf{x}\mathbf{,}\mathbf{y}\mathbf{,}\mathbf{f}\right.\right)\sim \mathrm{N}\left(M\left({\mathbf{f}}^{\ast }\right),V\left({\mathbf{f}}^{\ast }\right)\right)$(9)

(10) $M\left({\mathbf{f}}^{\ast }\right)=K\left({\mathbf{x}}^{\ast },\mathbf{x}\right)K{\left(\mathbf{x},\mathbf{x}\right)}^{-1}\mathbf{f}$(10)

(11) $V\left({\mathbf{f}}^{\ast }\right)=K\left({\mathbf{x}}^{\ast },{\mathbf{x}}^{\ast }\right)-K\left({\mathbf{x}}^{\ast },\mathbf{x}\right)K{\left(\mathbf{x},\mathbf{x}\right)}^{-1}K\left(\mathbf{x},{\mathbf{x}}^{\ast }\right)$(11)

$P\left(\mathbf{f}\left|\mathbf{x},\mathbf{y}\right.\right)$ in Eq. 7(7) $P\left(\mathbf{f}\mathbf{,}{\mathbf{f}}^{\ast }\left|\mathbf{x},\mathbf{y},{\mathbf{x}}^{\ast }\right.\right)=\mathrm{N}\left(\left[\begin{array}{c}\mathbf{0}\\ \mathbf{0}\end{array}\right],\left[\begin{array}{cc}K\left(\mathbf{x},\mathbf{x}\right)& K\left(\mathbf{x},{\mathbf{x}}^{\ast }\right)\\ K\left({\mathbf{x}}^{\ast },\mathbf{x}\right)& K\left({\mathbf{x}}^{\ast },{\mathbf{x}}^{\ast }\right)\end{array}\right]\right)$(7) does not follow a normal distribution. The Laplacian technique was introduced to approximate $P\left(\mathbf{f}\left|\mathbf{x},\mathbf{y}\right.\right)$ by an optimal normal distribution $\mathrm{Q}\left(\mathbf{f}\left|\mathbf{x},\mathbf{y}\right.\right)$:

(12) $P\left(\mathbf{f}\left|\mathbf{x},\mathbf{y}\right.\right)\cong \mathrm{Q}\left(\mathbf{f}\left|\mathbf{x},\mathbf{y}\right.\right)\propto exp\left(-\frac{1}{2}{\left(\mathbf{f}-\stackrel{\wedge }{\mathbf{f}}\right)}^T{\mathrm{\Sigma }}_{\mathrm{f}}^{-1}\left(\mathbf{f}-\stackrel{\wedge }{\mathbf{f}}\right)\right)$(12)

where $\stackrel{\wedge }{\mathbf{f}}$ and ${\mathrm{\Sigma }}_{\mathrm{f}}$ denote the mean vector and covariance matrix, respectively. They are given by:

(13) $\stackrel{\wedge }{\mathbf{f}}=\underset{\mathbf{f}}{\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{a}\mathrm{x}}P\left(\mathbf{f}\left|\mathbf{x},\mathbf{y}\right.\right)$(13)

(14) ${\mathrm{\Sigma }}_{\mathrm{f}}=-\mathrm{\nabla }\mathrm{\nabla }logP\left(\mathbf{f}\left|\mathbf{x},\mathbf{y}\right.\right)\left|{ }_{\mathbf{f}=\stackrel{\wedge }{\mathbf{f}}}\right.$(14)

In order to calculate $\stackrel{\wedge }{\mathbf{f}}$ and ${\mathrm{\Sigma }}_{\mathrm{f}}$, $P\left(\mathbf{f}\left|\mathbf{x},\mathbf{y}\right.\right)$ can be factored using the Bayes’ theorem:

(15) $P\left(\mathbf{f}\left|\mathbf{x},\mathbf{y}\right.\right)=\frac{P\left(\mathbf{y}\left|\mathbf{f}\right.\right)P\left(\mathbf{f}\left|\mathbf{x}\right.\right)}{P\left(\mathbf{y}\left|\mathbf{x}\right.\right)}$(15)

Only $P\left(\mathbf{y}\left|\mathbf{f}\right.\right)$ and $P\left(\mathbf{f}\left|\mathbf{x}\right.\right)$ are dependent on f and left to the derivation of $\stackrel{\wedge }{\mathbf{f}}$. $P\left(\mathbf{y}\left|\mathbf{f}\right.\right)$ follows a normal distribution below:

(16) $P\left(\mathbf{f}\left|\mathbf{x}\right.\right)\propto exp\left(-\frac{1}{2}{\mathbf{f}}^T{\mathrm{\Sigma }}^{-1}\mathbf{f}\right)$(16)

Through the above manipulation, Eq. 12(12) $P\left(\mathbf{f}\left|\mathbf{x},\mathbf{y}\right.\right)\cong \mathrm{Q}\left(\mathbf{f}\left|\mathbf{x},\mathbf{y}\right.\right)\propto exp\left(-\frac{1}{2}{\left(\mathbf{f}-\stackrel{\wedge }{\mathbf{f}}\right)}^T{\mathrm{\Sigma }}_{\mathrm{f}}^{-1}\left(\mathbf{f}-\stackrel{\wedge }{\mathbf{f}}\right)\right)$(12) was further converted to the following formula:

(17) $\stackrel{\wedge }{\mathbf{f}}=logP\left(\mathbf{y}\left|\mathbf{f}\right.\right)-\frac{1}{2}{\mathbf{f}}^T{\mathrm{\Sigma }}^{-1}\mathbf{f}$(17)

However, the above equation can hardly be solved analytically. The Newton method was utilized based on the following iterative equation:

(18) $\underset{t+1}{\overset{\wedge }{\mathbf{f}}}=\underset{t}{\overset{\wedge }{\mathbf{f}}}-{\left(\mathrm{\nabla }\mathrm{\nabla }\mathrm{l}\mathrm{o}\mathrm{g}\mathrm{P}\left(\mathbf{y}\left|{\stackrel{\wedge }{\mathbf{f}}}_t\right.\right)-{\mathrm{\Sigma }}^{-1}\right)}^{-1}\left(-\mathrm{\nabla }\mathrm{l}\mathrm{o}\mathrm{g}\mathrm{P}\left(\mathbf{y}\left|{\stackrel{\wedge }{\mathbf{f}}}_t\right.\right)-{\mathrm{\Sigma }}^{-1}{\stackrel{\wedge }{\mathbf{f}}}_t\right)$(18)

The covariance matrix was then achieved via

(19) ${\mathrm{\Sigma }}_{\mathrm{f}}={\left(\mathrm{\nabla }\mathrm{\nabla }\mathrm{l}\mathrm{o}\mathrm{g}\mathrm{P}\left(\mathbf{y}\left|\stackrel{\wedge }{\mathbf{f}}\right.\right)-{\mathrm{\Sigma }}^{-1}\right)}^{-1}$(19)

Once the calculations of $\stackrel{\wedge }{\mathbf{f}}$ and ${\mathrm{\Sigma }}_{\mathrm{f}}$ were finished, $P\left(\mathbf{f}\left|\mathbf{x},\mathbf{y}\right.\right)$ was estimated from Eq. 12(12) $P\left(\mathbf{f}\left|\mathbf{x},\mathbf{y}\right.\right)\cong \mathrm{Q}\left(\mathbf{f}\left|\mathbf{x},\mathbf{y}\right.\right)\propto exp\left(-\frac{1}{2}{\left(\mathbf{f}-\stackrel{\wedge }{\mathbf{f}}\right)}^T{\mathrm{\Sigma }}_{\mathrm{f}}^{-1}\left(\mathbf{f}-\stackrel{\wedge }{\mathbf{f}}\right)\right)$(12) . Finally, the probability of y* = + 1 was achieved according to Eqs. 6(6) $P\left({\mathbf{y}}^{\ast }=+1\left|{\mathbf{f}}^{\ast }\right.\right)=\int F\left({\mathbf{f}}^{\ast }\right)P\left({\mathbf{f}}^{\ast }\left|\mathbf{x},\mathbf{y},{\mathbf{x}}^{\ast }\right.\right)d{\mathbf{f}}^{\ast }$(6) and 8.

The classifier should be trained in advance using ship hull and false alarm samples. The samples were extracted through candidate hull detection method introduced above from 18 GF-1 images from 2013 to 2015 over the coastal regions of Guangdong Province, China. The detected candidate hulls were labeled manually as ship hulls and false alarms. In order to increase the robustness of the classifier, each sample was rotated 7 times with an interval of 45º. In total, 4064 ship hull and 4192 false alarm samples were generated.

2.2.2 Wake detection

Ship wakes were detected in subimages with ship hulls at the center. Linear wake detection followed the method proposed by Liu et al. (2021). Given the periodic wave structure, striped wakes were detected in the Fourier transform domain, in which striped wakes appeared as bright contrasts with respect to surrounding water. Wake detection then turned into peak point detection. Natural waves may also present as periodic structure and hamper the detection of striped wakes. However, characteristics of striped wakes and natural waves can be easily distinguished. Striped wakes usually form a cone shape (Fig. 4a) and a crescent shape (Fig. 4b) in the image before and after the Fourier transform, respectively. In contrast, natural waves exhibit no regular shape in the original image (Fig. 4c) while a nearly circle when the Fourier transform was applied (Fig. 4d). Based on this, a striped wake detection method was designed and the entire procedure is shown in Fig. 5. The following steps were carried out.

Figure 4. Simulated images with the presence of striped wakes (a) and natural waves (c). The corresponding images after Fourier transform are shown in (b) and (d).

Figure 5. Flow chart for striped wake detection.

1. A subimage was obtained by cropping the original image with a candidate hull at the center.

2. The frequency image was generated using the fast Fourier transform and shifting the zero-frequency component to the center.

3. Connected components were acquired by threshold segmentation using the Otsu method (Otsu 1979). Whether a component corresponded to a striped wake was determined by its shape features, including length, maximum width, and their ratio. After implementing the above steps to simulate wakes with different imaging conditions using the method proposed in Liu, Deng, and Zhao (2019), the threshold of each shape feature was obtained through statistics of shape features for connected components in the frequency domain. Gaussian noises were added to provide close enough to actual circumstance. Connected components whose length, width, and length-width ratio exceeded the thresholds were then eliminated. If more than one pair of connected components were left, the pair of connected components with the largest area would be reserved.

4. Based on the principle of the fast Fourier transform, the wave number kt and the propagation direction Φt for the center of the transverse wake were calculated after a striped wake was detected through

(20) $k_t=2\pi \sqrt{{\left(\frac{x_k}{N\mathrm{\Delta }x}\right)}^2+{\left(\frac{y_k}{M\mathrm{\Delta }y}\right)}^2}$(20)

(21) ${\mathrm{\Phi }}_t=arctan\left[\left(\frac{y_k}{M\mathrm{\Delta }y}\right)/\left(\frac{x_k}{N\mathrm{\Delta }x}\right)\right]$(21)

where xk and yk are coordinates for the pixel with the maximum value in the connected component corresponding to the striped wake; M and N are the number of rows and column, respectively; Δx and Δy are the resolutions in the row and column direction, respectively, both of which are 2 m in this study. The striped wake in far region can be considered as freely propagating gravity wave (Liu and Deng 2018). The propagation velocity was calculated according to the dispersion relationship of gravity wave in deep water via where g is the gravity acceleration. Here, the value of ship velocity was equal to ct according to the linear model of Kelvin wakes (Zilman, Zapolski, and Marom 2015; Oumansour, Wang, and Saillard 1996).

• (5) Fourier transform images were centrally symmetric. This means that wakes may propagate in a direction of Φt or Φt + 180. In order to confirm the propagation direction, the subimage was divided equally into two parts. If −45º<Φt < 45º, left and right parts were obtained. Otherwise, upper and lower parts were obtained. The striped wake detection was repeated for each part. The propagation direction was determined from which part the striped wake was observed in. If striped wakes existed in both parts, they were actually natural waves and identified as false alarms.

It should be noted that striped wakes were mainly used to distinguish between cargo ships and warships as depicted below. Although some weak striped wakes generated by slow ships may not present a crescent shape in the image after the Fourier transform and were left out, it would not influence ship classification.

2.2.3 Ship detection and classification

Most previous studies focused on determination of true ships from candidate hulls using machine learning or deep learning classifiers. In this study, hull classification and wake presence were coupled to improve detection accuracy. The classifier produced the probability that a candidate hull belonged to a true ship, rather than classification labels. The probability would be large when the number of wakes is large and the category of wakes is also large. As a rule of thumb, the number of wakes would increase the probability: (1) by 0.1 if only one turbulent wake existed, (2) by 0.2 if one turbulent wake and one Kelvin arm existed, (3) by 0.4 if one turbulent wake and two Kelvin arms existed or the striped wake was detected or internal waves appeared. Finally, candidate hulls with a probability more than 0.5 were identified as true ships. Furthermore, ship information was complete when wakes were involved and some false alarms could be eliminated through expert decision.

After ship detection, targets identified as true ships were further classified as fishing vessel, motorboat, cargo ship, or warship. The comprehensive information on ship hulls and wakes was extracted from the visual inspection and statistical analysis of large amount of ships with various Froude numbers and background as reported in our previous works (Liu and Deng 2018; Liu, Zhao, and Qin 2021). The results showed that each ship category had characteristic wakes and there were significant differences among the categories. The hull and wake features for different kinds of ships are summarized in Table 3.

Table 3. Hull and wake features for different kinds of ships.

Ship category	Hull feature	Wake feature
Fishing vessel	Short	Only turbulent wake appears
Motorboat	Very short	A turbulent wake and two bright narrow V-shape Kelvin arms appear simultaneously
Cargo ship	Long, low length-width ratio	A turbulent wake and 0–2 classical Kelvin arms exist, striped wake and internal wave may present
Warship	Long and thin, high length-width ratio	A turbulent wake, 0–2 classical Kelvin arms and complete striped wake exist at cruising speed and narrow V-shape Kelvin arms appear at full speed

It should be noted that the four ship types were categorized by their hull and wake features presented in optical images, rather than the actual types of ships. These criteria classified ships using their features shown in optical remote-sensing imagery and assured that most ships were included (Liu and Deng 2018). From Table 3, it can be seen that the same kind of ships may present different features of hull and wake while the same feature of hull and wake may correspond to different types of ships. A fuzzy classifier following the soft voting of the ensemble learning that returned the sum of predicted probabilities was designed to address the situation. Each hull or wake feature was treated as a classifier and the probability that a ship belonged to each category was predicted through expert decision and listed in Table 4 where wake features denote the wake existence. The final class label was then derived from the category with the largest averaged probability.

Table 4. Probability that a ship was classified as one of the categories based on hull and wake features.

Feature	Fishing vessel	Motorboat	Cargo ship	Warship
L^a<=30 m	0.5	0.5	0	0
L>30 m	0	0.2	0.4	0.4
L/W^a<=4	0.4	0.3	0.3	0
L/W>4	0	0	0.2	0.8
Classical Kelvin arm	0	0	0.4	0.6
Narrow V-shape arms	0	0.8	0	0.2
Striped wake	0	0	0.2	0.8
Internal wave	0	0	1	0

^aL and W denote length and width of ship hull, respectively.

In this study, the bright turbulent regions right after ship hulls were treated as part of ship hulls. Turbulent wakes were abandoned since they can be generated by all moving ships except when internal waves appeared, which had no influence on the classification. The hull length of fishing vessels and motorboats is relatively small while cargo ships and warships are relatively long. Thresholds of hull length and length-width ratio were experimentally set to 30 m and 4, respectively. A ship with length larger than 30 m may be a motorboat that can generate long bright turbulent regions. The likelihood to produce classical Kelvin arms by warships is higher than by cargo ships. A higher probability can thus be expected for a ship to be classified as a warship than as a cargo ship when classical Kelvin arms exist. Narrow V-shape arms can be observed when a ship moves very fast. The speed for motorboats of small hulls can easily reach more than 30 kt with most narrow V-shape arms. The speed of warships can also amount to 30 kt but for a limited number of cases. The vote was then set to 0.8 and 0.2 for motorboats and warships, respectively. The detected striped wakes in this study were obvious and generated by fast-moving ships. Only warships and some cargo ships, such as container ships, met this condition and got the votes. Only cargo ships can generate internal waves and got 100% of the vote when internal waves appeared.

Each feature was treated equally and assigned the same weights. Wakes that were not detected were dropped out during the final probability calculation. The category with the highest averaged probability referred to the final class label for each ship. It should be noted that each feature was entitled to exercise a veto with only one vote. In other words, if a feature yielded a probability of 0 for a category, the ship to be classified would no longer be included in the specific category. In addition, if a ship was classified as warship, there must be narrow V-shape arms or striped wakes as a result of the large velocity in most cases.

2.3 Evaluation criteria

Ship detection is a binary classification problem in essence. The predicted samples were divided into four categories, namely, (1) True positive (TP): The samples predicted as ships were true ships; (2) True negative (TN): The samples predicted as false alarms were true false alarms; (3) False positive (FP): The samples predicted as ships were false alarms; (4) False negative (FN): The samples predicted as false alarms were ships.

Several metrics were then calculated to assess the performance of the proposed method in this study. They were defined as follows:

(22) $R=\frac{TP}{TP+FN}$(22)

(23) $P=\frac{TP}{TP+FP}$(23)

(24) $A=\frac{TP+TN}{TP+TN+FP+FN}$(24)

(25) $S=\frac{TN}{TN+FP}$(25)

where R, P, A, and S represent recall, precision, overall accuracy, and specificity, respectively. The larger metrics, the better performance.

The F-measure (F) can provide a way of combining recall and precision and was also employed to comprehensively evaluate the detection performance (Chinchor and Sundheim 1993):

(26) $F=\frac{\left({\beta }^2+1\right)\times \mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}\times \mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}}{{\beta }^2\times \mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}+\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}}$(26)

where β² denotes the relative importance of recall over precision. It was set to 1, which means that recall and precision are of equal weight. The larger F, the better performance.

3. Results

Authentic ship hull shapes are favorable for false alarm elimination, wake detection, and ship classification from satellite imagery. In this regard, the PFT algorithm and hull refining module were utilized to obtain shapes and locations of candidate targets. Examples of ship hulls from visual inspection are shown in Figure 6 to illustrate the effectiveness of PFT and hull refining. The ship hulls were accurately located by the PFT algorithm and the clutter noises were suppressed. On the other hand, adjacent areas of ship hulls were also involved in the salient regions resulting in dilatant candidate hulls with irregular boundaries. After hull refining, shapes of ship hulls were restored well.

Figure 6. The saliency detection and hull refining results of ship hulls.

Representative results of ship detection and classification are shown in Figure 7. Although many speckle noises and wind-induced waves existed, anchored ships can be accurately recognized, as displayed in Figure 7(a,b). The brightness of ship hulls demonstrates discrepancy. There were more than one ships in Figure 7c. The small ship in the middle of the image was successfully detected from complicated background with only one turbulent wake (Figure 7d) and classified as fishing vessel. The other ship was not the focus of attention and the detection result was not shown. As shown in Figure 7(e,f), the ship hull was originally small but covered by surrounding bright turbulent regions, which led to the big detected hull. The ship was correctly classified as motorboat thanks to the narrow V-shape Kelvin arms. A ship longer that 30 m with a turbulent wake and a Kelvin arm can be clearly seen in Figure 7(g,h). It was classified as cargo ship. A small turbulent region besides the ship was excluded from the detected ship hull. Similarly, Figure 7(i,j) show another cargo ship with a turbulent wake and a Kelvin arm in the other direction. It was a container ship by the lattice-like hull. The Kelvin arm in the 2-m resolution image presents as cusp waves with distinct crests and troughs. The crests dominate the reflectance result in a bright line in the 8 m-resolution NIR image. The ship in Figure 7k was identified as a false target only by the hull, which was successfully fixed by the detected turbulent region shown in Figure 7i. The ship was finally classified as a cargo ship by combining the hull information. Figure 7(m,n) illustrate a ship moving fast with obvious transverse wave. The striped wake was nearly eliminated by frequency domain filtering with the ship hull unchanged in the subimage. This demonstrates that the striped wake was detected through searching peak regions in the Fourier transform domain. Combined with large length and length-width ratio, the ship was classified as warship. There were two moving ships close to each other with one on the Kelvin arm of the other, as illustrated in Figure 7(o,p). The hull and wake detection of the two ships were weakly influenced by each other and both ships were classified as cargo ships. Some transverse waves could be seen near the right Kelvin arm of the lower ship with alternating peaks and troughs, which made the Kelvin arm invisible in the NIR image where linear wakes were detected.

Figure 7. Ship detection and classification results for different scenario. The green polygons were boundaries of ship hulls. Blue and white lines demonstrate dark and bright linear wakes, respectively.

To our best knowledge, it is the first time that ship wakes have been used to improve hull detection accuracy and classify ships. Ship detection was conducted separately to compare with state-of-the-art coarse-to-fine ship detection methods using optical images of similar resolution, as reported in Shi et al. (2014), Yang, Xu, and Li (2017), Dong, Liu, and Fang (2018), and Nie et al. (2020). The algorithms and parameters for candidate target extraction were different among them, most of which used the Otsu method (Otsu 1979) to obtain thresholds for binarization of saliency maps; whereas this was unsuitable for imagery covering a large area. In order to focus on false alarm elimination, the candidate targets extracted from this study were fed into the four state-of-the-art algorithms. Furthermore, the training samples in this paper were collected for traditional machine learning methods and far from enough for deep learning methods. Therefore, the comparison between results from our method and those from the deep learning methods is beyond the scope of this study. The overall performance of each method is listed in Table 5. The method proposed in this paper shows the best performance, as demonstrated by the largest recall, precision, F, overall accuracy, and specificity. It is also worth noting that recall and precision for the proposed method based on only hull information are increased by 1.6% and 9.5%, respectively, when both hull and wake information were included, which demonstrated that wakes contribute to increase the accuracy of ship detection.

Table 5. Comparison between the performance for the method proposed in this study and state-of-the-art methods.

Method	R	P	F	A	S
Shi et al. (2014)	75.6%	46.1%	0.57	93.9%	94.9%
Yang, Xu, and Li (2017)	75.0%	42.9%	0.55	93.2%	94.3%
Dong, Liu, and Fang (2018)	80.8%	65.3%	0.72	96.6%	97.5%
Nie et al. (2020)	61.5%	21.3%	0.32	85.5%	86.9%
This study without wake	88.5%	78.6%	0.83	98.1%	98.6%
This study	90.1%	88.1%	0.89	98.8%	99.3%

The statistics results for length, length-width ratio, and area of targets identified as true ships are shown in Figure 8, where 1 pixel corresponds to 2 m. The area of the detected true ships ranged from 41 to 8373 pixels with a mean of 649.3 pixels and a standard deviation of 1035.9 pixels. The maximum count locates at the initial bins. More than half of ships fell in the first three bins. The length of detected true ships ranged from 10 to 240 pixels with a mean of 46.0 pixels and a standard deviation of 38.5 pixels. The distribution of length is similar to that of the area except that the maximum count was found in the bin of 20–25 pixels. More than 80% ships were shorter than 65 pixels. The length-width ratio of detected true ships ranged from 1.1 to 7.7 with a mean of 3.25 and a standard deviation of 1.34. The length-width ratio was close to a normal distribution and the maximum count locate in the bin of 2.6–2.75. The overall length-width ratios were lower than they actually are since the turbulent regions around ships were identified as ship hulls.

Figure 8. Statistics of the length, length-width ratio and area of targets identified as true ships.

Most ships were small- to medium-sized in the high-resolution satellite images according to the distribution of length and area. Meanwhile, most false alarms in the images were also small, such as white foams caused by breaking waves, buoys, and floating debris. Focusing on small targets would lead to many false alarms left. It is difficult to balance the tradeoff. In the original methods proposed by Yang, Xu, and Li (2017), targets with areas less than 100 pixels were all removed, resulting in a high precision of 87.8% but very low recall of 36.9%. Nevertheless, using the shape filters employed in this paper, many small ships were retained as indicated by the increased recall of 75.0%. On the other hand, the precision significantly decreased to 42.9% due to many survived false alarms. There was no shape filtering process in the other two methods and the small-sized false alarms were not removed effectively. Our proposed method balanced the recall and precision well even for small ships. The length and area of detected ships were as low as 10 pixels and 41 pixels, respectively. The statistics analysis shows that most small- to medium-sized ships can be identified with high accuracy.

Targets identified as true ships with detected wakes were further classified based on the combination of hull and wake features. Ship classification results are listed in Table 6. There are 99 moving ships with wakes that can be classified according to manual statistics. Among them, 83 ships were correctly classified and false classification happened for only 1 ship. The omission number was 15. The overall accuracy of correct classification was 83.8% while the proportion of false classification was 1.0%. Through manual validation, the omitted ships were not involved in the classification process since their wakes were not detected. The ship of false classification was a motorboat with a turbulent wake and only one obvious Kelvin arm and regarded as a cargo ship. All ships with correctly detected wakes were classified successfully and misclassification was caused by false wakes, which confirmed the effectiveness of the fuzzy classifier proposed in this paper.

Table 6. Classification results of ships with detected wakes.

Ship category	Manual statistics	Correct classification number	False classification number	Omission number
Fishing vessel	3	3	0	0
Motorboat	3	1	1	1
Cargo ship	91	77	0	14
Warship	2	2	0	0

4 Discussion

4.1 Feature and classifier selection

False alarm elimination using texture features is a key procedure for ship detection. Based on the extracted features, the ship detection problem can be considered as traditional classification task. According to whether the algorithm development relies on labeled samples provided by user, the classification algorithms can be mainly categorized into supervised and unsupervised approaches. Supervised classification algorithms can deliver more determined results and were employed in most ship detection methods. However, no uniform texture features and classifiers were adopted in existing detection methods. The combination of various texture features and classifiers would produce totally different results. In order to obtain optimal texture feature and classifier, typical rotation invariant texture features and classifiers were combined and tested here.

Moment invariants (MI) (Hu 1962), local binary pattern (LBP) (Ojala, Pietikainen, and Maenpaa 2002), radial gradient transform (RGT) (Dalal and Triggs 2005; Takacs et al. 2013), region covariance descriptor (RCD) (Tuzel, Porikli, and Meer 2006; Dong, Liu, and Fang 2018), speeded-up robust features (SURF) (Bay, Tuytelaars, and Van Gool 2006), and KAZE features (Alcantarilla, Bartoli, and Davison 2012) were proven to be effective and widely used as rotation invariant texture features for target classification. The considered classifiers covered various types of supervised learning methods, including artificial neural network (ANN) (Glorot and Bengio 2010), hyperplane-based support vector machine (SVM) with linear kernel function (Christianini and Shawe-Taylor 2000), decision tree-based random forest (RF) (Breiman 2001), and probabilistic-based GP (Rasmussen 2004). The texture features and classifiers are summarized in Table 7.

Table 7. Summarization of invariant texture feature and classifiers.

Texture feature	MI, LBP, RGT, RCD, SURF, KAZE
Classifier	ANN, SVM, RF, GP

All potential combinations of texture features were tested to train each classifier for ship hull detection. The test was only implemented to part of sample images to ensure the robustness of algorithms. The optimal texture feature combinations for each classifier are listed in Table 8 through exhaustive method. The largest F was achieved by the GP classifier trained by LBP, RCD, and KAZE features, which was used in this paper. The most popular texture features were RGH, RCD, and KAZE. Note that RCD can be found in each classifier. The MI feature was not selected by any classifier. The number of optimal texture features for each classifier varied between 3 and 4. Insufficient texture features would hinder distinguishing true ships from false alarms while excessive texture features would yield poor performance. A few but effective texture features were enough for binary ship classification.

Table 8. The optimal texture feature combinations for different classifiers.

Classifier	Texture feature	F
ANN	RGH+RCD+KAZE	0.843
SVM	LBP+RGH+RCD+SURF	0.813
RF	RGH+RCD+KAZE	0.857
GP	LBP+RCD+KAZE	0.862

4.2 Effects of key parameters

Adjustable parameters for linear wake detection were tuned according to Liu et al. (2021). The shape parameters for striped wake detection were obtained from simulation samples with no need for optimization since striped wakes were mainly used to distinguish between cargo ships and warships and high precision can meet the requirement. Only parameters involved in hull detection are necessary to be discussed, including SI and shape filtering threshold. The minimum and maximum of shape features for shape filtering were not tuned to avoid over-fitting and assure robustness. Therefore, the key parameter for hull detection, SI, was assessed below.

The threshold for SI (refer to as T_SI hereafter) was determined through whether a hull corresponded to the shape of a ship. Small T_SI usually led to swollen hulls that contained turbulent regions and foams, resulting in false hull retained. Nevertheless, the refined hulls may not only include the brightest parts of ships but also provide false shape features. In order to determine the optimal T_SI, different values were tested on sample images with ships at the center. The minimum enclosing rectangle of each ship was recorded manually. The Intersection-over-Union (IoU) between the detected hull, H, and the minimum enclosing rectangle, R, was employed to assess the performance. It was calculated from

(27) $\mathrm{I}\mathrm{o}\mathrm{U}=\frac{\mathrm{H}\cap \mathrm{R}}{\mathrm{H}\cup \mathrm{R}}$(27)

where H∩R and H∪R are the intersection and union areas between H and R, respectively. The larger IoU, the more optimal T_SI.

The mean IoU for all samples was calculated to test the performance of T_SI and IoU was used for convenience. The variation curve is depicted in Figure 9. With small T_SI, the shape feature filter condition SI > T_SI can be easily satisfied for candidate hulls. Refined hulls were derived from the binary segmentation by the initial threshold. As expected, IoU remained unchanged with T_SI. Shape feature filter started to vary as T_SI increased. The peak of IoU reached at T_SI = 1.4. IoU then decreased sharply to almost 0 with the continuous increase in T_SI. This can be explained by the fact that only the brightest parts of candidate hulls were left when T_SI was too large.

Figure 9. The minimum IoU of all samples with the change of T_SI.

5. Conclusion

A method for cascaded detection of ship hull and wake was proposed in this paper. Firstly, locations and shapes of candidate ship hulls were acquired by PFT algorithm and hull refining module from 2 m-resolution panchromatic images. Obvious false hulls were preliminarily removed through shape features composed of area, length, width, and length-width ratio. Texture features of candidate hulls were then extracted and imported into a Gaussian process classifier to obtain the probability of each candidate hull as a ship. Meanwhile, linear wakes and striped wakes were detected around all candidate hulls using 8 m-resolution NIR images and 2 m-resolution panchromatic images, respectively. The probability of a candidate identified as a ship increased if wakes were included. Finally, candidates with the probability higher than 0.5 were regarded as ships. True ships with wakes were further classified as fishing vessels, motorboats, cargo ships, and warships using a fuzzy classifier following the soft voting of the ensemble learning. The proposed method was implementing to multispectral high-resolution GF-1 PMS images. Recall, precision, overall accuracy, and specificity of hull detection amounted to 90.1%, 88.1%, 98.8%, and 99.3%, respectively, demonstrating better performance relative to other state-of-the-art coarse-to-fine ship detection methods. To our best knowledge, it is the first time that ship hull and wake were cascaded for ship detection and wakes were used to improve hull detection accuracy and classify ships. Factors influencing the accuracy of the developed method, including texture features and classifiers combination and key parameters of the method, were discussed.

Combination of hull and wake gives an opportunity to classify ships using images of relatively low resolution. The proposed method can also be applied to other satellite sensors of both high and moderate resolution. It should be noted that the ship classification was designed for normal traveling ships and not applicable for exceptional cases, such as low-speed motorboats, which is a considerable limitation of the proposed method. Nonetheless, it is of great significance in practice since static ships are mainly anchored in ports under supervision while moving ships in the open sea show higher uncertainty. In the future, a method that separate ship hulls from bright turbulent regions will be designed to obtain precise ship shapes and more ship categories will be distinguished. The newly designed method can be operationally implemented for real-time ship monitoring.

Disclosure statement

No potential conflict of interest was reported by the authors.

Data availability statement

The data that support the findings of this study are from Guangdong Data and Application Center for High-resolution Earth Observation System at http://gdgf.gd.gov.cn/GDGF_Portal/index.jsp.

Additional information

Funding

This work is supported by the Southern Marine Science and Engineering Guangdong Laboratory (Zhuhai) Grant [number SML2021SP308]; the Innovation Group Project of Southern Marine Science and Engineering Guangdong Laboratory (Zhuhai) Grant [number 311020004]; the National Natural Science Foundation of China Grant [number 42176173, 41901352, 41071230]; the China Postdoctoral Science Foundation Grant [number 2020M672938]; and the Guangdong Basic and Applied Basic Research Foundation Grant [number 2019A1515111151]. We would like to thank Guangdong Data and Application Center for High-resolution Earth Observation System for providing GF-1 data.