FastPFM: a multi-scale ship detection algorithm for complex scenes based on SAR images

Abstract

Synthetic Aperture Radar (SAR) is renowned for its all-weather capabilities, exceptional penetration, and high-resolution imaging, making SAR-based ship detection crucial for maritime surveillance and sea rescue operations. However, various challenges, such as blurred ship contours, complex backgrounds, and uneven scale distribution, can impede detection performance improvement. In this study, we propose FastPFM, a novel ship detection model developed to address these challenges. Firstly, we utilize FasterNet as the backbone network to reduce computational redundancy, enhancing feature extraction efficiency and overall computational performance. Additionally, we employ the Feature Bi-level Routing Transformation model (FBM) to obtain global feature information and enhance focus on target regions. Secondly, the PFM module is engineered to collect multi-scale target information effectively by establishing connections across stages, thereby improving fusion of target features. Thirdly, an extra target feature fusion layer is introduced to enhance small ship detection precision and accommodate multi-scale targets. Finally, comprehensive tests on SSDD and HRSID datasets validate FastPFM's efficacy. Compared to the baseline model YOLOX, FastPFM achieves a 5.5% and 4.4% improvement in detection accuracy, respectively. Furthermore, FastPFM demonstrates comparable or superior performance to other detection algorithms, achieving 92.1% and 83.1% accuracy on AP50, respectively.

KEYWORDS:

• Synthetic aperture radar (SAR) images
• complex scenes
• multi-scale ship detection
• YOLOX

1. Introduction

Advances in satellite imagery technology have made it easier to obtain high-resolution and wide-area photographs from space, which has improved assistance for a number of sectors, such as environmental monitoring, traffic management, and natural catastrophe identification. Furthermore, it has opened up promising opportunities for ship detection. In ocean surveillance, ship detection is essential because it makes it possible to perform duties like ship identification, tracking, and monitoring as well as to ensure maritime safety and facilitate rescue operations. These endeavours are paramount in safeguarding marine security, managing marine resources, and preserving the integrity of our oceans.

Traditional ship target detection methods (Leng et al., 2015; Liu et al., 2019) predominantly rely on Constant False Alarm Rate (CFAR) algorithms. However, these algorithms frequently found it difficult to handle complex coasts and marine clutter since they relied on human expertise and clutter data, which reduced their accuracy and adaptability. Therefore, these methods suffer from low detection precision and poor generalisation capacity in complicated scenarios, such as densely packed ships along the coast, buildings, beaches, tiny islands, and sea clutter, which can affect the detection accuracy of ships. Moreover, their performance may be limited under adverse weather conditions such as dense fog or nighttime. Therefore, in order to maintain the efficacy and security of maritime trade, a more accurate and efficient way of identifying ship targets is needed.

Ship identification has advanced to a new level thanks to the advancement of deep learning (DL) and convolutional neural network models (CNNs), which have addressed the shortcomings of earlier techniques. CNN-based ship detection has the advantages of more vital feature learning ability, better generalisation ability, better adaptability, and higher detection accuracy. The SAR technology utilises the synthetic aperture effect of the radar beam to obtain high-resolution images. In addition, it has all-weather imaging capability, which is almost independent of weather conditions (e.g. clouds, haze, etc.) and lighting conditions (e.g. night) (Li et al., 2021; Zhang & Hao, 2022). Deep learning-based SAR ship identification techniques have arisen as a result. This brings significant advantages for ship monitoring. Currently, one-stage ship detection (Tang et al., 2021) and two-stage ship detection methods (Lin et al., 2019; Zhao et al., 2020) are the two main research directions of DL-based SAR ship identification algorithms. Two-stage object identification methods take a two-step strategy, generating candidate regions first, then classifying and regressing position on those areas. While this approach achieves high detection accuracy, it is slow and computationally expensive, contradicting the “lightweight” concept and the real-time requirements of vessel detection. One-stage object identification algorithms, which offer simplicity and speed and are appropriate for real-time ship recognition during ship navigation, directly anticipate the position and kind of targets from the input image. Thus, we will adopt the one-stage detection method in this study.

Unfortunately, while SAR (Synthetic Aperture Radar) imagery brings developments and convenience to ship detection, it also presents several challenges. Firstly, sea waves and other disturbances frequently impact ship targets in SAR photos, resulting in hazy borders that make recognition and extraction difficult. Secondly, the water often looks dark in SAR photos, whereas ships are typically brilliant white. However, nearby islands, structures, and surrounding noise may also display comparable bright white hues, greatly enhancing the challenge of ship recognition (Zhao et al., 2023). Additionally, ships in motion can undergo significant geometric transformations in SAR images, resulting in blurred ship images as well as irregular patterns in the ship's silhouette (Wang et al., 2023a, 2023b). Lastly, in SAR images, the shape and size of ship targets may vary, and small ships dominate, resulting in an imbalanced distribution of ship scales and further increasing the challenges of target detection.

Since SAR ship images have some of the factors mentioned above that are unfavourable to the model detection performance, it brings a significant challenge to ship detection technology. First, the detailed features will be lost in the detection since the boat object in the SAR picture is hazy and susceptible to interference. FPN (Feature Pyramid Network) uses up-sampling and down-sampling to fuse different levels of features, so it may lead to the loss of detailed features after fusion or contain noise, which ultimately reduces the performance of the model detection. Secondly, tiny ships are only shown as a bright point in SAR photographs in real-world circumstances since small and medium-sized ships make up the bulk of SAR ship images. The distribution of ships along the shoreline or in narrow channels is more dense and affected by the buildings along the coastline. During detection, detecting dense small ships and differentiating between ships and buildings along the shoreline is challenging. Finally, many existing models have feature extraction networks and feature fusion networks that are computationally intensive and have poor robustness. These structures are usually operated by constant stacking with fixed up and downsampling. Because of this, the model's overall capacity for feature extraction and fusion is hampered, which lowers the model's capacity for generalisation and increases its processing power – a need that is insufficient for detection in real time.

To address the above challenges, we introduce FastPFM, a ship detection method building upon the YOLOX (You Only Look Once X) (Ge et al., 2021) algorithm to mainly target complex scenes and multi-scale scenarios. This method improves the model's ability to multi-scale ship combination of features, which improves the recognition of ships in SAR picture datasets and gathers more detailed contextual information, in addition to explicitly enhancing the model's extraction of features capability for tiny ships to alleviate the problem of low detection accuracy for large vessels caused by the dataset's uneven distribution of ship sizes.

The following is a summary of this paper's significant contributions:

• We proposed a YOLOX-based ship detection method, FastPFM, which adopts FasterNet as the foundation network of FastPFM to increase computational efficiency and detection accuracy and better utilise redundant data in feature maps. Additionally, we designed the FBM (Feature Bi-level Routing Transformation Module) module to divide high-level feature information into two parts in the backbone network and process them using BRAttention, followed by merging through a cross-stage structure. This design enables better capture of global features, achieving higher precision in ship detection.

• Designed a PFM (PConv Feature Module) module to utilise PConv convolution to reduce the model's computational burden and memory access. Furthermore, we employ a cross-stage connection structure to effectively gather the targets’ multiple scales and semantic characteristics, improving the object detection process's accuracy and robustness.

• Conducted model performance and generalisation tests compared to baseline YOLOX and other state-of-the-art (SOTA) methods on benchmark datasets HRSID (Wei et al., 2020) and SSDD (Zhang, Zhang, Li, et al., 2021). The experimental results demonstrate a significant improvement in detection accuracy with the FastPFM model compared to YOLOX. Furthermore, compared to other SOTAs, FastPFM performs better all around.

The remainder of this article is organised as follows: Part II provides an overview of important studies to our work; Part III discusses the proposed method; Part IV discusses the outcomes of the experiments and analysis, and Part V finishes with an overview of our work and future plans.

2. Related work

To provide readers a better knowledge of current breakthroughs in the field, this chapter focuses on introducing several traditional and DL-based SAR ship recognition algorithms, which played an important role in the development of our research technique.

2.1. Traditional ship detection algorithm

Conventional SAR ship target recognition algorithms heavily rely on the CFAR (Steenson, 1968), which dynamically adjusts thresholds based on variations in background noise, ensuring a consistent false alarm rate under different conditions. Despite effectively suppressing the influence of background clutter, leading to improved target detection accuracy, the CFAR algorithm is vulnerable to complicated coastlines, sea waves, and speckle noise since it relies on expert knowledge and handcrafted features, which may lead it to face lower detection accuracy and limited generalisation. Moreover, the effectiveness of handcrafted features in all situations is challenging to ensure, limiting overall robustness.

Many researchers work on enhancing the detection performance of the CFAR algorithm, reducing its sensitivity to complex shorelines, etc. Because the probabilistic neural network (PNN) model still needs to be enhanced in terms of recognition precision and speed in SAR pictures, Du et al. (Du et al., 2008) processed 8-bit and 16-bit SAR images, proposed a C-PNN model, and used a new CFAR algorithm that outperformed traditional PNN-based ship detection. Furthermore, because ship recognition techniques usually rely on high-resolution SAR pictures that are impacted by background clutter, many ship detections employ CFAR algorithms. Migliaccio et al. (Migliaccio et al., 2008) suggested a ship detection filtering approach using SLC pictures that can successfully cope with scattering patches in the image while maintaining a low false alarm rate. Traditional ship detection algorithms cannot recognise near-shore ships successfully due to the great similarity between ports and ships in terms of grey and texture properties. Zhai et al. (Zhai et al., 2016) recommended combining saliency and contextual information to solve this problem. They proposed a ship identification frame to reduce false alarms and hence improve the model's whole detection performance. Overall, these research are aimed at enhancing the model's accuracy and effectiveness in SAR ship identification, as well as addressing the numerous problems encountered.

Despite their advantages, traditional ship detection algorithms still have several drawbacks, such as limited versatility, poor adaptability to varied settings, and inability to satisfy the needs of immediate detection. Furthermore, typical ship detection method frequently make it difficult to effectively distinguish ships in multi-scale and complicated settings. Therefore, deep learning-based SAR ship detection algorithms are becoming dominant, gradually surpassing these traditional methods (Zheng, Li, et al., 2023).

2.2. Deep learning based SAR ship detection algorithm

Throughout recent times, DL-based techniques for synthetic-aperture radar (SAR) pictures have been developed to overcome the constraints of classic ship detection systems, and they can be divided into two categories: one-stage and two-stage ship detection algorithms. Deep learning is currently making strides with enormous amounts of labelled data. However, due to the shortage of large volumes of labelled data, developing semi-supervised ship detection algorithms is critical, limiting the broad application of deep learning (Rai et al., 2022). In the past few years, an increasing number of researchers have applied semi-supervised learning frameworks on SAR ship datasets to increase ship detection accuracy (Wang, Shi, et al., 2021; Zhou, Jiang, et al., 2023). Several research papers employ deep learning-driven ship identification approaches to ship trajectory prediction and container loading (Qian et al., 2022; Zong & Wan, 2022).

Despite significant advances in the use of deep learning in ship detection systems for SAR imagery, numerous hurdles and limitations remain. First, due to the unique nature of SAR images, such as image backdrop clutter and target complexity, building effective feature extraction and classifiers remains difficult. Second, while semi-supervised learning approaches help ameliorate the problem of insufficient labelled data, the question of how to efficiently use unlabelled data remains unanswered. Furthermore, applying deep learning approaches to domains like ship trajectory prediction and container loading necessitates additional research and exploration. Future research can concentrate on resolving these issues in order to increase the performance and dependability of ship detection and related applications.

2.2.1. Deep learning based two-stage SAR ship detection algorithm

The emergence of two-stage object detection methods has proven beneficial in addressing the limits of traditional methods. These methods enhance the performance and robustness of ship recognition by utilising deep learning's powerful feature learning capability, multi-scale detection capability, and efficient NMS (Non-maximum suppression). Currently, prominent two-stage object detection algorithms mainly contain Fast R-CNN (Girshick, 2015), Faster R-CNN (Ren et al., 2015), and Cascade R-CNN (Cai & Vasconcelos, 2019). Because traditional methods require segmenting the coast before detection but are inaccurate, resulting in poor detection performance, Lin et al. (Lin et al., 2019) put forward a Faster R-CNN-based structure that uses squeezing and stimulation techniques to improve ship detection efficiency. Furthermore, due to the effect of different ship sizes and various backdrops in SAR images, recognising multi-scale ships in complex surroundings remains difficult. As a result, Zhao et al. (Zhao et al., 2020) proposed an Attention-Receptive Pyramid Network (ARPN) by combining Receiving Field Block (RFB) and Convolutional Block Attention Module (CBAM), which improves the ability to identify of multiple scales ships in complicated background SAR images by enhancing vital information and suppressing interference caused by the surroundings. Zhang et al. (Zhang et al., 2021) proposed a novel Quad-Feature Pyramid Network (Quad-FPN) for SAR ship detection, made up of four unique FPNs, i.e. the fusion of features from multiple FPNs allows the model to contain more helpful ship shape information while reducing the influence of complex background. Because scattering noise in SAR images reduces the quality of the model for feature learning and impedes understanding of semantic features for target detection, Zhang et al. (X. Zhang et al., 2021) proposed a multi-task learning-based target detector (MTL-Det) for identifying ships in SAR pictures, which models the ship identification problem as three collaborative tasks, and by multi-tasking learning it can learn features of more discriminative targets.

Despite the benefits of precision, strong feature learning capabilities, and robust multi-scale detection abilities, two-stage object detection methods are relatively complex, slower, and have limited capabilities in detecting small objects. Hence, careful consideration of practical requirements and scenarios is essential when selecting a ship detection algorithm.

2.2.2. Deep learning-based one-stage SAR ship detection algorithm

The main one-stage ship identification systems are the YOLO series, SSD (Liu et al., 2016), FCOS (Tian et al., 2019), or RetinaNet (Lin, Goyal, et al., 2017). These approaches directly detect and locate objects, offering advantages such as lower computational complexity and faster detection times. For this reason, they are well-suited for real-time ship detection, making them frequently employed in recent works. However, ship detection is somewhat hampered by the unbalanced ship scale distribution in the SSDD and HRSID SAR datasets.

Improvements have been suggested by researchers as a solution to the scale imbalance issue. The introduction of the Feature Pyramid Network (FPN) by Lin et al. (Lin, Dollár, et al., 2017) addressed this issue by fusing multi-scale ship features using feature maps of various scales, thereby improving the model's capability to represent ship goals and effectively handling multi-scale ships. Several scholars have made improvements to FPN for detecting multi-scale ships. When merging semantic and spatial location, Zhu et al. introduced DB-FPN based on YOLOv5 (Zhu et al., 2021), which enhances the process by employing a recurring structure. However, the detection capabilities of DB-FPN are constrained in complicated scenarios. To enhance the detection capability of models in such scenarios, Zhou et al. (Zhou, Fu, et al., 2023) introduced the D-MFPN, which refines the expressiveness of feature maps by integrating a multi-layer feature pyramid network and attention modules. By combining multi-scale ship feature information through both bottom-up and top-down routes. Sun et al. (Sun, Leng, et al., 2021) introduced Bi-DFFM based on the YOLO algorithm framework, improving the recognition capabilities for multiple scales ships. Yasir et al. (Yasir et al., 2023) increased the accuracy of YOLOv5 for multi-scale ship detection by leveraging C3, the attention mechanism, and the PAFPN structure. The great majority of the ships in the SAR dataset are small ships, and the model's total detection capabilities is mostly determined by its detection accuracy. In order to solve the small target detection problem, Chen et al. (J. Chen et al., 2022) suggested a DFM based on the depth of difference. This DFM maps the various difference regions to the two-dimensional candidate region in accordance with the distance.

Besides the issue of imbalanced ship scale. Because there aren't many ships in the water, the majority of the anchor frames are unnecessary, which could make the model's computation take longer. Fu et al. (Fu et al., 2020) proposed a new method called Feature Balancing and Optimization Network (FBR-Net). The network employs a generalised anchorless frame strategy that contains a structure with an attention-guided balancing pyramid. The approach can recognise multi-scale ships by improving the semantic information and improving the object features through the use of a feature refinement module. In addition, several researchers have worked on improving classification and regression algorithms and enhancing detection speed and practical application techniques. To detect ships in SAR photos, Gao et al. presented a dense attention feature aggregation network without anchors (Gao et al., 2020). To improve the regression algorithms in the FCOS network, Sun et al. updated the object categorisation and bounding box regression techniques (Sun, Dai, et al., 2021). They proposed the CP module, which performed well at detecting ships in various datasets of high-resolution SAR images.

While many existing algorithms concentrate detection accuracy, they frequently overlook speed of detection and the technology's practical applications. Most algorithms are not applicable in real-world situations because they rely on strong desktop GPUs. The YOLO-V4-Light network was suggested by Jiang et al. (Jiang et al., 2021), which uses a multiple channels fusion SAR imagery processing technique to overcome these problems. Zheng et al. (Zheng, Zhang, et al., 2023) proposed the MCYOLOv5 ship detection algorithm as an improvement over the current algorithms. It utilises the lightweight network of MobileNetV3 to extract features and creates an effective CNeB feature fusion module, that further decreases the model complexity while enhancing the model's ability to extract and fuse features. Zheng et al. (Zheng et al., 2022) replaced the YOLOv4 feature extraction module with MobileNetV1, which resulted in a significant reduction in computation time while maintaining accuracy in recognition.

Notwithstanding the advantages of one-stage ship detection algorithms, such as speed, low computational complexity, and strong adaptability to small-scale objects, they suffer from relatively low target detection accuracy and weak ability to extract target context information details, resulting in inadequate detection ability in complicated backgrounds. To enhance the model's ability to extract global characteristics and balance computational effort, we thus suggest the FBM module.

3. Methods

In this section, we initially present YOLOX, our baseline model, in this section. Next, we showcase our suggested model's general architectural diagram. Lastly, we discuss the PFM-based multi-scale feature fusion pyramid, FasterNet, FBM, and PFM.

3.1. Baseline model YOLOX

Considering the target scattering, sparsity of ships, and small sample features in SAR images, the anchor-free mechanism, decoupled heads, and dynamic sample allocation strategy of YOLOX (Ge et al., 2021) make it an ideal baseline model. For this reason, we decided to use YOLOX as the foundational model in our study. Moreover, YOLOX is frequently used as a baseline model in ship detection tasks in some advanced methods (Peng & Tan, 2022; J. Zhang et al., 2023). Therefore, YOLOX is also used as a foundational model in this study.

The YOLOX network topology is shown in Figure 1 and is composed of three modules: the Head module for object identification, the Neck module for feature processing, and the Backbone module for feature extraction. YOLOX typically receives 640 * 640 * 3 three-channel RGB images and outputs boxes containing the boundaries of items found together with the pertinent category information. Furthermore, it employs data augmentation methods such as Mixup and Mosaic to enhance the detected objects’ visual backgrounds and mitigate the impact of batch size on the model.

Figure 1. Structure of the YOLOX network model. (BackBone: feature extraction network; PAFPN: feature fusion network; Head: target detection head; the rest are structures used in these three modules)

Backbone. Darknet53, which comprises of CBS, CSP-X, and SPP modules, is the foundation of YOLOX. Convolutional layers, BN layers, and the SiLU function make up the CBS module. The CSP-X module combines CBS modules, ResUnit modules, and Concatenation operations. The ResUnit module consists of CBS modules and addition operations. The SPP module transforms features of arbitrary sizes into fixed-size feature vectors.

Neck. YOLOX uses the PAFPN (Path Aggregation Feature Pyramid Network) topology of the neck portion. This structure takes into account the ambiguity of low-level target information and builds high-level semantic features at various scales through the use of top-down lateral connections. It introduces bottom-up structures to compensate for and reinforce localisation information. PAFPN integrates features from the S2, S3, and S4 layers outputted by the backbone network. Typically, shallow networks contain abundant ship-detail features, advantageous for detecting small vessels, while deeper networks usually possess rich semantic information, favouring the detection of large ships. The PAFPN feature fusion network effectively integrates both shallow and deep features, enabling the network to fully extract hierarchical features, thereby obtaining richer feature information, including strong semantic information, edges, and textures. However, since most ships in SAR images appear as small targets, the PAFPN does not integrate detailed features from the S1 layer, which might have a certain impact on detecting small vessels.

HEAD. YOLOX created the decoupled detection head module, which has several heads, in order to resolve the conflict between the classification and regression jobs. Initially, these heads use 1*1 Conv to minimise the number of channels with features to 256 for classification and regression applications. Then, an addition of two parallel pathways with two 3*3 Conv each. On the regression route, there is also an IoU branch. Another innovation of YOLOX is dynamic sample matching instead of an anchor frame-based approach.

3.2. Our proposed FastPFM network structure

Our suggestion is the FastPFM ship detection network model, which is built around the standard YOLOX architecture. FastPFM is composed of a Backbone, Neck, and Head module, as illustrated in Figure 2. First, FastPFM preprocesses the input image by adjusting its resolution to 640 by 640 pixels. Next, it normalises the image to aid in better model computation. Lastly, it uses data enhancement techniques like Mixup to combine multiple images with different coefficients, helping the model better learn the target's features and enhancing its robustness and generalisation capabilities. However, downsampling to the required resolution of the model during data preprocessing may cause some detailed features to be lost or become blurred, especially for small ships, which may affect their detection capability. Based on this problem, our multi-scale feature fusion network module of PFM can retain more detailed information during the detection process to obtain better semantic and spatial data, improving the overall network detection performance. Consequently, this enhances the network's detection performance as a whole. Furthermore, the pre-processing step involves adjusting the image resolution to achieve uniform image size, which simplifies the model processing and lowers the computational load on the model, speeding up training and inference.

Figure 2. The FastPFM network's overall framework. (BackBone: feature extraction network; Neck: feature fusion network; Head: target detection head; the rest are structures used in these three modules)

Specifically, compared to the standard YOLOX structure, FastPFM has several following improvements:

1. We replace the YOLOX backbone network Darknet53 with FasterNet (J. Chen et al., 2023) to address the issue of low computational efficiency. Moreover, Darknet53 only outputs feature maps from S2, S3, and S4 layers, lacking shallow detailed features, which results in poor performance in detecting small-sized ships. To tackle this, we add an S1 layer as an output layer in the backbone network, providing the necessary shallow detailed features to enhance the identification of tiny vessels.

2. FBM is designed to obtain richer global and contextual information while alleviating the loss of detailed features due to multiple downsampling.

3. Considering the presence of numerous small-sized ship targets in the HRSID dataset, We add the characteristics taken via the S1 layer within the backbone network to the Neck module. This extension enables the Neck module to capture detailed features of smaller targets effectively.

4. To increase feature fusion and broaden the network's receptive area, the PFM module is integrated into the Neck module. Additionally, the original CBS (Conv, BN, SiLU) module in YOLOX is redesigned using PConv, which improves the model's computational efficiency.

3.3. Our proposed FastPFM backbone network

3.3.1. FasterNet feature extraction network

The backbone network of the original YOLOX, Darknet53, used a Conv convolution, which is not only slow but also causes a lot of floating-point operations (FLOPs), increasing the model's latency. Equation (1)(1) $\begin{array}{r}\mathrm{Latency}=\frac{\mathrm{FLO}{P}_S}{\mathrm{FLOPS}}\end{array}$(1) illustrates the link between Latency, FLOPS, and FLOPs. FLOPs stands for floating-point operations, whereas FLOPS stands for floating-point operations per second. Thus, the computation's delay can be expressed as FLOPs divided by FLOPS. (1) $\begin{array}{r}\mathrm{Latency}=\frac{\mathrm{FLO}{P}_S}{\mathrm{FLOPS}}\end{array}$(1) The feature maps that YOLOX's Conv convolution produced were extremely redundant across channels, necessitating the use of additional convolution operations to refine the redundant features. As a result, the model computation's efficiency was reduced, its cost increased significantly, and its latency increased significantly. On the other hand, the FasterNet used in this work performs convolution operations on a subset of input channels while maintaining the integrity of the remaining channels by using PConv convolution as opposed to Conv. Thus, FLOPs are reduced while FLOPS is increased, which reduces latency and improves computational speed and accuracy. Figure 3(a) illustrates the implementation flow of PConv. To ensure continuous memory access, we calculate the first consecutive$\mathrm{Cp}$channels. Equation (2)(2) $\begin{array}{r}h\times w\times k^2\times C{p}^2\end{array}$(2) presents the FLOPs of PConv. (2) $\begin{array}{r}h\times w\times k^2\times C{p}^2\end{array}$(2) When the number of input channels of $\mathrm{Cp}$ is$1/4$, PConv's FLOPs are just $1/16$of Conv2D's FLOPs, and the number of memory accessed is only $1/4$ of Conv2D. The computational publicity for PConv access to memory is shown in publicity (3). (3) $\begin{array}{r}h\times w\times 2c_p+k^2\times c_p^2\approx h\times w\times 2c_p\end{array}$(3)

Figure 3. FasterBlock structure diagram.

As shown in Figure 2, four identical FasterBlock layers make up the FasterNet backbone network, and each FasterBlock layer is preceded by an additional layer (the first one is an Embedding layer, and the rest are Merging layers) for downsampling or channel expansion of the feature maps. This study mainly uses the FasterNet-M network, which has 3, 4, 18, and 3 FasterBlock layers in each stage. Figure 3 (b) illustrates the structure of a FasterBlock, composed of a PConv layer and two 1 × 1 Conv layers. Adding BatchNorm (BN) layers and the SiLu activation function between the two Conv layers enhances the network's generalisation ability and training efficiency. Replacing the YOLOX backbone network with the FasterNet backbone network maintains good feature extraction while reducing redundant computation and memory bearings.

Ship detection networks frequently use lower-level map features, which have narrow receptive fields, to detect small ships. This allows for the high-resolution capture of rich, detailed data. On the other hand, higher-level feature maps, which have greater global and semantic information and bigger receptive fields, are used to detect larger ships even though their resolution is lower. However, the standard YOLOX backbone network has a limitation in feature fusion. It primarily fuses features from the S2, S3, and S4 layers, overlooking the contribution of detailed features from shallow-level feature maps, which leads to unsatisfactory performance in detecting small ships. Although the shallow S2 layer in the YOLOX network provides valuable detailed information and clear resolution, its limited receptive field and weak semantic details may result in misclassifying small ships as background, impeding overall detection performance.

Due to the high proportions of small-sized vessels in some datasets, such as HRSID, the conventional YOLOX is ineffective at efficiently detecting these small ships. As a result, we integrate the shallow-level feature map from the FasterNet backbone network's S1 layer as an extra output layer into the feature fusion network, greatly enhancing the network's capacity to identify small ships.

3.3.2. Feature bi-level routing transformation module

Specifically, when analyzing the local feature extraction process for SAR images of small ships using the FasterNet backbone network, the downsampling effect may result in some small ships going undetected. Moreover, lower-level feature maps suffer from a limited receptive field, leading to inadequate capture of global information within convolutional neural networks. In order to address these issues and improve the model's capacity for feature extraction for multi-scale ships while simultaneously enhancing the precision of tiny ship detection, we provide the feature bi-level routing transformation module (FBM). This module may efficiently gather extensive global and contextual data without appreciably raising computational complexity, which will ultimately increase the precision and generalizability of the detection results.

Figure 4 depicts the overall architecture of FBM, which consists of a core module called Bi-Level Routing Attention (BRAttention) (Zhu et al., 2023). There are two phases in this attention mechanism. The image is divided into coarse-grained regions containing several pixels in the initial phase. The correlation between every pair of coarse-grained regions is computed based on the Q and K, resulting in a correlation matrix. Subsequently, this matrix undergoes sparse processing, where only the top-k elements with the highest values are retained, while the rest are set to zero. This step allows focusing on the most essential information about specific regions.

Figure 4. FBM module. (BRAttention: Bi-Level routing attention mechanism)

The coarse-grained sparse matrix obtained from the first stage is used for fine-grained self-attention calculation in the second stage. Each patch only attends to patches within the same coarse-grained region as itself, enabling more detailed and localised attention computation.

The specific implementation details of BRAttention are as follows:

1. Region division and input mapping: input feature map $X\in R^{H\ast W\ast C}$, the input feature map is divided into S*S regions, then each region contains a feature vector, $X^r\in R^{s^2\ast \left(\mathrm{HW}/S^2\right)\ast C}$ for feature map X reshapes from, then Q,K,V as shown in Equation (4)(5) $\begin{array}{r}{\mathit{A}}^r={\mathit{Q}}^r({\mathit{K}}^r{)}^T\end{array}$(5) : (4) $\begin{array}{r}\mathit{Q}\mathbf{=}{\mathit{X}}^{\mathit{r}}{\mathit{W}}^{\mathit{q}}\mathbf{,}\mathit{K}\mathbf{=}{\mathit{X}}^{\mathit{r}}{\mathit{W}}^{\mathit{k}}\mathbf{,}\mathit{V}\mathbf{=}{\mathit{X}}^{\mathit{r}}\mathit{W}\end{array}$(4)

where the relative weights of q, k, and v are $W^q,W^k,W^v\in R^{C\ast C}$.

1. Construct the index matrix between regions: first obtain the $Q^r,K^r\in R^{S^2\ast C}$ of the area by averaging each area of Q and K, and then calculate $A^r\in R^{S^2\ast S^2}$by equation (5)(6) $\begin{array}{r}{\mathit{I}}_r=\mathrm{topkIndex}\left({\mathit{A}}^r\right)\end{array}$(6) : (5) $\begin{array}{r}{\mathit{A}}^r={\mathit{Q}}^r({\mathit{K}}^r{)}^T\end{array}$(5)

The two terms in the matrix $A^r$represent the degree of semantic association between the two regions. Then, to create an index matrix $I_r\in N^{S^2\ast K}$, the $A^r$ matrix is pruned by keeping only the top-k connections for each region, as shown in Equation (6)(7) $\begin{array}{r}{\mathit{K}}^g=\mathrm{gather}\left(\mathit{K},{\mathit{I}}^r\right),{\mathit{V}}^g=\mathrm{gather}\left(\mathit{V},{\mathit{I}}^r\right)({\mathit{K}}^g,{\mathit{V}}^g\in R^{S^2\times \frac{\mathrm{kHW}}{S^2}\times C})\end{array}$(7) : (6) $\begin{array}{r}{\mathit{I}}_r=\mathrm{topkIndex}\left({\mathit{A}}^r\right)\end{array}$(6)

1. Fine-grained attention calculation: Since the corresponding small regions in the $I_r$ matrix are scattered over the whole feature map, the key-value pairs need to be collected first, as shown in Equation (7)(8) $\begin{array}{r}\mathit{O}=\mathrm{Attention}\left(\mathit{Q}\mathbf{,}{\mathit{K}}^{\mathit{g}}\mathbf{,}{\mathit{V}}^{\mathit{g}}\right)+\text{LCE}\left(\mathit{V}\right)\end{array}$(8) : (7) $\begin{array}{r}{\mathit{K}}^g=\mathrm{gather}\left(\mathit{K},{\mathit{I}}^r\right),{\mathit{V}}^g=\mathrm{gather}\left(\mathit{V},{\mathit{I}}^r\right)({\mathit{K}}^g,{\mathit{V}}^g\in R^{S^2\times \frac{\mathrm{kHW}}{S^2}\times C})\end{array}$(7)

Finally, we can calculate Attention on the collected key-value pairs as shown in Equation (8)(9) $\begin{array}{rl}\text{Precision }& =\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}\end{array}$(9) : (8) $\begin{array}{r}\mathit{O}=\mathrm{Attention}\left(\mathit{Q}\mathbf{,}{\mathit{K}}^{\mathit{g}}\mathbf{,}{\mathit{V}}^{\mathit{g}}\right)+\text{LCE}\left(\mathit{V}\right)\end{array}$(8) LCE(V) for the local context enhancement module, found in the literature. (C. Chen et al., 2022; Sun, Leng, et al., 2021)

Finally, the FBM module uses two branches to extract the input features. One branch retains the original features, which helps to maintain certain details and local characteristics of the original input data to avoid losing such information during the feature fusion process. The other branch passes through the BRAttention based on sparse sampling. On the one hand, this operation retains fine-grained detail information and enhances the fusion of context between individual regions, thus enhancing the model's capability for multi-scale ship detection. On the other hand, the BRAttention operation involves GPU-friendly dense matrix multiplication and is based on sparse sampling, which is conducive to improving computational efficiency.

By adding FBM after the S4 layer of FasterNet, the model's expressiveness and generalisation abilities are enhanced. It can also learn the correlation of particular regions or features, lessen the impact of ship detection in complex scenes, and suppress noise and extraneous information, like clutter in the ocean. Furthermore, the contextual data can be connected to improve the correlation between ships through continuous learning, which will improve the network's overall detection capabilities and the ability to recognise multi-scale ships.

3.4. PFM-based multi-scale feature pyramids

The YOLOX feature fusion network, PAFPN, effectively combines characteristics from several feature pyramid levels, leading to improved target detection speed and fine-grained feature path design. However, this efficient fusion poses challenges during training due to its complexity.

The features of the samples are less since there are fewer ships overall in a SAR image. There are multi-scale elements on the ships. In order to improve the network model's detection performance for tiny targets, this study inputs the features derived from the S1 layer into the PAFPN in the FasterNet backbone network and constructs an additional layer to fuse the input features from the S1 layer. But this design necessitates additional model computation. Thus, the PFM (PConv Feature Module) module is introduced in this study to decrease the model computation and enhance the detection performance.

3.4.1. Partial convolution-based PFM module

Designing an effective model structure is critical to capture multiscale information, handling unbalanced data, and improving model performance when involved in multiscale ship inspection. Figure 5 depicts the framework of the PFM module, which draws inspiration from CSPNet (Wang et al., 2020). The module comprises two main components: PBS and FasterBottleNeck. To elaborate, the PFM module initially splits the input into two distinct sections. One section undergoes processing through PBS, reducing the channel size by half. Then, this section proceeds through the FasterBottleNeck, with the two branches cascading together. In order to return the channel size to the input size, PBS is finally used. Two 1 × 1 convolutions come after a PConv layer inside the FasterBottleNeck. After every convolutional layer, Batch Normalization and SiLu activation functions are applied to improve the model's nonlinear learning capabilities and avoid overfitting.

Figure 5. PFM module structure diagram.

Using PConv convolution within the PFM module reduces computational complexity and memory access frequency, improving representational power and enhancing feature fusion quality. Furthermore, considering the diverse sizes and shapes of small vessels, the imbalanced distribution between large and small vessel samples, and the low contrast with the surrounding environment, the cross-stage connection structure in the PFM module effectively captures multi-scale and semantic information, thereby enhancing accuracy and robustness in target detection. Consequently, the PFM module successfully balances model computation and accuracy improvements, making it highly beneficial for ship target detection tasks.

3.4.2. Feature pyramid for multi-scale fusion

In detecting SAR vessels, the model's detection accuracy is significantly affected by numerous small vessels, imbalanced multi-scale features, and complex background interferences such as ocean waves. Considering these circumstances, this paper proposes improvements to the original YOLOX feature fusion network, PAFPN.

Since detailed features are abundant in shallow networks and vessels in SAR images and occupy a small space, along with the prevalence of smaller vessel targets, a multi-scale feature pyramid structure can fuse and utilise multi-scale information across different levels. This approach aids in extracting distinguishing features between vessels and backgrounds, enhancing the model's ability to separate targets from backgrounds. Consequently, the model can more accurately locate and identify vessels, reducing instances of both missed detections and false alarms.

Therefore, this paper augments the feature fusion network with shallow-level features, integrating contextual semantic information from four feature maps sized at 144 × 144, 288 × 288, 576 × 576, and 1152 × 1152. The objective of this augmentation is to improve the model's multi-scale vessel detection performance in intricate backgrounds. The constructed PFM-based multi-scale feature fusion pyramid structure, illustrated in Figure 6, initially captures various details and features from input feature maps of different resolutions. Subsequently, it performs upsampling on lower-resolution features and connects them with higher-level feature maps. These upsampled feature maps are then downsampled and connected with the original, non-downsampled feature maps. Finally, the output is generated.

Figure 6. Multi-scale feature fusion pyramid based on PFM module.

Our model captures information of different scales across these various levels of feature maps, facilitating a comprehensive understanding of the scene for subsequent multi-scale vessel detection.

4. Experiment

In this chapter, we do numerous tests to verify the efficacy of the model presented in this work. The experimental setup and conditions are described in Section 4.1; the dataset and experimentation methods are described in Section 4.2; the evaluation metrics of the model's performance are described in Section 4.3; the ablation experiments are described in Section 4.4; the experiments comparing the performance of the FastPFM with the twelve target detection models are compared in Section 4.5; the experiments comparing the generalisation ability and near-shore detection ability of the FastPFM with the twelve target detection models are compared in Section 4.6.

4.1. Experimental environment

All experimental environments were implemented on the server for this study, utilising the hardware configuration and software environment outlined in Table 1. The hyperparameter settings of the proposed model FastPFM are detailed in Table 2.

Table 1. Experimental software environment and hardware configuration.

Platform	Configuration
Memory	80G
Programming Language	Python 3.8.10
CPU	Intel(R)Xeon(R)Gold6330CPU@2.00GHz
GPU	NVIDIA GeForce RTX 3090(24G)
GPU Accelerator	CUDA 11.3
Development Environment	Pytorch 1.10

Table 2. FastPFM hyperparameters.

Parameter	Configuration
Batch Size	8
Learning Rate	0.0001
Training Epochs	150
Neural Network Optimizer	AdamW

4.2. Datasets

In this investigation, we used two standard datasets, HRSID (Wei et al., 2020) and SSDD (Zhang, Zhang, Li, et al., 2021) to investigate the generalizability and efficacy of the FastPFM model. These datasets encompass diverse scenarios, including coastal areas, open seas, and variations in ship sizes and sea conditions. Consequently, they offer crucial support in constructing a reliable ship detection model. During practical applications, these datasets necessitate specific considerations, such as:

1. Complex sea waves: Due to the complexity of marine weather, various complex sea wave patterns can occur, which may cause misleading detection results by the model.

2. Indistinct features of small ships: These datasets contain many small ship targets with indistinct features, posing a challenge to the detection performance of the model.

3. Dense ship traffic in coastal ports: A high density of docked ships in coastal areas requires the model to have effective detection and discrimination capabilities for densely packed targets.

To address the problems above, the HRSID and SSDD datasets are crucial. These datasets include a wide variety of target samples and situations, enabling the model to be suitably trained and assessed under numerous difficult scenarios in order to determine the FastPFM model's performance and generalizability.

SSDD. The SSDD dataset, which consists of 1160 photos with resolutions ranging from 1 to 15 meters and typically at a size of 500 × 500 pixels, is an open image dataset for SAR ship detection. SSDD includes 2540 ships, with proportions of small, medium, and large ships at 60.2%, 36.8%, and 3%, respectively. It covers diverse scenes such as ports, coastal environments, and open sea surfaces, adding a certain level of challenge. The introduction of the SSDD dataset establishes a standardised and publicly available benchmark for evaluating and comparing various SAR ship detection algorithms, further advancing research and applications in the field of ship detection.

HRSID. The 800 × 800 pixel HRSID collection consists of 5604 SAR pictures with resolutions ranging between 0.5-3 meters. The dataset offers complex imaging scenes, including ports, docks, coastal environments, and simple sea surface scenes, each containing different numbers and sizes of ship targets. A total of 16,951 examples of ships are included, with small, medium, and large ships accounting for 54.5%, 43.5%, and 2% of the total. The HRSID is intended to address the shortcomings of current SAR ship datasets, notably for CNN-based ship detection. The SAR images in HRSID capture ship features with greater precision and detail when compared to the SSDD dataset, providing a more challenging and opportunistic platform for training and evaluating SAR ship detection algorithms.

Furthermore, to provide a more precise comparison of ship quantities across different sizes in both datasets, we present the statistics of ship quantities for three ship sizes in Figure 7's style of bar charts. Observing the charts, we can see that in both datasets, the number of medium and small ships far exceeds that of large ships. This data distribution characteristic enables the proposed model to perform well in recognising small ships and multiple scales identifying ships. However, the dataset's uneven distribution of ship sizes might impair the model's ability to recognise large ships.

Figure 7. Three-size ship count in two datasets.

4.3. Experimental evaluation metrics

This work employs the MS COCO (Lin et al., 2014) assessment standard to evaluate each model's effectiveness precisely. The assessment criteria used to gauge the efficacy of object detection are mean average precision (mAP), average recall (AR), and average precision (AP). It is feasible to evaluate several models and confirm the efficacy of the suggested FastPFM on the HRSID and SSDD thanks to these evaluation criteria. The following explains the calculation procedures and significance of these measures.

The four parameters, TP, FP, TN, and FN, are the most commonly used among these evaluation metrics. TP represents the number of correctly identified ships, FP is the number of incorrectly detected ships, TN is the number of correctly detected backgrounds, and FN is the number of missed ships.

Recall is the percentage of correctly predicted samples among all real objects, while accuracy is the percentage of correct samples among the target objects that the model predicts. Equations (9)(10) $\begin{array}{rl}\text{Recall }& =\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}\end{array}$(10) and (10)(11) $\begin{array}{r}\mathrm{AP}={\int }_0^{1}P\left(R\right)\mathrm{dR}\end{array}$(11) show how they can be computed differently. (9) $\begin{array}{rl}\text{Precision }& =\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}\end{array}$(9) (10) $\begin{array}{rl}\text{Recall }& =\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}\end{array}$(10) The formula of AP is shown in (11): P is Precision, and R is Recall. The value of AP can be obtained by calculating the area under the PR curve. This indicates that the model's precision and recall can be included in the AP computation, which can summarise the model's overall performance. (11) $\begin{array}{r}\mathrm{AP}={\int }_0^{1}P\left(R\right)\mathrm{dR}\end{array}$(11)

4.4. Ablation experiments

We ran experiments on the benchmark dataset HRSID in order to thoroughly assess the efficacy of the suggested method and the contributions of each module. To be more precise, we varied the ways in which we integrated the FasterNet, PFM, and FBM modules into the foundational YOLOX model and then evaluated the results. The experimental results are presented in Table 3. When using only the FasterNet as the backbone network in FastPFM, the model's accuracy improved by 2.7%, with a remarkable 11.4% increase in detection accuracy for large ships. The detection accuracy was then increased by 1.7% and 1.9% when the PFM and FBM modules were added to the FasterNet model compared to those without these modules. The proposed model's detection accuracy outperformed the YOLOX model by 5.4% when all modules above were combined.

Table 3. Ablation results on HRSID for different combinations. “Baseline” indicates the YOLOX model we used, and “FasterNet,” “PFM,” and “FBM” represent the various modules mentioned above. No.1-5 indicates different combinations, whereas No.5 shows our proposed FastPFM model. “√”means the module chosen. AP₅₀ refers to the average accuracy at the IoU threshold of 0.5, and AP_S, AP_M, and AP_L are the average accuracies for small, medium, and large targets, respectively. The best results are in bold.

No.	Baseline	FasterNet	PFM	FBM	AP50	APS	APM	APL
1	√				86.7	61.7	55.7	4.6
2	√	√			89.4	66.4	65.7	16.0
3	√	√	√		91.1	67.9	68.4	30.9
4	√	√		√	91.3	68.8	67.3	31.6
5	√	√	√	√	92.1	68.6	69.3	31.5

Notably, the limited presence of large-sized ship samples in the HRSID dataset adversely affected the performance of the baseline YOLOX model, yielding a modest detection accuracy (AP_L) of only 4.6% for large ships. However, the proposed FastPFM method in this study significantly improved the detection accuracy for large ships, achieving a remarkable 26.9% enhancement compared to the YOLOX model. This result indicates the successful mitigation of the challenge arising from the dataset's imbalanced distribution of ship sizes, achieved by the novel approach proposed in this research.

These findings validate the fundamental contributions of each design module and show the overall validity and soundness of the suggested methodology. The experimental findings clearly demonstrate the method's capacity to handle the difficulties presented by the features of the dataset, such as the dearth of large ship samples. The usefulness of the suggested method and its capacity to achieve precision in the identification of multi-scale vessels are demonstrated by the success attained in detecting small, medium, and large vessels (6.9%, 13.6%, and 26.9% improvement over the baseline model, respectively).

Furthermore, we compare the PR(Precision-Recall) curves between FastPFM and YOLOX for clarity. As shown in Figure 8, the x-axis represents recall, and the y-axis represents precision. The average precision (AP) value is represented by the region beneath the curve, which is constrained by the curve and the axes. Notably, the FastPFM curve's enclosed area exceeds the YOLOX curve’s. These findings unequivocally show that the FastPFM model surpasses the YOLOX model in terms of overall performance.

Figure 8. Comparison of PR curves of FastPFM and YOLOX at Iou = 0.5.

4.5. Comparison experiment

To verify the resilience and detection efficacy of our algorithms, we run multiple comparative experiments. First, we use the HRSID and SSDD datasets to compare our model with the most advanced methods available today. Second, we validate the model generalisation capability on the SSDD dataset and specifically analyze complex scenarios like offshore and dense shipping lanes on the HRSID dataset to obtain a deeper understanding of the performance of our model for multi-scale target detection in complex backgrounds. These assessments clearly show how well our model performs in terms of efficacy and dependability.

4.5.1. Experimental comparison of FastPFM with existing models on HRSID

Furthermore, we do comparative experiments with 12 other important algorithms on the HRSID dataset in order to illustrate the performance of the FastPFM model. These include SSD, PVT (Wang, Xie, et al., 2021), FCOS (Tian et al., 2019), TOOD (Feng et al., 2021), YOLOX, YOLOF (Chen et al., 2021), DeformableDETR (Zhu et al., 2020), NAS-YOLOX (Wang et al., 2023), Swin-PAFF (Y. Zhang et al., 2023), YOLO-SD (Wang et al., 2022), FINet (Hu et al., 2022), and Zhu et al. (Zhu et al., 2022) proposed to algorithms. Using the HRSID and SSDD datasets, the FastPFM model suggested in this research is compared. Table 4 displays the outcomes of the experiment. To be more precise, the first seven of these techniques are SOTA techniques currently used in the target detection area, and the remaining five techniques are target detection algorithms designed especially for target detection on SAR ships.

Table 4. Results of multiple models’ detection on the HRSID dataset.

Method	AP	AP50	AP75	APS	APM	APL	Parameters/M
SSD(2016)	43.6	71.8	47.7	43.9	56.4	2.0	36.04
PVT(2021)	53.7	80.8	58.6	55.1	56.1	27.3	71.1
FCOS(2019)	56.9	84.7	63.8	58.6	62.1	14.2	32.02
TOOD(2021)	62.0	86.9	70.0	63.2	65.5	17.1	31.98
YOLOX(2021)	59.6	86.6	68.5	61.6	55.6	4.8	45.6
YOLOF(2021)	46.3	67.8	53.5	46.1	54.5	4.1	44.8
DeformableDETR(2021)	52.3	80.5	58.6	53.5	57.5	17.3	40.94
NAS-YOLOX(2023)	63.9	91.1	71.9	65.2	68.6	34.1	44.4
Swin-PAFF(2023)	64.6	91.3	73.3	65.7	67.9	-	45.7
YOLO-SD(2022)	59.5	83.7	67.6	62.3	51.9	1.3	59.6
FINet(2022)	-	90.5	79.6	69.1	-	-	-
Method(2022)	60.6	85.5	67.9	61.4	65.5	32.0	38.2
FastPFM(ours)	66.0	92.1	74.8	68.6	69.3	31.5	64.1

Table 4 displays the model's detection results for small, medium, and big ships, denoted as AP_S, AP_M, and AP_L, respectively. These results can accurately illustrate our model's detection capacity for multi-scale ships. The accuracy of our model reaches 92.1%, 68.6%, 69.3%, and 31.5% on AP₅₀, AP_S, AP_M, and AP_L, respectively. YOLOX increased by 5.5%, 7%, 13.7%, and 26.7% compared to the baseline model. These results are an excellent visual demonstration of the capability of our model for multi-scale ship detection. In particular, our proposed method achieves a detection performance of 68.6% for small ships, an average improvement of 9.8% with the compared models. This is in line with our model's apparent superiority over other models at the tiny ship detection level, where it is just 0.5% less than FINet. This suggests that FINet's feature interaction module is capable of successfully extracting small ship features. However, our model is 1.6% higher than FINet under the AP₅₀ metric. Our model achieves the best detection accuracy under the APM metric compared to other models, which is 9.2% higher than other models on average. It's also noteworthy that none of the analyzed models perform well in detecting huge ships. This could be because there are far less large ships in the dataset than there are small and medium-sized ships. As a result, the algorithms’ capacity to extract features and detect huge ships is weakened. Even with this issue, our suggested model outperforms the other models in terms of large ship detection accuracy by an average of 16.08%, which is enough to show that our model is capable of detecting large ships.

4.5.2. Comparison of FastPFM's generalisation ability on SSDD with existing models

One of the most important criteria for assessing a model is its capacity for generalisation. Using the SAR image dataset SSDD, which is not used for training, we compare the suggested model FastPFM with other more sophisticated detection methods in order to more thoroughly test its detection capacity. The results are displayed in Table 5. The table shows that FastPFM improves by 4.4%, 1.9%, 11.6%, and 5.9% over Baseline in AP₅₀, AP_S, AP_M, and AP_L measures, respectively, when the SSDD dataset is used for the generalisation ability test. FastPFM improves over Baseline to different extents in all of the metrics, and most of the metrics are better than the models compared. It is clear from the table that FastPFM's detection capability is almost optimal. Regarding multi-scale ship detection, our model is 14.65%, 16.69%, and 3.52% better than other models in AP_S, AP_M, and AP_L on average. This shows that FastPFM's ability to detect multi-scale ships is also better than other models under generalised capability verification. For large ships, the gain in detection is still negligible. In addition to the model's performance, the SSDD dataset's low data volume and lack of representativeness have a significant contributing factor. These outcomes show the FastPFM model's superior performance over alternative techniques and confirm its outstanding performance for ship target detection tasks.

Table 5. Detection results of FastPFM and other models on the SSDD dataset.

Method	AP	AP50	AP75	APS	APM	APL
SSD(2016)	30.6	69.5	25.5	34.5	34.2	7.2
PVT(2021)	32.5	69.2	25.5	34.6	25.3	7.4
FCOS(2019)	28.9	63.6	21.5	32.0	20.5	0.3
TOOD(2021)	31.7	66.4	24.8	34.3	23.5	2.9
YOLOX(2021)	45.9	78.7	48.5	50.2	32.7	1.4
YOLOF(2021)	26.4	60.2	17.6	30.7	17.8	0.1
YOLOv7(2022)	38.4	76.7	34.5	39.6	39.3	7.2
Retinanet(2018)	35.1	72.1	30.2	38.5	24.1	6.9
DeformableDETR(2021)	27.4	61.3	19.9	30.9	17.7	0.7
Swin_PAFF(2023)	46.4	80.3	47.6	49.2	41	-
FastPFM(ours)	50.2	83.1	52.0	52.1	44.3	7.3

4.5.3. Ship inspection results in complex nearshore environments

Due to the presence of many buildings in the nearshore image and densely docked multi-scale ships, this situation can significantly impact ship detection. To further test the model's ability to identify ability for complicated situations, this research compares FastPFM with ten other models for nearshore ship detection using the SSDD dataset. It should be noted that most advanced models, including ours, tend to have lower detection accuracy in nearshore environments, considering that the nearshore environments are interfered with by complex backgrounds such as shoreline buildings and densely docked ships. Table 6 displays the experimental findings. Using the AP₅₀ metric, the FastPFM model outperformed the baseline model YOLOX by 4%. For large, medium, and small objects, the detection accuracy is increased by 10.9%, 9.1%, and 3%, respectively. Furthermore, in the four measures of AP₅₀, AP_S, APM, and APL, respectively, FastPFM's detection accuracy outperforms the other ten models by an average of 12.9%, 15.96%, 9.6%, and 10.45%. These findings confirm that the FastPFM model outperforms other models in ship detection close to shore and further highlight the technology's remarkable durability and capacity for generalisation.

Table 6. Detection results of FastPFM and other models on the SSDD nearshore dataset.

Method	AP	AP50	AP75	APS	APM	APL
SSD(2016)	26.7	54.3	23.8	31.4	28.0	5.2
PVT(2021)	25.8	50.6	24.6	30.0	23.1	8.0
FCOS(2019)	28.9	56.7	26.1	33.7	19.6	0.2
TOOD(2021)	29.6	53.4	31.4	34.6	19.4	0.0
YOLOX(2021)	39.6	64.1	46.6	46.7	23.1	2.0
YOLOF(2021)	27.4	53.1	26.0	33.5	8.9	0.0
YOLOv7(2022)	28.1	58.2	26.5	32.5	28.1	-
Retinanet(2018)	31.2	59.7	28.5	32.1	29.4	-
DeformableDETR(2021)	19.8	41.2	16.6	22.0	15.3	1.8
Swin_PAFF(2023)	37.0	60.3	42.6	40.9	31.1	-
FastPFM(ours)	43.2	68.1	48.1	49.7	32.2	12.9

4.6. Visual results analysis

We chose three scenarios from the HRSID dataset for investigation in order to more clearly illustrate the detection capability of the model suggested in this paper. From these visualisations, it is intuitive to see that our model is superior to others. The three scenarios we selected for visual validation are Figure 9(a), a ship map near a port; Figure 9(b), a small ship map offshore. Figure 9(c) is a map of dense small ships in a narrow channel near shore in a complex background. The findings demonstrate that, even in complicated backgrounds, our suggested method is capable of extracting ship characteristics with high accuracy, particularly in near-shore locations where it is challenging for the human eye to discern between accurate ships and noise. In Figure 9(a), even though there are many buildings close to the shore with some similarity to the ship targets, FastPFM does not produce any misdetections, successfully avoiding misjudging the buildings on the shore, and does not miss detecting the ships densely docked in the harbour. In contrast, other models are easily affected by buildings on shore in this scenario, resulting in missed detection or misidentification. In Figure 9(c), due to the large amount of background interference and densely populated small ships, the model is subject to misdetection or missed detection. However, while other methods show obvious errors, our proposed FastPFM model only shows occasional misdetections. This is because of how the FBM and PFM modules are designed, and the FastPFM model can successfully enhance the identification of small boats that are widely dispersed. These visualisation results demonstrate that although FastPFM can still maintain high accuracy in the case of highly similar ship targets and shoreline building backgrounds, there will be detection errors for small, densely distributed ships in nearshore environments. However, it can still effectively detect multi-scale ship targets within an acceptable error range.

Figure 9. Demonstrates the HRSID dataset's detection results for several models. The green boxes show the target ships’ annotated positions in the dataset. The red boxes represent the expected boxes discovered by the models. The yellow boxes represent the detection boxes for missed targets, and the blue boxes show the detection boxes for false detection targets.

Furthermore, we have chosen three photos to illustrate the FastPFM model's capacity for generalisation: Figure 10(a) near-shore environment, Figure 10(b) dense ships in the far sea, and Figure 10(c) narrow channel. In these three environments, the Baseline model produces many misdetections and omissions, and our model can still identify overlapping ships and harbour vessels better than the Baseline model despite some omissions and misidentification of targets close to land.

Figure 10. Generalisation results of YOLOX and FastPFM on the SSDD dataset. The green boxes show the target ships’ annotated positions in the dataset. The red boxes represent the expected boxes discovered by the models. The yellow boxes represent the detection boxes for missed targets, and the blue boxes show the detection boxes for false detection targets.

5. Conclusion

This work presents the FastPFM ship detection model, which is based on YOLOX, to address false alarms and missed detections in ship detection in SAR images with complex backgrounds. With the goal of improving ship detection accuracy, the model uses FasterNet as its backbone network and adds the FBM and PFM modules to improve semantic features and optimise feature representation. We evaluate the FastPFM model on the HRSID dataset, comparing it against twelve other object detection models. The experimental results demonstrate FastPFM's proficiency in detecting small-sized ships, even with limited samples for large ships during training. Additionally, the generalisation test on the SSDD dataset reveals significant performance improvement compared to other models. In the detection experiments targeting coastal ships, the FastPFM model also performs satisfactorily.

We adopt the deep learning method to solve some problems in ship target detection and achieve good results, fully reflecting the application prospects of deep learning on ship detection tasks. Deep learning is able to replicate the significant process of human visual learning cognition by learning a vast quantity of sample data, allowing it to autonomously extract visual features from ship photographs and achieve automatic detection.

In summary, the approach presented in this research has promising applications in disaster monitoring, rescue, and maritime safety monitoring and traffic management. In marine safety monitoring and traffic management, the algorithm can detect abnormal situations such as illegal fishing vessels and piracy, thus improving the ability of maritime safety monitoring. It can also help detect the position and heading of ships and provide real-time ship traffic and sailing status to optimise ship traffic flow. Our method can detect vessels in distress in time for disaster monitoring and rescue, thus guiding rescue operations and improving rescue efficiency.

Despite the FastPFM model's success in identifying multi-scale ships in complex surroundings, tiny ships, there are still limitations. These include missed detections and false alarms when dealing with dense small ships in complex backgrounds. To address these limitations, future research should focus on two aspects. Firstly, investigating the angle-based SAR ship target recognition technique and strengthening the network model's security to increase the accuracy and safety of ships in complex circumstances (Chen et al., 2024; C. Chen et al., 2023; Han et al., 2020; Han et al., 2021; Li, Han, Weng, et al., 2022; Li, Han, Zheng, et al., 2022). Secondly, we strive to develop lightweight design approaches for the model to enable efficient deployment on mobile platforms. The goal of these further studies is to improve the existing approach and offer a ship detection solution that is more dependable, secure, and effective.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This research is supported by the National Key Research and Development Program of China under Grant 2021YFC2801001, the Natural Science Foundation of Shanghai under Grant 21ZR1426500, and the 2022 Graduate Top Innovative Talents Training Program at Shanghai Maritime University under Grant 2022YBR005.