EIoU-distance loss: an automated team-wise player detection and tracking with jersey colour recognition in soccer

Abstract

The surge in demand for advanced operations in sports video analysis has underscored the crucial role of multiple object tracking. This study addresses the escalating need for efficient and accurate player and referee identification in sports video analysis. The challenge of identity switching among players, especially those with similar appearances, complicates multi-player tracking. Existing algorithms relying on manually labeled data face limitations, particularly with changes in jersey colors. This paper introduces an automated algorithm employing Intersection over Union (IoU) loss and Euclidean Distance (EUD), termed EIoU-Distance Loss, to track players and referees. The method prioritizes identity coherence, aiming to mitigate challenges associated with player and referee recognition. Comprising BackgroundSubtractionMOG2 for player and referee detection and IoU with EUD for connecting nodes across frames, the proposed approach enhances tracking performance, ensuring a clear distinction between different identities. This innovative method addresses critical issues in sports video analysis, offering a robust solution for tracking players and referees in dynamic game scenarios. 

KEYWORDS:

• Soccer player
• jersey colour
• detection
• tracking
• computer vision

1. Introduction

In soccer videos, players and the referees are the most prominent subjects. Player and referee identification and tracking are crucial for acquiring a complete insight of a sports event and may be thought of as a multi-object detection, classification, and tracking task, with the objective of determining the position of every player at all times and reconstructing trajectories from multi-view video streams.

Recognising and distinguishing players in real sports videos presents a formidable challenge, primarily due to the players’ propensity to confuse opponents with abrupt changes in direction and speed. When compared to general person tracking scenarios, commonly used re-identification attributes like colour and gait prove less reliable in player tracking. Addressing player tracking with distinct identities involves tackling the following challenges: (I) Players often share a similar appearance due to wearing matching jerseys. Factors such as changes in body shape, unexpected movements, interference from spectators, and fluctuations in lighting further complicate the accurate tracking of players with unique identities. (II) The movement of players results in significant deformation of jersey numbers, particularly when they are at a distance from the cameras. This makes it challenging to discern and track the jersey numbers effectively. (III) Players frequently switch positions as a result of their unpredictable movements on the field. When player identification is inaccurate, it can lead to instances of mis-tracking.

Vision plays a pivotal role in achieving real-time object recognition and tracking in the context of soccer. Recent years have witnessed significant advancements in vision systems, boasting the capability to swiftly and accurately detect objects. The object detection and tracking methodology introduced here not only finds utility in real-time performance assessment systems but also in the domain of humanoid robotics. Performance measurement systems designed for both teams and individual players can unveil aspects of the game that elude the human eye. They enable the measurement of metrics such as player distance covered and relative positioning in relation to teammates and opponents. This valuable information serves multiple purposes, including the evaluation of individual player performance, assessment of fatigue levels, analysis of team tactics, and a thorough examination of adversaries’ strategies. To dissect the tactics employed by opposing players, an object detection method capable of identifying and categorising teammates is indispensable. This necessitates the extraction of the foreground, achieved by eliminating the background from the frames, a task facilitated by the built-in functions of OpenCV.

Research in this field currently grapples with limitations in its ability to effectively address pivotal challenges, including team player classification, referee detection and tracking, and identity switching. Consequently, there is a pressing need for the development of a novel system that employs deep representation to learn players’ identities and enhances the tracking mechanism by incorporating identity-related information. Furthermore, most contemporary methods for multiple object tracking rely on manually labelled bounding box data as input. Despite this, these state-of-the-art detectors and trackers often fall short when faced with the identical appearances of teammates, resulting in suboptimal player tracking outcomes. This gap underscores the lack of a comprehensive and reliable solution that seamlessly spans the entire spectrum of multi-object detection, identification, and tracking, a void that persists both in academic research and within the industry.

In summary, this study introduces a dependable method for recognising and tracking multiple players and referees in real-world scenarios. The primary contributions of this study include:

1. The introduction of an unsupervised player and referee detection and classification framework, eliminating the need for manual labelling. This framework detects players and referees and categorises them based on their respective teams.

2. The development of an efficient algorithm to tackle identity switching and enable the tracking of players and referees on a team-by-team basis, achieved through the utilisation of IoU and EUD methods.

This article clearly gives pertinent information as to what is being done in the field of player/referee detection and tracking in sports videos, with an emphasis on soccer video. The structure of this article is organised as follows: Section II provides a review of the existing literature related to detection and tracking. After that, Section III presents the proposed methodology of players/referee detection and tracking approaches. Section IV discusses results, experimental implementations. Finally, conclusion and scope for future research are provided in Section V.

2. Related work

In this section, we take a look at some of the most common architectures for object detection as well as some of the latest sports analysis techniques. Murthy et al. (2020) discussed deep learning-based supervised object detection mechanisms. A review article presented by Naik et al. (2022a) explored most of the aspects of academic research in sports activities.

2.1. Object detection

Faster R-CNN proposed by Ren et al. (2015) is a state-of-the-art object detection methodology that achieves remarkably promising performance in prominent datasets like COCO using a two-stage object detection approach. However, the computational cost is still considerable, necessitating the use of a GPU to run R-CNN better and faster. As a result, single shot detectors like YOLO family developed by Redmon et al. (2016, 2017, 2018) and in 2016, Liu et al. introduced the Single Shot Multibox Detector (SSD), which demonstrated enhanced speed and improved accuracy, as further evidenced in the findings of Huang et al. (2017). YOLO was a single convolutional neural network that recognised objects by predicting bounding boxes and class probabilities. However, detecting small objects in the crowed scene suggested by Singh et al. (2021) of a sport and achieving exact localisation for activity recognition as developed by Guan et al. (2011) remained a challenge. After that, SSD Liu et al. (2016) was offered as a way to improve YOLO approach. Due to the use of multi-scale feature maps and predefined boxes mechanism, SSD can identify tiny objects and improve localisation accuracy compared to YOLO. YOLO, on the other hand, was adapted to address its issue. The recent version, YOLO v4, used a more sophisticated deep architecture with additional layers and residual blocks to make detections at three different scales. Hu and Ramanan (2017) used context information such as hair and shoulder to distinguish little faces, while context information for balls or players was significantly poorer. R Zhang et al. (2022) introduced a method for converting features extracted by a pre-trained self-supervised feature extractor into a Gaussian-like distribution. In their work, B. Ding et al. (2023) introduced an unsupervised transmission-aware dehazing module designed to enhance visibility and mitigate depth-dependent noise propagation within the dehazing process. D. Yuan et al. (2022a, 2022b) have devised an Aligned Spatial–Temporal Memory network-based Tracking approach specifically tailored for the task of tracking Thermal Infrared targets and an adaptive spatial–temporal context-aware model within the DCF-based tracking framework which aimed at enhancing tracking precision and minimising the impact of boundary effects. These studies showed that expanding small objects are effective enough while requiring fewer processing resources.

2.2. Player detection

Vision-based player/ball detection proposed by Burić et al. (2018) is essential in sports applications for analysing ball trajectories developed by Theagarajan et al. (2018), Wei et al. (2015), player movements, player identification, and activity recognition developed by Miyata et al. (2017), Rematas et al. (2018) and Senocak et al. (2018). Real-time processing, a fundamental requirement for applications such as live broadcasting, as suggested by Wang et al. (2018) in their proposal and Augmented Reality or Virtual Reality, as explored by Lee et al. (2017) in their study, is a problem with 2 K Sports Videos. Reducing the size of 2 K videos to smaller dimensions was found to be effective, as demonstrated by the works of Wang et al. (2018) and Tijtgat et al. (2017). Another possibility was parallel computation, which necessitated more expensive equipment. This problem is exacerbated by the fact that multi-view 2 K sports videos have multiple views. Currently, there is limited research dedicated to object detection in multi-view 2 K sports videos, as exemplified by Shih’s work in 2017. Detecting soccer players are therefore challenging, especially when there are multiple entities involved. There are several methods for detecting players; Santhosh and Kaarthick (2019) investigated the use of a HOG colour-based detector that gives each image a player detection. For player detection, Ramanathan et al. (2016) used a CNN-based multibox detector. Johnson (2020) experimented with a more complex model, AlphaPose, an open-source multiperson pose estimator based on COCO dataset.

2.3. Player tracking

For player tracking across a game, Ramanathan et al. (2016) used a Kanade–Lucas–Tomasi feature tracker. Reidentification models developed recently take it a step further: In Arbués-Sangüesa et al. (2019) study, a multi-scale detection method was introduced to track players. This method involves the extraction of both geometric and content features, leading to the development of a multi-scale tracking system that operates with full-HD frames at a resolution of 1920 × 1080 pixels. Zheng et al. (2019) employed an embedding model to extract player attributes and a combination of IoU and embedding similarities. Zhang et al. (2020) introduced the k-shortest paths tracking technique to boost player tracking efficiency. However, it may be sensitive to dynamic field changes, impacting performance when players deviate from predefined paths. Liang et al. (2020) utilised similar schema and an embedding model to extract the numbers and colours of the players. Kamble et al. (2019) presented deep learning-based ball identification and tracking (DLBT) method for soccer videos. Based on IoU, this technique tracked players and the ball with a high precision of approximately 87.45 percent. Zikai Song et al. (2021) introduced a distractor-aware player tracking algorithm and a high-quality benchmark for soccer play tracking. Hurault et al. (2020) developed deep learning-based soccer player detection and tracking system that is resilient to small players in a range of circumstances. Kim (2019) suggested topographic surface technique to extract foreground region for consistently tracking multiple players in varying soccer match scenarios. Naik et al. (2022a, 2022b,  2023) employed supervised methodologies, including YOLOv3 and YOLOv4, to detect players, the soccer ball, and referees. Zheng (2022) introduced a target-tracking method for soccer using Hog feature-based Kernelized Correlation Filters in combination with deep CNNs. Kim et al. (2018) proposed an adaptive multiscale sampling approach for multiplayer tracking in soccer videos. A simple object detection algorithm detects objects as persons in the playfield but does not classify the players and the referees. Moreover, training is must for those object detection methods. In this paper, a novel approach is proposed to detect and track in which foreground was extracted from the background and followed by contour analysis is performed to detect objects and jersey colour is applied to classify players team-wise and the referee. To track the players and referee, IoU and Euclidian Distance based framework is developed. In this paper, we have leveraged OpenCV methods for the detection, and we have considered jersey colour for the automatic classification of team players and the referees. Furthermore, we have utilised handcrafted approaches to track these individuals with distinct identities.

3. Proposed methodology

To identify players and the referees in playfield, background subtraction methodology was used. BackgroundSubtractorMOG2 algorithm is the process of separating the foreground objects (moving objects) from the background (static or stationary objects) in a video or image sequence. The “MOG2” part of the name stands for “Mixture of Gaussians with Adaptive Learning”. It uses a statistical model to represent the background and foreground of an image or video sequence. Specifically, it models each pixel as a mixture of several Gaussian distributions. This allows it to adapt to changes in lighting conditions, making it suitable for dynamic environments. The algorithm continuously adapts and updates its model of the background over time. It dynamically adjusts the parameters of the Gaussian distributions based on the observed pixel values. This adaptability is essential for handling gradual changes in the scene, such as lighting changes and camera motion. Typically, a thresholding operation is applied to the likelihood of a pixel belonging to the background or foreground. Pixels with a low likelihood of belonging to the background are considered part of the foreground.

BackgroundSubtractorMOG2 has several parameters that can be fine-tuned to achieve the desired results. These parameters include the number of Gaussian components, learning rate, and the history of frames used for modelling. Proper parameter tuning is crucial to adapt the algorithm to different scenarios and achieve accurate background subtraction. The output of BackgroundSubtractorMOG2 is typically a binary mask, where each pixel is labelled as either foreground (object of interest) or background. This mask is then be used for object tracking, motion analysis. BackgroundSubtractorMOG, BackgroundSubtractorMOG2, and BackgroundSubtractorGMG are some of the built-in OpenCV functions. After detection and classification of players and referees, EUD + IoU based tracker was used to track players and referees. The block diagram of the proposed methodology to detect and track players and referees is shown in Figure 1 and the complete process is discussed in the following subsections.

Figure 1. Block diagram of proposed methodology for players and referee detection and tracking.

3.1. Team-wise player/referee detection and classification

In proposed research methodology, the determination of primary colours assigned to players and referees, corresponding to the colours of their respective bounding boxes, is achieved by combining equal HSV colours. To enhance adaptability across a range of sports scenes, we incorporate the BackgroundSubtractorMOG2 – an advanced background subtraction model leveraging a Gaussian mixture model – into each frame of the video sequence. In this intricate process, the initial frame serves as the background model, utilising subsequent frames to extract foreground pixels by discerning them from the modelled background. The resulting output from BackgroundSubtractorMOG2 undergoes thresholding, followed by the application of a kernel-based image smoothing technique to eliminate shadows. Subsequent morphological operations are applied to each frame to eliminate unwanted clutter and close small gaps between foreground objects. Players and referees are then categorised based on the colours extracted from foreground pixels, enabling the detection of any object in a frame using contour features. The outlines of entities are generated by connecting continuous bounding points with lines, facilitating shape analysis and object recognition. OpenCV offers various contour functions for contour detection, along with additional contour-related parameters such as area, centroid, and bounding box extraction. Following the application of these image processing techniques, structural analysis is conducted to gather information on player contours. These contour details are instrumental in determining the centroids and coordinates of the objects’ bounding boxes. The algorithm for classifying players and referees based on their jersey colours is visually depicted in algorithm-1 (Figure 2), providing a clear representation of our approach to object recognition and tracking in dynamic sports scenarios.

Figure 2. Proposed algorithm to classify team wise players and referee.

The approach employs IoU and EUD techniques for tracking both players and the referee. It achieves this by estimating the new positions of objects in the current frame, approximating the geometry of each detected object's bounding box, and using this information to match detections with existing objects.

3.2. Intersection over union based tracking with unique identity

The area of intersection (IoUArea) and IoU is calculated with respect to the centroid of the bounding box in each frame to the detected (bboxA) and predicted (bboxB) bounding boxes as shown in Figure 3 to detect and assign a unique identity to each player in consecutive frames which is shown in Figure 4(a). The identity is assigned to detected objects based on the threshold values of (IoUArea) and IoU. A threshold is specified for IoUArea and IoU to exclude any small elements detected. Therefore, the calculation of IoUArea and IoU is performed by the following equations. (1) $\begin{array}{rl}{X}_A& =\text{max}\left(\left(x_A-\frac{w_A}{2}\right),\left(x_B-\frac{w_B}{2}\right)\right)\equiv \left(x_B-\frac{w_B}{2}\right)\end{array}$(1) (2) $\begin{array}{rl}{Y}_A& =\text{max}\left(\left(y_A-\frac{h_A}{2}\right),\left(y_B-\frac{h_B}{2}\right)\right)\equiv \left(y_A-\frac{h_A}{2}\right)\end{array}$(2) (3) $\begin{array}{rl}{X}_B& =\text{min}\left(\left(x_A+\frac{w_A}{2}\right),\left(x_B+\frac{w_B}{2}\right)\right)\equiv \left(x_A+\frac{w_A}{2}\right)\end{array}$(3) (4) $\begin{array}{rl}{Y}_B& =\text{min}\left(\left(y_A+\frac{h_A}{2}\right),\left(y_B+\frac{h_B}{2}\right)\right)\equiv \left(y_B+\frac{h_B}{2}\right)\end{array}$(4) (5) $\begin{array}{rl}\mathrm{Io}{U}_{\text{Area}}& =(X_B-X_A)\times (Y_B-Y_A)\end{array}$(5) (6) $\begin{array}{rl}\mathrm{IoU}& =\frac{\mathrm{Io}{U}_{\mathrm{Area}}}{(w_A\times h_A)+(w_B\times h_B)-(\mathrm{Io}{U}_{\mathrm{Area}})}\end{array}$(6) where (X_A, Y_A), (X_B, Y_B) are centroids of detected (C_A) and predicted (C_B) bounding boxes as shown in Figure 3.

Figure 3. Calculation of IoUArea and IoU between detected and predicted bounding boxes.

Figure 4. (a) Assigning the identities to players and referees based on the threshold value of IoU in the consecutive frames. (b) Assigning the identities to players and referees by calculating the Euclidian Distance in consecutive frames.

3.3. Euclidean distance based tracking with unique identity

Within the outlined research methodology, we systematically evaluate the spatial distance between each identified player and the referee in consecutive frames by measuring the distance from the centroids of their respective bounding boxes. This intricate process incorporates the assignment of a distinct identity to each player and referee, a determination made based on the shortest Euclidean distance, visually exemplified in Figure 4(b). The allocation of identity to detected individuals hinges on a specified threshold value for the Euclidean Distance, introducing a crucial criterion for accuracy. Subsequently, the computation of Euclidean Distance is executed by employing precise equations tailored to proposed methodology. This comprehensive approach is designed to establish unequivocal and unique identities for players and the referee, thereby significantly enhancing the precision and reliability of our tracking system. (7) $\begin{array}{rl}C(1{)}_{\mathrm{bboxABF}(1)}^1& =\left(\left(x_{\mathrm{AB}(1)}^1+\frac{w_{\mathrm{AB}(1)}^1}{2}\right),\left(y_{\mathrm{AB}(1)}^1+\frac{h_{\mathrm{AB}(1)}^1}{2}\right)\right)=(c_{x(1)}^1,c_{y(1)}^1)\end{array}$(7) (8) $\begin{array}{rl}C(2{)}_{\mathrm{bboxABF}(2)}^1& =\left(\left(x_{\mathrm{AB}(2)}^1+\frac{w_{\mathrm{AB}(2)}^1}{2}\right),\left(y_{\mathrm{AB}(2)}^1+\frac{h_{\mathrm{AB}(2)}^1}{2}\right)\right)=(c_{x(2)}^1,c_{y(2)}^1)\end{array}$(8) (9) $\begin{array}{rl}C(3{)}_{\mathrm{bboxABF}(3)}^1& =\left(\left(x_{\mathrm{AB}(2)}^1+\frac{w_{\mathrm{AB}(2)}^1}{2}\right),\left(y_{\mathrm{AB}(2)}^1+\frac{h_{\mathrm{AB}(2)}^1}{2}\right)\right)=(c_{x(3)}^1,c_{y(3)}^1)\end{array}$(9) (10) $\begin{array}{rl}\mathrm{EU}{D}_{F12}& =\sqrt{{(c_{x(2)}^1-c_{x(1)}^1)}^2+{(c_{y(2)}^1-c_{y(1)}^1)}^2}\le \mathrm{Ma}{x}_{\mathrm{distance}}\end{array}$(10) (11) $\begin{array}{rl}\mathrm{EU}{D}_{F23}& =\sqrt{{(c_{x(3)}^1-c_{x(2)}^1)}^2+{(c_{y(3)}^1-c_{y(2)}^1)}^2}\le \mathrm{Ma}{x}_{\mathrm{distance}}\end{array}$(11) where $C(1{)}_{\mathrm{bboxABF}(1)}^1$, $C(2{)}_{\mathrm{bboxABF}(2)}^1$, $C(3{)}_{\mathrm{bboxABF}(3)}^1$ are the centroids of each detected bounding box in three consecutive frames calculated as shown in Figure 4(a).

In the context of proposed research methodology, the symbol $\mathrm{EU}{D}_{F12}$ represents the Euclidean Distance computation carried out between each bounding box identified in frame-1 and frame-2. Similarly, $\mathrm{EU}{D}_{F23}$ denotes the Euclidean Distance calculated between the bounding boxes detected in frame-2 and frame-3, as illustrated in Figure 4(b). This mathematical representation is integral to proposed methodology and aids in quantifying the spatial relationships between consecutive frames, contributing to a deeper understanding of the object-tracking process.

Therefore, to find out whether same player or referee is detected or not, minimum and maximum threshold values for IoU, EUD were defined and then identity was assigned in consecutive frames as shown in algorithm-2 (Figure 5).

Figure 5. Proposed tracking algorithm to assign the identity to players and referee.

3.4. Performance metrics

The following metrics were used to assess object detection and classification. The proposed tracking algorithm recognises and tracks three classes (Player_W, Player_B, Referee_R). Therefore, the metrics are described as follows.

Precision is defined as: (12) $\begin{array}{rl}P& =\frac{T_p}{T_p+F_p}\end{array}$(12) Recall is defined as: (13) $\begin{array}{rl}R& =\frac{T_p}{T_p+F_n}\end{array}$(13) Therefore, from precision and Recall, F1-score is defined as: (14) $\begin{array}{r}\text{F1-score}=2.\frac{P.R}{P+R}\end{array}$(14)

4. Results and experimental discussion

The experimental configuration is as follows: Windows 10 Pro operating system, Intel Xeon 64-bit CPU at 3.60 GHz, 64GB RAM, NVIDIA Quadro P4000 GPU, 8GB with 1792 Cuda-cores, CUDA10.0, CUDNN7.5 GPU acceleration library.

As BackgroundSubtractionMOG2 was utilised to detect objects (players and referee) in soccer field, it doesn’t need to train the model. The area of contour was calculated based on the object centroid and the coordinates of the objects bounding boxes, to remove small elements. The threshold value of the area to detect players and the referee was selected as greater than 1000 and less than 15,000. Jersey colour was used to classify team-based players as well as the referee. The performance of the suggested technique is discussed in this section, for multiple object classification and tracking and evaluated using the ISSIA dataset. Using ISSIA dataset, the technique proposed in the paper was compared to the benchmark which has been extensively used to track players in soccer videos. To show the accuracy and efficacy of the proposed methodology, it was compared to conventional techniques often used for tracking multiple objects, as shown in subsections below.

4.1. Qualitative analysis

This section presents the tracking results from a video sequence obtained using the ISSIA dataset, which was introduced by D'Orazio et al. (2009). To track efficiently, players and referees on the field required to be precisely spotted and categorised as Player_W, Plyer_B, and Referee_R. It is especially important that the location of each player within the squad is precisely detected and classified, and tracked with same identity. The identity of the players was changed from frame to frame when IoU is lost by tracking based on IoU as shown in yellow box in Figure 6 as a view of camera-2. Moreover, the identity of almost all the players and referees (Id-364, Id-365, and Id-356) was changed from frame to frame by tracking based on EUD as shown in yellow and sky blue box in Figure 7 as a view of camera-2. The identity of all the player and referee (Id-8, Id-19, and Id-23) remains same from frame to frame by tracking based on IoU and EUD as shown in yellow and sky blue box in Figure 8 as a view of camera-2.

Figure 6. Assigning the identity to the players and referee based on IoU while tracking on camera-2 from ISSIA dataset using proposed methodology (Frame-253 to Frame-278).

Figure 7. Assigning the identity to the players and referee based on EUD while tracking on camera-1 from ISSIA dataset using proposed methodology (Frame-39 to Frame-138).

Figure 8. Assigning the identity to the players and referee based on IoU and EUD while tracking on camera-2 from ISSIA dataset using proposed methodology (Frame-43 to Frame-136).

The proposed approach tracked team-wise players and the referee, and the tracking results from three distinct camera views from ISSIA dataset videos are presented in Figure 9.

Figure 9. Tracking performance of three different cameras views around the soccer field taken from ISSIA Dataset.

4.2. Quantitative analysis

Tracking results were evaluated using several performance measures in this subsection. To statistically examine the efficacy of the tracking algorithm, 500 frames were manually labelled as Player_W, Player_B, and Referee_R from the actual video of ISSIA dataset. The proposed tracking approach classifies and tracks three classes (Player_W, Player_B, and Referee_R), whereas the other methods identified and tracked either player or the ball. To commence, the comprehensiveness of the tracking method was assessed using five metrics which are precision, recall, f1-score, accuracy, and confusion matrix. Therefore the performance of the proposed methodology on ground truth labels is shown in Table 1, and the following confusion matrix is shown in Figure 10. Identity associations with respect to IoU and Euclidian Distance are shown in Table 2.

Figure 10. Confusion Matrix ground truths and predicted labels on ISSIA dataset.

Table 1. Performance of the proposed methodology on ground truth labels.

Id No:	Precision	Recall	F1-Score
1	1.0	0.88	0.93
2	0.88	0.88	0.88
4	1.0	0.90	0.95
5	0.89	0.94	0.92
6	1.0	0.78	0.88
7	1.0	0.95	0.97
8	0.71	0.91	0.80
9	0.75	0.86	0.80
10	1.0	0.92	0.96
12	0.94	0.94	0.94
14	0.90	1.0	0.95
15	0.92	0.92	0.92
16	1.0	1.0	1.0
17	1.0	1.0	1.0
18	1.0	1.0	1.0
19	1.0	0.80	0.89
20	0.75	1.0	0.86
21	1.0	0.92	0.96
25	0.75	1.0	0.86
26	1.0	1.0	1.0
27	0.80	0.92	0.86
33	1.0	0.33	0.50
34	1.0	1.0	1.0
35	1.0	1.0	1.0
Weighted Average	0.90	0.88	0.91
Accuracy	0.88

Bold values signifies the overall performance of proposed detection method.

Table 2. Identity assigning based on the IoU and EUD in 12 consecutive frames.

ID NO		1	12	22	15	17	9	23	24	25	26
F-32	IoU	0.85	0.73	0.87	0.95	0.95	1.0	0.97	0.88	0.85	1.0
	EUD	5.83	9.48	4.12	0.0	2.23	1.41	2.0	2.82	3.0	1.0
F-33	IoU	0.73	0.88	0.86	0.72	0.86	0.99	0.85	0.89	0.82	0.85
	EUD	9.48	3.60	4.0	8.60	5.0	1.41	3.0	2.23	5.0	5.0
F-34	IoU	0.74	0.83	0.75	0.98	0.87	0.87	0.82	0.77	0.92	0.80
	EUD	8.24	5.38	6.32	0.0	2.0	4.12	6.0	9.21	3.16	7.0
F-35	IoU	0.63	0.86	0.80	0.98	0.82	0.94	0.82	0.72	0.90	0.81
	EUD	11.1	4.47	4.12	1.41	4.47	2.23	3.60	10.2	4.0	6.0
F-36	IoU	0.86	0.85	0.82	0.91	0.81	0.90	0.69	0.93	0.99	0.90
	EUD	4.0	6.08	4.0	4.12	4.12	4.47	6.70	2.23	1.41	3.0
F-37	IoU	0.85	0.86	0.75	0.82	0.86	0.85	0.77	0.88	1.0	0.85
	EUD	4.24	6.40	7.0	5.0	2.82	4.12	5.83	3.16	1.0	4.47
F-38	IoU	0.83	0.91	0.76	1.0	0.76	0.89	0.68	0.93	1.0	0.95
	EUD	6.08	4.12	7.0	1.0	5.09	3.16	7.0	2.82	1.0	2.23
F-39	IoU	0.86	0.89	0.87	0.96	0.86	0.87	0.83	0.97	1.0	0.88
	EUD	5.0	5.0	5.0	2.23	4.0	3.16	4.0	1.41	2.0	3.60
F-40	IoU	0.78	0.84	0.92	1.0	0.94	0.86	0.79	0.91	0.92	0.85
	EUD	8.06	6.0	1.41	1.0	2.23	2.0	4.12	2.82	2.0	4.24
F-41	IoU	0.80	0.84	1.0	0.99	0.92	1.0	0.78	0.93	0.90	0.93
	EUD	7.28	5.0	1.0	1.0	1.41	0.0	6.08	2.23	2.23	2.23
F-42	IoU	0.67	0.83	1.0	0.96	0.89	0.97	0.99	0.98	0.85	0.92
	EUD	13.3	4.12	1.0	3.16	2.23	2.0	1.0	1.41	3.16	3.0
F-43	IoU	0.72	1.0	0.90	0.94	0.95	0.96	0.91	0.95	0.96	0.89
	EUD	12.0	0.0	4.12	2.82	1.0	2.0	2.82	2.23	1.41	3.60

4.3. Limitations of proposed approach

There are a few limitations to our approach. The algorithm demonstrates exceptional robustness when dealing with static backgrounds. The algorithm encounters difficulties when there is movement in the background, as player detection relies on background subtraction methodology. Identity switching becomes increasingly noticeable in scenarios with severe occlusions, given that identity assignment is contingent on the IoU and Euclidean distance methods.

4.4. Comparative analysis

It denotes the tracker's ability to forecast precise object coordinates, regardless of proficiency in recognising object configurations, maintaining consistent trajectories, and so on. In order to evaluate tracking performance quantitatively, the proposed technique was evaluated to six representative tracking algorithms which are proposed on the classes Player (P), Ball (B), and Referee (R) i.e. DLBT (Kamble et al. 2019), Tracklet-based Multi-Commodity Network Flow (T-MCNF) (Song et al. 2021), Small-Soccer Player (Hurault et al. 2020), Topography (Kim 2019), Deep-Player Track (Naik et al. 2022b) and Adaptive Multiscale approach (Kim et al. 2018) on ISSIA dataset depending on Precision, Recall, F1-score, and IoU/EUD metrics as shown in Table 3.

Table 3. The proposed tracking algorithm was compared with state-of-the-art approaches on the ISSIA dataset.

	(Kamble et al.	(Song et al.	(Hurault et al.	(Kim	(Naik et al.	(Kim et al.
Ref.	2019)	2021)	2020)	2019)	2022b)	2018)	Proposed
Class	P&B	P	P	P	P,R&B	P	P&R
Precision	0.93	0.63	–	–	0.93	–	0.90
Recall	0.73	–	–	–	0.95	–	0.88
F1-score	–	–	–	–	0.94	–	0.91
Accuracy	0.87	–	0.81	0.92	–	0.76	0.88
FPS	10	-	6.5	–	23	–	6.56

Bold values indicate superior performance of the proposed tracking methodology over others.

In contrast, the methodology described in this paper offers a solution to the previously mentioned problem by skilfully handling situations in which the jersey colours of players and the referee undergo changes. This encompasses instances where users are required to manually update the colour thresholds within the algorithm to accommodate the new jersey colours for each inning of soccer matches.

4.5. Industrial significance of the proposed approach

The “EIoU-Distance Loss” holds significant industrial relevance in the context of automated team-wise player detection and tracking with jersey colour recognition in soccer. This approach addresses a critical need in various industries and applications:

Sports Analytics: In the field of sports analytics, such as soccer, automated player detection and tracking are invaluable. It allows teams and coaches to gain insights into player performance, team dynamics, and opponent strategies. The ability to recognise jersey colours is essential for tracking players effectively.

Broadcasting and Media: Automated player tracking enhances the viewing experience for sports fans by providing real-time information about player movements and statistics. It contributes to better sports broadcasting and analysis, attracting more viewers and advertisers.

Security and Surveillance: Automated tracking systems can be applied in security and surveillance scenarios, including monitoring crowd behaviour during large sporting events. Recognising players and referees by their jersey colours can aid in crowd management and security measures.

Training and Coaching: Soccer teams and academies can use automated player tracking for training and coaching purposes. It assists in assessing player performance, making tactical decisions, and improving overall team strategies.

Merchandising and Fan Engagement: In the commercial aspect of sports, recognising players by their jersey colours can be used for personalised merchandise, fan engagement, and marketing campaigns.

In summary, the EIoU-Distance Loss method for automated team-wise player detection and tracking with jersey colour recognition in soccer has significant industrial applications across sports analytics, broadcasting, security, training, commercialisation, and technology integration. It enhances the overall experience for sports enthusiasts, professionals, and businesses involved in the sports industry.

5. Conclusion

This study introduces a comprehensive methodology for multi-object detection, categorisation, and tracking, commencing with foreground extraction from the background, followed by precise object detection. A notable aspect is the effective categorisation of team players and officials using a colour mask, specifically identifying Player_W, Player_B, and the Referee. Subsequently, the classification facilitates the computation of Intersection over Union (IoU) and Euclidean Distance (EUD) between detected and predicted bounding boxes across consecutive frames. The assignment of unique IDs to each player and referee plays a pivotal role in significantly enhancing tracking performance.

The study's findings are systematically presented in three main sections: (1) Visual representations of qualitative analysis results, considering the presence or absence of an embedded platform; (2) Quantitative analysis results conveyed through diverse performance metrics; (3) Comparative analysis, systematically benchmarking the proposed methodology against other state-of-the-art methods using the ISSIA dataset.

Acknowledging two inherent limitations, the study notes that (i) when a player is undetected in one frame but identified in a subsequent frame, a distinct ID is assigned and (ii) instances of player occlusion or crossover result in the assignment of different IDs to the involved players.

In the pursuit of enhancing tracking capabilities for future applications, specifically with the goal of averting instances of identity confusion among players within the same squad, our research advocates for the integration of a Bi-lateral transformation technique and the implementation of jersey number identification. The comprehensive nature of the proposed methodology underscores its adaptability, indicating potential utility across a spectrum of real-world scenarios. This extends beyond sports like basketball and rugby, emphasising its capacity to address tracking challenges in various contexts. The recommendation emerges from a recognition of the multifaceted demands of tracking technology and the need for a versatile solution that can effectively navigate diverse scenarios.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The authors reported there is no funding associated with the work featured in this article.