A Human-In-One-Loop Active Domain Adaptation Framework for Digit Recognition

ABSTRACT

Domain adaptation can effectively enhance a model’s performance on target domain data with limited data. However, when some target domain labels are obtainable, training the model with both source and target domain data simultaneously can lead to worse performance due to the lower density of target domain data. If a large amount of target domain data is labeled without discrimination, it will necessitate a considerable expenditure of human resources. To address this issue, this paper proposes a human-in-one-loop active domain adaptation framework based on Target Domain Feature Generation to solve the problems. The oracle participates in only one iteration of data labeling, and a target domain classifier will take over the subsequent rest iterations. An image generator based on multiple CycleGANs forms an iterative co-training mechanism, which can continuously generate more high-quality labeled fake target domain data in iterations to improve the performance of the target domain classifier. The Top-N labeled data selection method with high confidence is devised to select the most accurately predicted data for labeling, reducing manual labeling workload. This framework can achieve an average accuracy of 0.8869 on six domain pairs, doubling the classical domain adaptation method DSN, requiring only a small amount of manual labeling.

Introduction

The issue of digital classification is widely present in various application scenarios, such as meter recognition, road sign recognition, and invoice recognition. In a controlled experimental setting, these tasks achieve relatively good performance (Xiu et al. 2022; Zhang et al. 2022). However, there is often a discrepancy between the training data and the data from the actual production environment (Zhou et al. 2023). Domain adaptation is an important branch of transfer learning, suitable for situations where the target domain and the source domain only differ in data, without altering the task at hand (Zhao et al. 2020), and domain adaptation mainly achieves knowledge transfer based on shared features between the source and target domains, which can effectively improve the model’s performance on the target domain data even when training data is scarce (Cao et al. 2021).

Recently, the Unsupervised Data Adaptation (UDA) method has effectively improved generalization capabilities on the target domain by aligning distributions, but the model is unable to learn class discriminant boundary (Li, Li, and Yu 2024; Saito et al. 2019; Tan and Zheng 2023). In practical applications, it is feasible to label a small amount of target domain data. However, when both source and target domain data are used for training the model simultaneously, and some target domain labels are available, the small sample density in the target domain can affect the extraction of source domain features. This can lead to the model not always improving performance as expected and, in some cases, can even result in a decline in transfer performance (Saito et al. 2019; Zhou et al. 2023). Even if we can obtain instance data from the target domain, fully labeling the target domain data would require a significant amount of manpower, which is unrealistic in practical applications.

The previous method aimed to reduce the distribution shift between the source and target domains by applying adversarial learning and minimizing statistical discrepancies. This allowed the classifier trained on the source domain to be directly applied to the target domain (Ge et al. 2023). However, label mismatch was a significant issue faced by the previous method. Due to architectural limitations, the small sample size in the target domain and the difference in density between the source and target domains made it challenging for the model, which was based on adversarial learning, to effectively distinguish between different data in the source and target domains (Zhao et al. 2020).

To address the issues mentioned above, this paper proposes a human-in-one-loop active domain adaptation framework based on Target Domain Feature Generation (TDFG), as shown in Figure 1. The approach of using image style transfer was adopted to generate target domain images with correct labels. In this process, a certain amount of data was needed to learn the image style of the target domain. To achieve this, a human-in-the-loop method (HITL) was involved. HITL allows the model to better utilize unlabeled target domain data while improving performance. By incorporating human feedback, the model can refine its understanding of the target domain, thereby enhancing its ability to classify target domain data accurately.

Figure 1. The framework of human-in-one-loop active domain adaptation framework based on Target domain Feature Generation (TDFG). TDFG aims to enhance model performance on target domain data by involving human input in the data labeling process. By utilizing a co-training mechanism with an image generator and target domain classifier, high-quality labeled fake target domain data is continuously generated to improve classification accuracy.

Existing human-in-the-loop method faces the challenge of obtaining a reliable model using as few human resources as possible (Gómez-Carmona et al. 2024). To select high-value data for model correction, some past methods have focused more on the selection strategy (Konyushkova, Sznitman, and Fua 2017; Saito et al. 2019). In contrast, in terms of data annotation, the Target Domain Feature Generation (TDFG) approach adopted the Top-N labeling data selection method with high confidence (TopNHC), as shown in Figure 1, which involves annotating target domain samples with high-confidence predictions from the classifier. Through multiple rounds of training of high-confidence samples, the model achieved better performance in experiments.

Furthermore, to further reduce the amount of data annotation, the TDFG method utilized a group of image generators, as shown in the top part of Figure 1. By using a target domain image generator, it learned the features of target domain images and generated more target domain images. The TDFG method enhanced the model’s performance on the target domain by annotating fewer critical data points, thus reducing the consumption of human resources. This approach allows for an efficient use of human expertise, the human annotator in Figure 1, focusing on annotating the most informative examples that contribute the most to improving the model’s performance on the target domain.

Here is a summary of the entire framework: 1) To achieve human-in-one-loop, this framework is designed into two stages. The human annotator only needs to participate in the first stage of data labeling. In contrast, the second stage of data labeling will be undertaken by a target domain classifier with guaranteed performance. 2) To obtain robust migration performance, alleviating the phenomenon of label mismatch, an image generator based on multiple CycleGANs, which is the red component in Figure 1, is used to form an iterative training mechanism with a target domain classifier. In each iteration, the classifier, which is the green component in Figure 1, will make predictions on the target domain data and select a portion of labeled target domain data and source domain data to form the training data for the image generator. The trained image generator will generate more, and higher-quality labeled fake target domain data to improve the performance of the target domain classifier. The first iteration is the first stage of the framework, whose data labeling is done by the trained source domain classifier, the blue component in Figure 1, and requires manual correction of mislabeling. The subsequent iterations are the second stage of the framework, whose data labeling is entirely undertaken by the trained target domain classifier of the corresponding iteration. 3) To reduce the workload of manual labeling and to ensure the reliability of autonomous labeling in the second stage, this paper proposes a TopNHC method. Thus TDFG provides more reliable autonomous labels to ensure a virtuous cycle of iterative training of the target domain classifier and image generator.

The contribution of this paper is listed as follows. 1) We propose a human-in-one-loop active domain adaptation framework based on target domain feature generation (TDFG), which can achieve high-performance migration and only require a small amount of manual labeling in one iteration. 2) The image generator creates the target domain feature, which guarantees the quantity and quality of the generated data and minimizes the labeling workload. 3) We design a Top-N labeled data selection method with High Confidence (TopNHC), which can efficiently select valuable samples, providing a new and optional approach to achieving human-in-one-loop and reducing the manual labeling workload. 4) The TDFG framework can achieve the average classification accuracy of 0.8869 when facing six sets of data domain pairs, which is almost double the classical domain adaptation method DSN.

Related Works

Deep Domain Adaptation

Deep domain adaptation (Wang and Deng 2018) is to achieve knowledge transfer based on deep learning models, which can be generally divided into the following categories:

1. Discrepancy-based Deep Domain Adaptation. Such methods mainly focus on reducing the feature differences between domains so that the feature extractor can learn the shared features between different domains. Tzeng et al. (2014) proposed the Deep Domain Confusion (DDC) network, which uses Maximum Mean Discrepancy (MMD) (Borgwardt et al. 2006) to quantify inter-domain distances, and the MMD is added to the loss function of the network. This way, the feature differences between the source and target domain can be reduced. Based on DDC, joint MMD (Long et al. 2017), joint probability MMD (Zhang and Wu 2020), The Weighted MMD (Yan et al. 2017), and other measures have emerged to improve the performance of discrepancy-based deep domain adaptation continuously. Recently, the concept that multiple source domains represent a variety of distributions is a common scenario in practical applications. Through multi-source domain adaptation, the effectiveness of models in real-world scenarios can be significantly enhanced. Guided by this principle, Cheng et al. (2024) proposed the Deep Joint Semantic Adaptation Network to address the challenge of migrating from multiple source domains to a single target domain. This approach involves Joint Semantic Maximum Mean Discrepancy, a new maximum mean discrepancy-based metric, to uniformly optimize the cross-domain joint distribution of category-corresponded subdomains on multiple task-specific layers.

2. Generative Adversarial Networks (GAN) (Goodfellow et al. 2020) inspire adversarial-based Deep Domain Adaptation. Ganin et al. (2016) first applied the adversarial concept to domain adaptation by proposing Domain Adversarial Neural Networks (DANN). The feature extractor aims to extract shared features that can improve the performance of the task classifier, and the domain classifier aims to distinguish whether the extracted features belong to the source or target domain, which forms an adversarial mechanism to improve the migration performance of the model. Wang et al. (2019) added attention mechanism to DANN by arguing that not all regions in an image are worth migrating, so the images are divided into different sub-regions, and the domain classifier is decomposed into different sub-domain classifiers based on the regions, and only the worth parts are migrated. Bousmalis et al. (2017) proposed PixelDA, a domain adaptation method based on pixel space generation, which processes the source domain data to be similar to the target domain data by adversarial generation and trains the recognition model directly on the processed source domain data.

3. Reconstruction-based Deep Domain Adaptation. The idea of this method is to encode and then reconstruct the input data to ensure that the encoding process extracts the shared features and ensures that the features unique to each domain still exist after reconstruction simultaneously. Ghifary et al. (2016) used AutoEncoder (AE) (Lore, Akintayo, and Sarkar 2017) to extract features common to the source and target domains and then input them into a decoder to reconstruct the target domain samples, ensuring that the model can extract shared features while preserving some information specific to the target domain, thus improving the performance of the model for the target domain. Ghifary et al. (2015) focus on domain adaptation under multitasking, using an encoder corresponding to multiple decoders to reconstruct data from different domains. Bousmalis et al. (2016) discussed domain-shared features and domain-private features separately and limited the difference between domain-shared features and domain-private features to be large enough to extract domain-shared features effectively using the adversarial mechanism of feature extractor and domain classifier while making the integration of extracted domain-shared features and domain-private features can reconstruct the original data to ensure that no features are missed. In relation to this, in the area of handwritten digit generation, Anwar et al. (2021) proposed a new framework. This involves collecting user-written character shapes and then, based on prior knowledge, calculating local peaks from horizontal and vertical projection functions. The inclusion of human participation significantly enhances the speed at which numerical digits are generated.

Recently, Tan and Zheng (2023) introduced a method for active learning in object detection. In contrast to the approach described in Tan and Zheng (2023), our method focuses on object classification. It is important to note that object classification and object detection exhibit distinct performances in a human-in-the-loop (HITL) setting. Unlike the method proposed by Tan and Zheng (2023), the Target Domain Feature Generation (TDFG) does not train directly on unlabeled data from the target domain. Instead, it concentrates on learning the data of the target domain through image style transfer. This process involves using source domain data as input and generating new data in the style of the target domain, thereby creating accurate pairs of target domain data and labels. Li, Li, and Yu (2024) proposed an inter-domain mixup approach, employing a cross-domain alignment strategy that incorporates label information into model adaptation. This additional supervision aids in cross-domain feature alignment and reduces label mismatch issues. Contrary to Li, Li, and Yu (2024), TDFG employs a generator for producing target domain data, rather than using adversarial training. The generator group in TDFG simultaneously learns features from both the target and source domains, performing style transfer on target domain data to generate samples that are stylistically aligned with the target domain.

Human-In-The-Loop and Active Domain Adaptation

The purpose of human-in-the-loop (HITL) (Budd, Robinson, and Kainz 2021; Wu et al. 2022) is to improve the performance of machine learning algorithms and enhance human productivity through the interaction between humans and machine learning algorithms. Active learning and data selection are the foundations of human-in-the-loop. In active learning, it aims to optimize the model performance with a limited amount of labeled data (Ren et al. 2021). For an unlabeled dataset, the oracle will select a small portion of samples for manual labeling in an iteration. After each labeling iteration, the model will be trained and tested for performance. This process will be repeated until the model performance reaches the desired level. In data selection, depending on the practical scenarios, the data selection methods in active learning vary, and there are standard selection methods based on uncertainty, diversity, representativeness, reducing expected errors, maximizing expected label changes, and so on (Fu, Zhu, and Li 2013).

The performance of migration models can also be improved if labels are provided to the target domain data by human-in-the-loop. Thus, many works have combined active learning with domain adaptation, and the concept of active domain adaptation was born. Mayer and Timofte (2020) used the idea of adversarial generation in active learning to generate many uncertainty samples to be labeled to participate in model training. Su et al. (2020) based on DANN (Ganin et al. 2016) network, proposed to select and provide labels for target domain data that are difficult to accurately predict by classifiers based on classifier prediction entropy and domain discriminator cross-entropy, these data are considered to be less similar to the source domain data. Saito et al. (2018) used finetuning (Yosinski et al. 2014) as a migration method to design a tri-training data labeling method to enrich the target domain labeling: two independent source domain classifiers are used to predict the target domain data, and the labeling is considered accurate if the predictions of the two source domain classifiers agree. Prabhu et al. (2021) proposed the Clustering Uncertainty-weighted Embeddings (CLUE) data selection method for active domain adaptation, which uses uncertainty-weighted clustering to identify target data for labeling that are both uncertain under the model and diverse in feature space. Ma, Gao, and Xu (2021) proposed the Clustered Non-Transferable Gradient Embedding (CNTGE) strategy that uses the principles of transferability, diversity, and uncertainty to select and label target domain data.

Rangwani et al. (2021) proposed an ensemble-based information criterion that selects uncertain, diverse, and representative target domain data by sensing other samples in a target domain subset and labeling them. To effectively improve the performance of machine models, Wu et al. (2023) proposed a HITL-based approach that augmented the capabilities of autonomous driving systems. HITL has also been demonstrated to facilitate more efficient model compression to suit inference environments. Wang et al. (2023) introduced a human-in-the-loop method for the automatic generation of deep neural networks for mobile applications. Gómez-Carmona et al. (2024) conducted a quantitative study on HITL approaches, highlighting the importance of considering human factors in the design of more effective and flexible collaborative systems between humans and machines. Tan and Zheng (2023) introduced an active learning and semi-supervised approach to fully leverage unlabeled data, aiming to enhance the accuracy of deep object detection.

TDFG integrates human-in-the-loop with image style transfer, achieving superior performance while reducing labor costs. By utilizing this approach, TDFG effectively improves the performance of classification models and minimizes the consumption of human labor resources. This advancement contributes significantly to the related work in the field, demonstrating the potential of combining HITL strategies with advanced data augmentation techniques to address the challenges in machine learning applications.

Related Work Summary

The mentioned works above have creatively designed many transfer learning techniques and methods to improve migration performance and integrated domain adaptation and human-in-the-loop to promote the development of related fields. Recent methods have effectively extracted domain-invariant features by using semi-supervised approaches and minimizing the differences across domains. However, there is still considerable room for improvement when it comes to addressing the imbalance between the number of source domain samples and target domain samples, as well as the epistemic uncertainty, according to Kendall and Gal (2017), caused by the lack of target domain data. Furthermore, most of the active learning methods mentioned above focus only on data selection methods and have not yet recognized the feasibility of image style transfer in reducing labor costs. We designed TDFG to address these overlooked issues, employing image transfer for data augmentation to effectively alleviate the problem of data imbalance. Additionally, we utilized the TopNHC method to reduce the labor cost of data annotation.

Methodology

Define the source domain as ${\mathbb{D}}_S=\left\{{\mathbb{X}}_S,{\mathbb{P}}_S\right\}$, and the target domain as ${\mathbb{D}}_T=\left\{{\mathbb{X}}_T,{\mathbb{P}}_T\right\}$, where $\mathbb{X}$ denotes the data space, and $\mathbb{P}$ denotes the distribution of the corresponding data. Define the source domain task as ${\mathbb{T}}_S=\left\{{\mathbb{Y}}_S,f_S(\cdot )\right\}$ and the target domain task as ${\mathbb{T}}_T=\left\{{\mathbb{Y}}_T,f_T(\cdot )\right\}$, where $\mathbb{Y}$ denotes the label space and $\mathit{f}(\cdot )$ denotes the mapping from the data space to the label space. Define the source domain data as $X_S={\left\{\left(x_{S_i},y_{S_i}\right)\right\}}_{\mathit{i}=1}^{n_S}$, where $x_{S_i}\in {\mathbb{X}}_S$,$y_{S_i}\in {\mathbb{Y}}_S$, and the target domain data as $X_T={\left\{x_{T_i}\right\}}_{\mathit{i}=1}^{n_T}$, where $x_{T_i}\in {\mathbb{X}}_T$. The purpose of this paper is to train a ${\hat{f}}_{\mathit{T}}\left(\cdot \right)$ so that it is close enough to the mapping of the target domain data space to the labeling space $f_T\left(\cdot \right)$ in the function space to achieve the ability to accurately map the target domain data $X_T$ to the label space $Y_T$ in the presence of less feature commonality between the source and target domain.

Human-In-One-Loop Active Domain Adaptation Framework

This paper proposes an active domain adaptation framework based on Target Domain Feature Generation (TDFG) to solve the problem of poor migration performance with insufficient shared features and label mismatch problems. TDFG framework is designed into two stages to reduce the large, round-by-round manual labeling costs associated with human-in-the-loop. The first stage is active generation domain adaptation, where the human annotator only needs to participate in the first stage of data labeling. The process of the first stage is shown in Figure 2.

Figure 2. The first stage, active generation domain adaptation. In this stage, a trained source domain classifier will make predictions on the target domain data, and based on the prediction results, a portion of the target domain data is selected using the TopNHC data selection method (cf., section “Top-N labeled data selection method with high confidence”), and a subset of accurately labeled target domain data is obtained by mislabeling correction of the oracle. An image generator is trained using those accurately labeled target domain data and the source domain data. The image generator takes the source domain data as input and generates a large amount of labeled fake target domain data. The labeled fake target domain data and those accurately labeled target domain data are used to train the target domain classifier.

Framework Overview

The framework consists of two main stages: the first stage involves manual labeling by the human annotator, while the second stage utilizes a target domain classifier for autonomous labeling. An image generator based on multiple CycleGANs is employed to generate high-quality labeled fake target domain data, which is used to enhance the performance of the target domain classifier, as shown in the right part of Figure 2.

Training Details

In the first stage, the unlabeled target domain data $X_T={\left\{x_T^{⟨i⟩}\right\}}_{\mathit{i}=1}^{n_T}$ is predicted using the source domain classifier ${\hat{C}}_{\mathit{S}}\left(\cdot \right)$, which is trained by the source domain data $X_S={\left\{\left(x_S^{⟨i⟩},y_S^{⟨i⟩}\right)\right\}}_{\mathit{i}=1}^{n_S}$. Then the $n_{\mathit{true}}$ samples with the highest prediction confidence are selected by the TopHNC data selection method (cf., section “Top-N labeled data selection method with high confidence”) and manually corrected to obtain a set of accurately labeled target domain data $X_{{\mathit{T}}_{\mathit{true}}}^0={\left\{\left(x_T^{0;⟨i⟩},y_{{\mathit{T}}_{\mathit{true}}}^{0;⟨\mathit{i}⟩}\right)\right\}}_{\mathit{i}=1}^{n_{\mathit{true}}}$. This accurately labeled target domain data $X_{{\mathit{T}}_{\mathit{true}}}^0$ and the source domain data $X_S$ are used to train the image generator $G_{S\to T}^0\left(\cdot \right)$ (cf., section “Image generator”). The trained image generator $G_{S\to T}^0\left(\cdot \right)$ can generate a set of fake target domain data with labels $X_{{\mathit{T}}_{\mathit{fake}}}^0={\left\{\left({\mathit{x}}_{{\mathit{T}}_{\mathit{fake}}}^{0;⟨\mathit{i}⟩},y_{{\mathit{T}}_{\mathit{fake}}}^{0;⟨\mathit{i}⟩}\right)\right\}}_{\mathit{i}=1}^{n_{\mathit{fake}}}$ based on a set of source domain data. The labeled fake target domain data $X_{{\mathit{T}}_{\mathit{fake}}}^0$ and the accurately labeled target domain data $X_{{\mathit{T}}_{\mathit{true}}}^0$ will be mixed as the training data for the target domain classifier $C_T^0\left(\cdot \right)$, where the target domain classifier is initialized with the parameters of the trained source domain classifier ${\hat{C}}_{\mathit{S}}\left(\cdot \right)$. Equation (1)(1) $L_{C_S}=-\sum _{\mathit{i}=1}^{n_S}{y}_S^{⟨i⟩}log{C}_S\left(x_S^{⟨i⟩}\right)$(1) , Equation (2)(2) $L_{C_T}=-\sum _{\mathit{i}=1}^{{\mathit{n}}_{\mathit{fake}}}{y}_{{\mathit{T}}_{\mathit{fake}}}^{⟨\mathit{i}⟩}log{C}_T\left(x_{{\mathit{T}}_{\mathit{fake}}}^{⟨\mathit{i}⟩}\right)-\sum _{\mathit{i}=1}^{{\mathit{n}}_{\mathit{true}}}{y}_{{\mathit{T}}_{\mathit{true}}}^{⟨\mathit{i}⟩}log{C}_T\left(x_T^{⟨i⟩}\right)$(2) , and Equation (3)(3) $X_{{\mathit{T}}_{\mathit{true}}}^0={\left\{\left(x_T^{0;⟨i⟩},y_{{\mathit{T}}_{\mathit{true}}}^{0;⟨\mathit{i}⟩}\right)\right\}}_{\mathit{i}=1}^{n_{\mathit{true}}}=\mathrm{TopNHC}\left({\hat{C}}_{\mathit{S}}\left(\cdot \right),X_T,n_{\mathit{true}}\right)$(3) respectively define the loss function of the source domain classifier, the loss function of the target domain classifier, and the TopNHC data selection process in the first stage.

(1) $L_{C_S}=-\sum _{\mathit{i}=1}^{n_S}{y}_S^{⟨i⟩}log{C}_S\left(x_S^{⟨i⟩}\right)$(1)

(2) $L_{C_T}=-\sum _{\mathit{i}=1}^{{\mathit{n}}_{\mathit{fake}}}{y}_{{\mathit{T}}_{\mathit{fake}}}^{⟨\mathit{i}⟩}log{C}_T\left(x_{{\mathit{T}}_{\mathit{fake}}}^{⟨\mathit{i}⟩}\right)-\sum _{\mathit{i}=1}^{{\mathit{n}}_{\mathit{true}}}{y}_{{\mathit{T}}_{\mathit{true}}}^{⟨\mathit{i}⟩}log{C}_T\left(x_T^{⟨i⟩}\right)$(2)

(3) $X_{{\mathit{T}}_{\mathit{true}}}^0={\left\{\left(x_T^{0;⟨i⟩},y_{{\mathit{T}}_{\mathit{true}}}^{0;⟨\mathit{i}⟩}\right)\right\}}_{\mathit{i}=1}^{n_{\mathit{true}}}=\mathrm{TopNHC}\left({\hat{C}}_{\mathit{S}}\left(\cdot \right),X_T,n_{\mathit{true}}\right)$(3)

In the first stage, the idea of active learning is introduced to provide accurate labels for part of the target domain data so that more private but valuable features in the target domain data can be learned, which lays the foundation for the migration framework to be well applied to domains with less feature commonality. In addition, the image generator can learn the mapping from the source domain to the target domain, generating a more diverse and larger amount of fake target domain data with labels. The participation of these generated data enables the target domain classifier to learn target domain features more adequately and improve its performance and generalization ability. However, the labeled fake target domain data generated by the image generator in a single epoch may suffer from insufficient data volume and low data quality, resulting in the target domain classifier failing to achieve the desired migration performance. Therefore, the second stage of this framework is designed to compose the target domain classifier and the image generator into an iterative training mechanism, as shown in Figure 3.

Figure 3. The second stage, iterative generation training. The image generator and target domain classifier are designed to form an iterative generation training mechanism to generate more diverse and larger amounts of labeled fake target domain data for training the target domain classifier. In this stage, the target domain classifier with reliable performance directly performs the data labeling work without human involvement. The autonomously labeled data from the current iteration will train the image generator together with the source domain data and the accurately labeled target domain data. The labeled fake target domain data generated by the image generator will train the target domain classifier together with the accurately labeled target domain data. The performance of the image generator and target domain classifier will be continuously improved in iterations.

After training the target domain classifier and the image generator in the first iteration, the framework will enter the iterative generation training stage. The target domain classifier trained in $\mathit{j}$-th iteration $\widehat{C}\mathit{jT}(\cdot )$ will make predictions on the target domain data. Then the TopNHC data selection method is used to select a set of target domain data with high prediction confidence $X_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j}}={\left\{\left(X_T^{j;⟨i⟩},y_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j};⟨\mathit{i}⟩}\right)\right\}}_{\mathit{i}=1}^{n_j}$, and directly use the prediction results as the label of this set of data. Then use source domain data $X_S$, selected autonomous labeled target domain data $X_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j}}$ and accurately labeled target domain data $X_{{\mathit{T}}_{\mathit{true}}}^0$ to train the image generator $G_{S\to T}^j\left(\cdot \right)$ (cf., section “Image generator”). After the training, the image generator generates a large amount of fake target domain data with labels $X_{{\mathit{T}}_{\mathit{fake}}}^{\mathit{j}}$ according to the source domain data. The labeled fake target domain data $X_{{\mathit{T}}_{\mathit{fake}}}^{\mathit{j}}$ and the accurately labeled target domain data $X_{{\mathit{T}}_{\mathit{true}}}^0$ will be used as training data for the target domain classifier in the next iteration. The TopNHC data selection process and the loss function of the target domain classifier in the second stage are respectively shown in Equation (4)(4) $X_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j}}={\left\{\left(x_T^{j;⟨i⟩},y_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j};⟨\mathit{i}⟩}\right)\right\}}_{\mathit{i}=1}^{n_j}=\mathrm{TopNHC}\left(\hat{C}\mathit{Tj}\left(\cdot \right),X_T,n_j\right)$(4) and Equation (5)(5) $L_{C_T}^{\mathit{j}}=-\sum _{\mathit{i}=1}^{n_j}{y}_{{\mathit{T}}_{\mathit{fake}}}^{\mathit{j};⟨\mathit{i}⟩}log{C}_T^j\left(x_{{\mathit{T}}_{\mathit{fake}}}^{\mathit{j};⟨\mathit{i}⟩}\right)-\sum _{\mathit{i}=1}^{{\mathit{n}}_{\mathit{true}}}{y}_{{\mathit{T}}_{\mathit{true}}}^{⟨\mathit{i}⟩}log{C}_T^j\left(x_{{\mathit{T}}_{\mathit{true}}}^{⟨\mathit{i}⟩}\right)$(5) .

(4) $X_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j}}={\left\{\left(x_T^{j;⟨i⟩},y_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j};⟨\mathit{i}⟩}\right)\right\}}_{\mathit{i}=1}^{n_j}=\mathrm{TopNHC}\left(\hat{C}\mathit{Tj}\left(\cdot \right),X_T,n_j\right)$(4)

(5) $L_{C_T}^{\mathit{j}}=-\sum _{\mathit{i}=1}^{n_j}{y}_{{\mathit{T}}_{\mathit{fake}}}^{\mathit{j};⟨\mathit{i}⟩}log{C}_T^j\left(x_{{\mathit{T}}_{\mathit{fake}}}^{\mathit{j};⟨\mathit{i}⟩}\right)-\sum _{\mathit{i}=1}^{{\mathit{n}}_{\mathit{true}}}{y}_{{\mathit{T}}_{\mathit{true}}}^{⟨\mathit{i}⟩}log{C}_T^j\left(x_{{\mathit{T}}_{\mathit{true}}}^{⟨\mathit{i}⟩}\right)$(5)

In the second stage, the image generator forms a feedback incentive mechanism with the target domain classifier: the image generator can generate a large number of fake target domain data with reliable labels. The participation of these data can improve the performance of the target domain classifier by providing an increasing number and quality of target domain features. The increasing labeling accuracy of the target domain classifier with improving performance enables the TopNHC data selection method to select high-confidence samples in a more accurately labeled data set, and the reliability of autonomous labeling is thus continuously improved. These target domain data with high-accuracy autonomous labels are in turn used as training data for the image generator to improve its generation performance.

The process of the active domain adaptation framework based on TDFG is shown in Algorithm 1.

Algorithm 1. Active domain adaptation framework based on Target Domain Feature Generation (TDFG).

Input: source domain data $X_S={\left\{\left(x_{S_i},y_{S_i}\right)\right\}}_{\mathit{i}=1}^{n_S}$, target domain data $X_T={\left\{x_{T_i}\right\}}_{\mathit{i}=1}^{n_T}$

1: Initialization: Total number of iterations $\mathit{J}$

2: Use source domain data $X_S$ to train and obtain the source domain classifier ${\hat{C}}_{\mathit{S}}(\cdot )$

3: Use ${\hat{C}}_{\mathit{S}}(\cdot )$ to predict $X_T$, a set of target domain data is selected using the TopNHC data selection method according to the prediction result and manually corrected to obtain $X_{{\mathit{T}}_{\mathit{true}}}^0$

4: Use $X_{{\mathit{T}}_{\mathit{true}}}^0$ and $X_S$ to train and obtain the image generator ${\hat{G}}_{\mathit{S}\to \mathit{T}}^{0;}(\cdot )$

5: Use $X_S$ and ${\hat{G}}_{\mathit{S}\to \mathit{T}}^{0;}(\cdot )$ to generate a set of labeled fake target domain data $X_{{\mathit{T}}_{\mathit{fake}}}^0$

6: Use $X_{{\mathit{T}}_{\mathit{fake}}}^0$ and $X_{{\mathit{T}}_{\mathit{true}}}^0$ to train and obtain the target domain classifier ${\hat{C}}_{\mathit{T}}^0(\cdot )$, initialized by the trained source domain classifier ${\hat{C}}_{\mathit{S}}(\cdot )$ before training

7: $\mathit{j}\leftarrow 0$

8: repeat

9: Use $\hat{C}\mathit{Tj}(\cdot )$ to predict $X_T$, then use the TopNHC data selection method to select a set of target domain data and directly use the prediction result as its label to obtain $X_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j}}$

10: Use $X_S$, $X_{{\mathit{T}}_{\mathit{true}}}^0$ and $X_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j}}$ to train the the image generator ${\hat{G}}_{\mathit{S}\to \mathit{T}}^{\mathit{j};}(\cdot )$ and obtain ${\hat{G}}_{\mathit{S}\to \mathit{T}}^{\mathit{j}+1;}(\cdot )$

11: Use $X_S$ and the trained image generator ${\hat{G}}_{\mathit{S}\to \mathit{T}}^{\mathit{j}+1;}(\cdot )$ to generate a set of labeled fake target domain data $X_{{\mathit{T}}_{\mathit{fake}}}^{\mathit{j}}$

12: Use $X_{{\mathit{T}}_{\mathit{fake}}}^{\mathit{j}}$ and $X_{{\mathit{T}}_{\mathit{true}}}^0$ to train the target domain classifier $\hat{C}\mathit{Tj}(\cdot )$ and obtain ${\hat{C}}_{\mathit{T}}^{\mathit{j}+1}(\cdot )$

13: $\mathit{j}\leftarrow \mathit{j}+1$

14: until $\mathit{j}$ == $\mathit{J}-1$

Output: Trained target domain classifier $\hat{C}\mathit{TJ}(\cdot )$

Analysis of Manual Workload

The manual workload cost by TDFG is primarily manifested in the manual correction of incorrect labels. Humans need to review the accurately labeled data, even if no further action is required. However, with the emergence of various data labeling tools, the manual checking of accurately labeled data has become easy and quick, allowing this aspect of the workload to be disregarded. With this method, when deploying a deep learning model, the model parameters of the online system can be updated with less manpower, thereby achieving better performance.

Analysis of Time Complexity

To analyze the time complexity, we provide these annotations. $\mathit{G}$ represents the process related to the image generator, and $\mathit{C}$ the process related to the classifier. The superscript $\mathit{t}$ indicates training, while $\mathit{i}$ indicates inference. The subscript $\mathit{s}$ stands for source, and $\mathit{t}$ for target.

The time complexity for each step are listed as follows. In the first iteration: $\mathit{O}(C_s^t)$ is for training source domain classifier. Then, train the image generator $\mathit{O}(G^t)$. Next, predict with $\mathit{O}(C_s^i)$, as in algo.step.5. Then, generate images $\mathit{O}(G^i)$. After that, train $\mathit{O}(C_t^t)$, as in algo.step.6. This process is repeated in a for-loop $\mathit{J}$ times. The total complexity is $\mathit{J}\ast (\mathit{O}(C_t^i)+\mathit{O}(G^t)+\mathit{O}(G^i)+\mathit{O}(C_t^t))$. Since $\mathit{J}$ is a constant number, the complexity simplifies to $\mathit{O}(C_t^i)+\mathit{O}(G^t)+\mathit{O}(G^i)\text{break}+\mathit{O}(C_t^t)$.

$\mathit{O}(C_t^t)$ is equal to $\mathit{O}(\mathit{w}\ast \mathit{m}\ast \mathit{e})$, where $\mathit{w}$ represents the number of parameters, $\mathit{m}$ the number of training examples, and $\mathit{e}$ the number of training epochs.

$\mathit{O}(G^t)$ is $\mathit{O}(\mathit{w}\ast \mathit{m}\ast \mathit{T})$, where $\mathit{w}$ is the number of parameters, $\mathit{m}$ the number of training examples, and $\mathit{T}$ the number of training iterations.

Since inference time is required only once, and the number of training times depends on the batch number, the time complexity of entire pipeline can be simplified to $\mathit{J}\ast (\mathit{O}(G^i)+\mathit{O}(C_t^t))$.

Top-N Labeled Data Selection Method with High Confidence

This paper posits a hypothesis: if high-confidence samples contain errors, it indicates significant cognitive issues within the model. This means the model confidently misclassifies samples into the wrong categories. The root of this problem lies in the epistemic uncertainty (Kendall and Gal 2017) caused by the scarcity of data in the target domain. By involving human intervention, these confidently misclassified samples can be identified and corrected. In subsequent training rounds, the model, without updating its weights, will still make incorrect predictions on these samples. Consequently, during back-propagation when calculating the loss value, the batch with these errors will result in a larger loss compared to batches without sample errors. This leads to a more effective update of the model’s weights, thereby effectively reducing the model’s cognitive uncertainty and improving its accuracy. Therefore, this paper proposes a new data selection method: the Top-N labeled data selection method with High Confidence (TopNHC), to reduce the amount of manual labeling in the first stage and ensure the reliability of autonomous labels in the second stage. The TopNHC method selects samples predicted by a classifier with the highest confidence. The process is shown in Figure 4, taking the source domain classifier to predict the target domain data as an example.

Figure 4. Top-N labeled data selection method with high confidence (TopNHC). Take the source domain classifier to label the target domain data as an example. First, after the source domain classifier predicts the target domain data, the prediction result of each sample is a confidence sequence with a length equal to the total number of categories. Subsequently, the confidence sequence corresponding to each sample is maximized, and all the samples are reordered from the largest to the smallest by the maximum confidence. Finally, the top N samples with the highest maximum confidence are taken as the selected data.

TopNHC represents the first round of data screening in the TDFG, predominantly led by the machine. As shown in Figure 4, after the source domain classifier predicts each target domain sample, a confidence list of length equal to the total number of categories K is obtained, and the TopNHC data selection method uses this list as the basis for data selection. Firstly, the maximum value of each confidence list is recorded. Secondly, the target domain data are sorted from highest to lowest according to this value. Thirdly, the top N target domain data with the highest confidence are selected to form the selected data set. As shown in Equation (6)(6) $X_T^{\mathrm{T}\mathrm{o}\mathrm{pNHC}}=\mathrm{sort}\left(\max \left(\mathrm{softmaxPred}\left({\hat{C}}_{\mathit{S}}\left(\cdot \right),X_T\right)\right)\right)[:\mathit{N}]$(6) .

(6) $X_T^{\mathrm{T}\mathrm{o}\mathrm{pNHC}}=\mathrm{sort}\left(\max \left(\mathrm{softmaxPred}\left({\hat{C}}_{\mathit{S}}\left(\cdot \right),X_T\right)\right)\right)[:\mathit{N}]$(6)

Advantages of TopNHC

The Top-N labeled data selection method with High Confidence (TopNHC) has the following advantages: 1) The TopNHC data selection method saves manual labeling workload because the number of mislabels in the selected data is small. This is because the human annotator only needs to correct a small number of labels that a classifier predicted inaccurately. 2) The TopHNC data selection method can guarantee the reliability of the autonomous labeling in the second stage. High confidence is verified to have a positive correlation with high recognition accuracy. Since the TopNHC data selection method selects data that are predicted with high confidence, the proportion of being accurately predicted is higher among the selected data. 3) The TopNHC data selection method is possible to select confusing data. The data selected by the TopNHC data selection method are the samples predicted by the classifier with high confidence. If there is still a wrong prediction in such samples, these samples are considered to have a high transfer learning value. For example, if a digit 7 is classified as a digit 1 with high confidence, then correcting that mislabel and providing it to a classifier for learning can improve the classifier’s ability to discriminate between digits 1 and 7.

Image Generator

This section describes the input, output, and functions of the image generator, and this section will introduce its principle and implementation.

The image generator is implemented using multiple CycleGANs, each responsible for generating data for a specific category. Multiple CycleGANs are trained separately, with each CycleGAN operating independently. The structure of the CycleGAN, including the number of layers, the number of neurons, and other parameters, is identical to that proposed in the original paper (Zhu et al. 2017). The generator aims to translate source domain data into outputs that resemble the target domain data, ensuring that the generated data is indistinguishable from real target domain data. By training the source-to-target domain generator and target-to-source domain generator, the image generator can produce labeled fake target domain data with high quality We have provided the architecture of image generator, CycleGANs in Figure 3. Each CycleGAN performs learning and inference independently.

For target domain data generation, adversarial generation methods are generally preferred for implementation. However, most adversarial generation methods (Goodfellow et al. 2020) require pairs of training data, which are undoubtedly costly to acquire. In contrast, CycleGAN (Zhu et al. 2017) is more suitable for domain adaptation than other adversarial generation methods because of its ability to learn mappings from one domain to another without needing paired training data, as shown in Fig. ?. However, CycleGAN learns complete inter-domain mappings and cannot be directly specified to generate a specific category of target domain data. Therefore, to ensure that the generated target domain data are dependably labeled, and the volume of data is balanced across categories, this framework will implement an image generator based on multiple CycleGANs. Under this design, each CycleGAN only acts on one subset of source and target domain data with the same label. Since CycleGAN learns the complete mapping between two domains, the high quality of the generated labeled fake target domain data is still guaranteed even if partially mislabeled target domain samples exist. The process of the image generator designed in this framework is shown in Figure 5.

Figure 5. Image generator. The image generator is implemented using multiple CycleGANs. First, the source domain data and the labeled target domain data are split and reorganized by category to form the training data for each CycleGAN. Each CycleGAN is responsible for one category of data generation; its training data are the source domain data and the labeled target domain data belonging to the same category. In the inference generation period, its input is the source domain data, and its output is the generated labeled fake target domain data.

Take the $\mathit{j}$-th iteration as an example, $X_S^j$ and $X_T^j$ (including $X_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j}}$ or $X_{{\mathit{T}}_{\mathit{true}}}^0$) respectively denote the source domain data and target domain data with labels. $\mathit{K}$ denotes the total number of categories in the dataset, $D_T^{j;k}$, $D_S^{j;k}$, $G_{S\to T}^{j;k}$ and $G_{T\to S}^{j;k}$ respectively denote the target domain discriminator, source domain discriminator, source-to-target domain generator, and target-to-source domain generator for the $\mathit{k}$-th CycleGAN (corresponding to category $\mathit{k}$). The image generator takes the source domain data $X_S^j$ and the labeled target domain data $X_T^j$ as input. After splitting and reorganizing them by category, $\mathit{K}$ sets of data are obtained as the formal input data of the $\mathit{K}$ Cycle-GANs. Taking category $\mathit{k}$ as an example, the training and inference of the $\mathit{k}$-th CycleGAN are shown on the right panel of Figure 5.

For target domain data generation, the training paradigm of CycleGAN is to train a source-to-target domain generator so that the target domain discriminator cannot distinguish the generated labeled fake target domain data from the real target domain data. In contrast, after anti-generation by the target-to-source domain generator, the anti-generated target domain data must be similar enough to the original source domain data. The target-to-source domain generator and the source domain discriminator are trained with the same principle and form a complete training process with the former. The process of training the $\mathit{k}$-th CycleGAN model in the $\mathit{j}$-th iteration is shown in Equation (7)(7) $V_{T\to S}^{j;k}\left(G_{T\to S}^{j;k},D_S^{j;k},X_s^{j;k},X_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j};\mathit{k}}\right)=\sum _{\mathit{i}=1}^{n_j}\left\{\mathit{log}{D}_S^{j;k}\left(x_S^{j;k;⟨i⟩}\right)+\mathit{log}\left[1-D_S^{j;k}\left(G_{T\to S}^{j;k}\left(x_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j};\mathit{k};⟨\mathit{i}⟩}\right)\right)\right]\right\}$(7) to Equation (12)(12) $\underset{G_{S\to T}^{j;k}}{min}\underset{D_T^{j;k}}{max}\underset{G_{T\to S}^{j;k}}{min}\underset{D_S^{j;k}}{max}{V}_{cy\mathrm{lce}-gan}^{j;k}\left(G_{T\to S}^{j;k},G_{S\to T}^{j;k},X_S^{j;k},X_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j};\mathit{k}},D_S^{j;k},D_T^{j;k}\right)$(12) . Where Equation (7)(7) $V_{T\to S}^{j;k}\left(G_{T\to S}^{j;k},D_S^{j;k},X_s^{j;k},X_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j};\mathit{k}}\right)=\sum _{\mathit{i}=1}^{n_j}\left\{\mathit{log}{D}_S^{j;k}\left(x_S^{j;k;⟨i⟩}\right)+\mathit{log}\left[1-D_S^{j;k}\left(G_{T\to S}^{j;k}\left(x_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j};\mathit{k};⟨\mathit{i}⟩}\right)\right)\right]\right\}$(7) and Equation (8)(8) $V_{S\to T}^{j;k}\left(G_{S\to T}^{j;k},D_T^{j;k},X_S^{j;k},X_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j};\mathit{k}}\right)=\sum _{\mathit{i}=1}^{n_j}\left\{\mathit{log}{D}_T^{j;k}\left(x_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j};\mathit{k};⟨\mathit{i}⟩}\right)+\mathit{log}\left[1-D_T^{j;k}\left(G_{S\to T}^{j;k}\left(x_S^{j;k;⟨i⟩}\right)\right)\right]\right\}$(8) describe the value functions of training the target-to-source domain generator, source-to-target domain generator, source domain discriminator, and target domain discriminator. Equation (3.3) constrains the feature similarity between the original data and the result after domain mapping and inverse domain mapping. Equation (11)(11) $V_{cy\mathrm{lce}-gan}^{j;k}\left(G_{T\to S}^{j;k},G_{S\to T}^{j;k},X_S^{j;k},X_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j};\mathit{k}},D_S^{j;k},D_T^{j;k}\right)=V_{T\to S}^{j;k}+V_{S\to T}^{j;k}+\mathit{\lambda }{L}_{si\mathrm{milar}}^{j;k}$(11) and Equation (12)(12) $\underset{G_{S\to T}^{j;k}}{min}\underset{D_T^{j;k}}{max}\underset{G_{T\to S}^{j;k}}{min}\underset{D_S^{j;k}}{max}{V}_{cy\mathrm{lce}-gan}^{j;k}\left(G_{T\to S}^{j;k},G_{S\to T}^{j;k},X_S^{j;k},X_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j};\mathit{k}},D_S^{j;k},D_T^{j;k}\right)$(12) describe the overall value function of the CycleGAN ($\mathit{\lambda }$ is the hyperparameter) and the CycleGAN’s adversarial training process.

(7) $V_{T\to S}^{j;k}\left(G_{T\to S}^{j;k},D_S^{j;k},X_s^{j;k},X_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j};\mathit{k}}\right)=\sum _{\mathit{i}=1}^{n_j}\left\{\mathit{log}{D}_S^{j;k}\left(x_S^{j;k;⟨i⟩}\right)+\mathit{log}\left[1-D_S^{j;k}\left(G_{T\to S}^{j;k}\left(x_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j};\mathit{k};⟨\mathit{i}⟩}\right)\right)\right]\right\}$(7)

(8) $V_{S\to T}^{j;k}\left(G_{S\to T}^{j;k},D_T^{j;k},X_S^{j;k},X_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j};\mathit{k}}\right)=\sum _{\mathit{i}=1}^{n_j}\left\{\mathit{log}{D}_T^{j;k}\left(x_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j};\mathit{k};⟨\mathit{i}⟩}\right)+\mathit{log}\left[1-D_T^{j;k}\left(G_{S\to T}^{j;k}\left(x_S^{j;k;⟨i⟩}\right)\right)\right]\right\}$(8)

(9) $L^{\ast }=L_{si\mathrm{milar}}^{j;k}\left(G_{T\to S}^{j;k},G_{S\to T}^{j;k},X_S^{j;k},X_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j};\mathit{k}}\right)$(9)

(10) $L^{\ast }=\sum _{\mathit{i}=1}^{n_j}\left\{\left|\left|G_{T\to S}^{j;k}\left[G_{S\to T}^{j;k}\left(x_S^{j;k;⟨i⟩}\right)\right]-x_S^{j;k;⟨i⟩}\right|\right|+\left|\left|G_{S\to T}^{j;k}\left[G_{T\to S}^{j;k}\left(x_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j};\mathit{k};⟨\mathit{i}⟩}\right)\right]-x_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j};\mathit{k};⟨\mathit{i}⟩}\right|\right|\right\}$(10)

(11) $V_{cy\mathrm{lce}-gan}^{j;k}\left(G_{T\to S}^{j;k},G_{S\to T}^{j;k},X_S^{j;k},X_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j};\mathit{k}},D_S^{j;k},D_T^{j;k}\right)=V_{T\to S}^{j;k}+V_{S\to T}^{j;k}+\mathit{\lambda }{L}_{si\mathrm{milar}}^{j;k}$(11)

(12) $\underset{G_{S\to T}^{j;k}}{min}\underset{D_T^{j;k}}{max}\underset{G_{T\to S}^{j;k}}{min}\underset{D_S^{j;k}}{max}{V}_{cy\mathrm{lce}-gan}^{j;k}\left(G_{T\to S}^{j;k},G_{S\to T}^{j;k},X_S^{j;k},X_{{\mathit{T}}_{\mathit{model}}}^{\mathit{j};\mathit{k}},D_S^{j;k},D_T^{j;k}\right)$(12)

After each iteration of training, using the source domain data of a specific category as input, each CycleGAN’s source-to-target domain generator will generate a large amount (equal to the input source domain data) of fake target domain data with labels (the corresponding category is directly used as labels). Taking the $\mathit{j}$-th iteration, the $\mathit{k}$-th trained source-to-target domain generator ${\hat{G}}_{\mathit{S}\to \mathit{T}}^{\mathit{j};\mathit{k}}$ as an example, the generation process is shown in Equation (13)(13) $x_{{\mathit{T}}_{\mathit{fake}}}^{\mathit{j};\mathit{k}}={\hat{G}}_{\mathit{S}\to \mathit{T}}^{\mathit{j};\mathit{k}}\left(x_S^{j;k}\right)$(13) .

(13) $x_{{\mathit{T}}_{\mathit{fake}}}^{\mathit{j};\mathit{k}}={\hat{G}}_{\mathit{S}\to \mathit{T}}^{\mathit{j};\mathit{k}}\left(x_S^{j;k}\right)$(13)

Domain Classifier

The structure of the Source domain classifier and the Target domain classifier is the same as VGG11, and the number and type of neural network layers have not been modified. The target domain classifier plays a crucial role in the second stage of the framework, where it autonomously labels the target domain data. The classifier is trained iteratively along with the image generator to improve its performance using the generated fake target domain data. The TopNHC is used to select accurately labeled samples, reducing the need for manual intervention and ensuring reliable autonomous labeling.

Co-Training Mechanism

In each iteration, the target domain classifier makes predictions on the target domain data and selects a portion of labeled target domain data and source domain data for training the image generator. The trained image generator then generates more high-quality labeled fake target domain data, which is used to further train and improve the performance of the target domain classifier. This iterative process continues, with the first stage involving manual correction of mislabeling by the source domain classifier and subsequent stages relying on autonomous labeling by the target domain classifier.

Experiments

To verify the extent of manual labeling workload saving and the reliability of the selected labeled data when using the TopNHC, as well as the migration performance of the active domain adaptation framework based on TDFG when there exists less feature commonality between domains, experiments including labeled data selection and domain adaptation are designed for validation.

Four data domains are used in the experiments of this paper, namely MNIST(m), MNIST-M(mm), SVHN(sv), and SYNTH(sy), as shown in Figure 6.

Figure 6. Four data domains. The task for all four data domains is digit 0–9 classification. Among them, MNIST is the classical handwritten digit dataset with white characters on a black background. MNIST_M is obtained by random coloring based on MNIST. SVHN is the photographed door digits. SYNTH is a dataset consisting of printed digits of different colors placed on disturbed background images.

Among them, the feature differences between MNIST and MNIST-M are minimal (both are handwritten digits); the feature differences between SVHN and SYNTH are also relatively small (both are printed digits); there are partial feature differences between MNIST and SYNTH (differences between handwritten and printed digits), and the feature differences between MNIST and SVHN are significant (differences between handwritten and printed digits with camera interference in SVHN).

For a valid and fair experimental comparison, the experimental framework is set up as follows: The VGG-11 network (Simonyan and Zisserman 2014) is used to implement the source domain classifier and the target domain classifier, and the amount of labeled data selected from the target domain in the first stage is 5000. 6 iterations are performed in the second stage, and 500 target domain data are selected in the first iteration, and each subsequent iteration selects 500 more target domain data than the previous iteration. In the first round, we select an additional 500 data groups, which, when combined with the original 5,000 labeled data points, form a training set of 5,500 groups, as shown in Table 2. Ten independent CycleGANs are used to implement the image generator.

Data Selection Experiments

To measure the manual labeling workload of various data selection methods, the following data selection methods are used in the first stage of this framework for comparison: Random Sample (RS), Least Confident (LC), High Entropy (EN), and the Top-N labeled data selection method with High Confidence (TopNHC). Five sets of comparison experiments were designed; the variables are the sample size $\mathit{Nu}{m}_{\mathit{sample}}$ of the selected target domain data, which are 300, 800, 1500, 3000, and 5000. The comparison parameters are the actual labeled sample size $\mathit{Nu}{m}_{\mathit{label}}$, and the percentage of manual labeling workload reduction of the TopHNC method compared with other methods $\mathit{Nu}{m}_{\mathit{reduce}}$. The results of their comparison experiments are shown in Table 1.

Table 1. Compare the manual tagging effort required in the first phase of this framework with the use of different data selection methods. Concretely, five different data selection volumes from 300 to 5000, four data selection methods include Random Sample (RS), least confident (LC), high entropy (EN), and the Top-N labeled data selection method with high confidence (TopNHC), and two source and target domain pairs are MNIST and SVHN, and MNIST and SYNTH are involved in the comparison. The parameters being compared are the actual manual labeling volume $\mathit{Nu}{m}_{\mathit{label}}$ and the percentage reduction in manual labeling volume of the TopNHC data selection method compared to other data selection methods $\mathit{Nu}{m}_{\mathit{reduce}}$.

		m $\to$ sv		m $\to$ sy
$\mathit{nu}{m}_s$	selection method	$\mathit{nu}{m}_l$	$\mathit{nu}{m}_r$	$\mathit{nu}{m}_l$	$\mathit{nu}{m}_r$
300	RS	243	57.61%	243	94.65%
	LC	271	61.99%	266	95.11%
	EN	240	57.08%	234	94.44%
	TopNHC	103	–	13	–
800	RS	633	38.23%	641	81.90%
	LC	701	44.22%	706	83.56%
	EN	652	40.03%	634	81.70%
	TopNHC	391	–	116	–
1500	RS	1210	28.84%	1179	64.29%
	LC	1318	41.50%	1301	67.64%
	EN	1206	28.61%	1191	64.65%
	TopNHC	861	–	421	–
3000	RS	2410	19.04%	2394	40.23%
	LC	2612	25.38%	2563	44.17%
	EN	2434	19.93%	2403	40.45%
	TopNHC	1949	–	1431	–
5000	RS	4014	12.98%	3946	24.51%
	LC	4322	19.18%	4248	29.87%
	EN	4021	13.13%	3996	25.45%
	TopNHC	3493	–	2979	–

In Table 1, according to the $\mathit{Nu}{m}_{\mathit{label}}$ column, we can see that the TopNHC data selection method can achieve the minimum manual labeling amount regardless of the size of samples and the group of data for which the model is migrated. According to the $\mathit{Nu}{m}_{\mathit{reduce}}$ column, it can be seen that the TopNHC data selection method can save at most half of the manual labeling amount compared with other data selection methods, and at least more than a hundred data can avoid manual labeling. Therefore, it can be proved that the TopNHC data selection method can effectively reduce the manual labeling workload without being affected by the migrated data.

In the second stage of the framework, this paper proposes to replace manual labeling with autonomous labeling, so that manual involvement for data labeling is no longer required in this stage, further saving the manual workload. To verify the reliability of autonomous labeling, this paper compares the TopNHC data selection method with the three data selection methods mentioned above. Using two source and target domain data pairs, the framework is entirely run, and the accuracy of autonomous labeling achieved in each training iteration in the second stage is compared. The comparison results are shown in Table 2.

Table 2. Comparison of the autonomous labeling accuracy can be achieved using different data selection methods in the second stage. Concretely, the four data selection methods include Random Sample (RS), least confident (LC), high entropy (EN), and the Top-N labeled data selection method with high confidence (TopNHC), six iteration rounds from 5500 to 8000 total data selection volume, and two sets of source and target domain pairs including MNIST and SVHN, MNIST and SYNTH are involved in the comparison. The parameters being compared are the actual manual labeling volume $\mathit{Nu}{m}_{\mathit{label}}$ and the accuracy achieved by autonomous labeling $\mathit{AC}{C}_r$.

		m $\to$ sv		m $\to$ sy
method	$\mathit{nu}{m}_s$(epoch)	$\mathit{nu}{m}_l$	$\mathit{AC}{C}_r$	$\mathit{nu}{m}_l$	$\mathit{AC}{C}_r$
RS	5500(1)	1511	0.7253	330	0.94
	6000(2)	1380	0.77	334	0.9443
	6500(3)	1686	0.7406	316	0.9514
	7000(4)	1740	0.7514	321	0.9541
	7500(5)	1925	0.7433	354	0.9528
	8000(6)	2128	0.734	355	0.9556
	-(avg)	-	0.7441	-	0.9497
LC	5500(1)	3475	0.3682	1594	0.7102
	6000(2)	4009	0.3318	1776	0.704
	6500(3)	4676	0.2806	2082	0.6797
	7000(4)	4833	0.3096	2504	0.6423
	7500(5)	5143	0.3143	2399	0.6801
	8000(6)	5619	0.2976	2752	0.656
	-(avg)	-	0.317	-	0.6787
EN	5500(1)	1263	0.7704	338	0.9385
	6000(2)	1126	0.8123	293	0.9512
	6500(3)	1254	0.8071	288	0.9557
	7000(4)	1451	0.7927	364	0.948
	7500(5)	1678	0.7763	381	0.9492
	8000(6)	1901	0.7624	369	0.9539
	-(avg)	-	0.7869	-	0.9494
TopNHC	5500(1)	207	0.9624	53	0.9904
	6000(2)	203	0.9662	59	0.9902
	6500(3)	186	0.9714	72	0.9889
	7000(4)	233	0.9667	62	0.9911
	7500(5)	344	0.9541	79	0.9895
	8000(6)	303	0.9621	106	0.9868
	-(avg)	-	0.9638	-	0.9895

As shown in Table 2, the TopNHC data selection method can achieve the highest autonomous labeling accuracy in the two pairs of source and target domains. In particular, in the migration of MNIST to SVHN, the TopNHC data selection method can reach an average autonomous labeling accuracy of at least 20% points higher than the other three data selection methods. In addition, the TopNHC data selection method can guarantee an autonomous labeling accuracy higher than 0.95, i.e., it can guarantee that the autonomous labels provided by the classifier can have high reliability in the second stage.

Migration Performance Experiments

Seven sets of comparison experiments were designed to verify the migration performance of the active domain adaptation framework based on Target Domain Feature Generation. The variables include four migration models, DSN (Bousmalis et al. 2016), PiexlDA (Bousmalis et al. 2017), VGG-11, and active domain adaptation framework based on Target Domain Feature Generation (TDFG), and three types of training data: source domain data ($D_S$), source domain data mixed with the manually labeled target domain samples in the first stage ($D_{\mathit{ST}}$), and source domain data mixed with the manually labeled target domain data in the first stage and the labeled fake target domain data generated in the first iteration ($D_{\mathit{STF}}$). The target domain classification accuracies of various models and data types on six different pairs of data domains are compared. Table 3 shows the results of each group of model experiments.

Table 3. Comparison of migration performance of different domain adaptation methods. Four domain adaptation methods, including VGG-11, PiexlDA, DSN, and the active domain adaptation framework based on Target domain Feature Generation (TDFG), three forms of training data including source domain data ($D_S$), source domain data mixed with the manually labeled target domain samples in the first stage ($D_{\mathit{ST}}$), and source domain data mixed with the manually labeled target domain data in the first stage and the labeled fake target domain data generated in the first iteration ($D_{\mathit{STF}}$), and six sets of source and target domain pairs including MNIST and SVHN, MNIST and SYNTH, MNIST_M and SVHN, SVHN and MNIST, SVHN and MNIST_M, SYNTH and MNIST_M are involved in the comparison. The parameter to be compared is the migration performance of the models.

model	m $\to$ sv	m $\to$ sy	mm $\to$ sv	sv $\to$ m	sv $\to$ mm	sy $\to$ mm
VGG-11($D_{\mathit{ST}}$)	0.715	0.863	0.584	0.896	0.746	0.719
VGG-11($D_{\mathit{STF}}$)	0.671	0.819	0.618	0.91	0.761	0.712
PiexlDA($D_{\mathit{ST}}$)	0.732	0.847	0.716	0.834	0.745	0.746
PiexlDA($D_{\mathit{STF}}$)	0.769	0.843	0.731	0.864	0.745	0.807
DSN($D_S$)	0.398	0.676	0.465	0.794	0.704	0.815
DSN($D_{\mathit{ST}}$)	0.859	0.952	0.795	0.88	0.772	0.86
DSN($D_{\mathit{STF}}$)	0.83	0.944	0.805	0.924	0.791	0.873
TDFG	0.839	0.954	0.803	0.965	0.89	0.871

Comparing the first three columns of Table 3 individually, the performance of all models migrating from the source domain MNIST to the target domain SVHN with less feature commonality suffers a certain degree of performance degradation compared to migrating to the target domain SYNTH, whose features are less different from those of the source domain data MNIST. DSN($D_S$) is the most severe, with almost 30% degradation. This further illustrates that many domain adaptation methods that only extract shared features between domains may experience significant performance degradation when facing domains with less feature commonality.

According to Table 3, comparing the performance of TDFG and the traditional migration algorithm DSN($D_S$) separately, it can be seen that the migration performance of TDFG is higher than that of DSN for each domain pair, especially for domain pairs with less feature commonality such as MNIST to SVHN and MNIST-M to SVHN, the migration performance of TDFG is more than twice that of DSN($D_S$). It can be proved that the TDFG framework solves the severe degradation of migration performance for domain pairs. According to Table 3, comparing the performance of PiexlDA($D_{\mathit{ST}}$) and PiexlDA($D_{\mathit{STF}}$) separately, it can be seen that the labeled fake target domain data generated by the image generator are helpful for the improvement of the migration performance: in 5 cases out of 6 different data domain pairs, the migration model achieves higher migration performance thanks to the addition of the labeled fake target domain data. This can also be demonstrated by comparing the performance of DSN($D_{\mathit{ST}}$) and DSN($D_{\mathit{STF}}$) alone or VGG-11($D_{\mathit{ST}}$) and VGG-11($D_{\mathit{STF}}$) alone.

In the second stage of this framework, the iterative generation mechanism contributes to the joint performance improvement of the image generator and the target domain classifier. Since TDFG is a classifier implemented by VGG-11, according to Table 3, the performance of VGG-11 ($D_{\mathit{ST}}$), VGG-11 ($D_{\mathit{STF}}$), and TDFG are compared separately, and the accuracy achieved by the target domain classifier in the TDFG framework is greater than that of the VGG-11 network trained without iterative generation. This proves that the iteration generation mechanism incorporated in the second stage of our framework helps improve the performance of the target domain classifier.

Overall comparing the migration performance of various models in Table 3, the TDFG migration framework proposed in this paper can achieve the highest or second highest migration performance on various domain pairs, so it can be proved that, as an innovative domain adaptation framework, TDFG has good migration performance and has a specific practical application value.

To verify that our framework has both high migration performance and low manual labeling amount, we compare the amount of manual labeling when different domain adaptation methods use three data selection methods for active learning to achieve similar performance as TDFG, as shown in Table 4.

Table 4. Comparison of manual labeling amount at a similar performance. Four domain adaptation methods, including VGG-11, PiexlDA, DSN, and the active domain adaptation framework based on target domain feature generation (TDFG), four data selection methods include random sample (RS), least confident (LC), high entropy (EN), and the Top-N labeled data selection method with high confidence (TopNHC), and six sets of source and target domain pairs including MNIST and SVHN, MNIST and SYNTH, MNIST_M and SVHN, SVHN and MNIST, SVHN and MNIST_M, SYNTH and MNIST_M are involved in the comparison. The parameters to be compared include the samples that need to be labeled and the percentage of savings or increase in labeling workload compared to the TopNHC data selection method.

		samples to label (Percentage of savings or increase in labeling workload) migration performance
models	selection method	m$\to$sv	m$\to$sy	mm$\to$sv	sv$\to$m	sv$\to$mm	sy$\to$mm
		1724	3482	2540	220	868	1748
TDFG	TopNHC	0.8386	0.9544	0.8028	0.965	0.8896	0.8712
		5738	11839	3720	707	13297	1368
		$\uparrow$232.8%	$\uparrow$208.2%	$\uparrow$46.5%	$\uparrow$221.4%	$\uparrow$1432%	$\downarrow$21.7%
	RS	0.8358	0.9405	0.8073	0.96	0.8828	0.8788
		8408	15190	4274	1904	12601	2225
		$\uparrow$387.7%	$\uparrow$295.4%	$\uparrow$68.3%	$\uparrow$765.5%	$\uparrow$1352%	$\uparrow$27.3%
	EN	0.8299	0.9395	0.8089	0.9631	0.8869	0.8707
		7776	16806	4018	1959	13275	2313
		$\uparrow$351.0%	$\uparrow$337.4%	$\uparrow$58.2%	$\uparrow$790.5%	$\uparrow$1429%	$\uparrow$32.3%
DSN	LC	0.833	0.938	0.8093	0.96	0.8743	0.8691
		8314	12677	6002	1279	3284	3165
		$\uparrow$382.3%	$\uparrow$230.0%	$\uparrow$136.30%	$\uparrow$481.4%	$\uparrow$278.3%	$\uparrow$81.1%
	RS	0.8314	0.9405	0.8066	0.9654	0.8881	0.8695
		9838	15806	7669	4940	5521	5709
		$\uparrow$470.7%	$\uparrow$311.4%	$\uparrow$201.9%	$\uparrow$2145.5%	$\uparrow$536.1%	$\uparrow$226.6%
	EN	0.8317	0.9395	0.8092	0.9613	0.8897	0.8736
		10288	19971	7337	4420	6719	6782
		$\uparrow$496.8%	$\uparrow$491.8%	$\uparrow$188.9%	$\uparrow$1909.1%	$\uparrow$674.1%	$\uparrow$288.0%
PixelDA	LC	0.8316	0.941	0.8052	0.9516	0.8875	0.8763
		6702	11052	4487	1642	8135	1852
		$\uparrow$288.8%	$\uparrow$187.7%	$\uparrow$76.7%	$\uparrow$646.4%	$\uparrow$837.2%	$\uparrow$6.0%
	RS	0.8368	0.9517	0.802	0.9671	0.8888	0.8733
		8570	17806	5491	2200	8501	2485
		$\uparrow$397.1%	$\uparrow$363.5%	$\uparrow$116.2%	$\uparrow$900.0%	$\uparrow$879.4%	$\uparrow$42.2%
	EN	0.8333	0.9502	0.8094	0.9667	0.89	0.8747
		8622	15806	5122	2170	7688	2483
		$\uparrow$400.1%	$\uparrow$311.4%	$\uparrow$101.7%	$\uparrow$886.4%	$\uparrow$785.7%	$\uparrow$42.1%
VGG11	LC	0.8352	0.9487	0.8071	0.9617	0.8874	0.8764

According to Table 4, domain adaptation methods that use other data selection methods require more manual labeling to achieve the same performance. When using other methods to select the target domain data to be labeled, a maximum of 20 times more manual labeling is required to achieve the same level of migration performance as the TDFG framework. The only exception is the use of the random sampling method to select the target domain data to be labeled to improve the migration performance of the DSN model from SYNTH to MNIST_M. However, it only requires 21.7% less manual labeling than the TopNHC method, and overall, it is the TopNHC method that has the more significant potential to reduce the manual labeling workload. Therefore, it can be proved that the TopNHC data selection method in the TDFG framework can effectively reduce the manual labeling cost.

As a data selection method that efficiently selects the most accurately labeled data, the Top-N labeled data selection method with High Confidence (TopNHC) not only improves the migration performance of the model but also promotes a significant reduction of manual labeling workload on these datasets. In addition, the results of the experimental data indicate that the human-in-one-loop approach is effective. It also complements the active domain adaptation framework based on Target Domain Feature Generation (TDFG) so that the human could choose to participate in one iteration of manual data labeling. Moreover, the TopNHC data selection method can be considered simple and effective with practical application value.

In the experimental comparisons, we can see that TDFG can largely overcome the issue of label mismatch and can also to some extent narrow the performance gap between models under fully labeled target domain data. In this framework, the labeled fake target domain data generated by the image generator and the iterative generation training mechanism are beneficial for improving migration performance. Therefore, our framework is proven to have good migration performance and operational implementation value.

Conclusion and Future Work

This paper delves into the practicalities of domain adaptation, introducing an innovative human-in-the-loop domain adaptation framework termed Active Domain Adaptation based on Target Domain Feature Generation (TDFG). By incorporating active learning, it introduces the Top-N High Confidence (TopNHC) labeled data selection strategy, significantly reducing the manual labeling burden. This strategy not only labels a portion of the target domain data but also enhances the pool of learnable target domain features. An iterative training process is employed to boost migration performance.

In contrast to prior human-in-the-loop techniques, our framework significantly slashes manual labeling efforts, necessitating oracle involvement in just a single iteration of manual data labeling. Against conventional DA methods, including DSN and PixelDA, TDFG demonstrates marked performance enhancements across various domain pairs.

In real-world application scenarios, such as meter reading, models are typically expected to deliver exceptional performance with minimal manual effort. The engineers who solve problems are often dealing with similar issues for the first time. TDFG is a one-loop method, which means TDFG is valuable for new player who lack data across multiple scenarios. From the manual effort aspect, TDFG only requires manual involvement for mislabeling correction in the second stage. Besides, due to the improved performance of the target domain classifier and the property of TopNHC to select samples with high confidence, the manual workload can still be relatively low.

From the aspect of HITL and active learning, we think that the data selection method should generally be designed according to the application scenario. The innovation of TopNHC, and the key to its effectiveness, lies in its strategy to select data that the model predicts with a high degree of confidence, as opposed to choosing random or marginally challenging samples. The TopNHC method uses high-confidence samples for labeling because subsequent training can effectively correct the model’s erroneous perceptions. Moreover, the image generation part of the TDFG framework complements the above data to some extent and therefore improves the performance of the target domain classifier. Therefore, it can be considered that the TopNHC data selection method is highly bundled with the TDFG framework. When using the TopNHC data selection method in other human-in-the-loop machine learning scenarios, it is prudent to design a framework that can take advantage of the TopNHC data selection method.

There exist some limitations of the proposed TDFG framework. The application of CycleGAN in categorization tasks exhibits notable limitations, primarily due to its operational design that necessitates running distinct CycleGANs for different categories. This approach inherently leads to a linear increase in the manual effort required for label annotation as the number of categories escalates. Such a methodology is especially burdensome in tasks involving a vast array of categories, where the manual effort required for label assignment can become significantly extensive. Furthermore, the necessity for each category to undergo separate CycleGAN training introduces additional complexities. As the quantity of task categories surges, the training costs associated with CycleGAN networks escalate correspondingly. This increase in training requirements not only burdens computational resources but also results in a sluggish execution generation across multiple CycleGANs, presenting a critical bottleneck in the efficiency of the overall process.

The desirable future work of the TDFG framework can be listed as follows. Firstly, on the premise that the human-in-one-loop domain adaptation can be achieved using the TopNHC data selection method (the second stage of the TDFG framework does not require human involvement for data labeling), other data selection methods can be used in the first stage of the TDFG framework, which will slightly increase the workload of human labeling but is likely to reduce the number of iteration rounds in the second stage and accelerate the performance improvement of the target domain classifier due to their higher representativeness and diversity. Secondly, the components in the TDFG framework are low-coupled. Thus, many lightweight and efficient style migration methods can be used to replace the image generator implemented with CycleGAN, which has the potential to bring superior migration performance to the framework. Thirdly, a method to automatically measure precisely how many iterations should be performed in the second stage deserves to be investigated, addressing the possible adverse effects of manually specifying the number of iterations.

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Supplementary Material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/08839514.2024.2349410

Additional information

Funding

This work is supported by National Natural Science Foundation of China [U22A20106] Foshan Science and technology innovation special foundation [BK22BF001] and Foshan Higher Education Advanced Talents Foundation [BKBS202203].The data that support the findings of this study are available in MNIST at DOI:10.1109/MSP.2012.2211477. These data were derived from the following resources available in the public domain: https://yann.lecun.com/exdb/mnist/.