Advances in the field of developing biomarkers for re-irradiation: a how-to guide to small, powerful data sets and artificial intelligence

ABSTRACT

Introduction

Patient selection remains challenging as the clinical use of re-irradiation (re-RT) increases. Re-RT data are limited to retrospective studies and small prospective single-institution reports, resulting in small, heterogenous data sets. Validated prognostic and predictive biomarkers are derived from large-volume studies with long-term follow-up. This review aims to examine existing re-RT publications and available data sets and discuss strategies using artificial intelligence (AI) to approach small data sets to optimize the use of re-RT data.

Methods

Re-RT publications were identified where associated public data were present. The existing literature on small data sets to identify biomarkers was also explored.

Results

Publications with associated public data were identified, with glioma and nasopharyngeal cancers emerging as the most common tumor sites where the use of re-RT was the primary management approach. Existing and emerging AI strategies have been used to approach small data sets including data generation, augmentation, discovery, and transfer learning.

Conclusions

Further data is needed to generate adaptive frameworks, improve the collection of specimens for molecular analysis, and improve the interpretability of results in re-RT data.

KEYWORDS:

• Computational analysis
• machine learning
• AI
• reirradiation
• public data
• clinical data
• biomarkers
• data management

1. Introduction

Publications on re-irradiation (re-RT) have been increasing in oncology literature; however, data sets for use in analysis and validation and biomarkers that can guide the decision-making, administration, and prognosticate or predict response to re-RT have been slow to identify and none are currently used in the clinic [1–3]. Reasons for this include data heterogeneity and inherent biases concerning re-RT. There is also inclusion bias of patients with superior performance status, lower disease burden, and, thus, potentially superior biological tumor behavior. Inclusion bias may also be characterized by the ability to travel or seek second opinions and undergo management at centers with inherent re-RT expertise and trials. As a result, data on re-RT that emerges from these clinical scenarios is limited by significant bias and is defined as ‘small’ by data standards. Data in re-RT suffers from a lack of robust acquisition and curation outside of clinical trials, with small data defining this clinical space and limited computational analysis due to data volume, format type, and data siloes [4]. However, the definition of small data carries specific definitions depending on the context, i.e. it can be defined in terms of storage size or the number of instances, as will be discussed here for using small data in artificial intelligence (AI) approaches. Small data sets are necessary to study rare histologies or malignancies that disproportionally affect specific demographics or gender. In addition to being small in volume, small data may also refer to data that is narrow in variety and has faster data velocity than big data [5]. In the era of personalized medicine, small data complements big data in advancing healthcare as it examines individual data to identify causation and develop predictive models [6]. Notably, missing data or values are likely to affect small data sets more significantly than big data sets. A 2021 study examining data originating from registries [7] identified the percentage of missing data for variables of interest as 71% in lung cancer patients and 55% in breast cancer patients, with a discrepancy in survival between patients with missing data and those with complete data. Evolving treatment paradigms with heterogeneity in previous management and the overlapping administration of novel agents in conjunction with re-RT limit the number of instances with homogenous management. It is unlikely that the data landscape in re-RT will undergo significant alteration to improve data capture or homogeneity. The ability to harness existing imperfect data and mitigate shortcomings with computational analysis may help advance re-RT data toward identifying actionable biomarkers. In this review, we will examine existing re-RT publications and available data sets, and discuss AI strategies to approach small data sets that can be employed to maximize the use of re-RT data.

2. Publicly available data in re-RT

2.1. Trends in re-RT publications

Re-RT has grown in importance as some malignancies increasingly benefit from more prolonged survival, and the ability to administer re-RT safely is more feasible given technological improvements. Publications on re-RT have been steadily increasing, significantly growing in quantity between 2000 and the present (Figure 1). However, there is limited to no published literature on publicly available datasets in studies utilizing re-RT. According to clinicaltrials.gov as of June 2023, 29 clinical trials assessing re-RT in various disease sites and applications are recruiting. Approximately one-third (34.4%) are of Phase II design, with no Phase III studies currently recruiting. Of the 29 trials, they are predominately localized (78%) to the United States and Europe, with 13.7% recruiting in East Asia.

Figure 1. Publications on re-RT based on the Web of Science search for the terms “re-irradiation” and “reirradiation” [1].

Some tumor sites, such as CNS and head and neck, benefit from the bulk of the data in this space. Other sites, such as bladder, have relatively little data, and in many malignancies, much of the data is limited to reviews. In the last 5 years, data linkages are emerging in tumor sites and radiation techniques (Figure 2).

Figure 2. Sankey Chart evaluating the publication landscape in re-RT [1]. Head and neck and glioma feature most prominently in re-RT publications, while other cancer sites benefit from fewer articles and more reviews addressing re-RT. Linkages between technical approaches and their implementation in various oncologic sites are increasing but are concentrated in some oncologic sites and are relatively scarce in others, such as bladder cancer.

2.2. Available re-RT data with public data

To conduct a more in-depth evaluation of the landscape of existing re-RT data, we conducted a literature search using the National Institute of Health (NIH) institutionally available repositories: Web of Science, and Google Dataset (Figure 3). Search terms included: ‘re-irradiation,’ ‘reirradiation,’ and ‘reirradi*.’ Inclusion criteria was publications with associated data, involving human subjects and available in the English language, published until June 2023.

Figure 3. Diagram of the literature review process.

As of June 2023, 17 published articles on re-RT with associated data were identified using these repositories (Table 1). Two of the four prospective studies are of Phase II design [2,3]. The remainder are retrospective studies. The most common disease sites include CNS (6) and head and neck (5) consistent with re-RT data trends. The associated data were usually found within the published manuscript or supplemental material. Two datasets were conditionally available only upon request. Raw data were not readily available in many situations. As expected, the data were highly heterogenous, with a variety of re-RT doses and techniques. One study utilized brachytherapy [4], and four utilized proton or carbon ion therapy [5–7,13]. The use of these techniques in the re-RT setting is important but explored mostly in head and neck cancer and glioma, which are the sites with the most re-RT data overall (Figure 2). The included studies ranged from 22 to 1399 patients, the latter being a systematic review [11]. Most of the studies have endpoints that include local control (LC) and survival, with multivariate analyses to identify predictors, many of which include re-RT tumor volume, re-RT dose, patient age, and Karnofsky Performance Score (KPS). We note that most tumor sites carry little representation in re-RT, including common cancers such as prostate, breast, or brain metastases, as >50% of re-RT data is in head and neck and glioma (Figure 2). Overall, the studies identified a variety of variables associated with LC, progression-free survival (PFS), and overall survival (OS). In the studies on head and neck, in particular nasopharyngeal cancer, predictors of LC included tumor volume and re-RT dose. A study examining carbon ions showed that ≥81.9 GyE was associated with improved LC and age <60 years old had superior OS [13]. Huang et al. examined a comorbidity-based nomogram to predict survival in patients with recurrent nasopharyngeal carcinoma to aid in re-RT selection [20]. The nomogram included eight prognostic factors: comorbidity, recurrent tumor stage, patient age, prior radiation-induced toxicity, tumor volume, disease-free interval, and KPS. The authors suggest comprehensive nomograms may aid treatment selection between ideal candidates for re-RT vs. those better suited for palliative systemic therapy. This approach balances presumed survival outcomes with treatment tolerability. In the studies on re-RT in glioma patients, survival predictors included patient factors such as age and KPS, wherein younger age and higher KPS corresponded to increased survival. Tumor variables impacted survival, such as tumor volume, histology, grade, and IDH mutation status. Treatment parameters such as re-RT dose and longer time interval between first radiation to re-RT (>12–24 months) also impacted survival [3,8]. BED₁₀ >43 Gy correlated with increased survival following re-RT in glioma patients [8]. Additionally, concomitant receipt of chemotherapy or bevacizumab suggested increased survival [11].

Table 1. Existing re-irradiation datasets based on literature search. Glioma and nasopharyngeal cancer are the most common tumor sites represented.

Author/Date	Title	Study Design/ Number of patients	Disease	Re-irradiation dose	Data available	Predictors of control and survival
Navarria et al. [8]	Re-irradiation for recurrent glioma: outcome evaluation, toxicity, and prognostic factors assessment. A multicenter study of the Radiation Oncology Italian Association (AIRO)	R 300	Glioma	Median 16 Gy (single fraction SRS) Median 25 Gy in 5 fractions (hypofractionated SRT) Median 45 Gy in 15 fractions (multifraction)	Restricted access to data	OS: histology, interval time, KPS, age, and total dose of re-RT
Navarria et al. [3]	Re-irradiation for recurrent high-grade glioma (HGG) patients: Results of a single arm prospective phase 2 study	P 90	Glioma	Single fraction SRS 25 Gy in 1 fraction HSRT 37.5 Gy/7.5 Gy/5fractions or 49.5 Gy/3.3 Gy/15 fractions	Restricted access to data	OS/PFS: Time interval between first RT and re-RT, tumor grade, and IDH status
Jiang et al. [2]	Combination of Immunotherapy and Radiotherapy for Recurrent Malignant Gliomas: Results From a Prospective Study	P 30	Glioma	6 Gy in 3 fractions	List of antibodies and corresponding metal isotope labels used in the study	Treatment response: Age, LR, IDH1, CDC27, CD15, CD68 PFS: treatment response, age, recurrence pattern OS: treatment response, KPS, recurrence pattern
Kessel et al. [9]	Validation of an established prognostic score after re-irradiation of recurrent glioma	R 199	Glioma	Range 36–46 Gy	Survival curves based on KPS, histology, age, and interval time to reirradiation	OS: KPS and tumor grade
Kessel et al. [10]	Modification and optimization of an established prognostic score after re-irradiation of recurrent glioma	R 209	Glioma	Median 30 Gy (range 14–60 Gy)	Scoring scheme, survival, Univariate and multivariate analyses of the prognostic factors	OS: KPS and tumor grade
Kulinich et al. [11]	Radiotherapy versus combination radiotherapy-bevacizumab for the treatment of recurrent high-grade glioma: a systematic review	S 1399	Glioma	Range 30–107 EQD2, various techniques	Not able to be accessed	OS: Administration of bevacizumab
Huang et al. [12]	Development of a Comorbidity-Based Nomogram to Predict Survival After Salvage Reirradiation of Locally Recurrent Nasopharyngeal Carcinoma in the Intensity-Modulated Radiotherapy Era	R 469	Nasopharyngeal Cancer	50–70 Gy in 1.8–2.5 Gy/Fx daily using IMRT	5-year OS rate in locally recurrent nasopharyngeal carcinoma patients with rT3–4 based on treatment regimens	OS: Stage, age, toxicity, tumor volume, disease-free interval, KPS, comorbidities
Eekers et al. [5]	The benefit of particle therapy in re-irradiation of head and neck patients. Results of a multicenter in silico ROCOCO trial	R 25	Head and Neck	70 Gy using IMPT or IMIT	Data available from cancerdata.org EPTN International Neurological Contouring Atlas 2021 to consistently contour 10 OARs amygdala, caudate nucleus, cerebellum, corpus callosum, fornix, macula, optic tract, orbitofrontal cortex, periventricular space, pineal gland, and thalamus	N/A
Hu et al. [13]	Clinical outcomes of carbon-ion radiotherapy for patients with locoregionally recurrent nasopharyngeal carcinoma	R 206	Nasopharyngeal Cancer	Median 63 GyE (carbon ion)	Data available from biostudies.com	OS: Age LC: Tumor volume, BED
Liu et al. [14]	With or without reirradiation in advanced local recurrent nasopharyngeal carcinoma: a case-control study	R 88	Nasopharyngeal Cancer	58–70 Gy	Coded data with age, stage, EBV, re-RT dose, treatment volume, control/survival	OS: Significant toxic effect
Held et al. [7]	Ways to unravel the clinical potential of carbon ions for head and neck cancer reirradiation: dosimetric comparison and local failure pattern analysis as part of the prospective randomized CARE trial	P 16	Head and Neck Cancer	60 Gy RBE	Treatment parameters, organs at risk dose volume comparison, treatment plans with isodose distributions,	N/A
D’Agostino et al. [15]	Reirradiation of Locally Recurrent Prostate Cancer with Volumetric Modulated Arc Therapy	R 23	Prostate Cancer	Median 25 Gy in 5 fractions	Restricted access to data	N/A
Talliez et al. [16]	Studies of Intra-Fraction Prostate Motion During Stereotactic Irradiation in First Irradiation and Re-Irradiation	R 58	Prostate Cancer	36 Gy in 6 fractions (SBRT)	Results of the mixed linear regression of the deviation	N/A
Ren et al. [4]	Image-guided interstitial brachytherapy for recurrent cervical cancer after radiotherapy: A single institution experience	R 23	Cervical Cancer	Median EQD2 64 Gy	Patient characteristics and treatment outcomes Includes prior dose	OS: Tumor volume, Treatment free interval, Number of organs tumor invade, LC
Chung et al. [17]	Treatment Outcomes of Re-irradiation in Locoregionally Recurrent Rectal Cancer and Clinical Significance of Proper Patient Selection	R 41	Rectal Cancer	Median 50 Gy	Patient and tumor characteristics	OS: Tumor size and re-RT dose
Shirai et al. [18]	Prospective Study of Isolated Recurrent Tumor Re-irradiation With Carbon-Ion Beams	P 22	Salivary, Lung, Sarcoma, Hepatic cell, Rectal, Sigmoid and Anal Cancer	52.8–73.6 Gy (RBE)	Patient and tumor characteristics, radiation and re-RT treatment details, adverse events	OS and LC: Time interval to re-RT
Sutera et al. [19]	Stereotactic Body Radiation Therapy for Locally Progressive and Recurrent Pancreatic Cancer after Prior Radiation	R 38	Pancreatic Cancer	Median 24.5 Gy in 1–3 fractions (SBRT)	Univariate and multivariate tables for regional control, distant metastases. Univariate logistic regression for grade 3 and higher toxicity	OS: Receipt of surgery

Abbreviations: R = Retrospective, p = Prospective, S = Systematic review. SRS = stereotactic radiosurgery, HSRT = hypofractionated stereotactic radiation therapy, re-RT = reirradiation, RBE = relative biological effectiveness, GyE = Gray equivalent, IMRT = intensity modulated radiation therapy, IMPT= intensity modulated proton therapy, IMIT =intensity modulated ion therapy, KPS = Karnofsky Performance Score.

As stated previously, the most common disease sites where re-RT was examined in the repositories identified were glioma and nasopharyngeal carcinoma (NPC) (Figure 2, Table 1). These cancers have the potential to be aggressive and highly invasive depending on molecular features, grade, and stage. Both NPC and high-grade gliomas are believed to have intratumoral heterogeneity, potentially impacting prognosis, and therapy response [21–23]. With increasing molecular features guiding further subclassifications of glioma and NPC, there is a growing need and interest in identifying appropriate and tailored therapy [24,25]. Given that re-RT has an established role in select cases of recurrent glioma and NPC, molecular classifications may need to be considered in predicting response to re-RT [26,27]. Current published re-RT literature that looks at re-RT in gliomas is limited to single-institution retrospective or prospective phase II trials. To identify biomarkers predicting response to re-RT, larger datasets powered for such endpoints that are publicly available are needed for analysis.

3. Computational learning to harness small data

3.1. Strategies employed to approach small data sets

Given the limitations of small data sets and the lack of availability of larger detailed datasets for re-RT patients, computational methods must be harnessed to enhance the use of existing data (Figure 2) to arrive at clinically actionable biomarkers that can be validated as data sets emerge. Clinically actionable biomarkers of early detection, treatment selection, and prognosis currently in clinical use are based on larger data sets, including HER2 in breast cancer and PSA in prostate cancer [28–30]. Evidence-based clinical practice guidelines in oncology are derived from multiple trials and, when feasible, metanalyses with data acquired over several decades [31]. Data originating over the course of longer time periods may thus give the illusion of large data sets, though they may harbor heterogeneity and missing data. The reality is that there are limited published clinical trials investigating re-RT, and much re-RT is performed in clinical settings without data being robustly acquired or curated for later analysis. Furthermore, molecular data that could generate biomarkers is unavailable in most data sets involving re-RT. However, given examples in systemic management that include the addition of immunotherapy or radiopharmaceutical agents to the standard of care management where small data sets have been practice changing [32], there is reason to remain hopeful that small data sets in re-RT can also be practice altering if they are robust and generate validated markers. There is a realistic possibility that promising biomarkers can be identified via small data sets. For example, CA 19–9, a potential serum maker of early breast, and pancreatic cancer detection, was first established in a relatively small sample size (n = 37) [33,34]. Given this clinical need, approaches to small data sets (Table 2) have increased in prevalence, particularly in tumors such as bladder cancer, where the administration of RT with curative intent is less common and re-RT extremely rare. Markers of radiation response such as PD-L1, HiF-1a, and TP53 May 2001potentially be used in re-RT but data are limited, and inclusion of these in clinical trials would require further validation which is challenging [44].

Table 2. Strategies employed to approach small data sets.

Author/Date	Title	Technique	Data Source	Number of Data Samples	Data Type	Clinical setting	Method
Tsai et al. [35]	Considering Relationship of Proteins for Radiotherapy Prognosis of Bladder Cancer Cells in Small Data Set	Data generation	Real Training Set: Chao et al.[36] Virtual training set: random sample	Real training set: 36 Virtual training set: 100	Protein data	Bladder cancer	This study uses the Modified Extreme Value Theory (MEVT) method to estimate the population domain of the data set. It then uses randomly generated virtual samples to compensate for information gaps.
Lin et al. [36]	A new approach to generating virtual samples to enhance classification accuracy with small data—a case of bladder cancer	Data generation	Real Training Set: Tsai et al.[35]	Real Training set: 10 Virtual data set: 10	Gene protein data	Bladder cancer	This study uses the distance-based mega-trend-diffusion (DB-MTD) method to generate virtual samples. The DB-MTD method minimizes the influence of extreme values in small data sets when generating virtual samples.
Krause et al. [37]	Deep learning detects genetic alterations in cancer histology generated by adversarial networks	Data generation	The Cancer Genome Atlas	Training Set: 32	Synthetic histology images	Colorectal cancer	This study developed a conditional generative adversarial network (CGAN) to generate synthetic MSI and MSS colorectal cancer histology images. The CGAN was first trained for 50,000 iterations to generate synthetic images of microsatellite stable and microsatellite unstable colorectal cancer.
Ubaldi et al. [38]	Strategies to develop radiomics and machine learning models for lung cancer stage and histology prediction using small data samples	Data augmentation	Analysis set: A.R.N.A.S. Civico University Hospital of Palermo Training set: The Cancer Imaging Archive (TCIA)	Analysis Set: 47 Training Set: 130	CT images	Non-small Cell Lung cancer	This study uses radiomics to extract quantitative features from CT images that can be used to identify predictive signatures of tumor heterogeneity. These features were then used to develop machine learning models that predict the histology and stage of tumors.
Wodzinski et al. [39]	Training Deep Neural Networks for Small and Highly Heterogeneous MRI Datasets for Cancer Grading	Data discovery	Used 26 different MRI scanners and 125 different acquisition protocols	Training set: 174	Magnetic resonance images	Meningioma	This study trained deep neural networks by maximizing available data in the preprocessing, processing, and training steps. Preprocessing consisted of offline resampling, tumor cropping, and data augmentation. Data augmentation was done by affine transformations, random cropping and flipping, and random changing of hue, saturation, and contrast.
Zhu et al. [40]	Improving segmentation and classification of renal tumors in small sample 3D CT images using transfer learning with convolutional neural networks	Data discovery	KiTS19 Challenge	Analysis Set: 20 patients Training Set: 210 images	CT images	Renal carcinoma	This study developed a pipeline using transfer learning from publicly available data. The data was optimized by resampling all samples and augmenting the data through techniques such as scaling, rotation, brightness, contrast, gamma transformation, and introduction of Gaussian noise. In the post-processing, a connected component analysis was used to remove unnecessary components of the images.
Li et al. [41]	A fuzzy-based data transformation for feature extraction to increase classification performance with small medical data sets	Data discovery	UCI Repository	Training Set: 10, 20, 30, 50, 100, 200, 300 to 700	Medical data	Diabetes, breast cancer, Parkinson’s disease, echocardiogram, liver disorder, and bladder cancer	This study uses a fuzzy-based transformation method to extend original data into a higher dimensional feature space using attribute membership grade and the global data structure. Then, the optimal subset of features is extracted from the transformed data set and used as the input data for a support vector machine. This technique increases the information of the data set by adding additional classification information to the data set.
Rahul et al. [42]	Deep Feature Transfer Learning in Combination with Traditional Features Predicts Survival Among Patients with Lung Adenocarcinoma	Transfer learning	Training Set: ImageNet	Training Set: > 1,000,000	Analysis Set: chest CT scans Training Set: natural images	Analysis Set: lung cancer	This study used an existing convolutional neural network (CNN) to extract deep features of lung tumors from chest CT scans. The CNNs were pre-trained on ImageNet images of objects that are not necessarily medical.
Usman et al. [43]	Analyzing Transfer Learning of Vision Transformers for Interpreting Chest Radiography	Transfer learning	Training Set: ImageNet	Training Set: > 1,000,000	Analysis Set: chest x-rays Training Set: natural images	Analysis Set: pneumonia	This study performed transfer learning from ImageNet on a pre-trained Vision Transformer (Vit) and convolutional neural network (CNN) to classify chest radiographs as normal pneumonia cases. The ViT is first trained on large sets of natural images, then finetuned on medical image datasets, which are small. They found that the ViT produced better results than the CNN

Clinical learning and decision-making in medicine is a byproduct of pattern recognition with added evidence, both existing and emerging, and can be based on small data sets. The dataset size deemed small in the context of AI is subject to the question posed. The number of patients (‘instances’), and observations available for each instance (‘features’), has been described as subject to the rule of 10 [45], wherein a minimum of 10 events per predictor variable (EPV) should be used to optimize model convergence. Thus, when AI methods that aim to foster improvement in oncologic outcomes are considered, the need to examine large data sets or a greater number of instances to achieve stable results becomes a barrier to solution-driven discussion. However, small data sets in oncology are common, with major conclusions driven by data originating from data sets comprising less than 1000 instances. The typical number of patients in a clinical trial may be 10 to 30 patients (Phase I), 20–50 patients (Phase II), and 100 to 1000 patients (Phase III) [46]. Small data sets, defined as less than 10 observations per predictor variable [47], are thus common in oncology and the norm in re-RT (Table 1). Strategies employed to approach small data sets (Table 2) are important to consider and evaluate in re-RT datasets as they can be used as templates to create a dynamic framework for harnessing small data types in oncology to improve patient outcomes using AI.

3.2. Strategies to handle limited data scenarios for machine learning problems

There are various approaches to training machine learning models with small data sets. Some important strategies to tackle limited data problems for machine learning tasks are illustrated in Figure 4. These strategies are categorized into four groups that can be applied separately or as hybrids of several methods. Broadly, the strategies consist of data acquisition, cross-validation, feature selection, and learning methods [48–52]. Data acquisition aims to gather or generate data that can be used to train computational learning models by various means or combinations depending on the data type and clinical context [51]. Data acquisition includes discovery, augmentation, and generation [51]. Data discovery is concerned with searching for new related datasets to address a specific problem; this can include the curation of data sets by mining electronic health records and various data streams that may be siloed, such as radiation dosimetry data and quality of life or toxicity data. Data augmentation and generation strategies serve to maximize or to increase the amount of training data by generating virtual samples or augmenting data that already exists. Data generation was used, for example, by Tsai et al., who overcame insufficient protein data by analyzing the available dataset with modified extreme value theory (MEVT) and using the population domain to generate virtual samples [35]. The original and virtual samples were then used to train an artificial neural network. Another method used to generate virtual samples is distance-based mega-trend-diffusion (DB-MTD), which generates virtual samples while taking extreme values into consideration. In small data sets, the interpretation of extreme values is significant; thus, Lin et al. proposed the DB-MTD method to minimize the influence of extreme values while extending the range of possible virtual values [53]. The data generation process is generally employed when no available dataset is generated by synthetic data [51]. Data augmentation utilizes small sets by exploiting specific features of available data sets to maximize the amount of training data. For example, quantitative features can be extracted from medical images through radiomics and be used for tumor detection and classification. Ubaldi et al. used this technique and extracted radiomic features of computed tomography (CT) scans of non-small cell lung cancer patients to create a larger data set. These features were then used to train a machine-learning model to predict tumor histology and stage [38].

Figure 4. Strategies to handle limited data scenarios for machine learning problems.

Another approach, cross-validation (Figure 4), is a commonly used technique in machine learning and model evaluation for limited data scenarios [49]. It provides a robust and unbiased estimate of a model’s performance by partitioning the available data into multiple subsets to train and test machine learning models. Cross-validation helps mitigate issues such as overfitting and data bias [54,55]. K-fold, leave-one-out, or nested cross-validation can be used for limited dataset size-related machine learning problems [52]. Feature selection is crucial in machine learning, pattern recognition, and data analysis to identify the most relevant features or variables from a given dataset. The goal is to eliminate irrelevant or redundant features, which can improve model performance, reduce overfitting, and enhance interpretability. Several feature selection approaches, including filter, wrapper, and embedded methods, have been developed and applied to accomplish this task [48,56,57] (Figure 4). Filter feature selection methods aim to evaluate the relevance of features independently of any specific machine learning algorithm [58]. These methods assess the characteristics of individual features and their relationship to the target variable. Mutual information, chi-square, and minimum redundancy-maximum relevance (mRMR) are examples of filter feature selection methods. Wrapper feature selection methods aim to find the optimal feature subset that maximizes the performance (e.g. accuracy rate) of the specific model being used [58,59]. Recursive feature elimination (RFE) and genetic algorithms (GA) are examples of this method. Embedded feature selection methods integrate the feature selection process into the model training. These methods, such as Least Absolute Shrinkage and Selection Operator (LASSO) [60], simultaneously optimize both the feature selection and model parameters, resulting in a more efficient and effective feature subset.

Another strategy for tackling limited data problems is to use various learning strategies such as ensemble learning, transfer learning, semi-supervised learning, few-shot learning, and federated learning (Figure 1). Ensemble learning methods are a type of machine learning algorithms that improve the prediction performance by combining multiple model predictions [61]. They provide robustness, reducing bias, and better results than a single model. Transfer learning (e.g. fine-tuning) is a machine learning technique that leverages knowledge gained from one task/domain to improve learning and performance on a different and related task/domain [49,62]. Transfer learning is especially applied for medical images and deep learning-based classification/recognition tasks by employing pre-trained models [49]. Transfer learning offers several advantages, including reduced training time and improved generalization [63]. Semi-supervised learning is a machine learning paradigm that leverages labeled and unlabeled data to build predictive models and various learning tasks [64]. It aims to overcome the limitations of supervised learning, where labeled data may be difficult or expensive to obtain. Few-shot learning is a subfield of machine learning that deals with the challenge of learning new concepts or tasks with very limited labeled data [65]. In traditional machine learning, models require a substantial amount of labeled data for training, but in few-shot learning, the goal is to enable models to learn from only a few examples of each new class or task. Federated learning is a decentralized machine learning approach where models are trained collaboratively across multiple edge devices or servers without sharing the raw data [66]. This allows privacy-preserving and efficient model training by keeping the data local and distributed. The updates of local models are aggregated to create a global model, enabling learning from diverse strategies employed to approach small data sets and geographically dispersed data sources. Federated learning improves the generalizability of the model [49].

Learning methods such as transfer learning (Figure 4), in which machine learning models that have already been trained are finetuned for a different task, are crucial in working with re-RT data. This technique is demonstrated by Rahul et al. and Usman et al., who used machine learning models trained on ImageNet images to analyze medical images [42,43]. The medical image data sets are small; thus, they cannot be used to fully train machine learning models. However, millions of ImageNet images can be used to train the models. Although ImageNet only contains natural images that are not necessarily medical, the models serve as a basis that can be further improved to analyze medical images. Another technique in training machine learning models with small data sets is refining a specific portion of the training process. For example, Tappeiner et al. improved the quality of segmentation by using a stacked convolutional autoencoder as a shape prior to regularizing the network training. The shape prior guided the segmentation process based on the shapes of available medical images [67].

Finally, another approach to small data sets is to relax the rule of 10, wherein a minimum of 10 events per predictor variable (EPV) are necessary to lead to meaningful results [68]. This concept was tested using data from a cardiac trial of 673 patients in which 252 deaths had occurred, and seven variables were predictors of mortality [69], resulting in an EPV of (252/7 =) 36. This study evaluated EPV values of 2, 5, 10, 15, 20, and 25 and found no major problems for EPV of greater than 10. However, several method and simulation papers have examined minimum sample size, determining that the minimum number of events depends on the study and prediction model to be investigated with the number of patients (n) and the number of predictor parameters (p) subject to several criteria [70]. Per Riley et al., these criteria are comprised of predictor effect estimates as defined by a global shrinkage factor of ≥0.9, absolute difference of ≤0.05 in the apparent and adjusted R2, precise estimation of a margin of error ≤10% of the true value of the model’s residual standard deviation, and an estimation of the mean predicted outcome value (model intercept) [70]. Riley et al., describe an example model for lung function prediction in African American women; 25 predictor parameters would require 918 patients for 36.7 subjects per parameter. With respect to analyzing specific outcomes (time-to-event), Riley et al. explain that certain clinical scenarios require an EPV ranging from 4.8 to 23 depending on the clinical context [71]. This is particularly relevant in re-RT scenarios, given that the ability to leverage small data sets ultimately determines data separation and implementing penalties based on the strength of association [72].

3.3. Dimensionality of data

Re-RT data is defined by wide dimensionality, which is described in the context of the EPV, and this can pose significant difficulties. In many re-RT sets, not many patients are available for analysis, but the features known about each patient are many and, given emerging molecular data, growing rapidly. As dimensionality, or number of variables, increases, the number of samples necessary to accurately estimate an output also needs to increase [73,74]. However, as dimensionality increases, the differences between variables may become negligible, which is considered a ‘blessing of dimensionality’ [75,76] (Figure 5). The potential blessing of dimensionality needs to be maximized in oncology, particularly in the re-RT scenario, as data of increasingly larger dimensionality may continue to emerge. At the same time, the number of instances or events from where it will originate will remain small. The dimensionality that can be mined in re-RT scenarios is particularly relevant considering that re-RT is often implemented in superior biological tumor behaviors, i.e. the patient must be alive and clinically well enough to be offered and to be able to receive re-RT. This clinical scenario thus allows more aspects to be ‘known’ about the patient, the natural history of the disease, and biological alteration over time in response to management, which can mitigate the curse of dimensionality encountered in AI [80] since it is inevitably the other side of the coin of the blessing of dimensionality (Figure 5). This is the case since novel interventions in re-RT will result in large-scale omic panels and dosimetry parameters that will be based on very small patient cohorts, as is the case with digital medicine in general [81].

Figure 5. The blessing of dimensionality in small data sets [77–79]. In small data sets, the wide dimensionality of data, when available, can be leveraged using machine learning and deep learning to classify patients based on clinical, imaging, and molecular characteristics to arrive at features that can be employed for prediction and subsequently retraining. Conclusions can be employed to optimize data acquisition (e.g. on study protocols and in clinic on the standard of care, e.g. post-re-RT imaging fusion and analysis for response), as well as data curation (e.g. dose to organs at risk in the RT field).

3.4. Radiomics in re-RT

Imaging and biological data and dosimetric variables remain critical tools in the work-up and treatment of patients undergoing re-RT. There is limited published literature on radiomics in re-RT ([82,83]).

Magnetic resonance imaging linear accelerators (MRI-linac) may offer potential data acquisition for radiomic investigation particularly in re-RT [84,85]. Re-RT using SBRT may be employed for recurrent prostate cancer, wherein it is crucial to deliver ablative radiation doses to the target while limiting the dose to OARs, with the goals of tumor control and minimizing toxicity [84]. MRI-linac may improve disease detection, better visualization of pelvic anatomy and adaptive planning accounting for target size and shape, as well as organs at risk (OARs) [85]. MRI as an imaging modality may provide biologic data to predict tumor response [86].

It is anticipated that these features will increase in utilization with the aim of becoming routine in clinical practice and clinical trials [87]. To take advantage of emerging re-RT data, feature extraction across data sets must be possible to allow for predictive power [88].

3.5. Limitations for the acquisition and validation of biospecimens

Patients who receive re-RT often do so in the context of palliative care after many interventions, therefore tissue and biospecimens are rarely available for molecular analysis [89–95]. The ability to measure markers in serum as opposed to tissue in such scenarios would allow for realistic data acquisition leading to more robust computational studies and should become the norm in re-RT trials. As re-RT becomes more common and more data emerges, we anticipate that potential biomarkers identified in small data sets can attempt to be validated via computational analyses of open-access, publicly available platforms such as The Cancer Genome Atlas Program (TCGA),Chinese Glioma Genome Atlas (CGGA) and The International Cancer Genome Consortium (ICGC) for more meaningful results [96–98]. For such attempts to be successful, biomarkers found in re-RT small data sets can be contextualized with and compared to data from other radiation datasets and the administration of radiation in general, including in upfront management. Gene Vector for Each Sample (GVES) has been shown to be a reliable machine learning model to identify prognostic genes in small data sets by limiting the effect of sample size, especially for pancreatic cancer [99]. Computational methods such as GVES are being developed to limit the effects of small sample sizes for the development of biomarkers, but many more approaches noted in Table 2 need to be explored and compared in individual re-RT sets. An ongoing challenge with AI systems in clinical care relates to limitations of its interpretability, potentially improved with robust peer-reviewed metrics [100]. Given that novel treatments in systemic and radiation therapy are likely to be based on small data sets, computational learning needs to be leveraged and vetted for quality and interpretability in this setting.

4. Conclusions

Our study demonstrates a paucity of re-RT datasets overall. Given that there is no anticipated improvement in re-RT clinical scenarios with respect to robustly collected data on larger numbers of patients, we understand that data sets will continue to reflect heterogeneity, data missingness, and bias. However, we also anticipate a rapid growth in the number of features or variables that will become available in re-RT datasets, including with respect to omic data and radiation therapy dosimetry data complemented by imaging. As a result, we identified three areas of need: 1) It is critical to generate adaptive frameworks for working with imperfect data and small data sets in re-RT, and these need to be available for analysis in public repositories; 2) Biospecimen, such as serum, needs to be collected systematically on patients undergoing re-RT via clinical trials, and when possible, natural history protocols that gather, annotate, and collect specimens on patients receiving re-RT need wide implementation across all tumor sites; 3) The ability to aggregate data to analyze predictors of tumor response to re-RT will help to better identify the ideal patient populations suitable for re-RT in varied disease sites and AI methods need to be continuously explored, compared and vetted for interpretability in re-RT. Addressing these will further help oncologists and patients to make informed decisions about treatment approaches that balance potential toxicity from treatment with improvements in outcome and will lead to superior understanding of radiation biology and signaling pathways that can be targeted to improve outcomes.

5. Expert opinion

Despite increasing indications for re-RT, available data in this space remain small and heterogeneous rendering patient selection challenging. Identifying predictive and prognostic biomarkers for treatment response may aid in ideal patient selection and computational methods may be leveraged to maximally harness small data sets. The implementation of computational tools in this space is realistic given evolving methods for the organizational structure of RT data and an onus on secure data sharing and federated learning.

Key challenges in this area continue to be identifying optimal methods for data collection and curation, a lack of widespread tissue and biospecimen acquisition, limited phase III clinical trials, access to publicly available data, and innovative computational approaches that allow the generation of transferable clinical conclusions. Small data sets will continue to play an important role, particularly in rare diseases or conditions that disproportionately impact-specific patient populations based on demographics and thus both the collection of robust data sets and machine learning methods will need to evolve to ensure breakthroughs in evaluating small data sets.

Data can be augmented with enhanced use of objective treatment responses to re-RT, patient-reported outcomes (PRO) and health-related quality of life (HRQOL) which are essential components of oncologic care, particularly in the re-RT setting. Methods will need to be introduced to capture and mine data from patient reports, ideally digitizing paper questionnaires, and moving away from siloed data. The use of imaging, biological, and dosimetric data remains a critical tool in the work-up and treatment of patients undergoing re-RT. Methods to improve serum, as opposed to tissue markers, may provide more accurate information in the setting of re-RT. Ideally, serum collection will become routine as re-RT clinical trials increase. Health inequities exist within clinical trial enrollment, with certain demographics less represented than others. It is imperative to improve participant recruitment and enrollment in clinical trials. In the setting of re-RT, avenues to enhance data sharing may mitigate limitations present with retrospective reviews and single-institution experiences.

Advances have been made in the most common re-RT disease entities: head and neck and CNS, and in particular, glioma. Validated biomarkers include HPV and PD-L1 for select head and neck squamous cell carcinomas. Several efforts have been made to identify various biomarkers, radiologic features, and dosimetric predictors with research ongoing.

Re-RT data is defined by wide dimensionality which can be limiting. However, this aspect can also be favorably harnessed using computational approaches that aim at leveraging data acquisition, cross-validation, feature selection, and learning methods.

As the use of re-RT continues to grow, enhanced use of computational aids by collecting and structuring data for analysis is at the forefront of progress. It is anticipated that federated learning and innovation in this space as the field advances may provide a template for transferable advancement in other areas of medicine. There should be continued encouragement for multidisciplinary review to homogenize management and identify optimal patient selection with the byproduct of robust structured data.

Article highlights

• Given prolonged survival for certain malignancies and technological improvements of radiation delivery, the indications for re-RT have been increasing.

• Most re-RT data are limited to retrospective studies or single-institution experiences.

• Given the limitations of small data sets for re-RT patients, computational methods may be used to enhance existing data to arrive at clinically actionable biomarkers.

• Limitations to clinically use biomarkers include validation and identifying the optimal method to collect specimens for molecular analysis.

Abbreviations

CNS	=	Central Nervous System
CGGA	=	Chinese Glioma Genome Atlas
CV	=	Cross-Validation
DB-MTD	=	Distance-Based Mega-Trend-Diffusion
EPV	=	Events per Predictor Variable
GA	=	Genetic Algorithms
GVES	=	Gene Vector for Each Sample
GyE	=	Gray equivalent
HSRT	=	Hypofractionated stereotactic radiation therapy
IMIT	=	Intensity Modulated Ion Therapy
IMPT	=	Intensity Modulated Proton Therapy
IMRT	=	Intensity Modulated Radiation Therapy
KPS	=	Karnofsky Performance Score
LASSO	=	Least Absolute Shrinkage and Selection Operator
MEVT	=	- Modified Extreme Value Theory
mRMR	=	Maximum Relevance Minimum Redundancy
OS	=	Overall Survival
P	=	- Prospective
PFS	=	Progression-Free Survival
R	=	Retrospective
RFE	=	Recursive Feature Elimination
RBE	=	Relative Biological Effectiveness
Re-RT	=	Reirradiation
S	=	Systematic review
SRS	=	Stereotactic radiosurgery
TCGA	=	The Cancer Genome Atlas

Declaration of interests

The authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.

Reviewer disclosures

Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Author contributions

All authors have substantially contributed to the conception and design of the review article and interpreting the relevant literature, and have been involved in writing the review article or revised it for intellectual content. All authors have read and agreed to the published version of the manuscript.

Additional information

Funding

This paper was funded in part by the NCI NIH intramural program (ZID BC 010990).