Explainable data mining model for hyperinsulinemia diagnostics

ABSTRACT

In our research, we present a data mining model for the early diagnosis of hyperinsulinemia, potentially reducing the risk of diabetes, heart disease, and other chronic conditions. The dataset, gathered from 2019 to 2022 by Serbia's Healthcare Center through an observational cross-sectional study, includes 1008 adolescents. Medical datasets are often highly imbalanced and may contain irrelevant features that hinder predictive performance. To address these challenges in the medical data analysis, we propose a model employing Functional Principal Component Analysis (FPCA), which also accounts for outliers that could otherwise lead to the inclusion of irrelevant features. Unlike standard Principal Component Analysis (PCA), which is sensitive to the initial positions of cluster centers influencing the final outcome, our model integrates FPCA with K-Means clustering to improve the preprocessing stage. Additionally, we have incorporated the post-hoc explanatory method SHAP (SHapley Additive exPlanations) alongside algorithms such as Random Forest, XGBoost, and LightGBM to provide deeper insights into our model, identifying the most contributory features for the development of hyperinsulinemia. Experimental results showed that combining FPCA with K-Means clustering enhances the accuracy of the XGBoost classifier, with this model achieving an accuracy score of 0.99.

KEYWORDS:

• PCA
• FPCA
• K-Means
• SHAP
• Hyperinsulinemia

1. Introduction

Hyperinsulinemia represents a state of pre-type 2 diabetes mellitus and is characterised by significantly elevated insulin levels in the blood Figure 1. As this pathological entity can persist for years without pronounced symptoms, proper identification and verification of its presence, along with potential risk factors, during early adolescence can hold exceptional significance in preventing various conditions stemming directly from such a state. Hyperinsulinemia is a disorder that can emerge at any life age, including adolescence. It is imperative during these formative years to be vigilant about the potential onset of this disorder and to take proactive steps to prevent it. The adolescent phase is characterised by significant hormonal fluctuations, which can adversely affect pancreatic functionality and insulin regulation in the bloodstream. Moreover, this is a time when unhealthy lifestyle habits are prone to take root, including suboptimal dietary patterns and a lack of physical activity, both of which can substantially heighten the risk of various metabolic syndromes such as hyperinsulinemia. Adolescents at higher risk for hyperinsulinemia often include those with a family history of type 2 diabetes, elevated body mass, insulin resistance, inadequate physical activity, and unhealthy dietary habits (IDF Diabetes Atlas, 2021; Thomas et al., 2019).

Figure 1. Hyperinsulinemia process.

According to the International Diabetes Federation (IDF), if insulin values are greater than 15 µU/ml at 0 min and insulin values after the OGTT (Oral Glucose Tolerance) test are greater than 75 µU/ml at 120 min, or if the cumulative insulin value exceeds 300 µU/ml, hyperinsulinemia is diagnosed (Sun et al., 2022). As the prevalence of hyperinsulinemia accelerates globally and also in our region during this life period, the results of this research can hold considerable scientific and practical importance for pediatricians (Ryder et al., 2020). They can aid in strategizing the application of preventive and timely corrective measures to avert the onset of the mentioned pathological entity and the development of potential complications, primarily type 2 diabetes and cardiovascular diseases, extending into adulthood (Andes et al., 2020; Horta & de Lima, 2019).

Enhancing the diagnosis of hyperinsulinemia necessitates the adoption of innovative techniques. Data mining, with its capacity to uncover vital yet hidden patterns within vast databases, holds the promise of transforming the landscape of medical diagnostics (Savić et al., 2023; Vrbančič et al., 2022). Numerous data mining techniques and algorithms have been tailored to distil knowledge from medical databases for disease diagnosis (Chen et al., 2023; Sun et al., 2022). PCA, a simple yet potent non-parametric method, provides a pathway to extract pertinent insights from intricate datasets (Thenappan et al., 2022). In scenarios requiring the categorisation of vast datasets into user-defined clusters, the K-Means algorithm (Edeh et al., 2022) facilitates this by minimising the squared error function. However, susceptibility to outliers and elevated time complexity hinder its efficacy. Therefore, PCA plays a pivotal role in reducing dataset dimensions while conserving paramount information, thereby refining cluster centroids for improved accuracy. Importantly, the effectiveness of K-Means clustering is rooted in its capacity to group similar entities within a dataset. Any clusters that significantly deviate from the norm, indicating outliers, are identified as anomalies and then removed. To address these challenges, Functional-based Principal Component Analysis (FPCA) (Gecili et al., 2021) emerges as advantageous. FPCA identifies and eliminates irrelevant features, a key factor for unbiased outcomes and expedited training. Remarkably, FPCA's strength lies in minimal data loss despite dimension reduction, resulting in enhanced classification accuracy and computational efficiency compared to classical PCA (Pan et al., 2023). Within this landscape, the integration of machine learning algorithms like Random Forest, XGBoost, and LightGBM stands as the logical next step. Enriched by the insights from FPCA and K-Means clustering, these algorithms adeptly navigate the complexities of hyperinsulinemia diagnostics. Finally, for a deeper understanding and augmented transparency of the proposed model, the SHAP method is employed. It serves to elucidate the outcomes of intricate ensemble models such as Random Forest, XGBoost, and LightGBM, thus providing enriched insights. This multidimensional approach not only refines accuracy but also transforms the medical diagnosis landscape by harnessing the latent power of advanced technologies. To highlight the contribution of this study, the following main research objective with two sub-questions is posed:

RO: To what extent can unsupervised techniques such as PCA and FPCA improve K-Means clustering and provide foundation for input values of Random Forest, XGBoost and LightGBM contributing to higher classifiers accuracies?

RQ1: Among supervised models such as Random Forest, XGBoost, and LightGBM, which performs better compared to the baseline model, in this case, Logistic Regression, in terms of Recall, Precision, Accuracy, F1 score, and MCC?

RQ2: Using the SHAP method, what are the most informative features that the best-performing model among the three supervised models - Random Forest, XGBoost, and LightGBM - considers when predicting hyperinsulinemia?

As the prevalence of hyperinsulinemia in adolescence rapidly increases, and the significance and widespread application of a combination of unsupervised techniques and supervised models such as Random Forest, XGBoost, and LightGBM become evident, this study's findings hold exceptional scientific and practical value for pediatricians in crafting strategies for preventive and timely corrective measures. These strategies aim to prevent the onset of this pathophysiological entity and the development of potential complications, primarily type 2 diabetes mellitus and cardiovascular diseases, in later adult life.

The results of this research can significantly contribute to a better understanding of the risk factors that influence the occurrence of hyperinsulinemia with elevated glycemia in adolescents, especially those with excessive body weight, poor dietary habits, insufficient physical activity, positive family history, and psychoactive substance use. Timely identification of adolescents at risk for hyperinsulinemia using the latest artificial intelligence tools should become a daily and universally accessible practice because it is of great importance for their future and the health of society as a whole. Finally, the proposed model can be applied in detecting and identifying risk factors for the development of other chronic non-infectious diseases.

The rest of the paper is organised as follows: Section 2 gives an overview of the current research for improving medical diagnostics and of different health-care predictions using statistical methodologies and ML algorithms. Section 3 describes the proposed data mining model and ML algorithms, along with post-agnostic SHAP method that will be employed. Section 4 presents obtained results. Section 5 discusses the results. The concluding remarks and future directions are given in Section 6.

2. Related work

The domains of medical diagnostics and comprehensive prevention are experiencing swift and dynamic growth, driven by the application of advanced technologies that function as the foundational predictive models (Hassan et al., 2021; Rasha et al., 2023; Zhou et al., 2023) Exploring hyperinsulinemia holds significant importance due to its intricate connection with various health aspects beyond just diabetes type 2 (Halloun et al., 2022; Koch et al., 2021). Hyperinsulinemia, marked by an overproduction of insulin, is not merely a harbinger of type 2 diabetes; it is also implicated in a spectrum of other medical issues including obesity, metabolic syndrome, cardiovascular disease, and some forms of cancer. A deeper comprehension of hyperinsulinemia can shed light on the incipient phases of metabolic disorders, offering a window for early interventions that could halt the escalation into more debilitating diseases such as diabetes (Calcaterra et al., 2022). Given the prevailing emphasis on predictive model development for type 2 diabetes mellitus in prior literature, rather than hyperinsulinemia, our objective is to provide an overview and conduct a comparative analysis of their outcomes. This will be achieved through an exploration of the distinct techniques and models employed in these studies.

In (Bansal & Singhrova, 2022) the authors have used dataset for diabetes and PCA for dimensionality reduction and K-Means clustering technique to feed the voting classifier that employs soft voting, achieving 98.70% of accuracy. Moreover, dividing the famous, publicly available PIMA Indian diabetic dataset, the study (Khairunnisa et al., 2022) showed that a 5-fold cross validation-based evaluation shows that each proposed procedure enhances the K-Nearest Neighbors (KNN) algorithm. Additionally, they concluded that K-Means clustering is capable of increasing the accuracy of KNN from 81.6% to 86.7%. Combining K-Means and PCA improves the KNN accuracy to 90.9%. Another study (Arora et al., 2022) used the same PIMA dataset and proposed a novel architecture for predicting diabetes patients using the K-means clustering technique and Support Vector Machine (SVM). The features extracted from K-means are then classified using an SVM classifier achieving the accuracy of 98.7%. In the pursuit of diabetes prediction, the authors in (Ganguly & Singh, 2023) employed diverse machine learning assessments on a cluster-based dataset. They utilised the K-Means clustering algorithm for early detection of diabetes, analysing data from 165 diabetic patients. The most noteworthy precision, recall, and F1-score were achieved through the combination of K-Means, while the highest accuracy of 0.79 was attained using the random forest model.

Moreover, the authors in (Choubey et al., 2020) engaged in the processing and analysis of obtained datasets through two distinctive methodologies. On one hand, the initial approach involves the application of classification techniques including Logistic Regression, K-Nearest Neighbor, ID3 Decision Tree, C4.5 Decision Tree, and Naive Bayes. On the other hand, the second approach integrates PCA and Particle Swarm Optimization (PSO) algorithms for feature reduction prior to implementing the classification methods of the first approach. A comparative evaluation is conducted to juxtapose the diverse methodologies employed in this reseacrh. The yielded outcomes unequivocally demonstrate the superiority of the proposed technique over conventional classification approaches in terms of reduced computational time and heightened accuracy. Additionally, one study revealed that the classification performance of the k-nearest neighbour (KNN) method hinges on the selection of the k neighbours for a query. Optimizing KNN is challenging due to the difficulty in choosing suitable neighbours and the optimal k value. Furthermore, KNN's effectiveness is hampered by the simplistic majority voting approach. To overcome these issues, we introduce a novel adaptive k-nearest centroid neighbour classifier, termed AD-LAKNCN, which utilises average distance measures. Tests on 24 real-world datasets demonstrate that AD-LAKNCN outperforms nine leading KNN variants in classification accuracy (Wang & Zhang, 2022).

In an extensive examination detailed in (Chauhan et al., 2021) an inclusive survey of diabetes research papers spanning from 2018 to 2021 is undertaken. The accurate anticipation of diabetes has been achieved through the application of decision tree-based algorithms such as C4.5 AdaBoost and XGBoost. In dealing with vast datasets, individual machine learning techniques like PCA and K-Means emerge as valuable tools for feature selection and anomaly detection. This review also highlights the potent accuracy of diabetes prediction attained by combining supervised and unsupervised AI methodologies, notably employing K-Means and Support Vector Machine.

Furthermore, the analysis by (Muhammad et al., 2019) employed K-means clustering, utilising the first two principal components. Additionally, through correlation analysis between diabetes mellitus and specific illnesses, a significant association was unveiled – indicating heightened prevalence of kidney and hypertension issues among diabetes patients.

While K-means is lauded for its straightforwardness and adaptability to various data types, the algorithm's dependence on the initial placement of cluster centres can significantly affect the results. Properly executed, it can produce well-clustered datasets that enhance logistic regression analyses. Conversely, if clustering is not accurate, it can adversely affect the performance of the model. In (Yadav et al., 2021) and (Ye et al., 2020), the focus was precise diabetes prediction through PCA, K-Means, Random Forest, Multilayer Perceptron and Naïve Bayes. Steps encompass data preprocessing, PCA-driven feature extraction, and voting classifier-based classification in the diabetes prediction model.

Exploring the potential of utilising tongue images to achieve nuanced diabetic population classification, K-means emerges as a viable option. By employing K-means, the diabetic population can be effectively segregated into four distinct clusters defined by Vector Quantized Variational Autoencoder features. Remarkably, this approach yields a notable classification accuracy of approximately 88% (Li et al., 2022).

Ultimately, incorporating an explainable element when introducing novel predictive models holds significance, especially when these models are intended for use by experts who may not necessarily possess technical expertise in the domain. SHAP-TreeExplainer model that does not require reference data was proposed in (Lama et al., 2021) for predicting a prediabetes or T2D diagnosis 10 years after the data collection. Furthermore, SHAP can provide an interpretative framework for prediction models targeting specific medical conditions, such as Heart Failure stemming from coronary heart disease, by considering contributing risk factors associated with adverse outcomes (Wang et al., 2021).

Although the showcased studies demonstrate potential, they do not delve into the preliminary diabetes condition – hyperinsulinemia, through a synergistic integration of unsupervised methods like FPCA and K-Means, coupled with supervised models such as XGBoost, Random Forest, and LightGBM. Additionally, the studies that do investigate diabeteic conditions do lack with an exploration of concept explainability, a gap we address using the SHAP methodology.

3. Methodology

This section consists of the following sub-sections: Section 3.1 provides an overview of Exploratory Data Analysis (EDA) for the selected dataset. Section 3.2 focuses on data preprocessing. Section 3.3 describes the proposed data mining model, including both supervised models and unsupervised techniques. The final sub-section, 3.4, outlines the evaluation metrics used for assessing the presented model.

The constructed model is devised by synergistically integrating PCA, K-Means, and Logistic Regression as the baseline components, along with advanced models such as Random Forest, XGBoost, and LightGBM. A novel methodology is subsequently introduced that leverages PCA to transform the initial feature set, effectively mitigating the issue of correlation. This alleviates the challenge posed to the classification algorithm in identifying relationships within the data (Chang et al., 2023; Razvi et al., 2019). The integration of PCA serves to eliminate extraneous features, subsequently reducing training time and costs, while simultaneously enhancing model performance (Wang et al., 2022). Following the execution of PCA and FPCA analyses, the outcomes are then channelled into an unsupervised clustering phase facilitated by K-Means. This selection is influenced by K-Means’ inherent capacity to handle outliers. The resulting K-Means clusters are refined, and subsequently employed as the foundation for various supervised models, including Random Forest, XGBoost, and LightGBM. These models collectively contribute to the construction of our dataset's classification framework. The illustrative flowchart of the proposed model is showed in Figure 2.

Figure 2. Experimental setup (proposed data mining model).

3.1 Dataset description

The data including adolescents has been collected by the Healthcare Center in Serbia from 2019 to 2022 through an observational cross-sectional study. To clarify, our dataset consists of 1008 samples aged 12–17 years, all of whom underwent regular systematic examinations and were subsequently instructed to perform the OGT test. This indicates that each subject presented at least one indicator suggestive of potential hyperinsulinemia during their examination-examples include elevated glucose levels, high cholesterol, increased BMI and genetic predispositions for type 2 diabetes mellitus. Our dataset is comprehensive, capturing a total of 8 features representing medical diagnostic criteria, alongside a target class that indicates each individual's test status. Specifically, the dataset includes 336 instances with a positive test result and 672 with a negative result. It is imperative to highlight the substantial size of our dataset, which is significant, as all 1008 participants presented with at least one indicator for potential hyperinsulinemia. The set of features that significantly impact the overall clinical profile of hyperinsulinemia were meticulously selected by pediatricians conducting the regular systematic examinations. The Experimental Data Analysis (EDA) pertaining to the selected features is showcased in Table 1.

Table 1. EDA for the dataset features.

Fundamental participant traits	Experimental group (n = 336)	Control group (n = 672)	${\chi }^2$ Student's t-test	p
Gender (male/female) (n,%)	174 (51.8)/162 (48.2)	330 (49.1)/342 (50.9)	0.965	0.329^1
Age (years$\pm \mathrm{SD}$)	15.3$\pm 1.$4	13.7$\pm 1$.5	3.425	0.047^2
Body height (cm±SD)	161.4$\pm 12.5$	157.7$\pm 10.4$	5.036	0.001^2
Body weight (kg±SD)	69.5$\pm 12.7$	59.8 $\pm 9.4$	4.021	0.012^2
Waist circumference (cm±SD)	97.9±10.6	88.3±9.2	2.049	0.035^2
BMI (kg/$m^2\pm \mathrm{SD})$	27.4$\pm 4.2$	23.3$\pm 1.1$	11.247	0.000^2
Systolic blood pressure (mmHg±SD)	149.5±14.2	137.6±9.4	3.208	0.021^2
Diastolic blood pressure (mmHg±SD)	99.1±10.3	85.7±8.2	1.127	0.044^2
Heart rate	87.9±6.8	85.4±8.7	1.234	0.052^2

*SD – Standard deviation, ¹-${\chi }^2$ test, ² – Student's t-test.

3.2 Data preprocessing

Preprocessing medical data plays a pivotal role in the data mining process for disease prediction and diagnosis. This is because the presence of low-quality data can significantly compromise the accuracy and reliability of prediction outcomes. To enhance the utility and suitability of our initial dataset for hyperinsulinemia prediction, we systematically employed a range of preprocessing techniques. Initially, in dealing with missing data, a prevalent strategy is to impute absent values with the mean value of the corresponding attribute (Austin et al., 2021). In our machine learning framework, meticulous analysis identified 37 instances with missing data, each lacking a value for at least one feature. After thorough consideration, the decision was made to eliminate these instances, as their removal wouldn't detrimentally impact the credibility of the final results. These specific data points were also intentionally not substituted with the value of 0. This decision aligns with the guidance of medical experts who assert that, in the context of this attribute type, 0 is not a valid or appropriate value. Considering that, for instance, laboratory analysis parameters are provided in diverse parameter units for all input features, it becomes imperative to employ the min–max normalisation scaling technique within a specific narrow range [0, 1] (Borkin et al., 2019). This ensures the creation of a new scaled dataset based on the original, effectively harmonising the varied parameter units.

3.3 Model design

Our experimental process for the proposed model is structured into three well-defined phases to ensure a comprehensive evaluation. In the initial phase, we focus on enhancing data quality by performing dimensionality reduction on the already processed dataset. This reduction is carried out using two techniques: FPCA and PCA. These methods aim to distil the most crucial information from the data while preserving its essential characteristics. In the second phase, we employ the K-Means clustering algorithm on the subsets obtained from the previous phase's FPCA and PCA outputs. This clustering step is pivotal, as it not only aids in categorising the data but also serves to effectively eliminate outliers and rectify any misclassifications that may have occurred earlier. Finally, in the third phase, the data that has been accurately clustered and classified is primed for utilisation as input within a suite of supervised classifiers. These classifiers encompass advanced models such as Random Forest, XGBoost, and LightGBM. Additionally, a logistic regression model acts as our baseline for comparison. Following a thorough assessment of the model's efficacy, encompassing a comprehensive range of performance metrics including Recall, Precision, Accuracy, ROC, and MCC, the validation process attains a higher level of substantiation through factorial and multivariate analyses.

3.3.1 Unsupervised learning: FPCA and PCA

The process of Principal Component Analysis revolves around a series of distinct steps aimed at transforming the feature space by exploiting attribute relationships (Hasan & Abdulazeez, 2021). This transformation entails mapping the original feature space onto a lower-dimensional plane to achieve the desired dimension reduction, subsequently leading to an in-depth analysis of this newly formed feature space. PCA, being an unsupervised technique for dimensionality reduction, achieves this reduction by capitalising on the correlations between input features. The optimisation of the transformation matrix is grounded in identifying the most substantial disparities within the original space (Velliangiri & Alagumuthukrishnan, 2019). A pivotal principle governing the selection of Principal Components (PCs) derived from PCA is that directions associated with the greatest variances inherently encode the most salient information about classes. PCA often originates from a linear projection that maximises variance within the projected space (Anowar et al., 2021). A commonly employed strategy for PC selection involves setting a variance-explained threshold, often around 80%, and subsequently determining the number of components that generate a cumulative variance sum close to this established threshold. The integration of PCA into a dataset proves invaluable, especially in scenarios demanding unsupervised learning. This utility lies in its ability to efficiently initialise centroids for clustering algorithms, thus significantly enhancing the efficacy of the clustering process. Given that PCA yields a feature subspace optimizing variance along axes, a vital preprocessing step involves standardising the dataset to a unit scale (with mean = 0 and variance = 1). This standardisation step is pivotal to amplify the effectiveness of PCA results, which in turn, stands as a prerequisite for the optimal performance of numerous machine learning algorithms. The PCA analysis transforms the original values of variables into principal components together using the following formulas (1), (2) (Sun et al., 2019): (1) $Y=X\cdot C$(1) X is a set of n vectors, $\left(x_1,\cdots ,x_n\right)$ where each X_i element represents an instance of our dataset, and C is a square matrix of order n: $c_{\mathrm{ij}}=\left[\begin{array}{ccc}{c}_{11}& \cdots & c_{1n}\\ ⋮& ⋮& ⋮\\ c_{n1}& \cdots & c_{\mathrm{nn}}\end{array}\right]$ and (2) $\begin{array}{rl}{Y}_{\mathrm{ij}}& =\left[\begin{array}{c}{x}_1\\ ⋮\\ x_n\end{array}\right]\cdot \left[\begin{array}{ccc}{c}_{11}& \cdots & c_{1n}\\ ⋮& ⋮& ⋮\\ c_{n1}& \cdots & c_{\mathrm{nn}}\end{array}\right]\\ & =\left[\begin{array}{c}{y}_1\\ ⋮\\ y_n\end{array}\right],i=\overline{1,n},j=\overline{1,n},\mathrm{dimension}:\left(1,n\right)x\left(n,n\right)=\left(1,n\right).\end{array}$(2) When interpreting principal components, it is often useful to know the correlations of the original variables with the principal components. The correlation between variable X_i and principal component Y_j is, formula (3): (3) $K_{\mathrm{ij}}=\sqrt{c_{\mathrm{ij}}^2\cdot \mathrm{Var}\left(y_j\right)/{\sigma }_{\mathrm{ii}}}$(3) where ${{\sigma }_{\mathrm{ii}}}^2=(1/n-1)\cdot {\sum }_{i=1}^n{\left(x_i-\overline{x}\right)}^2$, $\overline{x}$ the mean value is calculated as: $\overline{x}={\sum }_{i=1}^n{x}_i/n$

PCA represents eigenvectors and eigenvalues of the covariance matrix. Eigenvectors will determine the directions of the new feature space, while eigenvalues will determine the magnitudes.

From $\left(\mathrm{\lambda I}-C\right)X=0$ in other words: $\mathrm{CX}=\mathrm{\lambda X}$ we define a set S of all vectors that satisfy this equation as, formula (4): (4) $S=\left\{x|\left(\mathrm{\lambda I}-C\right)X=0\right\}.$(4) FPCA is utilised to analyse and further reduce the dimensionality of data, specifically focusing on functional instances. Its primary goal is to determine the ideal number of functional instances to retain in the process. Once the central data within clusters is identified, a matrix of variations, denoted as C, is generated. This matrix involves the calculation of its elements as, formula (5):

$c_{\mathrm{ij}}=x_i\left(r_j\right)-\overline{x_i}\left(r_j\right)$, where $r_j$ are the values in the observed dataset. (5)

After that, the functional variance is calculated for each r defined as: $\mathrm{Var}\left(r\right)=\frac{1}{n}\cdot \sum _{i=1}^n{c}_{\mathrm{ij}}^2,$ where n is the number of instances in each cluster. In this way, less significant instances are eliminated, and only the main ones will be used as input for the K-Means clustering in the next phase of the experiment.

3.3.2 Unsupervised learning: K-Means clustering

The K-means clustering algorithm groups data into two clusters: a set of adolescents with hyperinsulinemia and a set of adolescents without hyperinsulinemia (Ripan et al., 2021). Firstly, k cluster centres are assigned values, and then the remaining values are grouped around them based on the Euclidean distance values. In other words, each point x_i is assigned to a specific cluster m (Alam & Muqeem, 2022), formula (6). (6) $c_i=\mathrm{argmi}{n}_j||x_i-k_j|{|}^2,$(6) where c_i is the index of the cluster to which the value x_i is assigned, and k_j is the centre of cluster m.

Then, the cluster centres are updated using, formula (7): (7) $k_j=\frac{1}{n_j}\sum _{i=1}^n{x}_i,$(7) where $n_j$ is the number of points in cluster j. (7)

For k = 2, the target variable contains two possible outcomes (positive – has hyperinsulinemia, and negative – does not have hyperinsulinemia).

3.3.3 Supervised learning: logistic regression, Random Forest, XGBoost, and LightGBM

In this research, logistic regression will serve as a baseline for evaluating the performance of Random Forest, XGBoost, and LightGBM. In contrast to more intricate models, logistic regression doesn't aim to uncover data patterns; instead, it predicts based on the most common label in the training set. Logistic regression offers several strategies, including most frequent, stratified, uniform, and constant. While being a straightforward classifier, the anticipation is that Random Forest, XGBoost, and LightGBM will surpass the baseline performance of logistic regression.

The Random Forest Classifier is a prominent ensemble model in machine learning classification. Comprising specialised decision tree configurations, it yields categorical outcomes. This classifier uses bootstrap aggregation to create n trees, averaging their predictions for the final outcome. While selecting split points, the algorithm prioritises effective features from a randomised subset (Alam et al., 2019; Xu et al., 2020). Adjustable parameters govern model size and foundational learning processes. Despite susceptibility to overfitting, its documented strengths include robustness, adaptability, and solid performance with outliers and missing values. In medical data analysis, Random Forest excels, effectively handling complex relationships and diverse variables in healthcare's intricate scenarios. In this research, the following parameters are used:

• n_estimators : the amount of trees within the forest;

• max_features: the amount of features that can be considered when looking for the ideal split;

• max_depth: the utmost depth of a tree;

• criterion: the function to compute the quality of a split.

XGBoost, an acronym for “Extreme Gradient Boosting”, is a pivotal algorithm in this study. It leverages gradient tree boosting, excelling in medical data analysis (Hassan et al., 2022). With rapid computation, high accuracy, and built-in regularisation against overfitting, it's well-suited for medical data. Handling intricate relationships, outliers, and imbalanced data, XGBoost's strength lies in healthcare complexities. Guided by an objective loss function, it optimises predictions, enhanced by boosting. This algorithm adeptly uncovers complex medical patterns and dependencies (Li et al., 2020; Zhang et al., 2020), formula (8): (8) $L^{\left(t\right)}=\sum _{i=1}^{n}l\left(y_i{\widehat{y_i}}^{\left(t-1\right)}+f_t\left(x_i\right)\right)+\mathrm{\Omega }\left(f_t\right)$(8) The formula (8) amalgamates gradient descent with Taylor approximation to optimise the loss function. With each iteration within the XGBoost framework, a novel model is crafted, focusing on refining the outcome using the error gradient derived from the preceding model (Zhang & Gong, 2020). The rationale for selecting XGBoost rests on its demonstrated superiority compared to alternative models. XGBoost has many parameters that can be tuned. For our research, we tuned the following parameters:

• subsample: denotes the fraction of observations to be randomly sampled for each tree;

• colsample_bytree: the subsample ratio of columns when constructing each tree;

• max_depth: the utmost depth of a tree;

• min_child_weight: defines the minimum sum of weights of all observations required in a child;

• learning_rate: the shrinkage you do at every step you are making.

LightGBM is a gradient boosting algorithm commonly applied to medical data. Its fundamental steps and concepts gain particular significance in the context of medical information. When dealing with medical data, LightGBM becomes a vital tool for analysis and prediction due to its ability to efficiently handle large datasets and achieve high predictive accuracy. The algorithm employs gradient boosting to iteratively enhance prediction models, combining weak models to form a robust predictor (Rufo et al., 2021). The gradient of the target function $L\left(y,F\left(x\right)\right)$ with respect to the function $F\left(x\right)$ is computed as, formula (9) (Ghourabi, 2022): (9) $\frac{\mathrm{\partial }L\left(y,F\left(x\right)\right)}{\mathrm{\partial }F\left(x\right)},$(9) where these gradients indicate the direction of the rate of change of the objective function with respect to the prediction.

For each node, the Gain is calculated as the value of the function before and after node splitting, as, formula (10): (10) $\mathrm{Gain}=L\left(\mathrm{left},F\left(\mathrm{left}\right)\right)-L\left(\mathrm{right},F\left(\mathrm{right}\right)\right)-L\left(\mathrm{node},F\left(\mathrm{node}\right)\right)$(10) where left and right denote nodes to the left and right of the selected node, and “node” represents the node before splitting. The node that yields the highest gain upon adding a new node is selected. For our research, we tuned the following parameters:

• num_leaves: controls model complexity, speed, performance;

• n_estimators: boosting rounds that affect model performance;

• max_bin: improve the accuracy;

• min_data_in_leaf: data points in leaf that can prevent overfitting and influence model robustness against noise;

• reg_alpha and reg_lambda: L1 and L2 regularisation prevents overfitting, enhances model's generalisation through penalty addition.

3.3.4 Post-agnostic method: SHAP

Employing model-agnostic approaches such as SHAP allows for the evaluation of machine learning models as opaque entities, focusing exclusively on their outputs and disregarding the need for internal model specifics. This characteristic renders SHAP and similar methodologies highly adaptable and applicable across a diverse range of model architectures, enhancing their utility in the field of explainable AI (Nohara et al., 2022). Employing SHAP post-hoc enables insights into feature contributions without knowing model internals. In medical data, SHAP's model-agnostic nature is beneficial. Medical data's complex relationships are unveiled, aiding feature influence understanding (Knapič et al., 2021). SHAP is adaptable across medical models, traditional or advanced, empowering practitioners to interpret and validate outcomes, leading to informed medical decisions. The assessment of the importance of variable i is determined by analysing how the inclusion of variable I in the set S impacts the value of the function eS. This contribution of variable I is represented by (i) and is calculated as a weighted average across all possible subsets S, formula (11): (11) $\mathrm{\varnothing }\left(i\right)=\sum _{S\subseteq \frac{\left\{1,\dots ,p\right\}}{\left\{i\right\}}}\frac{|S|!\left(p-1-|S|\right)!}{p!}\left(e_{S\cup \left\{i\right\}}-e_S\right).$(11)

Formula (18), has an equivalen to the following formula (12): (12) $\mathrm{\varnothing }\left(i\right)=\frac{1}{\mathrm{\Pi }}\sum _{\pi \in \mathrm{\Pi }}{e}_{\mathrm{before}\left(\pi ,i\right)\cup \left\{i\right\}-e_{\mathrm{before}\left(\pi ,i\right)}}.$(12) where Π represents the set of all orderings of p variables, and before(π,i) refers to the subset of variables that precede variable i in the ordering π. Each ordering in Π corresponds to a set of values eS that transition from e∅ to (x*). To summarise, when examining a specific ordering, the analysis reveals the impact of adding consecutive variables on the eS function, (Slack et al., 2021) is derived by averaging these contributions across all potential orderings.

3.4 Performance metrics

Recall, a performance metric, measures the fraction of accurately identified positives, showcasing the successful recognition of true anomalies. Precision, another metric, indicates the fraction of true positives divided by the total number of positive predictions, offering insight into the accuracy of identified anomalies. Additionally, metrics such as accuracy, and Matthews Correlation Coefficient (MCC) will be integrated into the assessment. To assess the overall performance, the F1 score, a harmonic mean of precision and recall, will be adopted for each model. Furthermore, the generation of confusion matrices will yield insights into how models misclassify both classes. Collectively, these metrics provide a thorough evaluation of model efficacy in medical data analysis (Thabtah et al., 2020).

4. Results

In this section, we present the results obtained in this study. Section 4.1 presents the results conducted through traditional statistical analysis, Section 4.2 showcases the outcomes achieved using the proposed data mining model, and Section 4.3 presents the results obtained through factorial analysis.

4.1 Statistical analysis

Table 2 shows the average glucose and insulin values along with the corresponding standard deviations for participants in both the first experimental group and the second control group. These parameters were measured at intervals of 0th, 30th, 60th, 90th, and 120th minutes. Subsequently, mean values and standard deviations were computed for adolescents within each group, in addition to the values of the HOMA-IR index (insulin resistance index). Utilising Student's t-test, an analysis was conducted to confirm the presence of statistically significant differences among participants between the observed groups during the execution of the OGTT test, as well as for the calculated HOMA-IR index values.

Table 2. OGTT values along with HOMA-IR index values.

OGTT	Experimental group (n = 336, %)	Control group (n = 672, %)	Student’s t-test	р
Glucose at 0 min (mmol/l) $\pm \mathrm{SD}$	7.2$\pm$1.2	6.3$\pm$0.8	2.024	0.027
Glucose at 30 min (mmol/l) $\pm \mathrm{SD}$	14.5$\pm$1.5	12.3$\pm 1$.5	2.862	0.005
Glucose at 60 min (mmol/l) $\pm \mathrm{SD}$	12.7$\pm$1.3	9.5$\pm$1.2	6.337	0.000
Glucose at 90 min (mmol/l) $\pm \mathrm{SD}$	9.2$\pm$1.1	8.3$\pm$1.5	2.845	0.009
Glucose at 120 min (mmol/l) $\pm \mathrm{SD}$	8.2$\pm 0$.8	7.1$\pm 1$.1	2.229	0.018
Insulin at 0 min (µIU/ml) $\pm \mathrm{SD}$	19.3$\pm 3$.4	16.8$\pm 1$.6	8.405	0.000
Insulin at 30 min (µIU/ml) $\pm \mathrm{SD}$	163.5$\pm 5.4$	151.2$\pm 6.1$	78.245	0.000
Insulin at 60 min (µIU/ml) $\pm \mathrm{SD}$	122.7$\pm 3$.7	113.3$\pm 4$.5	64.025	0.000
Insulin at 90 min (µIU/ml) $\pm \mathrm{SD}$	89.3$\pm 3$.2	83.1$\pm 3$.2	71.015	0.000
Insulin at 120 min (µIU/ml) $\pm \mathrm{SD}$	80.5$\pm 3$.3	67.3$\pm 2$.1	8.073	0.000
HOMA-IR$\pm \mathrm{SD}$	6.4$\pm 2$.2	5.3$\pm 1.5$	5.680	0.000

*bold values indicate the presence of statistical significance, SD – Standard Deviation.

Based on the statistical outcomes, insulin values exceeding 45 µU/ml were identified at the 0th minute in 18 participants (1.8%), while at the 120th minute, 318 participants (31.5%) exhibited insulin values surpassing 75 µU/ml. Notably, a cumulative insulin value surpassing 300 µU/ml was evident in 336 participants (33.3%), constituting the experimental first group. The second control group comprised 672 participants (66.7%), characterised by a cumulative insulin value below 300 µU/ml. For both participant groups, most hematological and biochemical parameters exhibited elevated values in comparison to reference standards. However, disparities emerged when analysing the results by gender and groups Table 3.

Table 3. Results of hematological and biochemical analyses.

	Experimental group (n = 336, %)		Control group (n = 672, %)
Parameters of hematological and biochemical analyses	male	female	male	female	Student's t-test	р
Leukocytes WBC (10*9/l) $\pm \mathrm{SD}$	14.3$\pm$2.7	16.8$\pm$3.5	11.5$\pm$2.4	12.7$\pm$3.3	3.452	0.001
Red blood cells RBC (10*12/l ) $\pm \mathrm{SD}$	3.5$\pm$1.5	3.7$\pm$2.1	4.2$\pm$2.5	4.5$\pm$2.6	2.956	0.004
Hemoglobin Hgb (g/l) $\pm \mathrm{SD}$	156$\pm$5	145$\pm$7	138$\pm$5	142$\pm$4	2.835	0.007
Hematocrit Htc (l/l) $\pm \mathrm{SD}$	0.626$\pm$0.6	0.548$\pm$0.9	0.533$\pm$0.5	0.427$\pm$0.7	2.335	0.030
МCV (fl) $\pm \mathrm{SD}$	98.4$\pm$10.3	92.5$\pm$11.7	89.1$\pm$12.2	94.2$\pm$9.4	0.997	0.318
МCH (pg) $\pm \mathrm{SD}$	36.7$\pm$3.3	34.5$\pm$4.5	33.2$\pm 2$.1	33.5$\pm$3.6	1.735	0.098
МCHC (g/l) $\pm \mathrm{SD}$	358.9$\pm$17.6	345.2$\pm$18.9	344.1$\pm$12.3	338.7$\pm$15.7	0.525	0.769
RDW (%)$\pm \mathrm{SD}$	17.2$\pm$2.7	16.7$\pm$3.1	16.3$\pm$2.8	15.9$\pm$3.6	1.025	0.165
Platelets PLT (10*9/l) $\pm \mathrm{SD}$	324.2$\pm$67.2	345.6$\pm$84.4	318.9$\pm$58.3	338.2$\pm$62.4	2.023	0.003
Segmented (%)$\pm \mathrm{SD}$	54$\pm$6.8	57$\pm$7.9	52$\pm 6$.2	55$\pm$7.5	0.752	0.386
MID (%)$\pm \mathrm{SD}$	9.8$\pm$1.5	10.2$\pm$2.4	8.7$\pm$1.4	9.3$\pm$1.9	0.851	0.356
Lymphocytes (%)$\pm \mathrm{SD}$	32.3$\pm$3.3	31.7$\pm$3.8	31.5$\pm$3.1	33.6$\pm$3.5	2.847	0.006
Sedimentation (mm/1.h) $\pm \mathrm{SD}$	17.8$\pm$2.2	18.2$\pm$2.5	15.4$\pm$1.8	14.8$\pm$2.4	3.205	0.036
CRP (mg/l) $\pm \mathrm{SD}$	18.3$\pm$5.3	21.5$\pm$6.7	16.3$\pm$4.2	14.7$\pm$77.9	14.599	0.000
Glucose (mmol/l) $\pm \mathrm{SD}$	7.6$\pm$1.6	8.4$\pm$2.6	6.7$\pm$1.3	7.3$\pm$1.8	3.829	0.024
Cholesterol (mmol/l) $\pm \mathrm{SD}$	7.11$\pm$4.2	8.27$\pm$5.4	5.93$\pm$3.6	6.08$\pm$4.4	17.774	0.000
HDL Cholesterol (mmol/l) $\pm \mathrm{SD}$	0.723$\pm$0.5	0.845$\pm$0.7	0.994$\pm$0.6	0.805$\pm$0.6	13.497	0.000
LDL Cholesterol (mmol/l) $\pm \mathrm{SD}$	3.58$\pm$1.2	3.92$\pm$1.7	3.23$\pm$1.3	3.31$\pm$1.5	15.961	0.000
Triglycerides (mmol/l) $\pm \mathrm{SD}$	3.71$\pm$2.8	4.26$\pm$3.4	2.87$\pm$2.6	2.99$\pm$3.3	13.980	0.000
Urea (mmol/l) $\pm \mathrm{SD}$	10.4$\pm$3.2	12.5$\pm$4.3	8.3$\pm$2.5	9.7$\pm$3.6	3.024	0.018
Creatinine (umol/l) $\pm \mathrm{SD}$	112.6$\pm$11.5	115.8$\pm$14.3	108.6$\pm$9.6	114.7$\pm$10.0	3.527	0.027
Total proteins (g/l) $\pm \mathrm{SD}$	94.3$\pm$3.5	102.7$\pm$4.4	85.7$\pm$3.2	89.2$\pm$3.3	3.027	0.007
Total bilirubin (umol/l) $\pm \mathrm{SD}$	19.8$\pm$4.3	23.7$\pm$5.2	18.3$\pm$3.8	19.5$\pm$4.1	6.024	0.000
AST(SGOT) (u/l) $\pm \mathrm{SD}$	35.6$\pm$3.2	38.4$\pm$3.9	33.8$\pm$2.8	35.4$\pm$3.1	3.418	0.023
ALT(SGPT) (u/l) $\pm \mathrm{SD}$	39.6$\pm$4.5	44.7$\pm$5.7	35.2$\pm$3.6	37.9$\pm$4.2	6.134	0.000
Sodium (mmol/l) $\pm \mathrm{SD}$	152$\pm$14.2	165$\pm$16.3	148$\pm$11.5	159$\pm$13.8	3.024	0.031
Potassium (mmol/l) $\pm \mathrm{SD}$	5.3$\pm$1.4	5.9$\pm$1.9	5.1$\pm$1.2	5.7$\pm$1.3	3.235	0.024
Chlorides (mmol/l) $\pm \mathrm{SD}$	111$\pm$9	114$\pm$13	105$\pm$9	109$\pm$11	2.956	0.012

* Measurements and Reference Values

EB – electronic counter, RN – calculated, WGR – Western Green, SPFT – spectrophotometric, ISE – ion-selective electrode

Cholesterol: 1 – optimal (<5.20 mmol/l); 2 – borderline high (5.20-6.18 mmol/l); 3 – high risk level (>6.20 mmol/l).

HDL cholesterol: 4 – optimal (>1.60 mmol/l); 5 – risky (<1.00 mmol/l).

LDL cholesterol: 6 – optimal (<2.60 mmol/l); 7 – near optimal (2.60-3.30 mmol/l);

8 – borderline high (3.40-4.11 mmol/l); 9 – high (4.12 mmol/l).

Triglycerides: 10 – optimal (<1.70 mmol/l); 11 – borderline high (1.70-2.20 mmol/l);

12 – high (2.20-5.60 mmol/l); 13 – very high (>5.60 mmol/l).

4.2 Results obtained using proposed data mining model

The original dataset used in this study consists of 1008 instances, divided into two subsets: the first subset is for training and contains 672 instances, while the second subset is for testing and contains 336 instances. The identical steps of the experimental setup, as depicted in Figure 2 are performed during both the training and testing phases.

Through the utilisation of PCA analysis, the original dataset's variable count was minimised to better facilitate subsequent stages involving a range of ML algorithms. To ascertain the technique offering superior dimensionality reduction while preserving crucial instance-specific details, a comparison was drawn between FPCA and PCA. In both analyses, essential variables were distilled, data was streamlined, and dimensionality was reduced without significant information loss. Particularly noteworthy is FPCA's remarkable effectiveness, rendering it the favoured option for enhancing precision in the K-means algorithm for cluster identification. For a comprehensive understanding of these findings, detailed outcomes are available in Table 4. Before implementing the model in practice, it is essential to assess the efficiency of the proposed model being utilised. Initially, performance metrics were examined, and the results are presented in Table 5. Subsequently, cross-validation was employed to predict the model's behaviour on new, unfamiliar data, thereby addressing issues of bias and substantial variability reduction. The results of the confusion matrix are provided. This encompasses true positives (TP = 222), true negatives (TN = 441), false positives (FP = 2), and false negatives (FN = 7) Figure 3.

Figure 3. Confusion matrix.

Table 4. FPCA and PCA technique performed on original dataset.

Method	FPCA	PCA
Feature ranking	Insulin(0)+Insulin(120) $\ge$300	Insulin(0)+Insulin(120) $\ge$300
	Body Mass Index	High cholesterol
	High cholesterol	Body Mass Index
	Insufficient physical activity	Unhealthy diet
	Unhealthy diet	Insufficient physical activity
	Genetic diseases	Consumption of psychoactive substances
	Consumption of psychoactive substances	Genetic diseases
	Other risk factors	Other risk factors

Table 5. Performance metrics.

Performance measure	Score
Recall	0.9901
Precision	0.9869
Accuracy	0.9867
MCC	0.9702
F1-score	0.9809

Subsequently, we refined our k-means clustering outcomes by eliminating inaccurately grouped data points. Should the augmented dataset exceed 80% of the original size, we would then advance to the realm of supervised classification. However, in instances where this threshold is not met, we would iterate through the k-Means procedure anew, pursuing an optimal dataset size, while ensuring that the first eight principal components (8PC's) maintain a correlation of at least 70%. Illustrated in Figure 4 are two discernible clusters denoting positive and negative outcomes, wherein cluster 1 corresponds to the negative outcome and cluster 2 pertains to the positive outcome.

Figure 4. K-Means clustering for the hyperinsulinemia outcome.

In a pursuit to conduct a more comprehensive evaluation of our proposed experimental model, we extended our analysis by employing three additional machine learning algorithms. This was executed across a spectrum of data variations, encompassing the original dataset, data subjected to PCA processing, data transformed through FPCA, and data subject to both FPCA and K-Means processing. From Tables 6 and 7 in both training and testing phase, it can be seen that FPCA + K-Means techniques improved the accuracy of posed algorithms. Additionally, FPCA alone shows to be a good procedure to improve the accuracy of each of the algorithms, while PCA was slightly reducing the accuracy result when applied alone.

Table 6. Accuracy on processed dataset compared on different ML models (training).

Algorithm	Original dataset	PCA processed	FPCA procesed	FPCA + K-Means proposed dataset
Logistic Regression	0.79	0.81	0.83	0.98
XG Boost	0.82	0.85	0.87	0.99
Random Forest	0.77	0.73	0.79	0.87
Light GBM	0.81	0.84	0.86	0.98

Table 7. Accuracy on processed dataset compared on different ML models (testing).

Algorithm	Original dataset	PCA processed	FPCA procesed	FPCA + K-Means proposed dataset
Logistic Regression	0.78	0.79	0.81	0.97
XG Boost	0.81	0.83	0.86	0.99
Random Forest	0.78	0.75	0.80	0.89
Light GBM	0.79	0.81	0.83	0.97

Using the SHAP feature importance method, it was possible to determine the influence of seven input features and their values in comparison with mean(SHAP value). From Figure 5, it can be concluded that the features that played a crucial role in determining whether a dolescent was classified with a positive or negative outcome were BMI (Body Mass Index) and High cholesterol. Additionally, other notable contributors encompassed Insufficient physical activity and an Unhealthy diet. Moreover, the influence of other risk factors, namely the adolescents’ residential demographic area and their age, ranked lowest among the influencing factors.

Figure 5. SHAP feature importance.

4.3 Results obtained using factorial analysis

Conducting exploratory factorial analysis, we once again validate the most influential risk factors. Within the experimental group exhibiting hyperinsulinemia, the pivotal risk factors included BMI and elevated cholesterol levels, serving as the primary factors responsible for the overall risk associated with hyperinsulinemia development. Subsequently, employing a similar approach through confirmatory factor analysis, we also unveiled an additional four significant factors: Insufficient physical activity, Unhealthy diet, Genetic diseases, and the consumption of psychoactive substances. The amalgamation of all other factors contributed to the cumulative risk linked to hyperinsulinemia with elevated glycemia. These outcomes are presented in Figure 6.

Figure 6. Risk factor importance – factorial analysis.

5. Discussion

The outcomes of the presented research have demonstrated that FPCA enhances the K-Means clustering algorithm, which has been a challenge in prior investigations, and this challenge is addressed through the proposed data mining model. A dataset containing information from 1008 adolescents was divided into two subsets: a training set with 672 adolescents and a testing set with 336 adolescents. The proposed approach was applied to both sets (Table 1).

In previous studies, the best result was achieved in (Zhu et al., 2019), where an accuracy of 97.4% on a sample of 678 subjects for diabetes detection and identification was attained. Employing PCA and K-Means to enhance logistic regression accuracy, the authors attained peak performance through the Pima Indian diabetes dataset. In (Wu et al., 2018), the amalgamation of K-Means and Logistic Regression produced a model accuracy of 95.4% among 589 subjects. Comparable research undertakings have revealed diminished precision and model reliability, constituting a primary hurdle in the domain of medical algorithm applications.

The FPCA technique proposed in our study effectively addresses the fundamental issue of prediction model accuracy. When combined with K-Means, it precisely classified instances. The medical dataset containing 1008 instances of adolescent data achieved a groundbreaking accuracy and reliability of 98.67% through the utilisation of the XG Boost algorithm. Among the experimental group, 33.3% of adolescents with hyperinsulinemia were identified, while 66.7% showed no presence of hyperinsulinemia. Employing the SHAP post-agnostic method, BMI was identified as the most influential risk factor present in all adolescents, followed by six other significant factors. A comparison with factor analysis revealed an identical prioritisation of risk factors as in the FPCA technique, though the model's accuracy was slightly lower at 95%. The results obtained from this research exhibit greater reliability and precision of the novel proposed model compared to previously introduced models and techniques. These insights will be robustly employed to establish frameworks and tailor healthcare policies in Serbia, yielding practical applications that resonate effectively within the medical landscape.

6. Conclusion

The main objective of our paper was to propose the data mining model and improve the accuracy of the supervised models that are frequently used in medical diagnostics.. The experimental segment of this study, which involved the implementation of unsupervised techniques for dimensionality reduction such as PCA, FPCA, along with K-Means clustering, aimed to enhance supervised models and address our primary objective:

“To what extent can unsupervised techniques such as PCA and FPCA improve K-Means clustering and provide foundation for input values of Random Forest, XGBoost and LightGBM contributing to higher classifiers accuracies?”.

It was found that the XGBoost model outperforms the baseline model along with Random Forest and LightGBM. While PCA is a widely recognised technique, its potential for enhancing K-Means clustering and, consequently, the performance of the classifiers, has not garnered sufficient attention. Our experiment effectively demonstrates that logistic regression and LightGBM models, even when having slightly lower accuracy (around 0.02%) could be promising predictors for hyperinsulinemia through the integration of FPCA and K-Means. The study's contribution lies in its capacity to attain significantly improved K-Means clustering outcomes, surpassing what other researchers have accomplished in similar investigations. Additionally, the XGBoost model exhibits enhanced predictive capabilities for hyperinsulinemia onset when contrasted with the outcomes of employing other algorithms, both within our study and in comparison to similar studies conducted by others. Another notable advantage is the model's proficiency in successfully handling new datasets. Utilising post-agnostic methods like SHAP, interpretability is established within the proposed models. This advancement aids medical professionals and healthcare experts in gaining a clearer comprehension of the outcomes when utilising such a diagnostic tool.

Finally, the outcomes of this research hold significance for the domain of hyperinsulinemia prediction across diverse industries. Detecting hyperinsulinemia early on can prove advantageous when the model is effectively integrated with healthcare policies. This integration facilitates the automatic initiation of precautionary measures in the event of an anticipated hyperinsulinemia risk. Leveraging this integration, the timely prediction of hyperinsulinemia can be ensured, thereby mitigating the adverse effects associated with its onset.

Finally, the insights gained from our experiment transcend their initial application in diagnosing hyperinsulinemia, offering valuable implications for a broader range of medical diagnostics. These insights stem from an extensive experimental setup that integrates various unsupervised and supervised learning techniques, alongside a selected hold-out approach for validation. Our dataset comprises authentic values linked to a wide array of indicators from significant laboratory analyses and specialist physician reviews. Furthermore, the interpretability of these models has been achieved by incorporating SHAP with XGBoost, which allows for an understanding of both the overarching patterns and the specifics underpinning individual predictions. Hence, we contend that an additional comparison would not contribute further value, given our thorough global and local analytical coverage.

6.1 Limitations and future directions

When exploring the datasets gathered on a national scale to predict hyperinsulinemia, it is important to consider the limitations associated with missing data. Although this study didn't encounter significant issues due to a small number of missing data points, the introduction of absent values can potentially introduce biases and hinder the precision of predictive models, particularly when dealing with a larger number of instances. Incomplete records arising from missing data can impede the thorough analysis and prediction process, given that these models rely on complete data to achieve accurate estimations. Moreover, the absence of data can result in diminished sample sizes, thereby compromising the statistical power of predictive models. When dealing with a smaller sample size, models may face challenges in detecting significant associations and patterns related to hyperinsulinemia diagnostics, consequently limiting the degree to which their findings can be extrapolated to the broader population. Furthermore, the task of imputing missing values presents challenges, as the chosen imputation methods can introduce assumptions and uncertainties that impact the accuracy and reliability of the projected outcomes.

The application of Shapley values to hyperinsulinemia diagnostics encounters a significant challenge owing to computational complexity, particularly in modern models like deep neural networks with high-dimensional inputs. This intricacy renders the exact computation of Shapley values impractical. Nonetheless, specialised implementations of SHAP are available for tree-based methods (e.g. Random Forest, XGBoost) and additive models, offering viable alternatives. Caution must be exercised, especially when considering particular versions of SHAP for estimation, such as KernelSHAP, which, while effective, can be computationally slow. Additionally, when features are interdependent, using Shapley values might lead to extrapolation into areas with low data density. To mitigate this, conditional versions have been developed, primarily tailored for tree-based models, but necessitating careful interpretation to avoid common pitfalls. Importantly, SHAP explanations do not yield sparse outcomes, as a non-zero Shapley value is attributed to each feature influencing the prediction, irrespective of its magnitude. If sparse explanations are preferred, alternative approaches like counterfactual explanations might prove more suitable (Su et al., 2023).

Given these limitations, future research in the realm of predicting hyperinsulinemia diagnostics using diverse datasets should delve into exploring deep learning models. Deep learning models, such as recurrent neural networks (RNNs), have demonstrated promise in handling missing data and extracting intricate patterns from extensive datasets. Particularly, the use of RNNs, like Fuzzy Cognitive Maps, could unearth hidden patterns within the data and facilitate various WHAT-IF simulations to determine important concepts. Harnessing the potential of deep learning models (Abunadi, 2022; Hu et al., 2022), researchers can potentially address the challenges posed by missing data and enhance the precision, dependability, and general applicability of predictive models for hyperinsulinemia diagnostics.

Institutional review board statement

The study was conducted in accordance with the Declaration of Helsinki, and approved with Protocol code: DZ-01-2294 issued on 02/07/2019 by the Healthcare Center in Serbia, and Protocol code: DZ-01-2646/1 issued on 11/08/2021.

Informed consent statement

Informed consent was obtained from all subjects involved in the study.

Acknowledgements

The authors would like to thank freepik.com website (accessible through the following link: https://www.freepik.com/) for providing free license for parts of the Figure 1. Hyperinsulinemia process. Muscle picture as a part of Figure 1. was designed by brgfx/Freepik. Pancreas picture as a part of Figure 1. was designed by macrovector/Freepik. Liver picture as a part of Figure 1. was designed by Freepik. Circulation or blood stream picture as a part of Figure 1. was designed by Freepik. Adipose tissue picture as a part of Figure 1 was designed by Freepik.

Data and code availability statement

The raw dataset used for this study is under a Non-Disclosure Agreement (NDA) and is therefore not available to the public. The code for the presented data mining model could be available upon reasonable request.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.