Interpretable machine learning for mortality modeling on patients with chronic diseases considering the COVID-19 pandemic in a region of Chile: A Shapley value based approach

Abstract

The main objective of this study was to evaluate and interpret different machine learning models to predict the probability of mortality from chronic oncological diseases (COD), chronic non-oncological diseases (CNOD), and COVID-19 in the Biobío Region, Chile from 2016 to 2022. In this study, the causes of death attributed to COD and CNOD were recognized as conditions that would have necessitated palliative care. Retrospective cohort study of mortality data from the Chilean Ministry of Health. A total of 57,623 mortality records due to chronic diseases were considered during the study years in the Biobío Region. Data characteristics included sociodemographic factors (age, gender, residence, place and date of death) and causes of death. Seven classification models were trained: Multinomial Regression, Random Forest, Decision Tree, Support Vector Machine, Naives Bayes, XGBoost and Neural Networks, to predict the probability of mortality from COD, CNOD and COVID-19 and the calibration, discrimination and accuracy of these models. Additionally, the Shapley Additive Explanations (SHAP) values were used to assess the interpretability of the models and the BorutaShap algorithm was used for variable selection. The XGBoost, Random Forest and Multinomial Regression models had the best prediction performances. In all prediction cases, XGBoost had a slight advantage over Random Forest and Multinomial Regression models, with an average global AUROC for repeated cross-validation of 0.624, 0.600, and 0.627, respectively. In addition, a global average Accuracy in favor of XGBoost of 0.642 compared to 0.641 and 0.633 of the models mentioned above. The variables selected by the BorutaShap method were age, place and date of death. XGBoost can be used to predict the probability of death from COD, CNOD and COVID-19 in mortality data from the Biobío Region. This model can be useful for allocating palliative care resources more effectively to the people who require it. Among the relevant variables for the prediction were the Date of death, Place of death and the Age.

KEYWORDS:

• COVID-19
• chronic diseases
• interpretability
• machine learning
• mortality
• palliative care
• Shapley value

1 Introduction

Worldwide, the population is progressively aging and chronic non-communicable diseases (CND) are the main cause of mortality (World Health Organization 2020). Each year 41 million people die from CND, representing 71% of mortality in the world. These diseases greatly affect people living in low- and middle-income countries, where more than three quarters of deaths (32 million) from CND occur. They are often associated with older age groups, but evidence shows that 15 million of all deaths attributed to CNDs occur between the ages of 30 and 69 (World Health Organization 2021a).

The World Health Organization (WHO) World Health Statistics Report (World Health Organization 2020), noted that CNDs account for 7 of the top 10 causes of death globally, revealing an exponential increase in their impact over the period from 2000 to 2019. The leading causes of death worldwide from CND in 2019 were cardiovascular diseases (17.9 million deaths, representing 44% of all CND deaths), cancers (9.3 million, 23% of all CND deaths), respiratory diseases (4.1 million, 10% of all CND deaths) and diabetes (1.5 million deaths, 4% of all CND deaths) (World Health Organization 2021b).

In the Region of the Americas, the main cause of death of the population is also associated with CND, deeply impacting the well-being of people and their families and at the same time generating a great social and economic burden (Sánchez-Herrera et al. 2013; de la Salud 2017). The latest epidemiological data in Chile indicated that the total number of deaths registered in the country increased by 19.2% in the last 11 years, from 91,965 in 2009 to 109,658 in 2019; the main groups of causes of death were tumors or neoplasms with 28,492 deaths (26%), followed by diseases of the circulatory system with 28,079 deaths (25.6%), diseases of the respiratory system with 13,864 deaths (12.6%), other external causes of morbidity and mortality with 8,065 deaths (7.4%), and diseases of the digestive system with 7,996 deaths (7.3%) (INE 2018). Based on these statistics, we inferred that chronic non-oncological diseases (CNOD) represent a large percentage of deaths compared to deaths from chronic oncological diseases (COD) that occurred in that period. This results in greater projected requirements for general and palliative care (PC henceforth), due to the fact that chronic diseases in an advanced state present different degrees of complexity and needs, and at the same time, is associated with a limited life expectancy and that will evolve progressively in the medium term toward the end of life (Batiste-Alentorn et al. 2017; Meléndez and Limón 2018).

The provision of care for COD in Chile is incorporated in the Pain Relief and Palliative Care Program (Pastrana 2012, 2021). The recent Law 21375 published (Ministerio de Salud 2021) in October 2021 is expected to allow the gradual incorporation of PC provision for the CNOD. The health context generated by the COVID-19 pandemic cannot be ignored, because it pushed the provision of health care for people with COD and CNOD to the background. It is unknown how this completely unforeseen health situation impacted the detection of chronic diseases that would have required PC and their mortality; therefore, it is important to analyze the predictive behavior of these groups (COD and CNOD) in the pre-pandemic and pandemic period of COVID-19.

Chow et al. (2001) and Han et al. (2012) pointed out that mortality prediction based on health professional judgment and/or experience is inaccurate and often optimistic, meaning that there is a tendency to overestimate patient survival in daily clinical practice. Therefore, to address this situation, various authors have proposed comparative studies between different machine learning (ML) statistical techniques and logistic regression techniques (Cao et al. 2019; Desai 2020). In particular, Cary Jr et al. (2021) used logistic regression and a multilayer perceptron (MLP), to predict the probability of mortality at 30 days and 1 year for Medicare program beneficiaries older than 65 years treated for hip fracture in rehabilitation centers. These authors concluded that MLP models can reduce cognitive load and increase the ability to calibrate local data, leading to clinical specificity in mortality prediction so that PC resources can be allocated more effectively. A similar conclusion was obtained by Avati et al. (2018) who validated a hybrid model of machine learning versus deep learning, to classify patients into two categories of results, Intensive Care Unit (ICU) and non-ICU.

To ensure the interpretability of the models, the Lundberg and Lee (2017) proposal was considered, which proposes Shapley Additive Explanations (SHAP) values, which are calculated using the idea of the Shapley value (Hart 1989). Finally, to perform the reduction of the number of explanatory variables was applied BorutaShap (Keany 2020). This methodology has been recently used by different authors including the following: Chieregato et al. (2022) proposes a hybrid model of machine learning/deep learning for COVID-19 forecasting using a CatBoost model and the Boruta algorithm with the importance of the SHAP feature as a metric. Smith and Alvarez (2021) proposes interpretability of the ML model using the Shapley values to predict the mortality of hospitalized patients with COVID-19 in a hospital of the region of Wuhan, China. Unlike these studies, this research has proposed evaluating ML models in the classification of COD, CNOD, and COVID-19 in order to identify the indirect need for PC and, in turn, guide the planning of the necessary resources for this type of care that has recently been implemented in our country.

The purpose of this study was to evaluate and interpret the different ML models designed to predict mortality in COD, CNOD and COVID-19 with chronic disease mortality data in people of all ages provided by the Department of Statistics and Information of Health (DEIS) of the Ministry of Health of Chile (Departamento de Estadíticas e Informaci’́on de Salud (DEIS) (2022)). This is the first study in Chile to propose the ML methodology for predicting mortality in people who died according to causes of the ICD-10 classification (International Statistical Classification of Diseases and Related Health Problems-10th Revision), used to indicate mortality in the population that would have required PC in the pre-pandemic and pandemic periods Batiste-Alentorn et al. (2017).

2 Methodology

A retrospective cohort study ((Acar et al. 2021)) was conducted to identify the ML models that exhibited the highest predictive power. The study utilized mortality data, which are free access and anonymous data obtained from the web page https://deis.minsal.cl//#datosabiertos Departamento de Estadíticas e Informaci’́on de Salud (DEIS) (2022). The data were deaths due to chronic diseases that would have required PC according to the ICD-10 Code from 2016 to 2022. This database consists of 57,623 chronic disease mortality records with 7 variables. The variable “Age group” was created from the variable “Age in years” according to the classification used by Ministerio de Salud GdC (2019). The dependent variable was the basic cause of death according to the ICD 10 classification (Table 2) and the other 6 variables corresponded to independent variables defined in Table 1.

Table 1 Independent Variable Records

Variable	Description	Values
Sex	Identifies biological sex	1: Man
		2: Woman
		9: Indeterminate
Age	Unit of measure of age	1: age in years
		2: age in months
		3: age in days
		4: age in hours
		0: Ignored
Place of death	Place where death occurs	1: Hospital or Clinic.
		2: House room.
		3: Other.
Date of death	Date of death	DD/MM/AA
Residency	Commune of residence deceased	16 Comunas
	Childhood	0: 0–9 year.
	Adolescence	1: 10–19 year.
Age group	Youth	2: 20–24 year.
	Adulthood	3: 25–64 year.
	Older Adulthood	4: +65 year

Table 2 ICD-10 codes for conditions identified as potentially having palliative care needs.

Underlying cause of death	ICD-10 codes
Malignant neoplasm	C00-C97
Heart & cerebrovascular disease	I00–I52 (excluding I12 and I13.1), I60–I69
Renal disease	N17, N18, N28, I12, I13.1
Liver disease	K70-K77
Respiratory disease	J06–J18, J20–J22, J40–J47, J96
Neurodegenerative disease	G10, G20, G35, G122, G903, G231
Alzheimer’s, dementia and senility	F01, F03, G30, R54
HIV/AIDS	B20–B24
COVID-19	U07

The dependent variable used in this study was basic cause of death, of which the records with chronic diagnoses of the Biobío Region were considered. For this classification, the proposal of Calvache et al. (2020) was followed, using the ICD-10 codes to identify deaths from COD, CNOD and COVID-19, respectively. Table 2 shows a description of the cause of death and ICD-10 codes used to classify deceased persons who would have required PC. From this classification, the dependent variable is defined as follows: $$Y=\{\begin{array}{cc}0,& \text{If person dies for COD }ó\text{C00-C96}\\ 1,& \text{ If person dies from CNOD, all other codes}\\ & \mathrm{\hspace{1em}}\mathrm{\hspace{1em}}\text{ excluding C00-C96 and }\\ 2,& \text{ If person dies from COVID-19 or U07}.\end{array}$$

The study population was randomly divided into a training cohort, in which the mortality risk algorithms were developed, and a validation cohort, in which the algorithms were applied and tested. The training cohort consisted of 70% of the data cohort, and the validation cohort consisted of the remaining 30%. It was randomly divided in our cohort in the records of deceased people so that there was no duplication between the training and validation sets. In this study, we compared the performance of models based on the accuracy, the ROC, and balanced accuracy, as well as the sensitivity, specificity, positive predicted value (PPV), and negative predicted value (NPV). Readers can refer to the definitions of each of these calibration measures in the Cary Jr et al. (2021) study. A Monte Carlo study consisting of 100 iterations was conducted to assess the precision level of each model using the configuration outlined at the beginning of this paragraph.

2.1 Models

The ML approach is the study of algorithms capable of improving or learning automatically through experience and the use of data (Mitchell 1997). The most common approaches to machine learning are supervised and unsupervised. In supervised learning, the algorithm can generalize or learn based on labeled data, either for (Hastie et al. 2001) classification or regression problems. On the other hand, unsupervised learning attempts to extract features and patterns from unlabeled (Hastie et al. 2001) data. ML techniques have been noted for their computational efficiency and for achieving better prediction results than those obtained with commonly used statistical techniques (Shin et al. 2021; Lang et al. 2022). Currently, ML is used in a variety of applications, such as finance, robotics, pattern recognition, natural language processing, medical diagnostics, recommendation algorithms for social networks, prediction of people’s mode of travel, among many others (Ray 2019). In this study, ML models were used to predict mortality and the need for PC in people who died of chronic diseases in the Biobío Region from 2016 to 2022. In particular, ML techniques were used, such as Artificial Neural Networks (ANN), Support Vector Machine (SVM), Decision Trees (DT), Random Forest (RF), Naives Bayes, XGBoost; and the predictions of these models were compared with the Multinomial Regression model. The models used are described below.

• Multinomial Regression Model: This model is also known as the with polytomous response, which is a generalization of the regression model logistic binomial (McCullagh and Nelder 1989) in which we want to estimate the probability whether or not the individual experiences a specific event, given a set of variables that explain the specific characteristics of individuals. In the case of the multinomial model, the endogenous variable has more than two alternatives to consider as possible answers (Fernández and Fernández 2004).

• Decision tree: It is a prediction model used in the field of artificial intelligence, whose objective is to create a model that predicts the value or class of an observation from decision rules obtained from previous data, so this technique can be used to predict categorical and/or continuous variables (classification and regression trees) (Breiman et al. 1994). The decision tree model is built from the description narrative of a problem, since it provides a graphic view of decision making, where the structure begins with a node called “root node”; then in this and in the internal nodes decisions are made based on different attributes; subsequently, the branches indicate the decisions made. At the end of the decision tree are the terminal nodes, which represent the result of following a combination of decisions. Alternatively, the terminal node may be associated with the probability that the final result will take on a certain value.

• Random Forest: It is a ML model of supervised learning for classification, which has the particularity of being an algorithm that works using different models that, as a result, are combined to obtain a model of the entire set (Hidalgo Ruiz-Capillas 2014). The method used in Random Forest to generate different models from the data is known as boostrap aggregatting, or bagging, this method generates a series of training dataset from the original data using bootstrap sampling, to later be used when training a single model with each of the training data.

• Support Vector Machine: They are a supervised classification method whose objective is to determine the optimal border between two groups and can be extended to a larger number. This method is used for classification or regression problems (depending on the response variable), but it is often used for classification. Given 2 or more labeled data classes, it acts as a discriminative classifier, formally defined by an optimal hyperplane that separates all classes (Vargas et al. 2012).

• Naive Bayes: It is a statistical classifier that predicts the class of membership based on probabilities. The probability of occurrence, given that it belongs to a particular class, assumes that the effects of one attribute are independent of the values of the other attributes (Chen et al. 2020). This classifier has been identified as a suitable classifier in the medical field, given the advantages of its high accuracy when applied to large databases and is especially useful, for example, in medical diagnoses. Other advantages include lower computational cost, easy to understand as it is a probability-based classifier, and requires a single processing step if the data is discrete (Hariz 2012).

• XGBoost (eXtreme Gradient Boosting): This model emerges as an efficient implementation of the Gradient Boosting method, developed by (Chen et al. 2016). Among the main features, the inclusion of penalty terms to avoid overfitting, proportional reduction of tree leaves, Newton-Raphson method for loss function and efficient implementation for multiprocessor training stand out.

2.2 BorutaShap through XGBoost

When ML techniques are used in decision-making processes, the interpretability of the models becomes important. In this context, Lundberg and Lee (2017) proposes the Shapley Additive Explanations (SHAP) methodology, which is based on cooperative game theory. The main idea is to calculate the Shapley values for the model, which give a fair allocation of gains between the variables according to their contribution to the prediction of the target variable. In this way, SHAP values can be used as a measure of variable importance. Once SHAP is defined, we propose using the BorutaShap introduced by Effrosynidis and Arampatzis (2021), which is a variable selection method that seeks to obtain a reduced subset of these, being discriminated from the SHAP value with the Boruta algorithm (Kursa and Rudnicki 2010). Having the subset of important variables, an interpretability analysis of the XGBoost model will be carried out using SHAP, in order to consistently know to what extent and magnitude the variables contribute to predicting the target variable.

2.3 SHAP summary plot

To represent the impact of the variables under study, a SHAP summary diagram (Lundberg et al. 2018) was used, which shows the importance of the function and a summary of the SHAP dependency diagrams. In the chart, the features are ordered by their importance. Each row graph is a summary SHAP dependency graph of each variable. Each point represents the SHAP value of an individual. The color of each point denotes the value of the function, from low (blue) to high (red). Black dots represent missing values. If the red points are plotted on the lower side and the blue points are plotted on the upper side, then the risk increases as the value increases. Since a SHAP summary plot shows the importance of feature values and a SHAP summary dependency plot, it is useful to get an overview of the SHAP analysis.

2.4 Preprocessing

Data preprocessing was initially conducted through coding for the response variable and categorizing the Age group variable. Resampling methods were not used, therefore the models were applied to the original imbalanced data. We used R version 4.2.2 (2022-10-31) as statistical analysis tool and packages such as: multinom, randomForest, ctree, svm, naivebayes, and nnet to build models, multinomial, RF, DT models, SVM, Naives Bayes, and ANN, respectively. For RF, the number of variables randomly sampled as candidates at each split was set to $\sqrt{6}$, and the number of trees to grow was set to 40. For SVM, “C-classification” and radial kernel were used, in the case of DT, the maximum depth of the tree is by default, i.e., maxdepth = Inf. Furthermore, we have implemented XGBoost using Python and leveraged SHAP values to compute the feature importance. The source codes for ANN, XGBoost and Naive Bayes algorithms contain comprehensive details, and they can be obtained by contacting the corresponding author or by referring to the data availability statement in the data set section.

3 Descriptive analysis of the results

3.1 Sample characteristics

The sample of mortality records due to chronic diseases was 57,623 people residing in the Bío Bío Region between the period 2016 to 2022. In general, the average age was 75 years; 51% of these records were men, 50% of the deaths occurred at home. Of the people who died from COD, 66% died at home and were over 65 years old (73%). Of the people who died from CNOD, 51% of these deaths occurred in hospitals or clinics and the patients were significantly older. See Table 3.

Table 3 Characteristics of the mortality data in the Biobío region for the period 2016–2022.

Characteristics	Overall	COD	CNOD	COVID-19	p-value*
	N = 57.623	n = 18.187	n = 34.924	n = 4.512
Sex					0.037
Man	29584(51.3%)	9426(51.8%)	17741(50.8%)	2417(53.6%)
Woman	28039(48.7%)	8761(48.2%)	17183(49.2%)	2095(46.4%)
Place of death					< 0.001
house room	28713(49.8%)	12023(66.1%)	15757(45.1%)	933(20.7%)
Hospital or Clinic	27262(47.3%)	5963(32.8%)	17805(51.0%)	3494(77.4%)
Other	1648(2.9%)	201(1.1%)	1362(3.9%)	85(1.9%)
Age group					< 0.001
Childhood	73(0.13%)	36(0.20%)	32(0.09%)	5(0.11%)
Adolescence	113(0.20%)	58(0.32%)	45(0.13%)	10(0.22%)
Adulthood	11505(20%)	4695(25.8%)	5786(16.6%)	1024(22.7%)
Older adulthood	45831(79.5%)	13349(73.4%)	29017(83.1%)	3465(76.8%)

p-value by comparing the differences between patient characteristics of “COD” and “CNOD” groups.

3.2 Model performance

This section summarizes the classification efficiency metrics of the previously defined models for the COD and CNOD classes in the context of the COVID-19 pandemic. To obtain an overview of the predictive performance of the models, a Monte Carlo study was conducted with a total of 100 iterations. In each iteration, the original sample was randomly divided into a training sample, which consisted of approximately 70% of the original sample size, and a test sample containing the remaining observations. Consequently, the performance of each method was assessed using Monte Carlo cross-validation, utilizing the confusion matrix. Table 4 displays the average and standard deviation of the cross-validation metrics obtained from the confusion matrices. These metrics were calculated from 100 iterations for each of the models proposed in this study.

Table 4 Classification performance table (average and standard deviation of the confusion matrix computed across 100 iterations).

Method	Class	Accuracy	AUROC	Sensitivity	Specificity	PPV	NPV	Balanced (SD)
		(SD)	(SD)	(SD)	(SD)	(SD)	(SD)	Accuracy
Multinomial regression model	0			0.382(0.006)	0.865(0.003)	0.567(0.006)	0.752(0.002)	0.624(0.003)
	1	0.641(0.003)	0.627(0.004)	0.842(0.004)	0.359(0.006)	0.669(0.002)	0.596(0.006)	0.600(0.003)
	2			0.130(0.013)	0.985(0.002)	0.419(0.019)	0.930(0.001)	0.557(0.006)
RF	0			0.370(0.012)	0.868(0.006)	0.564(0.006)	0.749(0.003)	0.619(0.004)
	1	0.639(0.002)	0.613(0.006)	0.850(0.008)	0.341(0.011)	0.665(0.002)	0.597(0.007)	0.596(0.003)
	2			0.095(0.020)	0.988(0.003)	0.406(0.026)	0.928(0.001)	0.541(0.009)
DT	0			0.373(0.049)	0.865(0.023)	0.563(0.012)	0.750(0.010)	0.619(0.013)
	1	0.639(0.002)	0.600(0.024)	0.855(0.029)	0.327(0.047)	0.662(0.008)	0.597(0.017)	0.591(0.010)
	2			0.088(0.112)	0.988(0.015)	0.258(0.172)	0.927(0.007)	0.538(0.049)
SVM	0			0.418(0.006)	0.847(0.002)	0.557(0.005)	0.759(0.002)	0.632(0.003)
	1	0.641(0.002)	0.608(0.003)	0.839(0.003)	0.353(0.005)	0.666(0.002)	0.588(0.005)	0.596(0.003)
	2			0.000(0.000)	1.000(0.000)	NaN	0.922(0.000)	0.500(0.000)
Naives Bayes	0			0.357(0.008)	0.862(0.006)	0.545(0.009)	0.744(0.002)	0.609(0.004)
	1	0.632(0.004)	0.629(0.005)	0.831(0.007)	0.361(0.007)	0.667(0.002)	0.581(0.008)	0.596(0.003)
	2			0.198(0.009)	0.976(0.001)	0.409(0.016)	0.935(0.001)	0.587(0.004)
XGBoost	0			0.389(0.020)	0.862(0.011)	0.566(0.009)	0.754(0.004)	0.625(0.005)
	1	0.642(0.003)	0.624(0.007)	0.843(0.013)	0.359(0.019)	0.669(0.004)	0.598(0.008)	0.601(0.004)
	2			0.109(0.019)	0.988(0.002)	0.426(0.018)	0.929(0.001)	0.548(0.009)
ANN	0			0.324(0.096)	0.884(0.037)	0.555(0.069)	0.740(0.018)	0.604(0.030)
	1	0.633(0.010)	0.600(0.028)	0.872(0.041)	0.283(0.089)	0.653(0.016)	0.580(0.069)	0.577(0.024)
	2			0.067(0.057)	0.991(0.007)	0.343(0.112)	0.926(0.004)	0.529(0.025)

From Table 4 it can be seen that favorable metrics have been obtained in the XGBoost, Random Forest and Multinomial Regression algorithms, the most favorable being the XGBoost for having a slightly higher predictive power with respect to the other algorithms. In particular, the accuracy, AUC and balanced accuracy metrics favor this algorithm. Therefore, it can be pointed out that with the data used, a greater advantage is identified in the use of the XGBoost method for the modeling of COD and CNOD mortality in the COVID-19 pandemic context. Moreover, the presence of NaN values in the SVM model’s PPV indicates the model’s challenges in accurately classifying instances as positive for the overlapping class Y = 2. This observation supports the findings of multiple authors, with specific emphasis on those highlighted by Vuttipittayamongkol et al. (2021), which emphasize the detrimental effects of class imbalance and class overlap on SVM performance. Based on the classification results for the Y = 2 class, it is evident that the performance of the ML models exhibits a significant degradation under such circumstances.

3.3 Variable selection using the BorutaShap algorithm

The most relevant variables for the classification of COD, CNOD and COVID-19 were analyzed. This was done using the BorutaShap algorithm. From the analysis, it was observed that the selected variables were 3 of the 6 under study, namely; Date of death, Place of death and Age.

Figure 1 represents the average of the SHAP values for the variables selected by the algorithm and for each class (colors). The interpretation is that the higher the average SHAP values are, the greater the contribution of that variable in predicting each class. We also observed that for the COVID-19 class, on average, the variable that contributed the most was Date of death, which is explained by the fact that the pandemic has developed in the last 2 years. On the other hand, for the COD class, the most influential variables were Place of death and Age. Finally, for CNOD, the most influential variable was Age.

Fig. 1. Feature Importance Plot: the 3 most important variables are listed and ordered by the gain method. The Date of death variable is the most important characteristic of the predictor using the gain method.

Figure 2 represents the SHAP values of each variable for the COD class. From this Figure, it is observed that the older the age, less impact the model will have in predicting the COD class. This is due to the higher prevalence of comorbidities in older people, making it more difficult for the model to distinguish between the COD and CNOD class. On the other hand, for the variable Place of death, the lower the values of the variable, the greater the contribution to predict the COD class; This is because, in general, the majority of people with oncological diagnoses die at home. In the opposite case, the higher the values of the variable Place of death, the lower the contribution of the model to predict the COD class, since the records of admissions to hospitals or clinics are not necessarily for this cause, but due to complications of this. Figure 3 shows the Shap values of each variable, only considering the CNOD class. It can be seen that the older the person, the greater the shap value and therefore the contribution to the model. At the same time, the younger the person, the less contribution they will have to the model’s prediction. It is observed that in the variable Place of death values greater than 0 of the function (1 and 2) have a greater contribution to predict the CNOD class.

Fig. 2. SHAP Summary Plot: The graph shows the 3 most important variables evaluated by the SHAP method and the effects of each characteristic on the COD classification.

Fig. 3. SHAP Summary Plot: The graph shows the 3 most important variables evaluated by the SHAP method and the effects of each characteristic on the CNOD classification.

Finally, Figure 4 represents the SHAP value of each variable, considering only the COVID-19 class. From the figure, it is interesting to highlight the variable Date of death, which has a very strong contribution but in a negative direction. For low values (first years after 2016), the contribution of this variable is low, while recent years (high values of the variable) contribute positively to predicting death from COVID-19. On the other hand, it is observed in the variable Age that the greater the magnitude, the greater the contribution of the variable to predict the COVID-19 class.

Fig. 4. SHAP Summary Plot: The graph shows the 3 most important variables evaluated by the SHAP method and the effects of each characteristic on the classification of COVID-19.

4 Discussion and conclusion

In this study, different models were proposed for the classification of mortality from chronic diseases in the pre-post pandemic period: Multinomial Regression, Random Forest, Decision Trees, Support Vector Machine, Naive Bayes, XGBoost and Neural Networks. Models with various performance measures were compared to capture their predictive and classification capabilities, with the XGBoost statistical technique obtaining a better result. On the other hand, the SHAP method was used to interpret the XGBoost model built with mortality data from chronic diseases in people of all ages in the Bío Bío region from 2016 to 2022 and SHAP summary figures were obtained as Figure of importance of the variables. For the selection of variables, the BorutaSHAP method was proposed, and from this analysis it can be mentioned that the most significant variables to predict or classify mortality are Date of death, Place of death and Age.

With respect to the SHAP value of each class, we concluded that in the case of COD, the variable Age has a smaller effect on the classification if people die at advanced ages, the Place of death does not have a significant effect on the power of classification, and the variable Date of death reflects that the higher the values, the lower the power of classification.

For the CNOD case, the higher Age the variable has a higher shap value and a greater contribution to the model, this means that the model contributes more significantly to predicts in older people; the variable Place of death has a greater impact to predict the CNOD class with place of death in hospitals or clinics. Finally, for the classification of mortality by COVID-19, we found that the variable Date of death has a very strong contribution only in the years of the pandemic (2020–2021). The variable Age has a greater effect in predicting the COVID-19 class only in deaths for older ages. It can be concluded that XGBoost techniques are useful for interpreting machine learning models and can describe the underlying relationships between features and results.

In the empirical evidence, there are no studies that have considered the pandemic context to predict the probability of mortality in this specific population of people with chronic diseases. Therefore, this study would be novel and a first approximation to discover the characteristics of this study population using ML models. Other authors have used ML models considering other population groups and without the pandemic context, Hsu et al. (2021), Cary Jr et al. (2021), among others.

These results indicate that the importance of the need for PC is determined by the place of death and age. The analysis of these results shows that almost half of the people with chronic diseases die at home, and therefore it is up to primary health care in Chile to provide the necessary PC. Considering that the new legislation aims to improve care at the end of life by expanding the spectrum of care to all chronic diseases, it is relevant that public policies in this area consider greater resources in primary health care. Currently in Chile, PC in primary care is delivered by general teams with a low level of specialization Departamento de Estadíticas e Informaci’́on de Salud (DEIS) (2022). Therefore, a greater focus on the preparation of exclusive and specialized teams in PC is required.

The prediction of mortality from COVID-19 shows that the most relevant variable is age, which implies that older people are the most vulnerable to this disease. This is also explained because older people live with more than one chronic disease at the same time, making them more susceptible to serious illness (Han et al. 2012). The foregoing reinforces with greater reason that the focus of PC resources should be especially on the elderly and through home care, thus ensuring a better quality of life at the end of life.

During the pandemic, Chile implemented strict quarantine measures and health services focused mainly on the care of patients with COVID-19 and very urgent cases related to other diseases. This context may have had an impact on the results related to place of death in different years. In addition, considering that, in Chile, the Palliative Care Program for patients in advanced stages of oncological diseases provides home care, the place of death becomes particularly significant for class 1 (person dies from CNOD). It is important to note that some geographic areas in the Biobío Region in Chile face challenges in accessing health centers, which may affect mortality rates from certain diseases, including COD and CNOD, which may affect a higher number of deaths that occur in the home. In Chile, Law 21375 was recently enacted, which includes PC for non-oncological chronic diseases. However, it is still in the early stages of implementation, and its impact on mortality and end-of-life care outcomes are still unknown.

4.1 Study limitations

Our study had some limitations. First; The data were obtained from an open database provided by the DEIS, which is a public institution in charge of health statistics. Mortality data obtained from https://deis.minsal.cl//#datosabiertos is the official and unified system for collecting and analyzing health data in Chile and has been effective for over a decade. However, it is important to mention that the analyzed database has a reduced number of variables, compared to the number of variables that could be used in training with ML models. This is due to a limitation in the entry of records of the sociodemographic variables available in the database by the authorized health authority. Secondly, in light of the obtained results, it is evident that all the models exhibit a weak performance when classifying the category of death due to COVID-19. This can be attributed to the imbalanced nature of the original dataset, where the Y = 2 class is a minority, as well as the overlap that exists between this class and classes 0 and 1. To address this issue, following the recommendations of previous authors (Vuttipittayamongkol et al. 2021), a possible solution is to employ re-sampling techniques to balance the data. Additionally, overlap-based methods, such as the k–Nearest Neighbor (KNN) rule (although it has faced criticism from other researchers (Sun and Wang 2011)) or the Support Vector Data Description (SVDD) algorithm (Tax and Duin 2004), can be considered. It is worth noting that the overlap class has not been mathematically well-characterized (Sun and Wang 2011), and a standardized measure to quantify the degree of overlap has not yet been established. We take it as a challenge for further research.

Disclosure statement

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability statement

The data that support the findings of this study are available in Open Science Framework, OSF at https://doi.org/10.17605/OSF.IO/2UPBW. These data were derived from the following resources available in the public domain: https://deis.minsal.cl//#datosabiertos.