Predicting intensive care need in women with preeclampsia using machine learning – a pilot study

ABSTRACT

Background

Predicting severe preeclampsia with need for intensive care is challenging. To better predict high-risk pregnancies to prevent adverse outcomes such as eclampsia is still an unmet need worldwide. In this study we aimed to develop a prediction model for severe outcomes using routine biomarkers and clinical characteristics.

Methods

We used machine learning models based on data from an intensive care cohort with severe preeclampsia (n=41) and a cohort of preeclampsia controls (n=40) with the objective to find patterns for severe disease not detectable with traditional logistic regression models.

Results

The best model was generated by including the laboratory parameters aspartate aminotransferase (ASAT), uric acid and body mass index (BMI) with a cross-validation accuracy of 0.88 and an area under the curve (AUC) of 0.91. Our model was internally validated on a test-set where the accuracy was lower, 0.82, with an AUC of 0.85.

Conclusion

The clinical routine blood parameters ASAT and uric acid as well as BMI, were the parameters most indicative of severe disease. Aspartate aminotransferase reflects liver involvement, uric acid might be involved in several steps of the pathophysiologic process of preeclampsia, and obesity is a well-known risk factor for development of both severe and non-severe preeclampsia likely involving inflammatory pathways..

KEYWORDS:

• Artificial intelligence
• clinical prediction model
• aspartate aminotransferase
• uric acid
• body mass index

Introduction

Preeclampsia (PE) is a pregnancy-specific disorder affecting 2–5% of all pregnancies and is an important cause of maternal and fetal mortality and morbidity (1,2). In Sweden, 180 mothers a year are treated in an intensive care unit (ICU) because of complications to PE (Swedish Intensive Care Register, SIR).

Specific and sensitive predictive algorithms would be valuable clinical tools to distinguish high-risk preeclampsia patients from patients that would be possible to treat more expectantly. An optimal clinical prediction model should contain patient characteristics, physiological parameters, and/or routine plasma biomarkers that are inexpensive, and simple to obtain even in a low-resource setting. Adding specific biomarkers and complex physiological measurements such as uterine artery Doppler index to the models may improve sensitivity and specificity, but the clinical availability of such analysis is still limited to university clinics in high-income countries.

The Fetal Medicine Foundation (FMF) first trimester prediction model consisting of maternal factors, mean arterial pressure, uterine pulsatility index and serum placental growth factor (PIGF) followed by administration of low-dose aspirin in high-risk patients, has been shown to reduce the incidence of pre-term PE by 62% (3,4). When PE is established, prevention of seizures, treatment of hypertension and timed delivery are the only treatment options available.

Severe forms of PE are however still not possible to predict. Eclampsia is one severe adverse outcome of preeclampsia associated with increased maternal mortality and morbidity. Treatment with magnesium sulfate reduces the occurrence of eclamptic seizures in patients with severe preeclampsia from 2% to less than 0.6% (5).

Severe hypertension is also a feature associated with adverse pregnancy outcomes increasing the risk for complications (6). The thresholds for treatment are trending lower, and the International Society for the Study of Hypertension in Pregnancy (ISSHP) has a therapeutic target of 130/80 mmHg, but national guideline recommendations may differ.

For late-onset preterm PE, planned delivery at 37 gestational weeks reduce maternal mortality and morbidity, without increasing neonatal mortality, but with increasing neonatal unit admissions (7). For early-onset PE, the decision to deliver is per definition associated with higher neonatal mortality and morbidity.

Previous studies on prediction models for maternal adverse events in PE are based on different combinations of biomarkers and/or clinical characteristics (8–13).

Machine learning models are gaining increasing popularity in research in general but also in PE research specifically (14). Preeclampsia is a heterogenous disease of great complexity and machine learning algorithms could make it possible to discover patterns and interactions between data that are not found using traditional multivariable regression models (15). Machine learning has previously been used in several studies for prediction of PE diagnosis as well as monitoring women during pregnancy to ensure perinatal health (16–23). A recent study by Schmidt et al. (14) showed that machine-based learning algorithms can improve prediction of PE associated adverse outcomes in pregnancy.

To the best of our knowledge, machine-based learning models have never been used for predicting severe organ failure postpartum with need of intensive care treatment because of PE.

Objective

The objective of this case-control study was to develop a pilot machine-learning based algorithm to predict severe organ failure in PE.

Methods

Study population

We included 41 women diagnosed with severe PE (PE, eclampsia) and/or HELLP (Hemolysis, Elevated Liver enzymes, Low Platelets) that needed intensive care treatment and 40 women diagnosed with PE but managed at the normal delivery unit as a control group. All women were treated in regional hospitals in Southern Sweden with the same inclusion criteria for ICU-admission.

The intensive care patients were previously enrolled in the Swecrit-study (Blood Samples From Critically Ill Patients and Healthy Controls) between 2015 and 2018 (ClinicalTrials.gov, Identifier: NCT04974775). All ICU-patients were informed about the possibility to have their samples destroyed if not willing to participate in the Swecrit-register. Preeclampsia was defined according the ISSHP guidelines as de novo hypertension after 20 gestational weeks accompanied by proteinuria and/or evidence of any maternal organ dysfunction such as acute kidney injury, liver dysfunction, neurological features, hemolysis, or thrombocytopenia, and/or fetal growth restriction (24). In some cases, PE became manifest intra-partum or early post-partum. Severe preeclampsia was defined as blood pressure >160 mmHg systolic or 110 mmHg diastolic. Early onset PE was considered as severe PE with delivery before 34 weeks’ gestation (25). The HELLP-syndrome, a serious manifestation of PE was also defined as a severe form of PE (24).

Outcome

Outcome was defined as admission to intensive care treatment or not.

Ethical permission

The study was approved by the Regional Ethical Review Board for studies in human subjects at Lund University, Lund, Sweden, Dnr 2019–0468, 2016/49 and for Swecrit 2015/267.

Model development

We included the numerical variables age, body mass index, systolic blood pressure, diastolic blood pressure, as well as routine blood samples platelets, alanine aminotransferase (ALAT), aspartate aminotransferase (ASAT), creatinine, uric acid, and hemoglobin. Due to the small dataset, we excluded the categorical variables severe preeclampsia, smoking, parity, multiple gestation, proteinuria, small for gestational age, induction, previous PE, chronic hypertension, previous kidney disease, diabetes mellitus, gestational diabetes mellitus, gestational hypertension, fetal sex, ethnicity, and family history even though some of them are known risk factors for PE (detailed demographics of the study-groups are shown in Table A1). The reason for this was that the coding of the categorical variables would have generated to many variables for the size of the dataset. Additional feature selection was performed during modeling based on their importance ranking.

To ensure an unbiased model development and independent internal validation, the data was randomly divided into a training-set (80%) and a test-set (20%) in a stratified fashion. Stratification ensured that the distribution of the outcome variable was approximately the same in the training-set and test-set. Hence, the patients in the training-set were not the same as the patients in the test-set for internal validation.

The model development was performed through an iterative process, chosen to identify the subset of features that gave the best predictive performance. In each iteration, the hyperparameters of the XGboost model was optimized. At the end of each iteration, the best model was used to determine feature importance, and the variable reported as least important was removed. Variable importance was calculated using Shapley Additive Explanations (SHAP) (26), which calculates the contribution of individual features to the prediction.

The hyperparameter tuning was automated using Optuna, an open-source hyperparameter optimization framework to automate hyperparameter search (27). Optuna was configured to use the Tree-Structure Parzen Estimator (TPE) to sample the hyperparameters (28). The TPE is a Bayesian optimization technique that intelligently searches for optimal hyperparameters. The hyperparameters and their sample-space are shown in Table A2. The number of trials were set to 500, i.e., for each iteration, 500 sets of hyperparameters are tested. To limit overfitting the models were trained using early stopping. The few missing values present in the data were handled internally by the XGboost model (29).

To evaluate model performance, the training data was split using repeated stratified K-fold cross-validation. This technique is similar to K-Fold cross-validation, where the data is divided into K-folds of equal size, and a model is trained on data from K-1 folds, leaving one to evaluate the model. After training K-models, using a different fold to evaluate the model each time, the cross-validation was complete. However, in repeated K-fold cross-validation this process was repeated multiple times, and for each repetition, the data was randomly re-divided into K-folds. Repeated stratified K-fold cross-validation, simply adds to the repeated K-fold cross-validation by ensuring that each fold has approximately the same distribution based on the outcome variable. In this study, 5 folds and 10 repetitions were used, resulting in a total of 50 training- and validation-sets.

The evaluation metrics used were accuracy and area under the curve (AUC). During model development, the model performance was defined as the mean of these metrics calculated on each validation set. Initially, models were compared based on the validation accuracy, and if two models had the same accuracy, AUC was used as the differentiator. The best-performing models in each iteration were saved and used to compute feature importance. Since the cross-validation approach generates 50 models, the importance of a feature was determined by averaging the SHAP-value from the different models. The iterative model development was completed when all but one feature had been removed. Internal validation on the test set was performed by creating an ensemble prediction from the models having the best predictive performance, i.e., each model provided predictions for the test set, and the final prediction for a subject was determined as the mean value of these predictions. A threshold of 0.5 was used to classify the binary prediction based on the mean prediction.

To give a better comparison between the cross-validated result and the ensemble prediction on the test set, an ensemble prediction was also performed on the cross-validation set. This cross-validation ensemble prediction was created by averaging the predictions for each subject whenever it was included in the validation set.

Results

Patient characteristics

The clinical and biochemical characteristics for the patient groups are presented in Table 1. Since uric acid levels varies with gestation week (30), a z-score for uric acid in the ICU-cohort and PE-controls was calculated and presented in Table 2a and 2b.

Table 1. Clinical and biochemical characteristics for the patient cohorts

	ICU-cohort n=41	PE controls n=40	p-value
Age	33 (31-35)	29 (28-31)	0.03*
BMI (kg/m^2)	27 (25-29)	27 (25-29)	1.0
ALAT (µkat/L)	0.58 (0.26-1.85)	0.2 (0.17-0.33)	0.0001**
ASAT (µkat/L)	0.81 (0.37-2.1)	0.38 (0.33-0.47)	0.0089**
Creatinine (µmol/L)	76 (62-93)	60 (52-77)	0.0042**
Uric acid (µmol/L)	453 (385-531)	377 (316-454)	0.0017**
Hemoglobin (g/L)	120 (111-132)	122 (114-128)	0.51
Platelets (x 10^9/L)	150 (122-194)	189 (135-213)	0.11
Systolic BP^1 (mmHg)	171 (164-178)	160 (155-165)	0.007**
Diastolic BP^2 (mmHg)	107 (103-112)	101 (98-103)	0.002**
Gestation days at delivery	250 (225-267)	273 (256-278)	0.002**
Antihypertensive treatment (number of drugs)	0 = n 7 1 = n 9 2 = n 15 3= n 10	0 = n 26 1 = n 13 3 = n 1	<0.001**

Values are shown as mean (95% confidence interval) for normal distributed data and median (quartiles) for not normal distributed data. A p-value <0.05 was considered significant.

*p-value <0.05, **p-value <0.01

1 Highest systolic blood pressure recorded during hospitalization or in maternity care (1-30 days prior to delivery).

2 Highest diastolic blood pressure recorded during hospitalization or in maternity care (1-30 days prior to delivery).

Creatinine is not significant in t-test (p=0.09), but in non-parametric test (Kolmogorov-Smirnov).

Table 2a. Uric acid corrected for gestational age in the patient cohorts.

Min	1st Qu.	Median	Mean	3rd Qu.	Max	Sd
-0.03571	2.87500	3.94340	4.09974	5.37500	11.52830	2.3608

Descriptive statistics of z-score for uric acid in the ICU-cohort

Table 2b. Uric acid corrected for gestational age in the patient cohorts.

Min	1st Qu.	Median	Mean	3rd Qu.	Max	Sd
-0.5536	0.9018	2.0089	2.2197	3.8125	5.2143	1.6881

Descriptive statistics of z-score for uric acid in the PE-controls

Reasons for admission to the intensive care unit beyond PE are shown in Table 3.

Table 3. Reasons for admission to the intensive care unit beyond PE

	Number of patients
Dysregulated hypertension	n = 5
HELLP syndrome	n = 13
Postpartum hemorrhage	n = 8
Eclampsia	n = 9
Infection	n = 1
Kidney failure	n = 3
Pulmonary oedema/heart failure	n = 2

Reasons for admission to intensive care for the forty-one patients in the intensive care cohort beyond PE-diagnosis.

In the control-group the diagnosis was PE or severe PE. Nine patients (22.5%) were early-onset PE (EPE) and the rest late-onset PE (LPE). In the ICU-cohort all patients were severe PE, 14 patients were EPE (34%) and the rest LPE.

None of the controls developed a seizure. Three patients were treated with Magnesium (Mg) in the control group. Nine ICU-patients developed eclampsia, but none of them were treated with Mg before the seizure.

Three patients in the control-group had a bleeding >500 ml and one >1000 ml.

Patients in the ICU-cohort had significantly higher systolic and diastolic blood pressure despite being treated with more than one antihypertensive agent.

The total number of days in hospital before delivery was 4,6 (mean) in the ICU-cohort and 4,9 (mean) for the controls.

A comparison of end organ failure between patients admitted to ICU vs. PE controls is shown in the supplemental section as Table A3.

Final model development

We used SHAP-values to rank the importance of the variables for every model. The best model was generated when using the variables ASAT, uric acid, and body mass index. This set of variables resulted in a cross-validation accuracy of 0.88 and an area under the curve (AUC) of 0.91. When internally validating this model on the test-set the accuracy was lower with 0.82 and an AUC of 0.85.

The cross-validated ensemble prediction resulted in an accuracy of 0.92, and AUC 0.90. The Receiver operating characteristic (ROC) curve, and confusion matrix for the cross-validated ensemble prediction and internal validation are shown in Figure 1 and 2.

Figure 1. Predictive performance of the cross-validated ensemble prediction. Left figure shows the receiver operating characteristic curve and the area under the curve. The right figure is the prediction confusion matrix.

Figure 2. Internal validation results. Left figure shows the receiver operating characteristic curve and the area under the curve. The right figure is the prediction confusion matrix.

Neonatal outcome

The patients in the ICU-cohort delivered earlier, at 250 (225–267) gestation days compared to 273 (256–278), p = 0.002, in the control group. The birthweight in the ICU-cohort was significantly lower, 2257 (1949–2565) g compared to 3043 (2720–3366) g, p < 0.001, in the controls. There was no difference in Apgar score at one or ten minutes between the groups, however, at 5 min there was a significant difference in Apgar score showing lower scores in the ICU-cohort (Table A4).

Discussion

When a patient presents with pregnancy hypertension and the suspicion of PE is raised, a standard package of clinical blood biomarkers is taken to confirm organ involvement. We included these standardized lab results together with other easily obtainable clinical characteristics, such as age, BMI, systolic and diastolic blood pressure.

In this study, we used machine-based learning models, to find patterns not recognizable by logistic regression that performed less well on our test-set. The best model was generated with the variables ASAT, uric acid, and body mass index with a cross validation-accuracy of 0.88 and an area under the curve (AUC) of 0.91. Our model was internally validated on a test-set where the accuracy was reduced to 0.82, with an AUC of 0.85. Still performing well considering the few parameters. In fact, adding more parameters to the model did not improve the accuracy.

The prediction model showed that ASAT and uric acid as well as body mass index, were the three parameters that in combination were most indicative of severe disease.

Aspartate aminotransferase, uric acid and BMI are, according to the results, parameters that clinicians should pay special attention to and follow their dynamics. Aspartate aminotranferase is present in all tissues except bone, with the highest level in skeletal muscle and liver. In hepatocellular injuries, both ASAT and ALAT are elevated, the magnitude of the later is usually greater because of the longer half-life of ALAT and a greater fraction of ASAT is bound to mitochondria. The elevated serum level of ASAT in PE is explained by the effect of hypoxia on the liver (31). Aspartate aminotransferase is also included in the full-PIERS model, and in the studies by Dröge et al., Tan et al., and Liao et al. (11,13,32).

Uric acid is often thought of as a marker of decreased kidney function. The vasoconstriction in PE leads to reduced renal blood flow which decreases the excretion of uric acid causing raised plasma levels (33). Uric acid has also been shown to be involved in the pathogenesis of PE through attenuation of normal trophoblast invasion and spiral artery vascular remodeling. The elevated levels usually pre-dates the onset of clinical symptoms (34).

Uric acid is included in prediction models for adverse pregnancy outcomes in studies by Lv et al. and Saleh et al. (9,33). In the last study, uric acid was one of the candidate markers, but not included in the final model, that instead included the protein to creatinine ratio, also related to kidney function. Maternal serum uric acid concentration has also been shown to be a good predictor of fetal/neonatal outcomes in women with PE/eclampsia (35,36).

High body mass index has been shown to have a strong correlation to PE, both severe and mild disease, likely mediated via inflammatory pathways and triglycerides (37–39).

There was no significant difference at group level for BMI as a single parameter, indicating that high BMI itself is not a predictor for need of ICU, although BMI is a known risk factor for PE. Rather, it is BMI in combination with ASAT and uric acid that seems to lead to a higher risk for the need of ICU-treatment.

In comparison with other prediction models, our results show high predictability with few components. A combination of biomarkers and different clinical parameters have been used in several studies to predict adverse outcomes in PE. The clinical parameters are varying from lab parameters alone as in our study, to also involve ultrasound-Doppler measurements, demographic data, and patient history (9,11,14). However, these models are more complex.

The sFlt/PIGF ratio is the most evaluated ratio of biomarkers for predicting adverse events in PE. In a study by Rana et al. the addition of the sFlt1/PlGF ratio to hypertension and proteinuria in patients presenting with PE before 34 gestational weeks, significantly improved the prediction for subsequent adverse outcome, with an area under the curve of 0.93 for hypertension, proteinuria, and sFlt1/PlGF, versus 0.84 for hypertension and proteinuria alone (12). In contrast, the PEACOCK-study, showed that the angiogenic biomarkers did not add predictive value to the clinical parameters regarding appropriate time for delivery for women with late preterm PE (40,41).

In the study by Schmidt et al., 114 variables were included, among them biomarkers such as soluble fms-like tyrosine kinase-1 (S-Flt-1), PlGF, and sonography data (umbilical artery pulsatility index, middle cerebral artery pulsatility index, mean uterine artery pulsatility index) (14). The outcome was the occurrence of adverse events throughout the remaining pregnancy and 2 weeks after delivery. By using machine learning an area under the receiver operating characteristic curve of 0.82 was reached, but this algorithm might still be too complex to implement clinically. More simple models using routine plasma biomarkers are therefore needed, especially in low- and middle-income countries.

A model with a clinical focus and the emphasis on clinical parameters is the full-PIERS model (preeclampsia integrated estimate of risk) (42). The fullPIERS-model including gestational age, chest pain/dyspnea, oxygen saturation (SpO2), platelet count, creatinine, and aspartate transaminase, was developed (42) to identify adverse events of PE within 48 h from hospital admission, and has been validated in several studies (42–44). However, some parameters such av chest-pain, dyspnea and saturation was unfortunately not well documented in the records of our patient-cohorts for a full comparison. When evaluating the full-PIERS model a problem has been missing values of e.g., oxygen saturation (8). The introduction of the Modified Early Obstetric Warning score (MEOWS) during the last years in Sweden might contribute to a more standardized way of documenting vital parameters in pregnant women, which could be useful, not only for medical reasons, but also for future research. Modified early obstetric warning scores have in previous studies been shown to be predictive of ICU-admissions and to reduce maternal morbidity. Additional larger studies are needed to validate them in the general obstetric population (45).

Our intensive care cohort and PE controls consisted of “real-life patients” with early-onset, as well as late-onset PE. Unfortunately, the cohorts in this study were too small to analyze EPE and LPE separately. In the future it would be necessary to externally validate the model in the two separate PE-forms.

No angiogenic biomarkers were used in our model since they are not yet routinely used in Swedish obstetric care.

The only adverse outcome in our study was the actual need of intensive care treatment, independent of the underlying medical reason. Most studies, for example von Dadelszen et al. used specified outcomes such as maternal mortality or one or more severe central nervous system, cardiopulmonary, hepatic, renal, coagulation/blood dysfunction or obstetric adverse events (42), which could be more specific since the level of care sometimes can be dependent on resources and individual medical decisions. However, all patients were admitted to ICU-departments in Southern Sweden, where access to high-dependency units for obstetric patients, as well as criteria for ICU-admission are similar. Sweden has 4.89 ICU-beds/100 000 citizens, which is amongst the lowest in Europe (Swedish Intensive Care Register) and the inclusion criteria for ICU-care are strict. Since PE is a very heterogenous disease, choosing ICU care or not as our main outcome, contributes to the avoidance of selection bias, and distinguish between severely ill patients needing continuous monitoring, compared to less ill patients possible to be managed at the delivery ward.

In a previous study, we analyzed the postpartum levels of the biomarkers hemopexin and alpha-1-microglobulin in the same ICU-cohort and found the levels to be lower in the severely ill patients as a sign of insufficient response to the oxidative stress, typically seen in PE (46). These biomarkers were not included in our current model because they were collected when the patients already had deteriorated postpartum. Future studies will add these biomarkers to the model to see if the predictability is improved.

Neonatal outcome was not included in the prediction-model. As previously presented, there was significant difference in birthweight between the groups as well as the Apgar points at 5 min, but no difference in Apgar score at 1 min, and most important at ten minutes.

A strength in this study is the internal validation of the model and that the parameters used are available in most countries and hospital settings.

A weakness is the retrospective approach. However, this is the chosen study design in most studies trying to develop prediction models for adverse outcomes in PE. The reason for this is that the event rate is rare, and it would take a large number of patients and time to achieve enough power to reach statistical significances. A preselected data set divided in cases and controls can therefore boost the model accuracy. A weakness is also the small sample size, but the study can be seen as a pilot study, and an external validation of the model in a new larger prospective cohort is needed before clinical implementation.

Since the data set is small, it raises concerns about the model’s ability to be generalized. Small datasets can lead to overfitting and poor representation of the underlying patterns in the data. However, the XGBoost method includes multiple regularization techniques which reduce the risk of overfitting, such as shrinkage, subsampling, tree-pruning, and the typical L1 and L2 regularization. Furthermore, by employing the variable selection scheme described in the model development section, we reduce the number of variables, hence reducing the risk of overfitting.

To assess the model’s generalization ability, we used cross-validation, which estimates its performance. However, this scheme might lead to a bias toward the cross-validation sets, hence we include a test set to obtain a reliable estimate of the best-performing model generalization ability. In the case of significant overfitting, the test set results would be poor.

Conclusion

When using machine-based learning analysis on a “real-life patient-cohort” of intensive care patients with severe PE and PE controls, the best area under the curve for predicting intensive care need was generated by the parameters ASAT, uric acid together with BMI.

These are simple routine parameters available in most clinical settings that could help clinicians to have a greater preparedness for severe complications in PE.

Abbreviation

PE	=	Preeclampsia
BMI	=	Body mass index
ASAT	=	Aspartate aminotransferase
ALAT	=	Alanine aminotransferase
SWECRIT biobank	=	Blood samples from critically ill patients and healthy controls in Sweden
HELLP	=	Hemolysis elevated liver enzymes low platelets
sFlt-1	=	Soluble fms-like tyrosine kinase-1
PIGF	=	Placental growth factor

CRediT authorship contribution statement

Conceptualization: C.E., S.R.H.

Methodology: O.B., C.E., S.R.H.

Software: O.B.

Formal analysis: O.B., C.E.

Investigation: C.E., O.B.

Resources: C.E., O.B., S.R.H.

Data curation: O.B., C.E.

Writing – original draft preparation: C.E., O.B.

Writing – review and editing: all authors.

Visualization: C.E., L.E., S.R.H.

Supervision: S.R.H., L.E.

Project administration: C.E, S.R.H.

Highlights

• A prediction model for estimating the risk of intensive care need in preeclampsia.

• Machine learning found correlations not shown by traditional logistic regression .

• The best parameters for predicting intensive care need were ASAT, uric acid and BMI.

Acknowledgments

This project was supported by the Gorthon Foundation, Helsingborg Hospital, Swedish ALF (Agreement concerning research and education of doctors), Swedish SUS Foundations and the Swedish Research Council (VR).

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/10641955.2024.2312165

Additional information

Funding

acquisition: C.E., S.R.H, L.E.All authors have read and agreed to the published version of the manuscript.