Development of machine learning models for predicting acute respiratory distress syndrome：evidence from the MIMIC-III and MIMIC-IV

doi:10.21203/rs.3.rs-3221576/v1

Download PDF

Research Article

Development of machine learning models for predicting acute respiratory distress syndrome：evidence from the MIMIC-III and MIMIC-IV

https://doi.org/10.21203/rs.3.rs-3221576/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Background

Acute Respiratory Distress Syndrome (ARDS) is a prevalent condition in the ICU with a mortality rate of 27% to 45%. Despite the Berlin definition being the current diagnostic standard, it has significant limitations. This study aims to establish and validate a novel machine learning-based prediction model for ARDS in ICU patients.

Methods

The data of suspected ARDS patients was extracted from the Medical Information Mart for Intensive Care (MIMIC)-III and MIMIC-IV databases. Ten-fold cross-validation was employed, utilizing machine learning algorithms such as K-nearest neighbor (KNN), support vector machine (SVM), Decision Tree Classifier (DTC), random forest (RF), extreme gradient boosting (XGBoost), light gradient boosting (LightGB), and categorical boosting (CatBoost) and logistic regression (LR) for model construction. Finally, the performance of these models was evaluated based on metrics including area under the ROC curve, calibration curve, and clinical decision curve.

Results

A total of 2,852 patients who met the exclusion criteria were included in the final study, of which 2078 patients developed ARDS.We established scoring models, such as LR, KNN, SVM, DTC, RF, XGBoost, LightGB, and CatBoost. The area under the receiver operating characteristic curve (AUC) values for each model were as follows: LR - 0.664, KNN - 0.692, SVM - 0.567, DTC - 0.709, RF - 0.732, XGBoost - 0.793, LightGB - 0.793, and CatBoost - 0.817. Notably, CatBoost exhibited superior predictive performance in discrimination, calibration, and clinical applicability compared to all other models.

Conclusions

The application of machine learning models has showcased their robustness in predicting ARDS. Notably, the CatBoost algorithm emerges as the most promising in terms of predictive performance.

Acute respiratory distress syndrome

Machine learning

Prediction model

Intensive care unit

Acute respiratory distress syndrome (ARDS) is the acute onset of hypoxaemia and bilateral pulmonary oedema due to excessive alveolocapillary permeability^{[1, 2]}. Although the incidence of ARDS is high in ICU^[3], the mortality of severe ARDS is still over 40%^{[4, 5]}. Early identification of ARDS is crucial to improve outcomes. At present, the Berlin definition is the consensus for diagnosing ARDS, but the existing studies show that it has big defects^[6–8]. Diffuse alveolar Injury (DAD) is the pathological hallmark of ARDS^[9] ,however, an autopsy study showed that only half of ARDS subjects actually have DAD, and there is a large difference between clinical and pathological diagnoses as Berlin definition^[10]. Approximately more than 60% of patients with severe ARDS according to the Berlin definition were reclassified as moderate, mild, or non-ARDS after 24 hours of standardized ventilation Settings (PEEP ≥ 10, FiO₂ ≥ 0.5)^[11]. The current Berlin definition of ARDS may lead to numerous false positive cases, resulting in non-ARDS patients receiving unnecessary or even harmful treatments.

There is growing concern among clinicians that the PaO₂/ FiO₂ may vary depending on ventilator settings, and the standardized value of PaO₂/ FiO₂ after specific ventilation Settings may be more diagnostic. A stable value of PaO₂/ FiO₂ after 24 hours of standardized ventilation Settings more accurately reflects the severity of ARDS^[11–13]. It is a better method to optimize ARDS patients' diagnosis and risk stratification. Although this approach improves diagnostic accuracy, delayed disease diagnosis may also lead to unbeneficial or harmful treatment for patients. Therefore, accurately identifying ARDS patients early according to PaO₂/ FiO₂ under standardized ventilation Settings has great clinical application value.

With the advancements in statistical theory and computer technology, the utilization of artificial intelligence (AI) in the medical domain, particularly in critical care medicine, has become increasingly prevalent^[14–16]. Abigail et al. ^[17]used the eXtreme Gradient Boosting (XGBoost) algorithm to predict the occurrence of ARDS, which could give an early warning of ARDS 48 hours in advance. Yang et al. ^[18]combined feature selection methods with machine learning algorithms to develop an improved ARDS diagnostic model using non-invasive physiological parameters, surpassing traditional linear models reliant on non-invasive pulse oxygen saturation. Compared with traditional regression analysis methods, machine learning algorithms has better clinical prediction accuracy and performance, which can help doctors make more accurate clinical decisions.

In the present study, we employed the PaO₂/ FiO₂ following 24 hours of standardized ventilation settings as a diagnostic marker for ARDS. We ainmed to develop and validate multiple machine learning (ML) models with the aim of predicting ARDS accurately, uimately identifying the model exhibiting the highest predictive performance.

Data source

We obtained patient data from the comprehensive Medical Information Mart for Intensive Care (MIMIC)-III and MIMIC-IV database. MIMIC is a large, freely-available database, comprising a wealth of information, including but not limited to demographic characteristics, vital sign monitoring, laboratory and microbiological test results, imaging studies, documentation of intake and output, medication records, length of stay, survival data, and records of discharge or mortality. ^{[19, 20]}

Participants

The selection of study participants is shown in Fig. 1. We included suspected ARDS patients meeting the Berlin definition^[21] from the database on the first day of ICU admission into the study. The exclusion criteria of the patients from the current study included the following conditions:(a) Age < 18 years old; (b) ICU hospital stay < 48 h; (c) without standard ventilatory setting (FiO₂ ≥ 0.5 with PEEP ≥ 10 cmH2O)

For patients with multiple hospitalizations or multiple ICU records during the same hospitalization, only the first ICU admission data of the first hospitalization.

Data extraction

Using Structured Query Language (SQL) PostgreSQL (version 9.6) to extract the patient data within 24 h after patient admission to the ICU from the MIMIC database. The content of the extracted data was demographic information, vital signs, laboratory parameters, and clinical parameters. (1) demographic information, including sex, age;(2)vital signs, including heart rate, temperature, respiratory rate, oxygen saturation (SpO₂), systolic blood pressure (SysBP), and diastolic blood pressure (DiasBP) (3)laboratory parameters, including anion gap, partial pressure of blood oxygen(PaO₂), arterial partial pressure of carbon dioxide(PaCO₂), partial thromboplastin time (PTT), prothrombin time (PT), international normalized ratio (INR),white blood cell(WBC), hematocrit, platelet, red cell distribution width(RDW), hemoglobin, lactate, base excess, bicarbonate, sodium, potassium calcium,chloride, creatinine, blood urea nitrogen (BUN), glucose. We included maximum and minimum values for analyzing some variables with multiple measurements.

The primary outcome of the study was the PaO₂/ FiO₂ under a standard ventilatory setting (FiO₂ ≥ 0.5 with PEEP ≥ 10 cmH₂O) after 24 h.

Statistical analysis

Continuous variables were summarized as the median with interquartile range and were compared using the Wilcoxon rank-sum test. Categorial variables were expressed as number and percentage and were compared using the Chi-square tests or Fisher’s exact probability Feature selection is an important step in model construction. We mainly use a filter approach^{[22, 23]}, where the idea is to "score" each feature, that is, assign a weight to each feature, which represents the importance of the feature, and then rank it according to the weight. After the preliminary screening of features, the wrapper method is used to continuously select feature subsets from the initial feature set, train the learner, and evaluate the subsets according to the performance of the learner until the best subset is selected.

After Feature selection, we generated eight ML models to predict the development of ARDS in ICU, namely, LR, KNN, SVM, DTC, RF, and XGBoost, LightGB, CatBoost. During the process of building the prediction model, we performed ten fold cross-validation on the training and validation sets to prevent overfitting. We randomly divided the entire dataset into 10 folds. Nine of them were used as the training set for model development, and the remaining one was used as the model validation set. Because each of these 10 folds was used as a validation set, the above procedure was repeated 10 times. Finally, we validated the performance of each model by using the AUC of the ROC curve, sensitivity, specificity, recall, accuracy, and F1 score, which were compared in the validation set.The clinical application was investigated by decision curve analysis (DCA).The calibration evaluation of the model was assessed using the calibration curve.

Statistical analyses were performed in Python, version 3.7.6. A significance level of p < 0.05 was considered statistically significant.

Baseline characteristics

In our study, a comprehensive analysis was conducted, encompassing a total of 2,825 individuals. Among the included individuals, a total of 2,078 individuals maintained their compliance with the diagnostic criteria for ARDS after 24 hours of standard ventilatory conditions (PEEP ≥ 10, FiO₂ ≥ 0.5). However, 747 individuals did not meet the required diagnostic criteria for ARDS. The divergent characteristics between the ARDS and non-ARDS groups are summarized in Table 1. Importantly, patients who met the diagnostic criteria for ARDS exhibited advanced age and significantly higher SAPS II scores, along with elevated rates of in-hospital mortality and 28-day mortality, compared to those who did not meet the diagnostic criteria. Within the ARDS group, several parameters exhibited substantial disparities compared to the non-ARDS group. Specifically, the minimum values of PH, Calcium,PaCO₂,bicarbonate and base excess, and the maximum values of bicarbonate and base excess, were markedly higher (p < 0.001). Conversely, the minimum values of PaO₂ and the maximum value of SpO₂ and PaO₂/ FiO₂, were notably lower among ARDS patients compared to their non-ARDS counterparts (p < 0.001).

Table 1

Characteristics of participants
Value	Overall	Non-ARDS	ARDS	p
	n = 2825	n = 747	n = 2078
gender (%)				0.017
male	1042 (36.9)	303 ( 40.6)	739 ( 35.6)
female	1783 (63.1)	444( 59.4)	1339 ( 64.4)
age (years)	59.31 (24.58)	56.79 (30.83)	60.22 (21.83)	0.001
Weight(kg)	92.51 (27.88)	92.21 (32.04)	92.62 (26.22)	0.727
Height(cm)	171.38 (9.12)	171.40 (9.24)	171.37 (9.08)	0.956
Heartrate_min (beats/minute)	75.40 (16.46)	76.45 (16.96)	75.02 (16.26)	0.041
Heartrate_max (beats/minute)	113.09 (21.83)	115.29 (20.84)	112.30 (22.12)	0.001
Sysbp_min (mmHg)	83.16 (14.89)	82.87 (15.33)	83.27 (14.73)	0.53
Sysbp_max(mmHg)	150.69 (25.13)	151.12 (24.11)	150.53 (25.49)	0.582
Diasbp_min(mmHg)	43.30 (10.36)	43.11 (10.99)	43.37 (10.12)	0.549
Diasbp_max(mmHg)	85.81 (19.51)	86.84 (19.81)	85.44 (19.39)	0.092
Meanbp_min(mmHg)	53.90 (14.10)	53.94 (14.15)	53.88 (14.09)	0.916
Meanbp_max(mmHg)	111.85 (36.04)	113.84 (36.43)	111.13 (35.87)	0.079
Respiratory rate_min	13.16 (4.48)	13.21 (4.46)	13.15 (4.49)	0.744
Respiratory rate_max	30.29 (7.26)	30.43 (7.18)	30.24 (7.29)	0.543
Temperature_min(℃)	36.14 (1.21)	36.04 (1.40)	36.18 (1.13)	0.007
Temperature,_max(℃)	37.94 (0.97)	37.96 (1.02)	37.93 (0.95)	0.483
SpO₂_min (%)	88.59 (9.66)	88.86 (9.81)	88.49 (9.61)	0.375
SpO₂_max (%)	99.65 (0.96)	99.85 (0.47)	99.58 (1.08)	< 0.001
PaO₂/ FiO₂	122.31 (57.02)	134.02 (67.17)	118.10 (52.29)	< 0.001
AnionGap_max	16.88 (5.40)	17.32 (5.44)	16.72 (5.37)	0.009
AnionGap_min	13.04 (3.78)	13.19 (3.60)	12.99 (3.84)	0.218
PaCO₂_max(mmHg)	54.56 (16.01)	54.23 (16.26)	54.68 (15.92)	0.503
PaCO₂_min(mmHg)	35.48 (8.44)	34.24 (8.37)	35.93 (8.43)	< 0.001
PH_max	7.41 (0.07)	7.40 (0.08)	7.41 (0.07)	0.172
PH_min	7.24 (0.12)	7.22 (0.12)	7.25 (0.12)	< 0.001
PaO₂_max(mmHg)	233.64(125.93)	246.51 (116.18)	229.01 (128.97)	0.001
PaO₂_min(mmHg)	69.09 (23.51)	74.07 (28.16)	67.30 (21.32)	< 0.001
Lactate_max (mmol/l)	3.94 (3.59)	4.13 (3.24)	3.87 (3.71)	0.081
Lactate_min (mmol/l)	1.96 (1.53)	2.02 (1.42)	1.93 (1.56)	0.157
Bicarbonate_max(mmol/l)	23.77 (4.56)	23.04 (4.46)	24.03 (4.57)	< 0.001
Bicarbonate_min(mmol/l)	20.42 (5.20)	19.48 (5.16)	20.76 (5.17)	< 0.001
Base excess_max(mmol/l)	0.23 (4.58)	-0.45 (4.54)	0.47 (4.57)	< 0.001
Base excess_min(mmol/l)	-5.92 (6.23)	-7.08 (6.35)	-5.51 (6.14)	< 0.001
INR_max (seconds)	1.67 (1.14)	1.64 (1.12)	1.69 (1.14)	0.305
INR_min (seconds)	1.33 (0.43)	1.30 (0.37)	1.34 (0.45)	0.024
PT_max (seconds)	17.72 (10.39)	17.28 (9.28)	17.88 (10.76)	0.179
PT_min (seconds)	14.58 (4.08)	14.22 (3.25)	14.70 (4.34)	0.006
PTT_max (seconds)	47.77 (31.35)	48.31 (32.02)	47.58 (31.11)	0.582
PTT_min (seconds)	32.00 (9.80)	31.70 (9.47)	32.11 (9.92)	0.32
WBC_max(K/ul)	17.04 (12.55)	16.77 (9.13)	17.14 (13.57)	0.482
WBC_min (K/ul)	11.57 (8.92)	11.23 (6.68)	11.69 (9.60)	0.228
Hemoglobin_max(g/L)	11.79 (2.22)	11.90 (2.19)	11.75 (2.22)	0.116
Hemoglobin_min(g/L)	9.99 (2.27)	10.00 (2.16)	9.99 (2.30)	0.967
Platelet_max (K/ul)	226.56(119.92)	238.39 (124.23)	222.31 (118.07)	0.002
Platelet_min (K/ul)	174.0(110.09)	180.59 (111.77)	171.72 (109.41)	0.059
Hematocrit_max(%)	35.50 (6.56)	35.79 (6.46)	35.40 (6.59)	0.16
Hematocrit_min(%)	29.59 (6.75)	29.52 (6.32)	29.62 (6.90)	0.753
RDW_max	15.26 (2.13)	15.13 (2.16)	15.31 (2.12)	0.047
RDW_min	14.77 (1.98)	14.66 (1.98)	14.82 (1.99)	0.059
Creatinine_max(mg/dl)	1.69 (1.59)	1.82 (1.87)	1.65 (1.47)	0.012
Creatinine_min(mg/dl)	1.31 (1.21)	1.36 (1.37)	1.29 (1.14)	0.17
Sodium_max (mg/dl)	140.93 (5.38)	141.01 (5.32)	140.90 (5.40)	0.651
Sodium_min (mg/dl)	137.59 (5.25)	137.37 (5.06)	137.67 (5.31)	0.182
Potassium_max(mg/dl)	4.71 (1.00)	4.76 (0.85)	4.70 (1.05)	0.163
Potassium_min (mg/dl)	3.91 (0.60)	3.87 (0.61)	3.93 (0.60)	0.033
Chloride_max (mg/dl)	108.33 (6.43)	108.99 (6.34)	108.09 (6.44)	0.001
Chloride_min (mg/dl)	104.01 (6.44)	104.41 (6.27)	103.86 (6.50)	0.048
Calcium_max (mg/dl)	8.42 (1.32)	8.43 (1.33)	8.41 (1.32)	0.756
Calcium_min (mg/dl)	7.63 (0.93)	7.52 (0.93)	7.67 (0.93)	< 0.001
BUN_max(mg/dl)	28.73 (21.43)	29.39 (23.42)	28.49 (20.66)	0.322
BUN_min (mg/dl)	23.60 (18.17)	23.61 (18.91)	23.60 (17.90)	0.984
Glucose_max(mg/dl)	183.7(102.79)	187.64 (111.02)	182.32 (99.65)	0.226
Glucose_min(mg/dl)	121.80 (42.73)	119.52 (39.53)	122.62 (43.80)	0.089
GCS	11.28 (4.65)	11.58 (4.50)	11.17 (4.70)	0.041
SAPS II	45.77 (15.21)	44.39 (15.33)	46.27 (15.14)	0.004
SOFA	9.25 (4.09)	9.15 (3.98)	9.28 (4.13)	0.457
in-hospital mortality(%)	644 (22.8)	137 ( 18.3)	507 ( 24.4)	0.001
28-day mortality (%)	708 (25.1)	154 ( 20.6)	554 ( 26.7)	0.001
Data are presented as %, mean ± SD or median (IQR)
Sysbp Systolic blood pressure, Diasbp Diastolic blood pressure, Meanbp Mean arterial pressure,SpO₂ oxygen saturation, PaO₂ partial pressure of blood oxygen, PaCO₂ arterial partial pressure of carbon dioxide, PPT,artial thromboplastin time,PT prothrombin time, INR international normalized ratio,WBC white blood cell,RDW red cell distribution width, BUN Blood urea nitrogen,GCS Glasgow coma scale,,SAPS III Simplified Acute Physiology Score III,SOFA Sequential organ failure assessment.p < 0.05 was statistically significant

Feature selection Fig. 2 displays the results of feature screening. Based on the weight attributed to each feature, we have identified the 20 variables most strongly associated with the occurrence of ARDS. These variables are listed as follows: PaO₂/ FiO₂, height, the maximum values of anion gap, PT, creatinine, hematocrit, SpO2, DiasBP, SysBP, heartrate and platelet; as well as the minimum values of respiratory rate, SpO2, temperature, PaO₂, platelet, BUN,WBC,RDW and glucose.

Model performance comparisons We developed a total of eight machine learning (ML) models to assess the risk of ARDS following ICU admission. The discriminative performance of these models is depicted in Fig. 3, illustrating the receiver operating characteristic (ROC) curves. Among the eight models evaluated, the CATBoost model demonstrated the highest predictive efficacy for ARDS, with an AUC of 0.817. Following closely were the lightGBM model (AUC = 0.793) and the XGBoost model (AUC = 0.792). Notably, the SVM model (AUC = 0.567) exhibited comparatively lower performance in predicting ARDS. The DTC model (AUC = 0.709), RF model (AUC = 0.731), KNN model (AUC = 0.692), and LR model (AUC = 0.767) also demonstrated varying degrees of prediction ability, surpassing the LR model (AUC = 0.664) used as a reference. To further evaluate the models, a detailed set of performance metrics can be found in Table 2. Notably, based on the findings from the decision curve analysis (DCA) curve presented in Fig. 4 ,the CATBoost model exhibits a substantial net benefit in terms of threshold probability compared to alternative models. In addition, the calibration curve showed that the CATBoost model was well calibrated showed in Fig. 5 .

Table 2

Model performance metrics
Model	AUC	Accuracy	Precision	Recall score	F1 socre
LR	0.664	0.745	0.705	0.745	0.685
SVM	0.567	0.651	0.656	0.650	0.653
KNN	0.692	0.733	0.715	0.733	0.722
DTC	0.709	0.728	0.681	0.728	0.628
RF	0.732	0.777	0.758	0.777	0.760
XGB	0.793	0.801	0.837	0.841	0.775
LightGB	0.793	0.841	0.783	0.831	0.803
CatBoost	0.817	0.797	0.806	0.797	0.801
LR logistic regression, SVM support vector machine, KNN:K-nearestneighbor, DTC:Decision Tree Classifier,RF: random forest,XGBoost extreme gradient boosting, LightGB light gradientboosting, CatBoost:categorical boosting

Discussion In this study, we conducted an extensive analysis incorporating eight prominent machine learning algorithms, namely LR, KNN,SVM,DTC,RF, XGBoost, LightGB, and CatBoost. We considered the PaO₂/ FiO₂ obtained after standardizing ventilator settings as the gold standard for the ARDS diagnostic model. The resulting ROC scores for the models were as follows: LR − 0.664, KNN − 0.692, SVM − 0.567, DTC − 0.709, RF − 0.732, XGBoost − 0.793, LightGB − 0.793, and CatBoost − 0.817. Remarkably, the CatBoost algorithm demonstrated the highest predictive performance among the evaluated models.

Due to the intricate etiology and pathophysiology of the condition ,the diagnosis of ARDS based on the initial PaO₂/ FiO₂ faces significant challenges^[24]. The PaO₂/ FiO₂, influenced by various external factors including conventional mechanical ventilation parameters such as FiO2 and PEEP, fails to accurately reflect the true oxygenation status of the body. At high intrapulmonary shunt (QS/QT) and FiO₂ levels above 0.4, the relationship between PaO₂/ FiO₂ and FIO2 remains relatively constant. However, when QS/QT falls between 0.1 and 0.3, PaO₂/ FiO₂ undergoes significant changes as FiO₂ is altered^[25]. When PEEP is equal to or greater than 10 cm H₂O, it can decrease intrapulmonary shunt and significantly increase PaO₂, resulting in an increase in PaO₂/ FiO₂^[26]. Due to the lack of stability of oxygenation index, when it is used as one of the diagnostic criteria for ARDS, it will inevitably lead to misdiagnosis, and some studies have also proved this point. In a multicenter randomized controlled clinical trial^[27], 66% of patients initially meeting the ARDS criteria no longer met the criteria after 24 hours under standard ventilation settings (FiO2 > 0.5, PEEP > 5cm H₂O) .In two prospective observational studies from Hernu et al. ^[28]and Casser et al^[29], the Berlin criteria failed to identify subgroups with different degrees of lung injury based on unnormalized baseline PaO2/FiO2 values. The Berlin criteria failed to classify patients into categories of varying severity with significantly different mortality rates.Several autopsy reports ^{[10, 30, 31]} have investigated pathological findings and found no association between the Berlin criteria and DAD in over 50% of patients with moderate and severe ARDS. Interestingly, the study^[30] further reported a higher likelihood of DAD on autopsy examination among patients who met the clinical criteria for ARDS for an extended duration. DAD was observed in 27% of patients who met the clinical criteria for ARDS for less than 72 hours, compared to 62% of patients who met the criteria for ARDS for more than 72 hours. These observations highlight the unreasonableness of diagnosing ARDS based on the oxygenation index at admission, given its inherent instability. Patients categorized as severe ARDS based on the Berlin criteria may receive invasive and aggressive treatments that provide no benefit or could potentially be harmful. This is because a significant proportion of patients transition to less severe forms of ARDS after 24 hours of ventilation. In the ESICM Guidelines for ARDS^[32], experts also note that some minimum period of stabilization and stability prior to diagnosing ARDS is likely appropriate; Although the length of this period remains uncertain, and a longer stable period increases specificity, it impedes early therapeutic intervention.

Villar and coworkers ^[27]demonstrated in a prospective observational cohort (n = 282) that the PaO₂/ FiO₂ obtained on ventilator settings (FiO₂ > 0.5, PEEP > 5cm H₂O) 24 h of ARDS onset may allow a better risk classification. The PaO₂/ FiO₂ determined on these settings no later than 24 h after ARDS onset, stratified patients into mild (PaO₂/ FiO₂ [200; n = 47, mortality 17%), moderate ARDS (PaO₂/ FiO₂ 100–200; n = 149, mortality 40.9%), and severe ARDS (PaO₂/ FiO₂ < 100 (n = 86, mortality 58.1%) (p = 0.00001) .In another study^[11] by Villar, 478 patients with moderate to severe ARDS diagnosisd by the Berlin definition, assessing the PaO2/FiO2 ratio after 24 hours of standardized ventilator settings (PEEP > 10, FiO2 > 0.5). The hospital mortality rate significantly differed based on PaO₂/ FiO₂ categories under standardized methodology. However, this correlation significantly improved when patients met the PaO₂/ FiO₂ criteria after 24 hours of sustained ARDS diagnosis. In view of the aforementioned observations, the lack of standardized methods for assessing oxygenation criteria can result in misdiagnosis of ARDS patients and misjudgment in treatment selection. Current research suggests that optimizing ARDS diagnosis and risk stratification can be achieved by utilizing the PaO₂/ FiO₂ after 24 hours of standardized ventilator application, providing a more accurate reflection of ARDS patient prognosis.

The advancement of artificial intelligence and the emergence of big data have propelled ML to achieve remarkable milestones in disease diagnosis^{[33, 34]}. In contrast to traditional algorithmic models, machine learning demonstrates efficient processing capabilities for large-scale and complex datasets, leading to higher accuracy and improved prediction ability in addressing complex problems^{[35, 36]}. In recent years, the application of machine learning in the clinical diagnosis of ARDS has witnessed a notable surge. Abigail et al. ^[17]utilized the XGBoost algorithm to predict the onset of ARDS. The model exhibited an area under the receiver operator characteristic curve (AUC) of 0.827, 0.810, and 0.790 at 12 hours, 24 hours, and 48 hours prior to the onset of ARDS, respectively. This implies that an early warning for ARDS can be provided up to 48 hours in advance. Ding et al. ^[37]conducted a secondary analysis of a multicenter prospective observational cohort study and employed a random forest model to predict the occurrence of ARDS. Ultimately, the model achieved AUC of 0.82. In a study by Yang et al.^[18], a combination of feature selection methodology and machine learning algorithms was employed. They established a non-invasive physiological parameter model to estimate the P/F ratio for identifying ARDS. Among the models tested, XGBoost demonstrated the highest performance with an AUC of 0.9128, surpassing the traditional linear model utilizing non-invasive pulse oxygen saturation (SpO₂/FiO₂, S/F) (AUC = 0.7738). The incorporation of machine learning techniques into clinical prediction models holds promise for enhancing the accuracy of models compared to traditional statistical approaches.

The study presents several notable advantages. Notably, we introduce a novel approach by utilizing PaO2/FiO2, measured after 24 hours of standardized ventilator setting, as the diagnostic standard for ARDS, marking a pioneering contribution in our field. This unique approach enhances the accuracy of our model, surpassing other models based on conventional gold standard selection. Furthermore, the incorporation of machine learning techniques in prediction models has significantly elevated their performance. Our comprehensive analysis encompasses eight distinct machine learning algorithms. Comparatively, ensemble learning models such as XGBoost, LightGB, and CatBoost have demonstrated remarkable enhancements in predictive capabilities when contrasted with the traditional logistic model.

Our study, however, has certain limitations that should be acknowledged. Firstly, it is important to note that our research utilized a single-center retrospective study design. Retrospective studies inherently carry a degree of bias, and this should be taken into consideration when interpreting the results. Furthermore, the lack of participation from multiple centers and external validation hampers the overall reliability of our model. In order to enhance the robustness of our findings, future investigations should involve collaborations with multiple centers and seek external validation. It is also worth mentioning that although previous studies have established associations between biomarkers and the diagnosis of acute respiratory distress syndrome (ARDS), unfortunately, such data was not available in the mimic database used for our research. Therefore, the inclusion of these variables in future diagnostic models holds promise for further improving the performance and accuracy of the model.

In summary, machine learning models serve as reliable and robust tools for predicting the occurrence of ARDS. Among the various predictive models investigated, the CATBoost model emerges as the most effective, providing clinicians with the means to promptly and accurately identify individuals at risk of ARDS.

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Availability of data and materials

Our data was obtained from the Intensive Care Medicine Information Mart (MIMIC) database.The study involving human participants was reviewed and approved by the Institutional Review Board at the Beth Israel Deaconess Medical Center.

Conflict of interest

The authors declare that there is no conflict of interest.

Acknowledgements

Not applicable.

Meyer NJ, Gattinoni L, Calfee CS. Acute respiratory distress syndrome[J]. Lancet. 2021;398(10300):622–37.
Sun H, Zhang T, Gao J. Extracellular Vesicles Derived from Mesenchymal Stem Cells: A Potential Biodrug for Acute Respiratory Distress Syndrome Treatment[J]. BioDrugs. 2022;36(6):701–15.
Liu Q, Wu J, Zhang X, et al. Circulating mitochondrial DNA-triggered autophagy dysfunction via STING underlies sepsis-related acute lung injury[J]. Cell Death Dis. 2021;12(7):673.
Huang Z, Zheng J, Huang W, et al. The effects and safety of omega-3 fatty for acute lung injury: a systematic review and meta-analysis[J]. World J Surg Oncol. 2020;18(1):235.
Zhang X, Sugita S, Liu A, et al. Therapeutic effects of high molecular weight hyaluronic acid in severe Pseudomonas aeruginosa pneumonia in ex vivo perfused human lungs[J]. Am J Physiol Lung Cell Mol Physiol. 2021;321(5):L827–36.
Fan E, Brodie D, Slutsky AS. Acute Respiratory Distress Syndrome: Advances in Diagnosis and Treatment[J]. JAMA. 2018;319(7):698–710.
Villar J, Blanco J, Kacmarek RM. Acute respiratory distress syndrome definition: do we need a change?[J]. Curr Opin Crit Care. 2011;17(1):13–7.
Villar J, Slutsky AS. GOLDEN anniversary of the acute respiratory distress syndrome: still much work to do![J]. Curr Opin Crit Care. 2017;23(1):4–9.
Cardinal-Fernandez P, Ortiz G, Chang CH et al. Predicting the Impact of Diffuse Alveolar Damage through Open Lung Biopsy in Acute Respiratory Distress Syndrome-The PREDATOR Study[J]. J Clin Med, 2019,8(6).
Lorente JA, Cardinal-Fernandez P, Munoz D, et al. Acute respiratory distress syndrome in patients with and without diffuse alveolar damage: an autopsy study[J]. Intensive Care Med. 2015;41(11):1921–30.
Villar J, Blanco J, Del CR, et al. Assessment of PaO(2)/FiO(2) for stratification of patients with moderate and severe acute respiratory distress syndrome[J]. BMJ Open. 2015;5(3):e6812.
Villar J, Kacmarek RM, Perez-Mendez L, et al. A high positive end-expiratory pressure, low tidal volume ventilatory strategy improves outcome in persistent acute respiratory distress syndrome: a randomized, controlled trial[J]. Crit Care Med. 2006;34(5):1311–8.
Villar J, Perez-Mendez L, Lopez J, et al. An early PEEP/FIO2 trial identifies different degrees of lung injury in patients with acute respiratory distress syndrome[J]. Am J Respir Crit Care Med. 2007;176(8):795–804.
Guo F, Zhu X, Wu Z, et al. Clinical applications of machine learning in the survival prediction and classification of sepsis: coagulation and heparin usage matter[J]. J Transl Med. 2022;20(1):265.
Vasey B, Ursprung S, Beddoe B, et al. Association of Clinician Diagnostic Performance With Machine Learning-Based Decision Support Systems: A Systematic Review[J]. JAMA Netw Open. 2021;4(3):e211276.
Wardi G, Carlile M, Holder A, et al. Predicting Progression to Septic Shock in the Emergency Department Using an Externally Generalizable Machine-Learning Algorithm[J]. Ann Emerg Med. 2021;77(4):395–406.
Le S, Pellegrini E, Green-Saxena A, et al. Supervised machine learning for the early prediction of acute respiratory distress syndrome (ARDS)[J]. J Crit Care. 2020;60:96–102.
Yang P, Wu T, Yu M, et al. A new method for identifying the acute respiratory distress syndrome disease based on noninvasive physiological parameters[J]. PLoS ONE. 2020;15(2):e226962.
Johnson AE, Pollard TJ, Shen L, et al. MIMIC-III, a freely accessible critical care database[J]. Sci Data. 2016;3:160035.
Johnson A, Bulgarelli L, Shen L, et al. MIMIC-IV, a freely accessible electronic health record dataset[J]. Sci Data. 2023;10(1):1.
Ranieri VM, Rubenfeld GD, Thompson BT, et al. Acute respiratory distress syndrome: the Berlin Definition[J]. JAMA. 2012;307(23):2526–33.
Sun X, Qourbani A. Combining ensemble classification and integrated filter-evolutionary search for breast cancer diagnosis[J]. J Cancer Res Clin Oncol, 2023.
Zeng X, Cai J, Liang C, et al. Prediction of stock price movement using an improved NSGA-II-RF algorithm with a three-stage feature engineering process[J]. PLoS ONE. 2023;18(6):e287754.
Villar J, Perez-Mendez L, Kacmarek RM. The Berlin definition met our needs: no[J]. Intensive Care Med. 2016;42(5):648–50.
Feiner JR, Weiskopf RB. Evaluating Pulmonary Function: An Assessment of PaO2/FIO2[J]. Crit Care Med. 2017;45(1):e40–8.
Kacmarek RM, Villar J, Sulemanji D, et al. Open Lung Approach for the Acute Respiratory Distress Syndrome: A Pilot, Randomized Controlled Trial[J]. Crit Care Med. 2016;44(1):32–42.
Villar J, Perez-Mendez L, Blanco J, et al. A universal definition of ARDS: the PaO2/FiO2 ratio under a standard ventilatory setting–a prospective, multicenter validation study[J]. Intensive Care Med. 2013;39(4):583–92.
Hernu R, Wallet F, Thiolliere F, et al. An attempt to validate the modification of the American-European consensus definition of acute lung injury/acute respiratory distress syndrome by the Berlin definition in a university hospital[J]. Intensive Care Med. 2013;39(12):2161–70.
Caser EB, Zandonade E, Pereira E, et al. Impact of distinct definitions of acute lung injury on its incidence and outcomes in Brazilian ICUs: prospective evaluation of 7,133 patients*[J]. Crit Care Med. 2014;42(3):574–82.
Thille AW, Esteban A, Fernandez-Segoviano P, et al. Comparison of the Berlin definition for acute respiratory distress syndrome with autopsy[J]. Am J Respir Crit Care Med. 2013;187(7):761–7.
Thille AW, Penuelas O, Lorente JA, et al. Predictors of diffuse alveolar damage in patients with acute respiratory distress syndrome: a retrospective analysis of clinical autopsies[J]. Crit Care. 2017;21(1):254.
Grasselli G, Calfee CS, Camporota L, et al. ESICM guidelines on acute respiratory distress syndrome: definition, phenotyping and respiratory support strategies[J]. Intensive Care Med; 2023.
Howarth M, Bhatt M, Benterud E, et al. Development and initial implementation of electronic clinical decision supports for recognition and management of hospital-acquired acute kidney injury[J]. BMC Med Inform Decis Mak. 2020;20(1):287.
Sun L, Yang L, Liu X, et al. Optimization of Cervical Cancer Screening: A Stacking-Integrated Machine Learning Algorithm Based on Demographic, Behavioral, and Clinical Factors[J]. Front Oncol. 2022;12:821453.
Ong JJ, Castro BM, Gaisford S, et al. Accelerating 3D printing of pharmaceutical products using machine learning[J]. Int J Pharm X. 2022;4:100120.
Westhues CC, Mahone GS, Da SS, et al. Prediction of Maize Phenotypic Traits With Genomic and Environmental Predictors Using Gradient Boosting Frameworks[J]. Front Plant Sci. 2021;12:699589.
Ding XF, Li JB, Liang HY, et al. Predictive model for acute respiratory distress syndrome events in ICU patients in China using machine learning algorithms: a secondary analysis of a cohort study[J]. J Transl Med. 2019;17(1):326.

No competing interests reported.

Download PDF

Version 1

posted

You are reading this latest preprint version

Development of machine learning models for predicting acute respiratory distress syndrome：evidence from the MIMIC-III and MIMIC-IV

Status:

Version 1

Abstract

Figures

Introduction

Methods

Data source

Participants

Data extraction

Statistical analysis

Results

Baseline characteristics

Conclusion

Declarations

References

Additional Declarations

Status:

Version 1