A machine learning model for prediction of successful extubation in patients admitted to the intensive care unit

doi:10.21203/rs.3.rs-1949159/v1

Download PDF

Research Article

A machine learning model for prediction of successful extubation in patients admitted to the intensive care unit

https://doi.org/10.21203/rs.3.rs-1949159/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Background and objective:

Successful weaning from mechanical ventilation is important for patients admitted to intensive care units (ICUs); however, models for predicting real-time weaning outcomes remain inadequate. Therefore, this study was designed to develop a machine learning model using time series ventilator-derived parameters with good accuracy for predicting successful extubation.

Methods

Patients with mechanical ventilation between August 2015 and November 2020 admitted Yuanlin Christian Hospital in Taiwan were retrospectively included. The ventilator-derived parameter time series dataset was collected before extubation. Recursive Feature Elimination (RFE) was applied to choose the most important features. Machine learning models of logistic regression, random forest (RF), and support vector machine were adopted for predicting extubation outcomes. In addition, the synthetic minority oversampling technique (SMOTE) was employed to address the data imbalance problem. Area under receiver operating characteristic (AUC), F1 score, and accuracy along with 10-fold cross-validation were used to evaluate prediction performance.

Results

In this study, 233 patients were included, of whom 28 (12.0%) failed extubation. Moreover, the six ventilatory variables per 180-s dataset had the optimal feature importance. The RF exhibited better performance than others with an AUC of 0.976 (95% confidence interval [CI], 0.975–0.976), an accuracy of 94.0% (95% CI, 93.8–94.3%), and an F1 score of 95.8% (95% CI, 95.7–96.0%). The difference in performance between the RF with original and SMOTE dataset was small.

Conclusion

The RF model demonstrated good performance for predicting successful extubation of mechanically ventilated patients. This algorithm makes a precise real-time extubation outcome prediction for a patient at different time points.

Extubation

Intensive care unit

Machine learning

Mechanical ventilation

Prediction model

Although invasive mechanical ventilation is an important part of critical care treatment, prolonged mechanical ventilation results in ventilator-associated complications, morbidity, mortality, and increased hospitalization costs [1, 2]. As soon as the incitation factor causing respiratory failure begins to improve, weaning and extubation are essential next steps, and the ability to determine whether a patient is ready for extubation is crucial [3]. However, extubation is not always successful, and the literature shows that approximately 5.2–20% fail this process and require reintubation [4–6]. Extubation failure has consequences, such as the need for tracheotomy, the occurrence of pneumonia, and pulmonary damage induced by mechanical ventilation [7].

The discontinuation of ventilatory support can be challenging for physicians mostly because the pathophysiology of weaning failure is complex and not fully understood. Clinicians only have a sensitivity and specificity of 57% and 31%, respectively, for predicting failed extubation [8]. Nowadays, the rapid shallow breathing index (RSBI) (f/Vt) is the most commonly used ventilator weaning predictor [9]. However, the pooled sensitivity for RSBI of less than 105 breaths/min/L in predicting extubation success was 83%, whereas its specificity was 58%. This suggests that RSBI has only a moderate ability to rule out extubation success and does not sufficiently predict successful extubation [10]. Furthermore, numerous parameters have been reported as predictors of weaning outcomes, such as cough strength, duration of mechanical ventilation, and diaphragmatic function [11–13]. However, none of these clinical parameters had a better, real-time, and comprehensive ability to predict extubation outcomes.

Outcome prediction methods using machine learning models have recently been developed in many areas of healthcare research [14–16]. Traditional statistical methods mainly focus on the relationships between variables. In contrast, machine learning can be applied to big data using different approaches and make more accurate predictions [17]. Recently, some studies have explored the ability of machine learning models to accurately predict extubation outcomes [18–22]. Most studies have used demographic data (e.g., age, sex, duration of mechanical ventilation, reasons for respiratory failure), physiological data (e.g., Acute Physiology and Chronic Health Evaluation [APACHE] II score, respiratory rate [RR], blood pressure, heart rate, peripheral oxygen saturation, temperature), and laboratory data (e.g., lymphocytes, hematocrit, glucose, and sodium) [18–21]. Fabregat et al. applied time series data from clinical information systems, patient demographics, and medical records to create different machine learning algorithms [22]. However, most of these features are not real-time and might be poorly correlated with the weaning outcomes at the time of extubation.

Therefore, this study was designed to develop and validate a machine learning model using time series parameters of patients obtained from the mechanical ventilator in seconds with good accuracy to predict successful extubation. To this end, the model did a time-phased and immediate prediction within minutes to hours before extubation. Additionally, the trend change of extubation outcome prediction for patients would be provided to clinicians in further work.

Study participants

This study was conducted in the mixed medical/surgical intensive care unit (ICU) of Yuanlin Christian Hospital from August 1, 2015 to November 30, 2020. This is a 250-bed local community hospital with 20 ICU beds for adults. The inclusion criteria were as follows: (1) patients aged ≥ 20 years; (2) those diagnosed with acute respiratory failure who received invasive mechanical ventilation for more than 24 consecutive hours; (3) and those who had undergone extubation during hospitalization in the ICU. The exclusion criteria were as follows: (1) patients who received mechanical ventilation with tracheostomy or nasal endotracheal tube; (2) those who had undergone accidental extubation or self-extubation; and (3) those who were difficult to wean from mechanical ventilation for more than 21 days and further transferred to the Respiratory Care Center. The data were retrospectively collected and then analyzed. Therefore, informed consent was specifically waived, and the study was approved by the Institutional Review Board of Changhua Christian Hospital (approval no.: 210716). All methods have been performed according to the Declaration of Helsinki.

Weaning procedure

The weaning process followed the established protocol written based on the recommendations of the literature and medical guidelines [23]. Sedatives, hypnotics, and narcotics were discontinued for at least 8 h before the weaning procedure. The patients were identified as ready for weaning based on clinical stability criteria. The weaning process was started in the pressure-support ventilation (PSV) mode, followed by a spontaneous breathing trial (SBT) in a T-piece for 30 min. In patients who passed the SBT, the RSBI was calculated. The cuff leak test was performed before extubation.

Data collection

We collected the patients’ clinical characteristics (i.e., demographics, APACHE II score, reasons for intubation, and length of hospital stay) and weaning profiles from their medical records. The patients’ ventilation parameters (i.e., expiratory tidal volume (Vte), RR, peak airway pressure (Ppeak), mean airway pressure (Pmean), positive end-expiratory pressure (PEEP), and fraction of inspiration oxygen (FiO₂)) provided by the ventilator were collected in a time series dataset in seconds before extubation. Extubation failure was defined as the need for reintubation within the first 48 h after extubation. Post-extubation respiratory failure was defined as patients who had two or more of the following parameters: RR of more than 35/min; arterial oxyhemoglobin saturation of less than 90% despite adequate additional oxygen; pH of less than 7.20 and decreased from onset; hypercapnia with PaCO₂ of more than 45 mmHg; decreased level of consciousness; and abdominal paradox [24]. In the extubation success cohort, 4-h time series data before extubation was collected. Because of imbalanced datasets between the extubation success and failure cohorts, we hypothesize that early extubation would still be failure in the extubation failure cohort. Simultaneously, this may not be certain in the extubation success cohort. Therefore, we used 12-h time series data before extubation from the ventilator in the extubation failure cohort and divided it into every 4-h time evenly. With this process, we got a three-fold expansion in the extubation failure cohort.

Data processing

For data processing, first, we used the averaging method to separate the patient ventilation parameters into per 1, 30, 60, 120, 180, and 300 s, and we squared and calculated the degree of difference in the ventilation parameters to make dimension increases.

Feature extraction

For feature extraction, we used the Information Gain method to calculate the entropy of the time series dataset of ventilation parameters for extubation success or failure [25]. The entropy value is expressed between 0 and 1, where 0 indicates that the field cannot be partitioned for extubation success or failure, whereas 1 indicates that the field can be effectively divided. We made the raw data and calculated the degree of difference in the ventilation parameters to calculate the feature importance (FI) in the per 1-, 30-, 60-, 120-, 180-, and 300-s datasets from information gain algorithm results. To choose the most important features, Recursive Feature Elimination (RFE) was applied. RFE uses a classifier to rank the features and recursively removes the weakest features [26]. The process of removing the weakest features continues until the number of required features is reached.

Modeling

The ventilator-derived parameter time series dataset was divided into two parts: testing (30 min before extubation) and training (the remaining 210 min) (Fig. 1). Python (version 3.7) was used to perform all analyses. In the training stage, we used 10-fold cross-validation to train the model. The 10-fold cross-validation is divided into 10 datasets, and one dataset was used as a validation set, and the remaining datasets were used as training sets. To create predictive models, we used the following supervised machine learning methods, which are the most prominent and up-to-date machine learning methods for classification problems: logistic regression, random forest (RF), and support vector machine (SVM). Using maximum likelihood estimation to produce the regression coefficient, the logistic regression model accurately predicts the probability of the binary dependent variable. A decision tree approach is a tree-like model of decisions that can mathematically forecast the optimum option. A simple decision tree was created using an optimized version of the Classification and Regression Tree algorithm [27]. To find the split point, we used the Gini index as a statistic. The Gini index measures the likelihood of erroneously classifying a dataset at random. Because the drawbacks of the simple decision tree are instability and the possibility of overfitting, RF was used to improve prediction performance [27]. RF is an ensemble classifier that uses majority voting to aggregate many decision trees. SVM, a method for defining a high-dimensional boundary that clearly classifies data points, was also used. To address the issue of data imbalance, the method endorsed in literature of synthetic minority oversampling technique (SMOTE) were used [28]. We evaluated the prediction performance using area under the receiver operating characteristic curve (AUC), sensitivity, specificity, positive predictive value, negative predictive value, F1 score, and accuracy, along with the 10-fold cross-validation.

Statistical analysis

Continuous variables are expressed as medians and interquartile ranges (IQRs) (the spread difference between 25th and 75th percentiles), and categorical variables are presented as percentages. The chi-square test and Mann–Whitney U-test were used to detect differences for categorical and continuous variables. Differences with p-values of less than 0.05 were considered statistically significant. To better interpret the model, we employed the SHapley Additive exPlanations (SHAP) method to provide consistent and locally accurate attribution values for each feature within the prediction model [29]. The distribution of probability density was provided to better discriminate the extubation success and failure groups. All statistical analyses were performed using Python language and MedCalc (version 20.027; MedCalc, Mariakerke, Belgium).

In this study, 233 patients met the inclusion criteria, of whom 28 (12.0%) failed extubation (Supplementary Fig. 1). Demographic, clinical, and weaning profiles are shown in Table 1. APACHE II score and diagnoses at ICU admission were comparable between the extubation success and failure groups. Weaning profiles, including maximal inspiratory (Pimax) and expiratory (Pemax) pressures and mean RSBI values, were not statistically different between the two cohorts. The duration of hospital stay was significantly longer in the extubation failure group (median: 18.3 vs. 14.7 days; p = 0.006).

The dataset from the time series of the ventilation parameters is shown in Supplementary Table 1. The extubation failure cohort was more likely to have a lower Vte and higher Ppeak and Pmean than the extubation success cohort. The average IQR of PEEP and FiO₂ became smaller in the extubation success cohort. The mean RR values were not statistically different between the two cohorts. No statistically significant differences in ventilator-derived parameters were observed between the training and testing datasets.

Feature selection algorithms applied to the training dataset containing the twelve variables (Fig. 2). Based on the output of the feature selection and the literature review, the accuracy reached a pleatua after six selected variables in whether per 1-, 30-, 60-, 120-, 180-, and 300-s datasets. The ventilatory variables had the highest importance of characteristics per 180 s. The most importance six variables contributing to extubation outcomes were FiO₂ (FI: 0.144) and Ppeak (FI: 0.142), PEEP (FI: 0.105), Pmean (FI: 0.066), RR (FI: 0.027), and then Vte (FI: 0.025) in descending order (Table 2). The predictive effect is poor when using the degree of difference. We used the raw data of six ventilator-derived parameters as input variables.

We used the machine learning methods of logistic regression, RF with original and SMOTE datasets, and SVM with all variables per 180 s as input variables to predict extubation outcomes. The prediction performance of these four machine learning models is shown in Table 3. RF with original dataset exhibited an AUC of 0.976 (95% confidence interval [CI], 0.975–0.976), an accuracy of 94.0% (95% CI, 93.8–94.3%), and an F1 score of 95.8% (95% CI, 95.7–96.0%). Similarly, RF with SMOTE dataset showed an AUC of 0.979 (95% CI, 0.978–0.980), an accuracy of 92.9% (95% CI, 92.7–93.1%), and an F1 score of 94.9% (95% CI, 94.7–95.0%), indicating that the RF model has performance superior to that of the logistic regression and SVM models. Furthermore, the difference in performance between the RF with original and SMOTE dataset was small. This implied that the effect of data imbalance was less important in our dataset.

The performance of the RF model with training, validation, and testing datasets was presented in Supplementary Table 2. There was no obvious decrease in the AUC and accuracy of the RF model from training to validation or testing dataset. In other words, the overfitting issue was little in our RF model.

To identify the features that influenced the prediction model the most, we depicted the SHAP summary plot of RF (Supplementary Fig. 2), which depicts how high and low feature values were relative to SHAP values in the training dataset. According to the prediction model, the higher the SHAP value of a feature, the more likely the success of extubation. The most important predictive feature of successful extubation was the Ppeak value before extubation. For ventilatory characteristics, lower Ppeak and Pmean and higher Vte and RR in the last 30 min are predictive of extubation success.

To the best of our knowledge, this is the first study of a machine learning model for real-time prediction of successful extubation of mechanically ventilated patients in an ICU. The RF model demonstrated good prediction performance with an AUC of 0.976 (95% CI, 0.975–0.976) and an accuracy rate of 94.0% (95% CI, 93.8–94.3%). Considering that all features came from time series ventilatory variables routinely detected using the mechanical ventilator, this model can provide real-time prediction of extubation outcomes for a single patient at different time points. Moreover, it can be easily applied to improve clinical decision making for intensivists.

This study is unique as it developed a real-time model for predicting successful extubation. All previous machine learning studies involving mechanically ventilated patients to predict extubation failure have used electronic health records, including patient characteristics, laboratory data, vital signs, and ventilatory characteristics at a single time point [18, 20–22, 30–35]. These models could only identify factors associated extubation failure in a single patient during their hospital stay but could not make a concise prediction in different time points for the same patient. This decreased the practicality of these models in critical care. Li et al. have developed a time-phased machine learning model for predicting the onset of sepsis in critical care with high prediction performance [36]. Correspondingly, we developed and proposed a model for predicting the probability of extubation success in real time for weaning patients from the mechanical ventilator. Importantly, it could be easily employed using only ventilatory variables to improve the treatment decision for critical care physicians.

Patterns of spontaneous breathing of mechanically ventilated patients are potential markers of weaning outcomes. Variability in breathing during weaning from mechanical ventilation may be useful for clinical decision making [37]. In recent studies, low respiratory variability was associated with weaning failure during mechanical ventilation [38]. Patients who required noninvasive or invasive mechanical ventilation within 48 h had a considerably reduced quantitative variability of tidal volume during a 30-min SBT, according to Bien et al. [39]. Similarly, Sarlabous et al. have developed an entropy-based technique to accurately detect patient–ventilator asynchronies [40]. Correspondingly, we used a machine learning approach using ventilatory parameters derived from time series dataset to predict extubation success with an accuracy of 94.0% (95% CI, 93.8–94.3%).

Although artificial intelligence (AI) technologies have made remarkable advances in various fields, the use of AI algorithms with the black-box issue in healthcare remains rare due to physicians’ tendency to act only after understanding the rationale behind the results [41]. Given the potential for catastrophic consequences of a bad medical decision, particularly in critical care medicine, the black-box aspect leads physicians to distrust the AI model when no rationale is provided [42]. However, due to nonlinear relationships in the data, it makes it difficult for physicians to realize how to interpret these multiple features simultaneously. We believe that this study provides a satisfactory prediction of extubation outcomes for mechanically ventilated patients in the first step. Further work to interpret the major features and fluctuations of parameters that the algorithm prioritizes is urgently needed.

This study has several limitations that must be addressed. First, there is still debate about the definition of extubation failure. The definition adopted in this study is reintubation within 48 h after extubation. The use of noninvasive ventilation or high-flow oxygen therapy, with the potential to prevent reintubation, was excluded. Further studies to distinguish post-extubation management using a different time interval (e.g. 72 h after extubation) should be conducted. Second, the ventilator weaning mode used in this study is PSV. However, there are several weaning modes used in clinical practice. More studies on extubation failure prediction using other weaning modes are needed. Third, because this is a single-center study, it requires external validation. In contrast, the data we used in this study were routinely collected in a real-world setting. Fear of generalization should be alleviated to a large extent.

In summary, this is the first study that has used a machine learning method to predict extubation success using ventilator-derived parameters in mechanically ventilated patients. The RF model had the highest accuracy among the models examined in this study. This algorithm predicts extubation outcomes in real time for a single patient at different time points.

Ethical Approval and Consent to participate

The study was approved by the Institutional Review Board of Changhua Christian Hospital (approval no.: 210716). The Institutional Review Board waived the need for informed consent considering the retrospective nature of data collected. The Declaration of HELSINKI and its subsequent revisions were followed.

Consent for publication

Not applicable.

Data Availability Statement

Data are available from the corresponding author on reasonable request.

Competing interests

The authors declare no competing interests.

Funding

This research was funded by National Chung Hsing University and Changhua Christian Hospital Joint Research Program (No. NCHU-CCH 11003).

Authors' contributions

K.Y.H. and J.L.X. contributed to the data analysis. H.C.C.,M.H.H.,C.L.C. contributed to the data collection. C.H.L. contributed to data interpretation. Y.L.H. takes responsibility for the content of the manuscript, including the data and analysis.

Acknowledgements

Not applicable

Conflicts of interest

The authors declare no competing interests.

M.D. Zilberberg, B.H. Nathanson, J. Ways, A.F. Shorr, Characteristics, Hospital Course, and Outcomes of Patients Requiring Prolonged Acute Versus Short-Term Mechanical Ventilation in the United States, 2014–2018*, Crit Care Med. 48 (2020) 1587–1594. https://doi.org/10.1097/ccm.0000000000004525.
A. Jubran, B.J.B. Grant, L.A. Duffner, E.G. Collins, D.M. Lanuza, L.A. Hoffman, M.J. Tobin, Long-Term Outcome after Prolonged Mechanical Ventilation. A Long-Term Acute-Care Hospital Study., Am J Resp Crit Care. 199 (2019) 1508–1516. https://doi.org/10.1164/rccm.201806-1131oc.
J. Xie, G. Cheng, Z. Zheng, H. Luo, O.C. Ooi, To extubate or not to extubate: Risk factors for extubation failure and deterioration with further mechanical ventilation, J Cardiac Surg. 34 (2019) 1004–1011. https://doi.org/10.1111/jocs.14189.
A.W. Thille, J.-C.M. Richard, L. Brochard, The Decision to Extubate in the Intensive Care Unit, Am J Resp Crit Care. 187 (2013) 1294–1302. https://doi.org/10.1164/rccm.201208-1523ci.
K. Nitta, K. Okamoto, H. Imamura, K. Mochizuki, H. Takayama, H. Kamijo, M. Okada, K. Takeshige, Y. Kashima, T. Satou, A comprehensive protocol for ventilator weaning and extubation: a prospective observational study, J Intensive Care. 7 (2019) 50. https://doi.org/10.1186/s40560-019-0402-4.
A.W. Thille, G. Muller, A. Gacouin, R. Coudroy, M. Decavèle, R. Sonneville, F. Beloncle, C. Girault, L. Dangers, A. Lautrette, S. Cabasson, A. Rouzé, E. Vivier, A.L. Meur, J.-D. Ricard, K. Razazi, G. Barberet, C. Lebert, S. Ehrmann, C. Sabatier, J. Bourenne, G. Pradel, P. Bailly, N. Terzi, J. Dellamonica, G. Lacave, P.-É. Danin, H. Nanadoumgar, A. Gibelin, L. Zanre, N. Deye, A. Demoule, A. Maamar, M.-A. Nay, R. Robert, S. Ragot, J.-P. Frat, H.-W.S.G. and the R.R. Network, Effect of Postextubation High-Flow Nasal Oxygen With Noninvasive Ventilation vs High-Flow Nasal Oxygen Alone on Reintubation Among Patients at High Risk of Extubation Failure: A Randomized Clinical Trial, Jama. 322 (2019) 1465. https://doi.org/10.1001/jama.2019.14901.
B.D. Benneyworth, C.W. Mastropietro, E.M. Graham, D. Klugman, J.M. Costello, W. Zhang, M. Gaies, Variation in extubation failure rates after neonatal congenital heart surgery across Pediatric Cardiac Critical Care Consortium hospitals, J Thorac Cardiovasc Surg. 153 (2017) 1519–1526. https://doi.org/10.1016/j.jtcvs.2016.12.042.
A. Tulaimat, B. Mokhlesi, Accuracy and Reliability of Extubation Decisions by Intensivists, Respir Care. 56 (2011) 920–927. https://doi.org/10.4187/respcare.00851.
A.R. Baptistella, F.J. Sarmento, K.R. da Silva, S.F. Baptistella, M. Taglietti, R.Á. Zuquello, J.R.N. Filho, Predictive factors of weaning from mechanical ventilation and extubation outcome: A systematic review, J Crit Care. 48 (2018) 56–62. https://doi.org/10.1016/j.jcrc.2018.08.023.
V. Trivedi, D. Chaudhuri, R. Jinah, J. Piticaru, A. Agarwal, K. Liu, E. McArthur, M.C. Sklar, J.O. Friedrich, B. Rochwerg, K.E.A. Burns, The Utility of the Rapid Shallow Breathing Index in Predicting Successful Extubation: A Systematic Review and Meta-analysis, Chest. (2021). https://doi.org/10.1016/j.chest.2021.06.030.
W.-L. Su, Y.-H. Chen, C.-W. Chen, S.-H. Yang, C.-L. Su, W.-C. Perng, C.-P. Wu, J.-H. Chen, Involuntary Cough Strength and Extubation Outcomes for Patients in an ICU, Chest. 137 (2010) 777–782. https://doi.org/10.1378/chest.07-2808.
J.-R. Jiang, S.-Y. Yen, J.-Y. Chien, H.-C. Liu, Y.-L. Wu, C.-H. Chen, Predicting weaning and extubation outcomes in long-term mechanically ventilated patients using the modified Burns Wean Assessment Program scores., Respirol Carlton Vic. 19 (2014) 576–82. https://doi.org/10.1111/resp.12266.
S. Farghaly, A.A. Hasan, Diaphragm ultrasound as a new method to predict extubation outcome in mechanically ventilated patients, Aust Crit Care. 30 (2017) 37–43. https://doi.org/10.1016/j.aucc.2016.03.004.
F. Xu, C. Wan, L. Zhao, S. Liu, J. Hong, Y. Xiang, Q. You, L. Zhou, Z. Li, S. Gong, Y. Zhu, C. Chen, L. Zhang, Y. Gong, L. Li, C. Li, X. Zhang, C. Guo, K. Lai, C. Huang, D. Ting, H. Lin, C. Jin, Predicting Post-Therapeutic Visual Acuity and OCT Images in Patients With Central Serous Chorioretinopathy by Artificial Intelligence, Frontiers Bioeng Biotechnology. 9 (2021) 649221. https://doi.org/10.3389/fbioe.2021.649221.
L. Pasquini, A. Napolitano, M. Lucignani, E. Tagliente, F. Dellepiane, M.C. Rossi-Espagnet, M. Ritrovato, A. Vidiri, V. Villani, G. Ranazzi, A. Stoppacciaro, A. Romano, A.D. Napoli, A. Bozzao, AI and High-Grade Glioma for Diagnosis and Outcome Prediction: Do All Machine Learning Models Perform Equally Well?, Frontiers Oncol. 11 (2021) 601425. https://doi.org/10.3389/fonc.2021.601425.
E.K. Czyz, H.J. Koo, N. Al-Dajani, C.A. King, I. Nahum-Shani, Predicting short-term suicidal thoughts in adolescents using machine learning: developing decision tools to identify daily level risk after hospitalization., Psychol Med. (2021) 1–10. https://doi.org/10.1017/s0033291721005006.
F. Emmert-Streib, Z. Yang, H. Feng, S. Tripathi, M. Dehmer, An Introductory Review of Deep Learning for Prediction Models With Big Data., Frontiers Artif Intell. 3 (2020) 4. https://doi.org/10.3389/frai.2020.00004.
H.-J. Kuo, H.-W. Chiu, C.-N. Lee, T.-T. Chen, C.-C. Chang, M.-Y. Bien, Improvement in the Prediction of Ventilator Weaning Outcomes by an Artificial Neural Network in a Medical ICU, Respir Care. 60 (2015) 1560–1569. https://doi.org/10.4187/respcare.03648.
M.H. Hsieh, M.J. Hsieh, C.-M. Chen, C.-C. Hsieh, C.-M. Chao, C.-C. Lai, Comparison of machine learning models for the prediction of mortality of patients with unplanned extubation in intensive care units, Sci Rep-Uk. 8 (2018) 17116. https://doi.org/10.1038/s41598-018-35582-2.
T.-L. Tsai, M.-H. Huang, C.-Y. Lee, W.-W. Lai, Data Science for Extubation Prediction and Value of Information in Surgical Intensive Care Unit, J Clin Medicine. 8 (2019) 1709. https://doi.org/10.3390/jcm8101709.
Q.-Y. Zhao, H. Wang, J.-C. Luo, M.-H. Luo, L.-P. Liu, S.-J. Yu, K. Liu, Y.-J. Zhang, P. Sun, G.-W. Tu, Z. Luo, Development and Validation of a Machine-Learning Model for Prediction of Extubation Failure in Intensive Care Units, Frontiers Medicine. 8 (2021) 676343. https://doi.org/10.3389/fmed.2021.676343.
A. Fabregat, M. Magret, J.A. Ferré, A. Vernet, N. Guasch, A. Rodríguez, J. Gómez, M. Bodí, A Machine Learning decision-making tool for extubation in Intensive Care Unit patients, Comput Meth Prog Bio. 200 (2021) 105869. https://doi.org/10.1016/j.cmpb.2020.105869.
D.R. Ouellette, S. Patel, T.D. Girard, P.E. Morris, G.A. Schmidt, J.D. Truwit, W. Alhazzani, S.M. Burns, S.K. Epstein, A. Esteban, E. Fan, M. Ferrer, G.L. Fraser, M.N. Gong, C.L. Hough, S. Mehta, R. Nanchal, A.J. Pawlik, W.D. Schweickert, C.N. Sessler, T. Strøm, J.P. Kress, Liberation From Mechanical Ventilation in Critically Ill Adults: An Official American College of Chest Physicians/American Thoracic Society Clinical Practice Guideline Inspiratory Pressure Augmentation During Spontaneous Breathing Trials, Protocols Minimizing Sedation, and Noninvasive Ventilation Immediately After Extubation, Chest. 151 (2017) 166–180. https://doi.org/10.1016/j.chest.2016.10.036.
S.K. EPSTEIN, R.L. CIUBOTARU, Independent Effects of Etiology of Failure and Time to Reintubation on Outcome for Patients Failing Extubation, Am J Resp Crit Care. 158 (1998) 489–493. https://doi.org/10.1164/ajrccm.158.2.9711045.
S. Kullback, R.A. Leibler, On Information and Sufficiency, Ann Math Statistics. 22 (1951) 79–86. https://doi.org/10.1214/aoms/1177729694.
X. Chen, J.C. Jeong, Enhanced Recursive Feature Elimination, Sixth Int Conf Mach Learn Appl Icmla 2007. (2007) 429–435. https://doi.org/10.1109/icmla.2007.35.
L. Breiman, J.H. Friedman, R.A. Olshen, C.J. Stone, Classification And Regression Trees, 1984. https://doi.org/10.1201/9781315139470.
N.V. Chawla, K.W. Bowyer, L.O. Hall, W.P. Kegelmeyer, SMOTE: Synthetic Minority Over-sampling Technique, Arxiv. 16 (2011) 321–357. https://doi.org/10.1613/jair.953.
S. Lundberg, S.-I. Lee, A Unified Approach to Interpreting Model Predictions, Arxiv. (2017).
T. Otaguro, H. Tanaka, Y. Igarashi, T. Tagami, T. Masuno, S. Yokobori, H. Matsumoto, H. Ohwada, H. Yokota, Machine Learning for Prediction of Successful Extubation of Mechanical Ventilated Patients in an Intensive Care Unit: A Retrospective Observational Study, J Nippon Med Sch. 88 (2021) 408–417. https://doi.org/10.1272/jnms.jnms.2021_88-508.
Y. Jia, C. Kaul, T. Lawton, R. Murray-Smith, I. Habli, Prediction of weaning from mechanical ventilation using Convolutional Neural Networks, Artif Intell Med. 117 (2021) 102087. https://doi.org/10.1016/j.artmed.2021.102087.
M.-Y. Lin, C.-C. Li, P.-H. Lin, J.-L. Wang, M.-C. Chan, C.-L. Wu, W.-C. Chao, Explainable Machine Learning to Predict Successful Weaning Among Patients Requiring Prolonged Mechanical Ventilation: A Retrospective Cohort Study in Central Taiwan, Frontiers Medicine. 8 (2021) 663739. https://doi.org/10.3389/fmed.2021.663739.
M.H. Hsieh, M.J. Hsieh, A.-C. Cheng, C.-M. Chen, C.-C. Hsieh, C.-M. Chao, C.-C. Lai, K.-C. Cheng, W. Chou, Predicting weaning difficulty for planned extubation patients with an artificial neural network, Medicine. 98 (2019) e17392. https://doi.org/10.1097/md.0000000000017392.
M.-H. Hsieh, M.-J. Hsieh, C.-M. Chen, C.-C. Hsieh, C.-M. Chao, C.-C. Lai, An Artificial Neural Network Model for Predicting Successful Extubation in Intensive Care Units, J Clin Medicine. 7 (2018) 240. https://doi.org/10.3390/jcm7090240.
L.M. Fleuren, T.A. Dam, M. Tonutti, D.P. de Bruin, R.C.A. Lalisang, D. Gommers, O.L. Cremer, R.J. Bosman, S. Rigter, E.-J. Wils, T. Frenzel, D.A. Dongelmans, R. de Jong, M. Peters, M.J.A. Kamps, D. Ramnarain, R. Nowitzky, F.G.C.A. Nooteboom, W. de Ruijter, L.C. Urlings-Strop, E.G.M. Smit, D.J. Mehagnoul-Schipper, T. Dormans, C.P.C. de Jager, S.H.A. Hendriks, S. Achterberg, E. Oostdijk, A.C. Reidinga, B. Festen-Spanjer, G.B. Brunnekreef, A.D. Cornet, W. van den Tempel, A.D. Boelens, P. Koetsier, J. Lens, H.J. Faber, A. Karakus, R. Entjes, P. de Jong, T.C.D. Rettig, S. Arbous, S.J.J. Vonk, M. Fornasa, T. Machado, T. Houwert, H. Hovenkamp, R.N. Londono, D. Quintarelli, M.G. Scholtemeijer, A.A. de Beer, G. Cinà, A. Kantorik, T. de Ruijter, W.E. Herter, M. Beudel, A.R.J. Girbes, M. Hoogendoorn, P.J. Thoral, P.W.G. Elbers, J. Koeter, R. van Rietschote, M.C. Reuland, L. van Manen, L. Montenij, J. van Bommel, R. van den Berg, E. van Geest, A. Hana, B. van den Bogaard, P. Pickkers, P. van der Heiden, C. van Gemeren, A.J. Meinders, M. de Bruin, E. Rademaker, F.H.M. van Osch, M. de Kruif, N. Schroten, K.S. Arnold, J.W. Fijen, J.J.M. van Koesveld, K.S. Simons, J. Labout, B. van de Gaauw, M. Kuiper, A. Beishuizen, D. Geutjes, J. Lutisan, B.P. Grady, R. van den Akker, T.A. Rijpstra, W.G. Boersma, D. Pretorius, M. Beukema, B. Simons, A.A. Rijkeboer, M. Aries, N.C.G. van den Oever, M. van Tellingen, A. Dijkstra, R. van Raalte, Predictors for extubation failure in COVID-19 patients using a machine learning approach, Crit Care. 25 (2021) 448. https://doi.org/10.1186/s13054-021-03864-3.
X. Li, X. Xu, F. Xie, X. Xu, Y. Sun, X. Liu, X. Jia, Y. Kang, L. Xie, F. Wang, G. Xie, A Time-Phased Machine Learning Model for Real-Time Prediction of Sepsis in Critical Care, Crit Care Med. 48 (2020) e884–e888. https://doi.org/10.1097/ccm.0000000000004494.
A.J.E. Seely, A. Bravi, C. Herry, G. Green, A. Longtin, T. Ramsay, D. Fergusson, L. McIntyre, D. Kubelik, D.E. Maziak, N. Ferguson, S.M. Brown, S. Mehta, C. Martin, G. Rubenfeld, F.J. Jacono, G. Clifford, A. Fazekas, J. Marshall, C.C.C.T.G. (CCCTG), Do heart and respiratory rate variability improve prediction of extubation outcomes in critically ill patients?, Critical Care Lond Engl. 18 (2014) R65. https://doi.org/10.1186/cc13822.
O.F.C. van den Bosch, R. Alvarez-Jimenez, H.-J. de Grooth, A.R.J. Girbes, S.A. Loer, Breathing variability—implications for anaesthesiology and intensive care, Crit Care. 25 (2021) 280. https://doi.org/10.1186/s13054-021-03716-0.
M.-Y. Bien, S.-S. Hseu, H.-W. Yien, B.I.-T. Kuo, Y.-T. Lin, J.-H. Wang, Y.R. Kou, Breathing pattern variability: a weaning predictor in postoperative patients recovering from systemic inflammatory response syndrome, Intens Care Med. 30 (2004) 241–247. https://doi.org/10.1007/s00134-003-2073-8.
L. Sarlabous, J. Aquino-Esperanza, R. Magrans, C. de Haro, J. López-Aguilar, C. Subirà, M. Batlle, M. Rué, G. Gomà, A. Ochagavia, R. Fernández, L. Blanch, Development and validation of a sample entropy-based method to identify complex patient-ventilator interactions during mechanical ventilation., Sci Rep-Uk. 10 (2020) 13911. https://doi.org/10.1038/s41598-020-70814-4.
J.H. Chen, S.M. Asch, Machine Learning and Prediction in Medicine — Beyond the Peak of Inflated Expectations, New Engl J Medicine. 376 (2017) 2507–2509. https://doi.org/10.1056/nejmp1702071.
L. McDonald, S.V. Ramagopalan, A.P. Cox, M. Oguz, Unintended consequences of machine learning in medicine?, F1000research. 6 (2017) 1707. https://doi.org/10.12688/f1000research.12693.1.

Table 1. Demographic characteristics and weaning profiles of the patients under study.

Characteristic	All (n = 233)		Success (n = 205)		Failure (n = 28)		p
Age, years	73.0	(61.8–81.3)	73.0	(61.0–82.0)	72.5	(64.5–80.5)	0.980
Gender (%)
Male	147	(63.1)	132	(64.4)	15	(53.6)	0.267
Female	86	(36.9)	73	(35.6)	13	(46.4)
APACHE II score	21.0	(16.0–26.0)	21.0	(16.0–26.0)	20.0	(16.0–25.5)	0.982
Hospital stay, days	15.2	(10.1–22.3)	14.7	(9.7–21.7)	18.3	(14.3–25.4)	0.006
Mechanical ventilation duration, days	3.8	(2.3–7.1)	3.8	(2.2–7.4)	5.4	(3.0–7.0)	0.311
Reasons for intubation (%)
Pneumonia	106	(45.5)	94	(45.9)	12	(42.9)	0.369
ICH	28	(12.0)	24	(11.7)	4	(14.3)
Post surgery	25	(10.7)	23	(11.2)	2	(7.1)
AMI	19	(8.2)	18	(8.8)	1	(3.6)
Septic shock	19	(8.2)	18	(8.8)	1	(3.6)
Medical emergency	19	(8.2)	16	(7.8)	23	(10.7)
Stroke	9	(3.9)	6	(2.9)	3	(10.7)
Others	8	(3.4)	6	(2.9)	3	(7.1)
Weaning profiles
Pimax	24.0	(-32.0–40.0)	24.0	(-32.0–40.0)	-6.0	(-40.0–40.0)	0.281
Pemax	44.0	(32.0–60.0)	45.0	(34.0–60.0)	41.0	(24.0–60.0)	0.353
RSBI_average	56.2	(37.9–79.3)	57.4	(37.4–86.4)	50.0	(41.4–69.0)	0.543
RSBI_pass (%)	200	(85.8)	173	(84.4)	27	(96.4)	0.087
Cuff leak test_average	346	(160–500)	350	(162–500)	265	(139–476)	0.574
Cuff leak test_pass (%)	198	(85.0)	175	(85.4)	23	(82.1)	0.655

AMI, acute myocardial infarction; APACHE II score, Acute Physiology and Chronic Health Evaluation II score; ICH, intracerebral hemorrhage; Pemax, maximal expiratory pressure; Pimax, maximal inspiratory pressure; RSBI, rapid shallow breathing index.

Table 2. Feature importance of time series ventilator-derived parameters in the mean of different seconds.

Ventilatory parameters	Per 1 s	Per 30 s	Per 60 s	Per 120 s	Per 180 s	Per 300 s
FiO₂	0.108	0.127	0.133	0.140	0.144	0.128
Ppeak	0.083	0.121	0.131	0.139	0.142	0.140
PEEP	0.043	0.076	0.088	0.099	0.105	0.113
Pmean	0.034	0.057	0.066	0.071	0.066	0.065
RR	0.022	0.025	0.026	0.027	0.027	0.031
Vte	0.018	0.024	0.024	0.025	0.025	0.027
FiO₂_diff	0	0	0.002	0.002	0.004	0.005
Ppeak_diff	0	0.004	0.038	0.002	0.003	0.004
PEEP_diff	0	0.003	0.004	0	0.006	0
Pmean_diff	0	0.004	0.003	0	0	0
RR_diff	0.001	0.009	0.007	0.008	0.005	0.005
Vte_diff	0.001	0.007	0.006	0.006	0.008	0.010

Diff, degree of difference; FiO₂, fraction of inspiration oxygen; PEEP, positive end-expiratory pressure; Pmean, mean airway pressure; Ppeak, peak airway pressure; RR, respiratory rate; sq, squared; Vte, expiratory tidal volume.

Table 3. Performance comparison among different machine learning models.

Model

AUC

(95% CI)

Sensitivity (%)

(95% CI)

Specificity (%)

(95% CI)

PPV (%)

(95% CI)

NPV (%)

(95% CI)

F1 score (%)

(95% CI)

Accuracy (%)

(95% CI)

Logistic Regression

0.730

(0.729 – 0.730)

14.7

(14.3 – 15.0)

96.6

(96.5 – 96.6)

63.7

(63.2 – 64.2)

73.4

(73.3 – 73.5)

83.4

(83.4 – 83.5)

72.8

(72.7 – 72.9)

Random Forest original

0.976

(0.975 – 0.976)

87.5

(87.0 – 87.9)

96.7

(96.5 – 96.9)

91.6

(91.1 – 92.1)

95.0

(94.8 – 95.1)

95.8

(95.7 – 96.0)

94.0

(93.8 – 94.3)

Random Forest SMOTE

0.979

(0.978 – 0.980)

92.5

(92.1 – 92.9)

93.0

(92.7 – 93.3)

84.4

(83.9 – 85.0)

96.8

(96.6 – 97.0)

94.9

(94.7 – 95.0)

92.9

(92.7 – 93.1)

Support Vector Machine

0.860

(0.858 – 0.861)

51.3

(51.0 – 51.6)

97.3

(97.2 – 97.4)

88.5

(88.2 – 88.8)

83.0

(82.9 – 83.1)

89.6

(89.5 – 89.6)

83.9

(83.8 – 84.0)

AUC, area under receiver operating characteristic; CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value

No competing interests reported.

SupplementaryFigure1.tif
Supplementary Figure 1.Flowchart of study subject enrollment.
SupplementaryFigure2.tif
Supplementary Figure 2.The summary plot of the SHAP value of all ventilator-derived features in the RF model. FiO₂, fraction of inspiration oxygen; PEEP, positive end-expiratory pressure; Pmean, mean airway pressure; Ppeak, peak airway pressure; RR, respiratory rate; Vte, expiratory tidal volume.
SupplementaryTable1.docx
SupplementaryTable2.docx

Download PDF

Version 1

posted

You are reading this latest preprint version

A machine learning model for prediction of successful extubation in patients admitted to the intensive care unit

Status:

Version 1

Abstract

Background and objective:

Methods

Results

Conclusion

Figures

Introduction

Methods

Study participants

Weaning procedure

Data collection

Data processing

Feature extraction

Modeling

Statistical analysis

Results

Discussion

Conclusions

Declarations

References

Tables

Additional Declarations

Supplementary Files

Status:

Version 1