A total of 16,865 adult admissions were included in this study. Of these patients, 118 (0.7%) experienced cardiac arrest in a general ward (Table 1).
Table 1. Characteristics of the Study Population
Characteristic
|
Model Derivation
|
Model validation
|
Study period
|
Aug. 2016 ~ Nov. 2018
|
Dec. 2018 ~ Sep. 2019
|
Total patients
|
11,762
|
5,103
|
Patients with IHCA
|
81
|
37
|
Age, mean ± SD
|
63.8 ± 19.9
|
63.7 ± 20.5
|
Male sex (%)
|
5,875 (49.9)
|
2,293 (44.9)
|
Weight, mean ± SD
|
63.2 ± 14.7
|
63.3 ± 17.6
|
We used two tasks to test the performance of our proposed TEWS. We then compared the results of our system with those of other classification algorithms. The tasks are detailed as follows.
First Task: Prediction of IHCA Using Features (Vital Signs) Recorded in One Time Window (TW; 8 h), Three TWs (24 h), and Six TWs (48 h)
In the 1TW (8 h) group, we applied one set of five vital signs (i.e., features obtained in one TW) to predict IHCA events by using the proposed TEWS. The performance of the TEWS model was then compared with that of the MEWS and other classifiers, as displayed in Fig. 1. The support vector machine (SVM) and logistic regression algorithms had the highest AUROC values (0.729 and 0.721, respectively), followed by gradient boosting (0.712) and the TEWS (0.688). However, no classifier adequately outperformed the MEWS.
In the 3TW (24 h) group, we applied features recorded in three TWs (24 h) to predict IHCA events by using the TEWS. Each TW included a single value; therefore, three TWs with measurements for 5 vital signs contained a total of 15 features. The AUROC value of the TEWS (0.762) was superior to those of the logistic regression (0.730), random forest (0.676), MEWS (0.649), and other algorithms.
In the 6TW group (48 h), we applied features recorded in six TWs (48 h) to predict IHCA events by using the TEWS. The AUROC value of the TEWS (0.808) was superior to those of gradient boosting (0.768), SVM (0.747), random forest (0.733), and other algorithms.
When we used features from a single TW, most of the classification algorithms exhibited similar performance levels. The AUROCs of these models were within 0.62–0.73 (AUROC of MEWS: 0.65). When we used more TWs, some algorithms exhibited improved performance levels. Our TEWS demonstrated more favorable prediction in the 6TW (AUROC = 0.808, area under the precision–recall curve [AUPRC] = 0.052) than the MEWS did (AUROC = 0.649, AUPRC = 0.015).
Second Task: Prediction of IHCA Using Features Selected Through Sequential Backward Selection (SBS)
The TEWS had the most favorable performance in the first task when six TWs (48 h) were included. Because six TWs comprise 30 features, we sought a means of reducing the required features without compromising performance. We selected the most relevant features in the six TWs by using an SBS algorithm. These selected features are presented in Table 2. The first TW was the time closest to the cardiopulmonary resuscitation time for patients who were IHCA-positive. Heart rate, respiratory rate, and systolic blood pressure were determined to be the most relevant features for predicting IHCA events. The top five features were heart rate in the first, fourth, and fifth TWs and respiratory rate and systolic blood pressure in the first TW.
Table 2. Features Selected Through Sequential Backward Selection (SBS)
|
6th TW
|
5th TW
|
4th TW
|
3rd TW
|
2nd TW
|
1st TW
|
Heat rate
|
|
√
|
√
|
|
|
√
|
Temperature
|
|
|
|
|
|
|
Respiratory rate
|
|
|
|
|
|
√
|
Systolic blood pressure
|
|
|
|
|
|
√
|
Diastolic blood pressure
|
|
|
|
|
|
|
Furthermore, we applied the five selected features to the TEWS model and the other algorithms. A comparison of the AUROCs and AUPRCs for the algorithms the five features were introduced into is presented in Fig. 2. The TEWS (AUROC = 0.875, AUPRC = 0.087), Adaboost (AUROC = 0.958, AUPRC = 0.110), and logistic regression (AUROC = 0.845, AUPRC = 0.050) achieved their highest performance.