Population
We included 177 patients (63 [58–71] years, 58 females, 49 deaths). 23 patients were included during the first outbreak and 154 during the second (see flow chart diagrams, Fig. 1). The first 89 patients were used for algorithm training (30 intubations) and 88 as a validation set (29 intubations). Their basic baseline characteristics are listed in Table 1:
Table 1
Number
|
177
|
Age
|
63 [58–71]
|
Sexe ratio F/M
|
58 / 119 (33% / 67%)
|
Intubation
|
59 (33 %)
|
Length of Stay
|
8 [4–17]
|
Death
|
49 (27.6 %)
|
Algorithm accuracy
Among the tested algorithms, linear algorithms have the best performance in the validation set (Table 2).
Table 2
algorithms performances. AUC: Area under curve, PPV: Positive Predictive Value, Se: Sensibility. Algorithms: LD : Linear Discriminant, LOG Logistic regression classifier, SVM: SupportVector Machine, BN: Bayesian Naïve, CART: Classification And Regression Tree, BAG: bagging tree ensemble, RUS : RUSBosst algorithm, Neural : Neural Network algorithm.
|
Training
|
Validation
|
|
Accuracy
|
AUC
|
PPV
|
Se
|
Accuracy
|
AUC
|
PPV
|
Se
|
LD
|
86.5
|
0.97
|
85.2
|
76.7
|
92.0
|
0.96
|
84.4
|
93.1
|
LOG
|
87.6
|
0.97
|
85.2
|
76.7
|
92.0
|
0.96
|
84.4
|
93.1
|
SVM
|
88.8
|
0.97
|
83.3
|
83.3
|
90.9
|
0.95
|
81.8
|
93.1
|
BN
|
88.8
|
0.96
|
85.7
|
80.0
|
89.8
|
0.93
|
81.2
|
89.7
|
CART
|
94.4
|
0.95
|
90.3
|
93.3
|
87.5
|
0.90
|
80.0
|
82.8
|
BAG
|
91.0
|
0.97
|
84.0
|
81.3
|
89.8
|
0.93
|
81.2
|
89.7
|
RUS
|
91.0
|
0.88
|
84.0
|
81.4
|
84.1
|
0.91
|
72.7
|
82.8
|
NEURAL
|
92.1
|
0.98
|
89.7
|
86.7
|
88.6
|
0.94
|
80.6
|
86.2
|
We choose the logistic regression classifier due to its performances, generalizability and also because it is easy to deploy and use for the community. Its parameters are listed in Table 3:
Table 3
Logistic regression classifier parameters.
|
Estimate
|
SE
|
tstat
|
p-value
|
Intercept
|
36,2
|
15,8
|
2,2
|
0,02
|
BFmax
|
0,18
|
0,077
|
2,4
|
0,02
|
BF20
|
-1,16
|
2,8
|
-0,41
|
0,68
|
SpO2min
|
-0,51
|
0,19
|
-2,7
|
0,007
|
SPO2-90
|
6,6
|
3,2
|
2,05
|
0,04
|
S 24 Score
Patients that were intubated had a higher S24 score at least 80 hours before intubation (Fig. 2-A), and their score grows 24h before intubation continuously. We found the same difference between intubated and non-intubated patients in the validation cohort, ensuring its validity (Fig. 2-B). The S24 score was significantly higher the last 48 hours before intubation and remain high until intubation.
MS 24 score
MS24 score is the maximum of S24 score, disregarding the first 24 h of the stay. MS24 was highly correlated with the occurrence of intubation (Fig. 3-A). The event of intubation grows with MS24 score. The validation cohort follows the same law (Fig. 3-A), validating the score. We established a probability law that grows continuously with MS24 score (Fig. 3-B). The MS24 allows distinguishing three severity situations (green-orange-red). The cut-offs of these categories are established arbitrarily by using the probability law's natural inflexion points corresponding to a cumulative incidence of intubation of 30 and 76%.
Tableau 4
Number of intubation according to MS24 score.
The MS24 score allows distinguishing between three levels of intubation risk but also between different patients behaviours:
Green
(MS24 < 25) corresponds to patients that respond well to the therapeutics, and therefore their hourly score may increase but returns quickly to zero with a low risk of intubation (3 %). Figure 4-A shows a typical S24 evolution of such patients. The patient situation had a mild increase in S24 score but returned to a low score. One of the intubated patients was the first admitted patient in our ICU for COVID-19.
Orange
(MS24 ≤ 60) corresponds to patients that are unstable with large increases in the hourly score value that return more or less quickly to 0 with an increased risk of intubation (30%)(Fig. 4-A and B). It is difficult to say whether patients intubated in this category were intubated prematurely or, on the contrary, were intubated before they worsened. However, the patient in 4-A stabilized and improved dramatically, while patient in 4-B keeps worsening after a temporary improvement.
Red
(MS24 > 60) corresponds to highly unstable patients with a prolonged higher hourly score and an increased risk of intubation (76 %). Figure 4-C shows the typical evolution of red category patient. The increase of S24 score is continuous, and the patient stayed 40 hours at the maximum severity score before intubation.
MS 24 Score and ICU Length of stay
We studied the link between MS24 and the length of stay in ICU to use our score for triage. We excluded six intubated patients who died prematurely due to uncontrolled situation (cardiac arrest and pulmonary embolisms) for this study. MS24 was correlated to the ICU length of stay (Fig. 5-A). We found that a MS24 score superior to 20 was highly predictive of an ICU stay greater than 5 day with an accuracy of 88.8% (PPV = 93.0%, Sensibility = 90.7%, AUC = 0.95) (Fig. 5B and C). For intubated patients, 58/59 patients had a S24 score greater than 20 in the first 24 hours. Therefore, a prolonged ICU was predicted on the first day.