Study sample
A total of 1199 (609 training, 304 test, 207 validation by time, and 79 validation by site) participants were enrolled. The mean (SD) age was 35.1 (16.4) years old and 599 patients (50.0%) were female. There were some differences in basic clinical characteristics of patients across training, test and 2 validation samples. And the difference between the training group and the test and validation groups demonstrates the predictability and extrapolation of the model (Table 1).
Table 1
Patient characteristics by sample groups.
Characteristics | Aggregate | Training | Test | Validation_time | Validation_site | P_VALUE |
Total | 1199 | 609 | 304 | 207 | 79 | |
inhos_death n(%) | 454 (37.9) | 247 (40.6) | 117 (38.5) | 87 (42.0) | 3 (3.8) | < 0.0001 |
female n(%) | 599 (50.0) | 297 (48.8) | 172 (56.6) | 93 (44.9) | 37 (46.8) | 0.0444 |
age mean(std) median(q1,q3) | 35.1(16.4) 34(23, 47) | 34.7(16.5) 34(23, 46) | 34(16.2) 30.5(23, 46) | 36.1(17.2) 34(25, 48) | 40.1(12.8) 40(29, 48) | 0.0104 |
age_lt_50 n(%) | 963 (80.3) | 495 (81.3) | 246 (80.9) | 159 (76.8) | 63 (79.7) | 0.5614 |
age_ge_50 n(%) | 236 (19.7) | 114 (18.7) | 58 (19.1) | 48 (23.2) | 16 (20.3) | 0.5614 |
ALT1 mean(std) median(q1,q3) | 46.8(114.6) 18(12, 33) | 46.8(102.2) 18(12, 34) | 40.2(81.1) 16(12, 30.5) | 62.1(187.4) 19(12, 41) | 32(32.5) 20(13, 32) | 0.4286 |
ALT1_lt_100 | 1073 (89.5) | 546 (89.7) | 274 (90.1) | 179 (86.5) | 74 (93.7) | 0.3053 |
ALT1_ge_100 | 126 (10.5) | 63 (10.3) | 30 (9.9) | 28 (13.5) | 5 (6.3) | 0.3053 |
AST1 | 57.1(139) 23(17, 39) | 58.9(146) 23(18, 41) | 53.3(117.6) 23(17, 37.5) | 64.7(168.3) 23.2(18, 41) | 38.3(39.3) 22(17, 32) | 0.3409 |
AST1_lt_60 | 982 (81.9) | 498 (81.8) | 249 (81.9) | 170 (82.1) | 65 (82.3) | 0.9992 |
AST1_ge_60 | 217 (18.1) | 111 (18.2) | 55 (18.1) | 37 (17.9) | 14 (17.7) | 0.9992 |
BUN1 | 6.8(5.8) 5.2(4, 7.4) | 6.8(5.5) 5.4(4, 7.7) | 6.7(5.5) 5(3.9, 7.3) | 7.1(7.2) 5.2(4, 7.3) | 6.1(5.4) 4.9(4.1, 6.5) | 0.3161 |
BUN1_lt_10 | 1045 (87.2) | 530 (87.0) | 259 (85.2) | 180 (87.0) | 76 (96.2) | 0.0774 |
BUN1_ge_10 | 154 (12.8) | 79 (13.0) | 45 (14.8) | 27 (13.0) | 3 (3.8) | 0.0774 |
CK_MB1 | 29.4(38.6) 20.2(12.9, 33.6) | 31(39.6) 21(13.6, 36) | 31.3(40.3) 20.9(13.8, 32.7) | 28.6(37.7) 19.1(12.3, 33.3) | 12.1(14) 3.8(2, 25.5) | < 0.0001 |
CK_MB1_lt_50 | 1054 (87.9) | 518 (85.1) | 269 (88.5) | 189 (91.3) | 78 (98.7) | 0.0013 |
CK_MB1_ge_50 | 145 (12.1) | 91 (14.9) | 35 (11.5) | 18 (8.7) | 1 (1.3) | 0.0013 |
CR_1 | 108.4(124.7) 66(51, 117) | 108(108.2) 67(51, 123) | 109(123.1) 66(49.5, 117) | 114.4(152.9) 66(52, 105.5) | 92.6(163.1) 60.7(51.4, 81.6) | 0.4877 |
CR_1_lt_150 | 985 (82.2) | 492 (80.8) | 251 (82.6) | 168 (81.2) | 74 (93.7) | 0.0441 |
CR_1_ge_150 | 214 (17.8) | 117 (19.2) | 53 (17.4) | 39 (18.8) | 5 (6.3) | 0.0441 |
DBiL1 | 10.2(22.3) 5.1(3.4, 7.7) | 11(23.5) 5.1(3.6, 7.8) | 9.1(14.1) 5.4(3.5, 8.1) | 11.2(30.7) 4.8(3.3, 6.8) | 6.4(6.3) 4(2.9, 6.1) | 0.0033 |
DBiL1_lt_20 | 1080 (90.1) | 541 (88.8) | 277 (91.1) | 188 (90.8) | 74 (93.7) | 0.4418 |
DBiL1_ge_20 | 119 (9.9) | 68 (11.2) | 27 (8.9) | 19 (9.2) | 5 (6.3) | 0.4418 |
GGT1 | 50.7(95.8) 21(14, 47.8) | 50.6(101.3) 19(13.4, 37.3) | 43.4(74.1) 18.5(13.4, 40) | 56(119.1) 23.9(14, 50) | 65.6(42.1) 61.4(47, 79.2) | < 0.0001 |
GGT1_lt_150 | 1134 (94.6) | 569 (93.4) | 291 (95.7) | 196 (94.7) | 78 (98.7) | 0.1714 |
GGT1_ge_150 | 65 (5.4) | 40 (6.6) | 13 (4.3) | 11 (5.3) | 1 (1.3) | 0.1714 |
MPV | 9.2(3.6) 8.9(8, 9.9) | 9.1(1.5) 9(8.1, 9.8) | 9.2(1.6) 8.9(8, 10.2) | 9.6(8.1) 8.6(7.9, 9.7) | 9.2(1.1) 9.2(8.3, 9.9) | 0.0487 |
MPV_lt_10d5 | 1004 (83.7) | 518 (85.1) | 238 (78.3) | 179 (86.5) | 69 (87.3) | 0.0256 |
MPV_ge_10d5 | 195 (16.3) | 91 (14.9) | 66 (21.7) | 28 (13.5) | 10 (12.7) | 0.0256 |
N1 | 83.8(13.4) 88.9(79.3, 92.5) | 84.3(13.2) 89.2(80.2, 92.6) | 84.3(13.8) 89.5(81.1, 92.6) | 82.7(13.7) 87.9(76.2, 92.7) | 81(11.1) 84.1(74.2, 89) | 0.0007 |
N1_lt_80 | 310 (25.9) | 149 (24.5) | 68 (22.4) | 60 (29.0) | 33 (41.8) | 0.0029 |
N1_ge_80 | 889 (74.1) | 460 (75.5) | 236 (77.6) | 147 (71.0) | 46 (58.2) | 0.0029 |
PCT | 0.2(0.1) 0.2(0.1, 0.2) | 0.1(0.1) 0.1(0.1, 0.2) | 0.1(0.1) 0.1(0.1, 0.2) | 0.2(0.1) 0.2(0.1, 0.2) | 0.1(0.1) 0.1(0.1, 0.2) | < 0.0001 |
PCT_lt_0d2 | 903 (75.3) | 483 (79.3) | 240 (78.9) | 119 (57.5) | 61 (77.2) | < 0.0001 |
PCT_ge_0d2 | 296 (24.7) | 126 (20.7) | 64 (21.1) | 88 (42.5) | 18 (22.8) | < 0.0001 |
PLT | 179.4(90) 172(113, 236) | 173.6(87.7) 165(109, 226) | 170.2(90.5) 157(107, 218) | 218.4(88) 211(167, 271) | 158.1(84) 161(80, 220) | < 0.0001 |
PLT_lt_80 | 155 (12.9) | 79 (13.0) | 49 (16.1) | 11 (5.3) | 16 (20.3) | 0.0006 |
PLT_ge_80 | 1044 (87.1) | 530 (87.0) | 255 (83.9) | 196 (94.7) | 63 (79.7) | 0.0006 |
WBC1 | 14.1(8) 11.8(8.6, 17.3) | 14.4(8.4) 11.9(8.6, 18.5) | 14.4(7.9) 12.4(9.2, 17.7) | 13.2(7.1) 11.1(8.3, 16.5) | 12.3(6.5) 10.6(8.3, 14.4) | 0.0373 |
WBC1_lt_20 | 979 (81.7) | 484 (79.5) | 250 (82.2) | 173 (83.6) | 72 (91.1) | 0.0642 |
WBC1_ge_20 | 220 (18.3) | 125 (20.5) | 54 (17.8) | 34 (16.4) | 7 (8.9) | 0.0642 |
LDH1 | 279(151) 236.1(205, 299.3) | 289.4(102.8) 273.3(218.4, 333) | 310.8 (165.4) 259.7(235.5, 320.5) | 356.2(253.3) 265.1(221.7, 363.3) | 219.3(58.1) 209.6(182.8, 234.8) | < 0.0001 |
LDH1_lt_240 | 1101 (91.8) | 579 (95.1) | 277 (91.1) | 184 (88.9) | 61 (77.2) | < 0.0001 |
LDH1_ge_240 | 98 (8.2) | 30 (4.9) | 27 (8.9) | 23 (11.1) | 18 (22.8) | < 0.0001 |
ingestion_volume | 61.1(67.5) 40(20, 80) | 65.1(69.9) 40(20, 100) | 55.8(63.4) 30(15, 80) | 61.5(72) 30(15, 80) | 49.8(47.3) 40(20, 60) | 0.2922 |
Ingestion_volume_lt_100 | 927 (77.3) | 452 (74.2) | 244 (80.3) | 161 (77.8) | 70 (88.6) | 0.0141 |
Ingestion_volume_ge_100 | 272 (22.7) | 157 (25.8) | 60 (19.7) | 46 (22.2) | 9 (11.4) | 0.0141 |
LOS | 7.3(6.8) 5(2, 11) | 7.4(7) 5(2, 10) | 7.9(6.9) 6(2, 11.5) | 6.9(6.1) 5.1(2, 10.7) | 5.5(5.8) 3(1, 8) | 0.0045 |
In-hospital Death Events
The rates of in-hospital death were 40.6% (95% CI, 36. 6%-44.6%) and 38.5% (95% CI, 33.0%-44.2%) respectively for the training and test samples. The average in-hospital death was 37.9%. The median (interquartile range [IQR]) length of hospital stay was 5 (2–11) days (Table 1).
Risk Factor Selection And Test
The MCMC simulation selected 8 candidate factors with a posterior probability of at least 0.95 (Table 2), including age, ingestion volume, CK-MB, PLT, WBC, N, GGT and Cr (Fig. 2). The risk model based on the 8 risk factors and the training group demonstrated good discrimination, calibration and fit. The overall C statistic was 0.926 (95% CI, 0.891–0.924) for the risk model (Fig. 3). The mean observed in-hospital death ranged from 3.3% in the lowest predicted quintile to 99.2% in the highest predicted quintile, a range of 95.9% and the explained variation was 0.4999 (Fig. 4). Moreover, the P value of the Hosmer-Lemeshow goodness-of-fit test was 0.4262 in the training group and 0.9078 in the test group, indicating that the model fitted well with the observed cohort (Fig. 5).
Table 2
Final risk prediction model for in-hospital death event based on training group
Variable | Estimate | StdErr | Score_R* |
WBC1_ge_20 | 2.743299958 | 0.42703288 | 19 |
CK_MB1_ge_50 | 2.35461446 | 0.6455247 | 16 |
N1_ge_80 | 2.162847223 | 0.438185092 | 15 |
Ingestion_volume_ge_100 | 2.018090939 | 0.3082356 | 14 |
CR_1_ge_150 | 1.638231401 | 0.356155153 | 11 |
age_ge_50 | 1.427619459 | 0.329849932 | 10 |
GGT1_ge_150 | 1.387451403 | 0.566479088 | 9 |
PLT_lt_80 | 0.921076865 | 0.331850564 | 6 |
total | | | 100 |
*Risk scores were calculated by dividing a risk factor's coefficient by the sum of all coefficients, multiplying by 100, and rounding to the nearest integer. |
Additionally, model performance in the test sample was comparable to that in the training sample. The overall C Statistic was 0.895 (95% CI 0.855–0.928) in the test sample (Fig. 3). The rate of observed in-hospital death ranged from 4.6% in the lowest predicted quintile to 98.2% in the highest predictive quintile, the explained variation was 0.4182 for test group (Fig. 4).
Furthermore, in the latent class analysis, 609 patients in the training group were assigned into 5 classes based on the combination of the 8 risk factors (Fig. 6). For this analysis the area under the ROC curve was 0.877 (95% CI, 0.832–0.906), and the mean observed outcome rate ranged from 0.0% in the lowest rating group to 99.2% in the highest rating group. The spearman correlation coefficient between the predicted quintile based on the Logistic model and the latent class analysis was 0.754 (95% CI, 0.715–0.874).
Risk Score System
We observed that the risk factor-specific points ranged from 19 (WBC > = 20) to 6 (PLT < 80) (Table 2). WBC > = 20, CK-MB > = 50, N > = 80, Ingestion volume > = 100, and Cr > = 150 were the top 5 factors with an odds ratio more than 5.0 (Fig. 2). The training group has mean (SD) risk score of 26.6 (21.6). For the test group, the mean (SD) score was 24.8 (21.1). In addition, in the training sample, 18.4%, 59.9% and 21.7% of patients were stratified into the high, average and low-risk groups, with a corresponding probability of 0.985, 0.365, and 0.03 for in-hospital death respectively (Fig. 7). And the stratification for the test sample was not markedly different from that of the training group (Fig. 7, Table 3).
Table 3
Patient risk stratification based on risk score
Risk group | Training (n = 609) | Test (n = 304) | Validation_time (n = 207) | Validation_site (n = 79) |
| Patients # (%) | Probability of in-hospital death, mean (std) | Patients #(%) | Probability of in-hospital death, mean(std) | Patients #(%) | Probability of in-hospital death, mean(std) | Patients #(%) | Probability of in-hospital death, mean(std) |
High (46+) | 112 (18.4) | 0.98 (0.02) | 47 (15.5) | 0.99 (0.02) | 28 (13.5) | 0.98 (0.02) | 2 (2.5) | 0.97 (0.04) |
Average (15–45) | 365 (59.9) | 0.36 (0.28) | 189 (62.2) | 0.34 (0.25) | 123(59.4) | 0.38 (0.27) | 48 (60.8) | 0.33 (0.24) |
Low (0–14) | 132 (21.7) | 0.03 (0.02) | 68 (22.4) | 0.02 (0.02) | 56 (27.1) | 0.03 (0.03) | 29 (36.7) | 0.03 (0.02) |
Risk Model Validation
For the validation groups, the in-hospital death events rates were 42.0% (95% CI, 35.2%-49.1%) and 3.8% (95% CI, 0.8%-10.7%) for validation_time and validation_site respectively (Table 1). The observed in-hospital death rates ranged from 2.3%, 0.0% in the lowest predicted quintile to 94.9%, 12.5% in the highest predictive quintile, the explained variation was 0.3798, and 0.2421 for validation by time and validation by site respectively. Meanwhile, the overall C Statistic was 0.891 (95% CI 0.848–0.932) and 0.827 (95% CI 0.455-1.000) (Fig. 3) and the p value of Hosmer and Lemeshow's Goodness of Fit Test was 0.9671 and 0.9999 for the two independent groups (Fig. 5).
For the validation groups, the mean (SD) of risk scores were 23.5 (19.7), 16.3 (14.1), validation by time and validation by site group, respectively. In the validation_time group, 27.1%, 59.4% and 13.5% were classified into the low-, average- and high-risk groups, respectively, with corresponding probabilities of 0.98, 0.38 and 0.03 for in-hospital death events (Fig. 7 and Table 3). In the validation_site group, 36.7%, 60.8% and 2.5% were classified into low-, average- and high-risk groups, with a correspondence probability of 0.03, 0.33 and 0.97 respectively. The probabilities for in-hospital death events were identical to the training group (Fig. 7 and Table 3).