Development and validation of predictive models for mortality of cases with COVID-19 (Omicron BA.5.2.48 and B.7.14): A retrospective study

doi:10.21203/rs.3.rs-2862947/v1

Background

With the emergence of new COVID-19 variants (Omicron BA.5.2.48 and B.7.14), predicting the mortality of infected patients has become increasingly challenging due to the continuous mutation of the virus. Existing models have shown poor performance and limited clinical utility. This study aims to identify the independent risk factors and develop a practical predictive model for mortality among patients infected with new COVID-19 variants.

Methods

Demographic, clinical, and laboratory data of COVID-19 patients were retrospectively collected at our hospital between December 22, 2022, and February 15, 2023. Logistic regression (LR), decision tree (DT), and Extreme Gradient Boosting (XGBoost) models were developed to predict mortality. Those models were separately visualized via nomogram, decision trees, and Shapley Additive Explanations (SHAP). To evaluate those models, accuracy, sensitivity, specificity, precision, Youden’s index, and area under curve (AUC, 95% CI) were calculated.

Results

A total of 987 cases with new COVID-19 variants (Omicron BA.5.2.48 and B.7.14) were eventually included, among them, 153 (15.5%) died. Noninvasive ventilation, intubation, myoglobin, INR, age, number of diagnoses, respiratory, pulse, neutrophil, and albumin were the most important predictors of mortality among new COVID-19 variants. The AUC of LR, DT, and XGBoost models were 0.959, 0.878, and 0.961, respectively. The diagnostic accuracy was 0.926 for LR, 0.913 for DT, and 0.977 for XGBoost. XGBoost model had the highest sensitivity (0.983) and specificity (0.940).

Conclusion

Our study developed and validated three practical models for predicting mortality in patients with new COVID-19 variants. All models performed well, and XGBoost was the best-performing model.

COVID-19

Mortality

Prognostic model

XGBoost

SHAP

The coronavirus disease 2019 (COVID-19) pandemic has become a global emergency, posting not only a challenge to the healthcare system but also disrupting the global culture, society, and economy. According to statistics from the World health organization (WHO), there have been 760,360,956 cumulative COVID-19 cases until March 2023[1].

COVID-19 can cause severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2), which belongs to the β-coronavirus family [2]. As an RNA virus, the SARS-CoV-2 is easy to mutate, leading to failures in established detection and treatment methods. So far, several new variants of concern (VOC) have emerged, such as the Alpha variant (lineage B.1.1.7) [3], Beta (lineage B.1.351) [4], and Gamma (lineage P.1, B.1.1.7 and B.1.351) strain [5]. After the vaccine becomes available for public use in 2020, the Delta variant (lineage B.1.617.2) [6] and Omicron (lineage B.1.1.529) [7, 8] variants were discovered throughout the world. Moreover, the increasing cases of asymptomatic infection and loosening of COVID-19 management will promote the continuing dissemination of COVID-19 for a long time in the future.

Since its discovery, COVID-19 was characterized by high rates of severe illness rate, hospitalization and mortality. In 2019, the sudden influx of patients into the hospital crashed the medical and health system. Patients experienced respiratory tract involvements ranging from mild sore throat to severe pneumonia and tended to progress rapidly to fatal complications, such as acute respiratory distress syndrome (ARDS) and multiple organ dysfunction (MODS) [9, 10]. In the first year of COVID-19, despite the government published data on 6 million deaths [11], the WHO [12] and the Institute for Health Metrics and Evaluation (IHME) [13] reported 14.9 million and 18.2 million excess deaths in 2020, respectively.

Although the virulence of SARS-CoV-2 was weakened by mutation and more approaches have been developed to protect or treat COVID-19, experts concern that patients may not have long-lasting protective immunity, and recurrent infection of SARS-CoV-2 may lead to increases in death rates. As for 2023, the number of newly reported death cases is still over 4000 per week globally [1]. Considering the cost and effect, we need a tool to identify population with high risk of progression and mortality.

There are some existing prediction models for risk factors associated with death. At the very start, clinicians directly used the simplified acute physiology score version 3 (SAPS 3), a model widely used in intensive care unit (ICU) to evaluate disease severity and predict in-hospital mortality [14]. However, the SAPS 3 has proved to be inaccurate in estimating the mortality of COVID-19 inpatients in various pilots across regions [15, 16]. Thus, researches started to build custom-designed predictive model for COVID-19.

By initial stage clinical features extracted from the patient was modeled by reinforcement algorithm, such as Logistic regression (LR), Cox regression and gradient boosting decision tree (GBDT). Schmidt et al. discovered that high-sensitive troponin I concentration, fasting glycaemia, immature neutrophils count and high urine protein concentration were predictors for progression from moderate or severe to critical COVID-19 with a multivariable logistic regression model amoung 190 patients [17]. By using Cox regression model, Atlam et al. reported that the age, muscle pain, pneumonia and throat pain are most important predictors for the inpatient mortality [18]. Many algorithms are available to build helpful predictive models. However, most of them are difficult to interpret and apply in clinical practice. Nowadays, artificial intelligence (AI) models were developed to predict the outcomes of COVID-19 patients [18–20]. AI can recognize radiographic images with COVID-19 patterns. However, the imaging features are not obvious in the early stage, which limits the usage of these models.

During the study period, the main new variants of COVID-19 in Sichuan Province were Omicron BA.5.2.48 (66.37%) and B.7.14 (22.71%) [21]. Given the multiple rounds of infection and mutation, we require new and more effective method to screening high-risk and potentially high-risk groups for admission death-on-admission. Therefore, we conduct a comprehensive analysis to identify the independent risk factors for short-term mortality of COVID-19 pneumonia caused by the new variants and to determine the impact of predictive variables in quantifying the probability of short-term death among hospitalized patients with new COVID-19 variants.

Study design and participants

This retrospective study was conducted in the respiratory disease wards of our hospital between December 22, 2022, and February 15, 2023. A total of 1029 COVID-19 patients who tested positive for SARS-CoV-2 by Nucleic Acid Test (NAT) were included in the study. Four levels of disease severity for COVID-19 according to the Chinese Clinical Guidance for COVID-19 Pneumonia Diagnosis and Treatment (7th version) were defined: mild, moderate, severe, and critically ill [22]. In this study, we classified the severe and critically as severe cases, while the rest two levels as non-severe cases. Patients who were still in hospitalization after the study deadline or died within 24 h were excluded.

Data Collection

Demographic, clinical, and laboratory data were collected within 24 hours of admission and outcomes were extracted from electronic medical reports by the research team. The main outcome was the mortality of participants. Patients with unknown survival outcomes who discharged by themselves were followed up with the outcome 15 days after hospital discharge by phone call.

Demographic variables included age, gender, height, weight, nation, and history of drink and smoking. Clinical data included the severity of COVID-19 disease, number of diagnoses, the activity of daily living (ADL) measured by Modified Barthel Index [23], vital signs on admission (temperature, pulse, respiratory, blood pressure, and pain), clinical symptoms(fever, cough, dyspnea, distress, myalgia, stethalgia, headache), comorbidities (hypertension, diabetes, respiratory distress syndrome (RDS), chronic obstructive pulmonary disease (COPD), respiratory failure, phlebothrombosis, anemia, malnutrition, hypoproteinemia, organ transplant and more). The study also collected information on the modes of respiratory support (noninvasive ventilation, intubation, or tracheotomy tube) during the hospitalization.

Blood test data was collected within 24 hours of admission, including complete blood Count (erythrocyte count, hemoglobin, hemameba count, blood platelet count, neutrophil count, lymphocyte count), biochemical parameters (total protein (TP), alanine aminotransferase (ALT), aspartate aminotransferase (AST), AST/ALT, total bilirubin, albumin, glucose, creatinine, carbamide, glomerular filtration rate(GFR), lactic dehydrogenase (LDH)), electrolyte level (Na, K, Cl, Ca, Mg), coagulation index (prothrombin time (PT), international normalized ratio (INR), thrombin time (TT), fibrinogen), cardiac biomarkers (myoglobin, creatine kinase MB isoenzyme (CK-MB), troponin, brain natriuretic peptide (BNP)), procalcitonin and other factors (procalcitonin (PCT), C-reactive protein (CRP), IL-6) and blood gas analysis (Partial Pressure of Carbon Dioxide (PCO₂), arterial oxygen partial pressure (PaO₂), oxygen saturation (SPO₂)). The data were reviewed by two researchers and a third researcher adjudicated any differences and reverified outliers and missing values with electronic medical reports.

Statistical Analysis

Continuous variables, which are normal distribution, were described as mean (SD) and compared using an unpaired, 2-tailed t-test. Continuous variables, which are not normal distribution, were expressed as medians with interquartile ranges (IQRs) and compared using the Wilcoxon rank sum test. The Shapiro-Wilk test was used to estimate the normality of data. Categorical variables were reported as absolute values and percentages and compared using the Chi-square test and Fisher’s test.

Variables with > 20% missing values were deleted. We imputed other missing values by random forest (missForest, R package, version 4.2.1). Because candidate variables exist in high-dimensional complex collinearity, we used the Lasso regression model to select the most useful prognostic risk factors of death in COVID-19 patients. Lasso regression constructs a penalty function to select the main variables. Some variables, whose coefficients are compressed to 0 due to not strong correlation with the dependent variable, are eliminated. These independent variables strongly related to the dependent variable are screened and obtain corresponding coefficients. Tenfold cross-validation was used to select the optimal parameter (λ) based on the minimum criteria (λ min), and the 1 standard error of the minimum criteria (λ 1SE).

The variables, screened by Lasso regression, were used to formulate LR model. Then we formulated a nomogram to visualize the results of the LR model. All variables were included to construct a decision tree (DT) model in order to explore different predictors. Extreme Gradient Boosting (XGBoost) model was established based on the results of Lasso regression to predict the outcome of hospitalized patients with COVID-19. Further, we used Shapley Additive Explanations (SHAP) method to interpret the XGBoost model. SHAP value visually exhibited each feature’s importance and contribution to mortality.

To evaluate the prediction ability of different models, we conducted confusion matrix of each model to compare the actual values and predictive values and plotted the area under the receiver operating characteristic (ROC) curves. In the same time, we compared the accuracy, sensitivity, specificity, precision, Youden’s index, and area under curve (AUC, 95% CI) of those models.

All statistical analyses, except for XGBoost and SHAP, were implemented via R (version 4.1.3) with R software packages “dplyr”, “epiDisplay”, “gtsummary”, “missForest”, “glmnet”, “rms”, “car”, “rpart”, “partykit”, “caret”, “InformationValue”, and “pROC”. XGBoost and SHAP were carried out by python (version 2.7) with python software packages “numpy”, “pandas”, “xgboost”, and “shap”. A P-value lower than 0.05 (two-sided) was regarded as statistically significant.

Demographic and clinical characteristics

During the study period, 1029 patients with COVID-19 were admitted to our institution. Forty-two patients were excluded due to loss of follow-up, leaving a total of 987 patients were eventually included, 153(15.5%) died and 834(84.5%) survived. The median age of the patients was 69 years (interquartile range [IQR]: 54 to 81 years). There were 672 (68.1%) male patients. A total of 497 (50.4%) were classified as severe and 490 (49.6%) as non-severe cases. The majority of the patients had comorbidities, such as hypertension (411, 41.6%), hypoproteinemia (329, 33.3%), electrolyte disturbances (352, 35.7%), and diabetes (251, 25.4%). The most common COVID-19-related symptoms were cough (84.3%) and dyspnea (59.9%), followed by chest distress (17.9%) and myalgia (9.5%). Table 1 provided a comparison of characteristics between the survivors and the deceased patients.

Table 1

Demographics, clinical characteristics and laboratory findings of survival and death COVID-19 patients
Variable	Total	Survival	Death	Test stat	P
Age				-	< 0.001
median(IQR)	69(54,81)	67 (52,79)	82 (69,86)
Number of diagnoses				-	< 0.001
median(IQR)	9(6,13)	8 (5,12)	13 (10,15)
Respiratory				-	< 0.001
median(IQR)	20(20,22)	20 (20,21)	22 (20,24)
Pulse (n = 985)				-	< 0.001
median(IQR)	88(78,100)	87 (78,98)	98 (85,109)
Systolic pressure (n = 986)				-	< 0.001
median(IQR)	129(114,143)	127 (113,141)	135 (118,150)
Diastolic pressure (n = 986)				-	0.258
median(IQR)	76(68,85)	76 (68,85)	75 (66,85)
BMI (n = 718)				-	0.764
median(IQR)		23.3 (20.8,25.7)	23.1 (20.7,25.6)
ADL (n = 986)				-	< 0.001
median(IQR)	55(35,75)	60 (40,80)	30 (10,40)
Pain (n = 960)				-	0.683
median(IQR)	0(0,0)	0 (0,0)	0 (0,0)
Erythrocyte count, ×10⁹/L				-	< 0.001
median(IQR)	4.0(3.5,4.5)	4.1 (3.5,4.5)	3.7 (3.2,4.2)
Hemoglobin, g/L				-	< 0.001
median(IQR)	118(104,133)	120 (106,133)	109 (96,126)
Hemameba count, ×10⁹/L				-	< 0.001
median(IQR)	7.1(5.2,10.3)	6.9 (5.1,9.6)	9.7 (6,13.5)
Soterocyte count, ×10⁹/L	(n = 986)			-	< 0.001
median(IQR)	178(130,247)	184 (136,250)	147 (101,209)
Neutrophil count, ×10⁹/L				-	< 0.001
median(IQR)	5.3(3.6,8.2)	5.1 (3.5,7.5)	8.3 (4.9,12)
Lymphocyte count, ×10⁹/L				-	< 0.001
median(IQR)	0.8(0.5,1.3)	0.8 (0.5,1.3)	0.6 (0.3,0.9)
Total bilirubin, µmoI/L	(n = 985)			-	0.02
median(IQR)	9.2(6.7,12.7)	9 (6.7,12.3)	9.8 (7.1,14.4)
ALT, IU/L	(n = 985)			-	0.528
median(IQR)	25(15.2,44)	25 (16,43)	26 (14,44)
AST, IU/L	(n = 986)			-	< 0.001
median(IQR)	29(20,44)	27 (20,40)	39 (26,61)
AST/ALT	(n = 985)			-	< 0.001
median(IQR)	1.2(0.9,1.7)	1.1 (0.8,1.6)	1.8 (1.2,2.4)
Total protein, g/L	(n = 985)			-	< 0.001
median(IQR)	64.4(59.3,69.7)	64.8 (59.6,70)	61.6 (57.4,67.3)
Albumin, g/L	(n = 986)			-	< 0.001
median(IQR)	35(31.7,38.9)	35.6 (32.4,39.5)	31.9 (28.6,34.7)
Creatinine, µmoI/L	(n = 986)			-	< 0.001
median(IQR)	81(64,115)	78 (63,103)	109 (81,194)
Carbamide, ml/min	(n = 986)			-	< 0.001
median(IQR)	6.5(4.5,10.5)	5.9 (4.3,9.2)	13 (7.4,21.2)
GFR, ml/min/m²	(n = 986)			-	< 0.001
median(IQR)	78.5(51.4,93.9)	82 (57.9,95.2)	53.1 (26.6,76.7)
LDH, mmol/L	(n = 979)			-	< 0.001
median(IQR)	256(199,334)	241 (194,314)	349 (270,445)
Na, mmol/L	(n = 984)			-	0.646
median(IQR)	136,(132.9,139.5)	136.6 (133.3,139.4)	136.4 (132.3,140.3)
K, mmol/L	(n = 984)			-	0.001
median(IQR)	3.8(3.4,4.2)	3.8 (3.4,4.2)	4 (3.5,4.5)
Cl, mmol/L	(n = 984)			-	0.955
median(IQR)	102(98.6,105.8)	102.2 (98.7,105.5)	101.6 (97.7,106.5)
Ca, mmol/L	(n = 977)			-	< 0.001
median(IQR)	2.1(2, 2.2)	2.1 (2,2.2)	2 (1.9,2.2)
Mg, mmol/L	(n = 971)			-	0.001
median(IQR)	0.9(0.8,0.9)	0.8 (0.8,0.9)	0.9 (0.8,1)
Prothrombin time, s	(n = 956)			-	< 0.001
median(IQR)	11.5(10.8,12.4)	11.4 (10.7,12.2)	12.3 (11.5,13.5)
INR	(n = 954)			-	< 0.001
median(IQR)	1.1(1,1.1)	1 (1,1.1)	1.1 (1.1,1.2)
Thrombin time, s	(n = 944)			-	0.118
median(IQR)	16.7(15.8,17.5)	16.6 (15.8,17.4)	16.9 (15.9,17.6)
Fibrinogen, g/L	(n = 956)			-	0.303
median(IQR)	4.4(3.5,5.5)	4.4 (3.4,5.6)	4.5 (3.6,5.4)
DIC, mg/l FEU	(n = 940)			-	< 0.001
median(IQR)	1.2(0.6,2.8)	1 (0.5,2.3)	2.2 (1.1,8.5)
Myoglobin, ng/ml	(n = 919)			-	< 0.001
median(IQR)	48.4(27.1,96.7)	41.6 (23.6,75.8)	165 (83.4,411.7)
CK-M, ng/ml	(n = 918)			-	< 0.001
median(IQR)	1.2(0.7,2.2)	1.1 (0.6,1.9)	2.6 (1.1,4.9)
Troponin, ng/L	(n = 918)			-	< 0.001
median(IQR)	15.9(9.3,28.5)	13.6 (8.4,24.6)	39.7 (22.2,100.1)
BNP, ng/L	(n = 878)			-	< 0.001
median(IQR)	481.8(165,1513)	364.5 (120,1138.2)	1878 (663,4922.2)
PCT, ng/ml	(n = 949)			-	< 0.001
median(IQR)	0.1(0.1,0.4)	0.1 (0,0.2)	0.6 (0.3,2.7)
CRP, mg/L	(n = 893)			-	< 0.001
median(IQR)	49(14.5,105)	33.6 (10.7,86.1)	114 (66.6,175.8)
IL-6, pg/ml	(n = 874)			-	< 0.001
median(IQR)	22.7(7.4,73.4)	16.9 (5.1,49.4)	52.1 (23.6,183)
PCO_2, mmHg	(n = 791)			-	0.773
median(IQR)	33.4(29.4,36.5)	32.9 (28.8,37.5)	32.4 (27.9,39.6)
PaO_2, mmHg	(n = 796)			-	0.003
median(IQR)	93.1(76.1,120.7)	91.3 (74,128.9)	87.2 (63.2,118.7)
SPO_2, mmHg	(n = 790)			-	< 0.001
median(IQR)	98.4(96.9,99.3)	98.5 (96.6,99.5)	97.7 (94.7,99.1)
Gender				16.96	< 0.001
Female	315(31.9)	288 (34.5)	27 (17.6)
Male	672(28.1)	546 (65.5)	126 (82.4)
Nation				3.88	0.144
Han	918 (93)	770 (92.3)	148 (96.7)
Tibetan	53 (5.4)	49 (5.9)	4 (2.6)
Others	16 (1.6)	15 (1.8)	1 (0.7)
Severity of COVID-19				62.54	< 0.001
Non-severe	490(49.6)	459 (55)	31 (20.3)
Severe	497(50.4)	375 (45)	122 (79.7)
Respiratory failure				100.85	< 0.001
No	615 (62.3)	575 (68.9)	40 (26.1)
Yes	372 (37.7)	259 (31.1)	113 (73.9)
Hypertension				9.51	0.002
No	576 (58.4)	504 (60.4)	72 (47.1)
Yes	411 (41.6)	330 (39.6)	81 (52.9)
Diabetes				8.1	0.004
No	736 (74.6)	636 (76.3)	100 (65.4)
Yes	251 (25.4)	198 (23.7)	53 (34.6)
Hypoproteinemia				52.94	< 0.001
No	658 (66.7)	595 (71.3)	63 (41.2)
Yes	329 (33.3)	239 (28.7)	90 (58.8)
Electrolyte imbalance				49.8	< 0.001
No	635 (64.3)	575 (68.9)	60 (39.2)
Yes	352 (35.7)	259 (31.1)	93 (60.8)
COPD				0.01	0.933
No	828 (83.9)	700 (83.9)	128 (83.7)
Yes	159 (16.1)	134 (16.1)	25 (16.3)
Organ transplant				11.3	< 0.001
No	868 (87.9)	721 (86.5)	147 (96.1)
Yes	119 (12.1)	113 (13.5)	6 (3.9)
Anemia				8.68	0.003
No	766 (77.6)	662 (79.3)	104 (68.4)
Yes	221 (22.4)	173 (20.7)	48 (31.6)
Malnutrition				2.83	0.092
No	926 (93.8)	787 (94.4)	138 (90.8)
Yes	61 (6.2)	47 (5.6)	14 (9.2)
Cogh				0.92	0.337
No	155 (15.7)	127 (15.2)	28 (18.3)
Yes	832 (84.3)	707 (84.8)	125 (81.7)
Dyspnea				7.62	0.006
No	396 (40.1)	350 (42)	46 (30.1)
Yes	591 (59.9)	484 (58)	107 (69.9)
Distress				2.91	0.088
No	810 (82.1)	677 (81.2)	133 (86.9)
Yes	177 (17.9)	157 (18.8)	20 (13.1)
Myalgia				8.22	0.004
No	893 (90.5)	745 (89.3)	148 (96.7)
Yes	94 (9.5)	89 (10.7)	5 (3.3)
Stethagia				3.05	0.081
No	915 (92.7)	768 (92.1)	147 (96.1)
Yes	72 (7.3)	66 (7.9)	6 (3.9)
Headache				0.34	0.559
No	925 (93.7)	780 (93.5)	145 (94.8)
Yes	62 (6.3)	54 (6.5)	8 (5.2)
Smoke				3.93	0.047
No	764(77.4)	655 (78.5)	109 (71.2)
Yes	223(22.6)	179 (21.5)	44 (28.8)
Drink				0.16	0.686
No	861(87.2)	726 (87.1)	135 (88.2)
Yes	126(12.8)	108 (12.9)	18 (11.8)
Noninvasive ventilation				198.29	< 0.001
No	781(79.1)	725 (86.9)	56 (36.6)
Yes	206(20.9)	109 (13.1)	97 (63.4)
Intubation				178.52	< 0.001
No	935(94.7)	824 (98.8)	111 (72.5)
Yes	52(5.3)	10 (1.2)	42 (27.5)
Tracheotomy tube				-	0.079
No	980(99.3)	830 (99.5)	150 (98)
Yes	7(0.7)	4 (0.5)	3 (2)

Factors Selection

Due to a high proportion of missing data (27.25%), body mass index (BMI) was excluded from further analysis. Other variables with missing values below 20% were imputed using a random forest model. Lasso regression analysis was then performed to select predictive factors of death in COVID-19 patients (Fig. 1). The tuning parameter (λ) was selected by tenfold cross-validation (Fig. 1A). Two dotted vertical lines were drawn at λ min and λ 1SE. The λ min was a lambda value that yielded the highest AUC, indicating good model performance. On the other hand, the λ 1SE resulted in a model with good performance but a minimum number of variables. Log (λ 1SE) was chosen as the optimal value, which was equal to -3.31. A coefficient profile plot was produced (Fig. 1B), with a dotted vertical line drawn at the λ 1SE criteria, resulting in 15 nonzero coefficients. The 15 predictive factors were age, number of diagnoses, respiratory, pulse, ADL, neutrophil, albumin, GFR, LDH, INR, myoglobin, CRP, respiratory failure, intubation, and noninvasive ventilation. The coefficients of 15 variables were listed in Table 2.

Table 2

Risk factors selected by LASSO regression
No.	Variables	Coefficient
2	Age	0.0036337476
3	Number of diagnoses	0.0204194568
4	Respiratory	0.0214225215
5	Pulse	0.0112186222
8	ADL	-0.0087008606
14	Neutrophil	0.0075261677
21	Albumin	-0.0068663566
24	GFR	-0.0047710931
25	LDH	0.0004261510
32	INR	0.0737720551
36	Myoglobin	0.0003993199
41	CRP	0.0011195632
49	Respiratory failure	0.1699494031
67	Noninvasive ventilation	1.6374886079
68	Intubation	1.9264314263

LR model

Based on the Lasso regression results, a LR model was constructed with a variance inflation factors (VIF) of less than 2 for the predictive factors. The LR model revealed that age (OR = 1.03, p = 0.001), number of diagnoses (OR = 1.07, p = 0.041), respiratory (OR = 1.13, p = 0.002), INR (OR = 2.29, p = 0.013), CRP (OR = 1, p = 0.009), respiratory failure (OR = 1.98, p = 0.027), pulse (OR = 1.04, p<0.001), noninvasive ventilation (OR = 11.92, p<0.001), intubation (OR = 18.48, p<0.001) were independent risk factors, while albumin (OR = 0.92, p = 0.004) and GFR (OR = 0.98, p = 0.002) were independent protective factor for the mortality of COVID-19 (Table 3).

Table 3

Results of LR model
Variables	SE	OR	CI	P
Age	0.009792	1.03	1.01–1.05	0.001
Number of diagnoses	0.033709	1.07	1-1.14	0.041
Respiratory	0.039221	1.13	1.05–1.22	0.002
Pulse	0.00783	1.04	1.02–1.05	<0.001
ADL	0.00677	0.99	0.97-1	0.049
Neutrophil	0.02672	1.04	0.99–1.1	0.124
Albumin	0.028592	0.92	0.87–0.97	0.004
GFR	0.005233	0.98	0.97–0.99	0.002
LDH	0.001038	1	1–1	0.11
INR	0.333025	2.29	1.19–4.4	0.013
Myoglobin	0.000346	1	1–1	0.066
CRP	0.001166	1	1-1.01	0.009
Respiratory failure	0.307106	1.98	1.08–3.61	0.027
Noninvasive ventilation	0.30509	11.92	6.55–21.67	<0.001
Intubation	0.506045	18.48	6.86–49.84	<0.001

To visualize the results of the prediction model, a nomogram was created using the independently associated factors from the LR analysis. The nomogram transformed the complex regression equation into a visual graph, making it easier for clinicians to interpret and apply the results (Fig. 2). The sum of the assigned number of points for each factor in the nomograms corresponded to the total ponts, and higher total points were associated with a higher mortality. According to nomograms, INR has the greatest impact on total points, while respiratory failure had the smallest impact.

DT Model

The DT model is visible, clinically operable, and easy to interpret due to its recursive tree-based decision system. First, we input all variables to build a DT model. When it split 12 times, and the error rate was minimum (xerror = 0.673), a certain but complex tree model was built (Supplementary Fig. 1). Second, we used cross-validation to trim back the full tree. We set the max nodes of split no more than 4 and chose the smallest error rate in order to obtain a meaningful and simple DT model (Fig. 3). Patients who are not noninvasively ventilated and intubated have a very low risk of death of 3.4%. However, Fig. 3 indicated a high risk of death of 93.8% for intubated patients whose troponin >18.7 ng/ml. Noninvasively ventilated patients whose myoglobin ≥ 76.6 ng/ml had a risk of death of 76.5% if their INR level is more than 0.995.

XGBoost model

The XGboost ranks the individual variables based on their relative influence, and the top 15 variables are presented in Fig. 4A. The intubation factor had the most potent predictive power, followed by noninvasive ventilation, myoglobin, ADL, and INR. However, the rank of feature importance could differ in different algorithms. To visually exhibit each factor's importance and contribution to mortality, the SHAP algorithm was conducted. Figure 4B presented whether that feature was high (in red) or low (in blue) for that observation according to the SHAP value, A higher SHAP value indicates a higher probability of mortality from COVID-19. For each feature, each dot was created according to its positive (red color) or negative (blue color) contribution to the model output of each patient, creating the dots line for each feature of all patients.

Comparisons of models’ performance

The performance of the LR, DT, and XGBoost models were compared using confusion matrices and ROC curves. As shown in Table 4, the XGBoost model had the highest AUC of 0.961, followed by the LR model with an AUC of 0.959 and the DT model with an AUC of 0.878. The XGBoost model also had the highest diagnostic accuracy of 0.977, followed by the LR model with an accuracy of 0.926 and the DT model with an accuracy of 0.913. Furthermore, the XGBoost model achieved the highest sensitivity of 0.983 and specificity of 0.940, indicating that it had a better ability to correctly identify patients with COVID-19 mortality risk compared to the other models. These results suggest that the XGBoost model is the most effective model for predicting mortality risk in COVID-19 patients. For more detailed performance metrics of each model, please refer to Table 5. Additionally, ROC curves for each model are presented in Fig. 5.

Table 4

Confusion matrixes of each model
		Logistic prediction		Decision Tree prediction		XGBoost prediction
		0	1	0	1	0	1
Actual values	0	802	32	812	22	825	9
Actual values	1	41	112	58	95	14	139

Table 5

Prediction accuracy of different models
	LR	Decision Tree	XGBoost
accuracy	0.926	0.918	0.977
auc (95%)	0.959 (0.947–0.972)	0.883 (0.850–0.916)	0.993 (0.988–0.998)
sensitivity	0.962	0.974	0.989
specificity	0.732	0.621	0.908
precision	0.951	0.812	0.939
Youdens Index	0.694	0.595	0.898

The COVID-19 pandemic has emerged as a global emergency, presenting a significant challenge not only to the healthcare system but also disrupting global culture, society, and economy. While various mortality prediction models have been created since the onset of the pandemic, they are often complex and challenging to interpret and apply. Therefore, we undertook a retrospective cohort study to develop a more practicable and accurate model. The study collected 987 SARS-CoV-2 cases admitted to our hospital between December 22, 2022, and February 8, 2023, and created 3 prediction models.

Our study identified several important predictors in our mortality prediction models, including intubation, non-invasive ventilation, myoglobin, number of diagnoses, CRP, INR, and age. These predictors have been well-documented in the literature [22, 24, 25], with patients with intubation or non-invasive ventilation or had higher levels of myoglobin and INR being at a higher risk of mortality, which was consistent with previous studies [24, 26, 27]. However, those features were studied in isolation in previous studies. In our study, the DT model indicated the relationship among intubation, non-invasive ventilation, myoglobin, INR, and mortality, and provided threshold values associated with increased mortality. This finding could assist physicians in making better clinical decisions to reduce mortality among COVID-19 patients who require intubation or non-invasive ventilation by controlling the level of myoglobin and INR. Bassam et al. found that intubated patients with COVID-19 using anticoagulants and dexamethasone had a low risk of death [24], and dexamethasone was found to reduce the level of myoglobin [28]. Our study provided new insights into the mechanism of dexamethasone’s action in reducing mortality. Therefore, it is suggested anticoagulants and dexamethasone could be used to decrease mortality among intubation or non-invasive ventilation patients.

Our study found that older age predicted a higher risk of death, which was consistent with previous study [29–31]. The increased risk of mortality in older patient could be attributed to the impairment of the immune system and increased risk for immunopathology [32]. We also found that number of diagnoses was related to death, which had been reported in previous studies [9, 29, 30]. Although gender and BMI were associated with the mortality of COVID-19 patients in other studies[33–36], they are included in our models. However, our study showed that the mortality proportion of male patients was higher than that of female patients, providing further evidence to support the hypothesis of male’s vulnerability to COVID-19 [26, 36]. BMI was not input in the model due to its high percentage of missing values (more than 20%) and lack of significant difference between the groups of the survival group and the non-survival group according to single-variable analysis.

Fumagalli et al. conducted a retrospective cohort study to develop a risk score for predicting in-hospital mortality in COVID-19 patients, and their results showed that comorbidities and the number of chronic diseases were independent predictors of mortality [37]. Consistent with their findings, we also found that COVID-19 patients with more comorbidities have a higher risk of mortality. The rapidity of COVID-19 progression is largely influenced by the presence of comorbidities and extrapulmonary organ injuries. According to the results of Zaim’ study [38], exposure to SARS-CoV-2 in individuals with comorbidities such as diabetes, CVDs, and COPD can be detrimental to the lungs, heart, kidneys, and liver, leading to acute respiratory distress syndrome, heart failure, arrhythmias, renal failure, and, eventually, mortality. The comorbidities create a vicious cycle of infection, which increases the mortality of COVID-19 patients after admission. Additionally, the number of complications is also significantly associated with significant mortality.

In this study, we applied machine learning algorithms to develop three prognostic models for predicting mortality in COVID-19 patients with new variants. All models performed well in the overall population, with an AUC of 0.878 to 0.961 and an accuracy of 0.913 to 0.977. Compared with LR and DT models, XGboost model performed best in predicting mortality. XGBoost, is efficient, flexible, and lightweight, and it uses second derivatives to make the loss function more accurate, while adding regular terms to avoid overfitting. In addition, most predictive models are challenging to interpret due to their black-box modeling strategies. In this study, we used nomograms to visualize LR model, and plotted the DT model. The SHAP algorithm was used to visually display the importance of each factor in predicting mortality for the XGboost model. These visualizations make the three models more visible, clinically operable, and easy to be interpreted.

There are several limitations to this study. First, the study population only included patients from one hospital. Second, these models were not externally validated, and their generalizability and implementation across different populations are unknown. Third, the post-discharge follow-up time was relatively brief, so further investigation is needed to determine whether these models can predict the long-term mortality of COVID-19 patients.

In this study, we developed three models to predict the mortality of patients with new COVID-19 variants. All models demonstrated good performance and practicality, with XGboost being the best-performing model.

WHO	World health organization
VOC	Variants of concern
ARDS	Acute respiratory distress syndrome
MODS	Multiple organ dysfunction
IHME	Institute for Health Metrics and Evaluation
ICU	Intensive care unit
LR	Logistic regression
GBDT	Gradient boosting decision tree
AI	Artificial intelligence
NAT	Nucleic Acid Test
ADL	Activity of daily living
DT	Decision tree
XGBoost	Extreme Gradient Boosting
ROC	Receiver operating characteristic

Ethics approval and consent to participate

All methods of the study were carried out in accordance with relevant guidelines and regulations. This clinical study was approved by the Ethics Committee on Biomedical Research, West China Hospital of Sichuan University (2023-181) and the requirement for informed consent was waived due to the retrospective nature of the study.

Consent for publication

Not applicable

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Competing interests

The authors declare that they have no competing interests.

Funding

None

Authors’ contributions

PFL, HLY, and NN designed the study. PFL and HLY drafted the manuscript and conducted the data analysis. HLY, JLC and NN critically revised the manuscript for important intellectual content. PFL, JYW, ALH, and YFM collected and verified the data. All authors approved the final version of the manuscript. The authors read and approved the final manuscript.

Acknowledgements

We thank all of our project partners for their collaboration and contributions. Xianyou Bao, Tazhen Langmu, Yan Xie, and Hanni Zhang provided facilitation support during the workshop of data extract, and Fanfan Shi supported the data interpretation.

WHO Coronavirus (COVID-19) Dashboard | WHO Coronavirus (COVID-19) Dashboard With Vaccination Data. 2023. https://covid19.who.int/table. Accessed 22 Mar 2023.
Chan JF-W, Kok K-H, Zhu Z, Chu H, To KK-W, Yuan S, et al. Genomic characterization of the 2019 novel human-pathogenic coronavirus isolated from a patient with atypical pneumonia after visiting Wuhan. Emerging Microbes & Infections. 2020;9:221–36.
Meng B, Kemp SA, Papa G, Datir R, Ferreira IATM, Marelli S, et al. Recurrent emergence of SARS-CoV-2 spike deletion H69/V70 and its role in the Alpha variant B.1.1.7. Cell Reports. 2021;35:109292.
Tegally H, Wilkinson E, Giovanetti M, Iranzadeh A, Fonseca V, Giandhari J, et al. Emergence and rapid spread of a new severe acute respiratory syndrome-related coronavirus 2 (SARS-CoV-2) lineage with multiple spike mutations in South Africa. preprint. Epidemiology; 2020.
Faria NR, Mellan TA, Whittaker C, Claro IM, Candido D da S, Mishra S, et al. Genomics and epidemiology of the P.1 SARS-CoV-2 lineage in Manaus, Brazil. Science. 2021;372:815–21.
Mlcochova P, Kemp SA, Dhar MS, Papa G, Meng B, Ferreira IATM, et al. SARS-CoV-2 B.1.617.2 Delta variant replication and immune evasion. Nature. 2021;599:114–9.
Classification of Omicron (B.1.1.529): SARS-CoV-2 Variant of Concern. https://www.who.int/news/item/26-11-2021-classification-of-omicron-(b.1.1.529)-sars-cov-2-variant-of-concern. Accessed 2 Apr 2023.
Desingu PA, Nagarajan K, Dhama K. Emergence of Omicron third lineage BA.3 and its importance. Journal of Medical Virology. 2022;94:1808–10.
Guan W-J, Liang W-H, Zhao Y, Liang H-R, Chen Z-S, Li Y-M, et al. Comorbidity and its impact on 1590 patients with COVID-19 in China: a nationwide analysis. EUR RESPIR J. 2020;55:null.
Huang C, Wang Y, Li X, Ren L, Zhao J, Hu Y, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. The Lancet. 2020;395:497–506.
Jha P, Brown PE, Ansumana R. Counting the global COVID-19 dead. The Lancet. 2022;399:1937–8.
The true death toll of COVID-19: estimating global excess mortality. 2023. https://www.who.int/data/stories/the-true-death-toll-of-covid-19-estimating-global-excess-mortality. Accessed 22 Mar 2023.
Wang H, Paulson KR, Pease SA, Watson S, Comfort H, Zheng P, et al. Estimating excess mortality due to the COVID-19 pandemic: a systematic analysis of COVID-19-related mortality, 2020–21. The Lancet. 2022;399:1513–36.
Moreno RP, Metnitz PGH, Almeida E, Jordan B, Bauer P, Campos RA, et al. SAPS 3--From evaluation of the patient to evaluation of the intensive care unit. Part 2: Development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med. 2005;31:1345–55.
Kurtz P, Bastos LSL, Salluh JIF, Bozza FA, Soares M. SAPS-3 performance for hospital mortality prediction in 30,571 patients with COVID-19 admitted to ICUs in Brazil. Intensive Care Med. 2021;47:1047–9.
Metnitz PGH, Moreno RP, Fellinger T, Posch M, Zajic P. Evaluation and calibration of SAPS 3 in patients with COVID-19 admitted to intensive care units. Intensive Care Med. 2021;47:910–2.
Schmidt W, Jóźwiak B, Czabajska Z, Pawlak-Buś K, Leszczynski P. On-admission laboratory predictors for developing critical COVID-19 during hospitalization – a multivariable logistic regression model. Ann Agric Environ Med. 2022;29:274–80.
Atlam M, Torkey H, El-Fishawy N, Salem H. Coronavirus disease 2019 (COVID-19): survival analysis using deep learning and Cox regression model. Pattern Anal Applic. 2021;24:993–1005.
Chung H, Ko H, Kang WS, Kim KW, Lee H, Park C, et al. Prediction and Feature Importance Analysis for Severity of COVID-19 in South Korea Using Artificial Intelligence: Model Development and Validation. J Med Internet Res. 2021;23:e27060.
Wang R, Jiao Z, Yang L, Choi JW, Xiong Z, Halsey K, et al. Artificial intelligence for prediction of COVID-19 progression using CT imaging and clinical data. Eur Radiol. 2022;32:205–12.
China- SARS-CoV-2 Genomes information in 2019nCoVR. https://ngdc.cncb.ac.cn/ncov/monitoring/country/China. Accessed 2 Apr 2023.
Li S, Lin Y, Zhu T, Fan M, Xu S, Qiu W, et al. Development and external evaluation of predictions models for mortality of COVID-19 patients using machine learning method. Neural Comput & Applic. 2021. https://doi.org/10.1007/s00521-020-05592-1.
Shah S, Vanclay F, Cooper B. Improving the sensitivity of the Barthel Index for stroke rehabilitation. Journal of Clinical Epidemiology. 1989;42:703–9.
Mahboub B, Bataineh MTA, Alshraideh H, Hamoudi R, Salameh L, Shamayleh A. Prediction of COVID-19 Hospital Length of Stay and Risk of Death Using Artificial Intelligence-Based Modeling. Front Med. 2021;8:592336.
Peñuelas O, del Campo-Albendea L, de Aledo ALG, Añón JM, Rodríguez-Solís C, Mancebo J, et al. Long-term survival of mechanically ventilated patients with severe COVID-19: an observational cohort study. Ann Intensive Care. 2021;11:143.
Radovanovic D, Coppola S, Franceschi E, Gervasoni F, Duscio E, Chiumello DA, et al. Mortality and clinical outcomes in patients with COVID-19 pneumonia treated with non-invasive respiratory support: A rapid review. Journal of Critical Care. 2021;65:1–8.
Zhu F, Li W, Lin Q, Xu M, Du J, Li H. Myoglobin and troponin as prognostic factors in patients with COVID-19 pneumonia. Medicina Clínica. 2021;157:164–71.
Minetto MA, Botter A, Lanfranco F, Baldi M, Ghigo E, Arvat E. Muscle fiber conduction slowing and decreased levels of circulating muscle proteins after short-term dexamethasone administration in healthy subjects. J CLIN ENDOCR METAB. 2010;95:1663–71.
Mishra P, Singh RK, Nath A, Pande S, Agarwal A, Sanjeev OP, et al. A novel epidemiological scoring system for the prediction of mortality in COVID-19 patients. Transactions of The Royal Society of Tropical Medicine and Hygiene. 2022;116:409–16.
Richardson S, Hirsch JS, Narasimhan M, Crawford JM, McGinn T, Davidson KW, et al. Presenting Characteristics, Comorbidities, and Outcomes Among 5700 Patients Hospitalized With COVID-19 in the New York City Area. JAMA-J AM MED ASSOC. 2020;323:2052–9.
Yang Q, Li J, Zhang Z, Wu X, Liao T, Yu S, et al. Clinical characteristics and a decision tree model to predict death outcome in severe COVID-19 patients. BMC Infect Dis. 2021;21:783.
Chen Y, Klein SL, Garibaldi BT, Li H, Wu C, Osevala NM, et al. Aging in COVID-19: Vulnerability, immunity and intervention. AGEING RES REV. 2021;65 null:101205.
Cummings MJ, Baldwin MR, Abrams D, Jacobson SD, Meyer BJ, Balough EM, et al. Epidemiology, clinical course, and outcomes of critically ill adults with COVID-19 in New York City: a prospective cohort study. LANCET. 2020;395:1763–70.
Kass DA, Duggal P, Cingolani O. Obesity could shift severe COVID-19 disease to younger ages. LANCET. 2020;395:1544–5.
Kuderer NM, Choueiri TK, Shah DP, Shyr Y, Rubinstein SM, Rivera DR, et al. Clinical impact of COVID-19 on patients with cancer (CCC19): a cohort study. LANCET. 2020;395:1907–18.
Li J, Chen Y, Chen S, Wang S, Zhang D, Wang J, et al. Derivation and validation of a prognostic model for predicting in-hospital mortality in patients admitted with COVID-19 in Wuhan, China: the PLANS (platelet lymphocyte age neutrophil sex) model. BMC Infect Dis. 2020;20:959.
Fumagalli C, Rozzini R, Vannini M, Coccia F, Cesaroni G, Mazzeo F, et al. Clinical risk score to predict in-hospital mortality in COVID-19 patients: a retrospective cohort study. BMJ Open. 2020;10:e040729.
Zaim S, Chong JH, Sankaranarayanan V, Harky A. COVID-19 and Multiorgan Response. CURR PROB CARDIOLOGY. 2020;45:100618.

No competing interests reported.

supplementaryFIGURE1.docx

Development and validation of predictive models for mortality of cases with COVID-19 (Omicron BA.5.2.48 and B.7.14): A retrospective study

Status:

Version 1

Abstract

Figures

Background

Methods

Study design and participants

Data Collection

Statistical Analysis

Results

Demographic and clinical characteristics

Factors Selection

LR model

DT Model

XGBoost model

Comparisons of models’ performance

Discussion

Conclusion

Abbreviations

Declarations

References

Additional Declarations

Supplementary Files

Status:

Version 1