A Novel Severity Score to Predict Inpatient Mortality in COVID-19 patients

Abstract

Introduction

COVID-19 is commonly mild and self-limiting, but in a considerable portion of patients the disease is severe and fatal. Determining which patients are at high risk of severe illness or mortality is essential for appropriate clinical decision making. We propose a novel severity score specifically for COVID-19 to help predict disease severity and mortality.

Methods

4,711 patients with confirmed SARS-CoV-2 infection were included. We derived a risk model using the first half of the cohort (n=2,355 patients) by logistic regression and bootstrapping methods. The discriminative power of the risk model was assessed by calculating the area under the receiver operating characteristic curves (AUC). The severity score was validated in a second half of 2,356 patients.

Results

Mortality incidence was 26.4% in the derivation cohort and 22.4% in the validation cohort. A COVID-19 severity score ranging from 0 to 10, consisting of age, oxygen saturation, mean arterial pressure, blood urea nitrogen, C-Reactive protein, and the international normalized ratio was developed. A ROC curve analysis was performed in the derivation cohort achieved an AUC of 0.824 (95% CI 0.814-0.851) and an AUC of 0.798 (95% CI 0.789-0.818) in the validation cohort. Furthermore, based on the risk categorization the probability of mortality was 11.8%, 39% and 78% for patient with low (0-3), moderate (4-6) and high (7-10) COVID-19 severity score.

Conclusion

This developed and validated novel COVID-19 severity score will aid physicians in predicting mortality during surge periods.

Introduction

The first confirmed case of COVID–19 in New York City was on March 1^st, 2020. Within a few short weeks all of the hospitals in the area were overwhelmed hitting a peak on April 6^th, 2020 of 6,377 confirmed positive cases that day. As of July 3^rd, 2020, there have been 18,535 deaths, 55,110 hospitalizations and a total of 213,212 cases in this city.(1) Given the population density of New York City, this region was the first in the United States to encounter the full impact of the pandemic. Over these few months much has been learned about the disease, its deadliness, and those who are at higher risk for dying.

In many people the disease is mild and self-limiting, but in a considerable portion of patients the disease is severe and fatal. Determining which patients are at high risk of severe illness or mortality is an essential part of understanding this illness. Prior reports from Wuhan identified certain comorbidities as diabetes, hypertension and coronary artery disease as patients more likely to present to their hospital.(2) They also discovered that patients with older age, higher Sequential Organ Failure Assessment (SOFA) score, and elevated d-dimers were significantly associated with inpatient mortality.(2) Further reports have shown other predictors of poor outcome such as acute kidney injury, acute hepatic injury, the need for mechanical ventilation, elevated c-reactive protein (CRP), interleukin–6 (IL–6), lymphocyte count, and Procalcitonin levels.(3–6)

COVID–19 is unique in its ability to not only cause sepsis, and multi-system organ failure, but also to cause a severe inflammatory response that can lead to systemic multi-vascular thrombosis.(7, 8) While the SOFA score is also predictive of mortality for COVID–19, it does not address the additional thrombotic mitigators of severe illness.(9) Other reports have recommended the use of the International Society of Thrombosis and Haemostasis (ISTH) Disseminated Intravascular Coagulation Score (DICS) which was initially developed to help predict the development of disseminated intravascular coagulation (DIC), and now being used to help guide the use of anti-coagulation for patients with COVID–19.(10–12) We propose a novel severity score specifically for COVID–19 combining elements of both of these scores to help predict disease severity and mortality.

Methods

This study is an observational cohort study validating a novel, simple COVID severity score to predict inpatient mortality risk in 4,711 patients with confirmed SARS-CoV–2 infection using a combination of presentation vital signs, and basic admission laboratory values. This model was created on patients presenting from March 1^st to April 16^th. We used the first half of patients during this period (n = 2355) as the “derivation cohort” in which the severity score was developed and internally validated. The second half of our cohort (n = 2356) was used to confirm the power of the prediction score, this part of the cohort was considered the “validation cohort”.

Inclusion criteria was defined as all patients admitted to a hospital within a large healthcare network that were positive by detection of SARS-CoV–2 RNA using real-time reverse transcriptase–polymerase chain reaction (RT-PCR) assay testing, performed within the hospital system or documented at an outside system prior to transfer. Patients evaluated in the emergency room but not admitted, or those that died in the emergency room, were excluded from the analysis, given the relative paucity of data. Most patients had only one admission, and we only considered the last hospitalization for those that had multiple admissions during this period.

After approval of this study by the Institutional Review Board for which consent was waived due to its retrospective nature, information on demographics, comorbidities, admission laboratory values, admission medications, admission supplemental oxygen orders, discharge and mortality were identified through a healthcare surveillance software package (Clinical Looking Glass [CLG]; Streamline Health, Atlanta, Georgia) Mortality data was collected by querying in-hospital deaths and deaths recorded in the National Death Registry. Laboratory measures were extracted by identifying those obtained-on-admission. Comorbidities were identified based on the International Coding Disease coding system (ICD 10).. The comorbidities chosen for this study are those used in the Charlson Comorbidity Index. Each patient’s medical record was queried for any diagnosis occurring within 5 years of their index admission. Laboratory values were selected based on their identification in recent articles as potential correlates of illness severity.

Statistical Analysis

Categorical variables were described using frequencies and proportions and compared using χ2 tests. Continuous values were expressed using mean ± standard deviation (SD), or median and interquartile range (IQR) and compared using Student’s t test or the nonparametric Mann–Whitney test.

For easier application to a risk score model, when performing multivariate logistic regression analysis, most continuous variables were converted to categories based on published data as follows: advanced age (≥ 60 years, ≥ 70 years, and ≥ 80 years), body mass index (<18.5 or >24.9 kg/m²), oxygen saturation (<94%), temperature (>38°C), mean arterial pressure (MAP <80 mmHg, <70 mmHg, <60 mmHg), white blood cell count (<4800 or >10,800 per mm³), Lymphocytes (<1000 per mm³), platelet count (≤ 150,000 per mm³), alanine aminotransferase (ALT > 40 U/liter), aspartate aminotransferase (AST > 40 U/liter), ferritin (>300 µg/liter), INR (>1.2), d-dimer (>3 mg/ml), creatinine (>150 µmol/liter), blood urea nitrogen (BUN) (> 35 mg/dL), glucose (<60 or > 500 mg/dL), sodium (<139 or >154 mmol/liter), interleukin–6 (IL–6) (>150 pg/ml), C-reactive protein (CRP) (>10 mg/liter), Procalcitonin (>0.1 ng/ml), and Troponin (>0.1 ng/ml).

Candidate predictors with P < 0.10 in univariate analyses were included a multivariate logistic regression. In addition, a backward stepwise bootstrap regression model, in which 1000 random samples patients were generated with replacement, was also performed to investigate the relative importance of each variable included in our model.(14) Frequencies of occurrence of each covariate in the final model were noted; if predictors occurred in 70% or more of the bootstrap models, they were retained in the final multivariate model. Beta regression coefficients and odds ratios (OR) were calculated with 95% confidence intervals (CI). The multivariate regression coefficients of the predictive factors were used to assign integer points for the prediction score. However, for the simplicity of the score we allocated points 1 to 3 in variables with multiple categories.

The discriminative power of the prediction score was assessed by calculating the area under the receiver operating characteristic (ROC) curves (AUC). All variables were used as continuous variables when calculating AUC. A predictor with an AUC above 0.7 was considered to be useful, while an AUC between 0.8 and 0.9 indicated good diagnostic accuracy. The classification and regression tree (CART) analysis was used to create risk categories according to total prediction score. When performing CART analysis, impurity function was used for splitting and cut-off points for continuous variables which were generated automatically based on statistical cost assumptions. Calibration of the risk score reflecting the link between predicted and observed risk, was evaluated by the Hosmer–Lemeshow goodness of fit test. A P value < 0.05 was considered statistically significant for all analyses. Data were analyzed using the STATA version 12 and IBM SPSS version 24.

Results

Distribution of socio-demographics, comorbidities, vital signs and laboratory values between the validation and derivation cohorts are shown in Table 1. A total of 2355 COVID–19 positive patients were treated in our hospital during the first half New York City outbreak (derivation cohort), from which 621 (26.4%) patients died. The validation cohort consisted of 2356 COVID–19 positive patients out of which 527 (22.4%) died.

The univariate analysis showed 22 potential predictors with a p<0.1 (Table 2).. Out of the 22 candidate predictors, 10 variables remained as independent predictors in the multivariate logistic regression analysis, including age (>60, >70 and >80 years), female sex, oxygen saturation <94%, mean arterial pressure (MAP) (<80, <70 and <60 mmHg), international normalized ratio (INR) >1.2, creatinine >150 µmol/liter, blood urea nitrogen (BUN) >30 mg/dL, interleukin–6 (IL–6) > 150 pg/ml mol/dL, C-reactive protein (CRP) >10, and procalcitonin >0.1 (Table 3)..

The bootstrap analysis revealed that, out of the 10 independent predictors of mortality, age, oxygen saturation, MAP, BUN, CRP, INR and procalcitonin were reproducibly selected in more than 70%. Due to the large number of missing data for procalcitonin (44%), this variable was excluded in order to avoid noise predictors. Allocation of points for the COVID–19 severity score was made based on Beta coefficients and BCa 95%CI, however for the simplicity of the score we allocated points 1 to 3 in subcategorized variables (Age & MAP) (Table 4).. The total prediction score ranges between 0 and 10 with a high score indicating high risk of in-hospital mortality.

A ROC curve analysis was performed in the derivation cohort (Figure 1),, the novel COVID–19 severity score achieved an AUC of 0.824 (95% CI 0.814–0.851) indicating a good discrimination for patients with higher risk of in-hospital mortality. Furthermore, the Hosmer–Lemeshow goodness of fit test of tenfold cross-validation did not reach statistical significance (P  =  0.244) indicating a good match of predicted risk over observed risk.

Finally, we applied the score to the 2356 patients in the validation cohort. The ROC curve analysis showed an AUC of 0.798 (95% CI 0.789–0.818) still indicating a good discrimination for our model (Figure 2A).. Then, we determined that low risk patients (0–3 points) had a 11.8% risk of mortality, moderate risk patients (4–7 points) had a 39% risk of mortality and high-risk patients (>7 points) had a 78% risk of mortality (Figure 2B)..

Discussion

We propose a novel scoring system to aid in the prediction of inpatient mortality for patients presenting with SARS-CoV–2 infection to hospital emergency rooms. The score is based on simple pragmatic demographic data, and presenting biomarker values. This score incorporates the unique constellation of various presentations in which COVID–19 can manifest in severe illness. We avoided incorporating mechanical ventilation use into the score as this was tied to a clinical decision which over time with more knowledge an approach that changed. While IL–6 also seems to predict mortality, we avoided incorporating this biomarker as it is a non-routine test, and was not available in a large percentage of our patient population.

The limitations of this study are its retrospective design, its cohort, which is primarily a minority urban population, and the epoch at which the data was required. Since the data and outcomes were recorded during the highest surge of the pandemic this may bias the results towards higher mortality as this was a great strain on treating hospitals at the time. Prior reports also have shown increased mortality in racial and ethnic minority patients.(13) Given the sociodemographic background of our patient population the score may again be biased towards higher mortality risk. The limitations of the study are also its strengths in that this is specifically applicable to minority urban centers that are suffering from large surge populations of infected patients.

While mortality prediction is neither perfect nor absolute, having a simple score to predict how severe a patient’s illness and hospital course will aid admitting and emergency room physicians’ triage severity and prognosis during surge periods. It can also be used to guide recommendations for palliative care consultation early in a patient’s hospital course.

Declarations

Author Contributions

David J Altschul: Conceptualization, Methodology, Investigation, Writing-Original Draft, Review & Editing. Santiago R Unda: Conceptualization, Methodology, Formal Analysis, Investigation, Writing-Original Draft, Review & Editing. Joshua Benton: Investigation, Methodology. Rafael de la Garza Ramos: Conceptualization, Methodology, Investigation, Methodology. Mark Mehler: Conceptualization, Investigation, Methodology. Emad Eskandar: Conceptualization, Methodology, Visualization, Supervision, Writing-Review & Editing

Competing Interests statements

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References

1.City DoHNY. COVID–19 Data https://www1.nyc.gov/site/doh/covid/covid–19-data.page: Department of Health, New York City; 2020 [N/A:[COVID–19 Data]. Available from: https://www1.nyc.gov/site/doh/covid/covid–19-data.page.

2.Zhou F, et al. Clinical course and risk factors for mortality of adult inpatients with COVID–19 in Wuhan, China: a retrospective cohort study. Lancet 395, 1054–62 (2020).

3.Yang X, et al. Clinical course and outcomes of critically ill patients with SARS-CoV–2 pneumonia in Wuhan, China: a single-centered, retrospective, observational study. Lancet Respir Med 8, 475–81 (2020).

4.Richardson S, et al. Presenting Characteristics, Comorbidities, and Outcomes Among 5700 Patients Hospitalized With COVID–19 in the New York City Area. JAMA 323, 2052–2059 (2020).

5.Cheng Y, et al. Kidney disease is associated with in-hospital death of patients with COVID–19. Kidney Int 97, 829–38 (2020).

6.Wu S, et al. Identification and validation of a novel clinical signature to predict the prognosis in confirmed COVID–19 patients. Clin Infect Dis. prognosis in confirmed COVID–19 patients [published online ahead of print, 2020 Jun 18]. Clin Infect Dis. 2020;ciaa793. doi:10.1093/cid/ciaa793 (2020).

7.Huang C, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. Lancet 395, 497–506 (2020).

8.Iba T, Levy JH, Levi M, Thachil J. Coagulopathy in COVID–19. J Thromb Haemost (2020) 10.1111/jth.14975. doi:10.1111/jth.14975.

9.Vincent JL, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med 22, 707–10 (1996).

10.Taylor FB, et al. Towards definition, clinical and laboratory criteria, and a scoring system for disseminated intravascular coagulation. Thromb Haemost 86, 1327–30 (2020).

11.Sivula M, Tallgren M, Pettilä V. Modified score for disseminated intravascular coagulation in the critically ill. Intensive Care Med 31, 1209–14 (2005).

12.Thachil J, et al. ISTH interim guidance on recognition and management of coagulopathy in COVID–19. J Thromb Haemost 18, 1023–6 (2020).

13.Price-Haywood EG, Burton J, Fort D, Seoane L. Hospitalization and Mortality among Black Patients and White Patients with Covid–19. N Engl J Med 382, 2534–43 (2020).

14.Hong W, et al. Development and validation of a risk prediction score for severe acute pancreatitis. J Transl Med. 2019;17, 146 (2019).

Tables

Table 1. Baseline characteristics of COVID-19 positive patients in the derivation and validations cohorts.

Missing data: Congestive heart failure (2%), Chronic Pulmonary Disease (3%), Oxygen saturation (4%), Temperature (3%), Mean arterial pressure (5%), D-dimer (21%), Platelets (3%), INR (9%), BUN (12%), Creatinine (3%), Sodium (4%), Glucose (29%), AST (5%), ALT (4%), WBC (3%), Lymphocytes (3%), IL-6 (66%), Ferritin (28%), CRP (14%), Procalcitonin (44%), and Troponin (14%).

Table 2. Univariate analysis of discharged and dead patients with Covid-19 in the derivation cohort.

Table 3. Multivariate logistic regression analysis for in-hospital mortality in the derivation cohort.

MAP: Mean Arterial Pressure, INR: International normalized ratio, BUN: Blood urea nitrogen, IL-6: Interleukin-6, CRP: C-Reactive protein.

Table 4. Point allocation for predictors of severe COVID-19

MAP: Mean Arterial Pressure, INR: International normalized ratio, BUN: Blood urea nitrogen, CRP: C-Reactive protein.