Development and External Validation of a Nomogram for Early Predicting in-hospital Mortality of Patients with SARS-CoV-2 Pneumonia: A Two-Center, Retrospective Analysis

Background At present, the death cases with SARS-CoV-2 pneumonia are continuing to increase globally. However, the information on death cases and predictive methods are substantial lacking. We aimed to develop a nomogram, which was validated by both internal and external cohorts, for early predicting mortality in hospitalized patients with SARS-CoV-2 pneumonia. Methods We retrospectively collected data on 1,540 patients conrmed SARS-CoV-2 pneumonia from two hospitals. Multivariate logistic regression analysis was performed to examine factors associated with in-hospital mortality. We investigated the mortality related risk factors and their weights, thereafter designed and validated a predictive nomogram model to facilitate early discrimination of in-hospital death. We assessed the nomogram performance by examining calibration (calibration plots and Hosmer– Lemeshow calibration test) and discrimination (AUROC). We also plotted survival curves and decision curves to evaluate the clinical usefulness of the nomogram. procalcitonin ≥ 0.1 ng/mL(OR = 4.972; 95%CI, 2.537– 9.746; P < 0.001), and presence of myocardial injury (OR = 2.289; 95%CI, 1.260–4.160; P = 0.007) on admission. Calibration curves showed good tting of the nomogram model with no statistical signicance (P = 0.740) by Hosmer-Lemeshow test. This predictive nomogram had better predictive ability than CURB-65 score in training set (AUROC = 0.956 vs 0.828,P < 0.001). The good predictive performance of the nomogram is suggested by calibration, discrimination, and survival curve analysis, whether in the training, internal or external validation set. The decision curve analysis showed that predicting mortality risk applying this nomogram would be better than having all patients or none patients. Conclusions This nomogram is a reliable prognostic method that can accurately and early predict in-hospital mortality in patients with SARS-CoV-2 pneumonia. It can guide clinicians to improve their abilities to evaluate patient prognosis, enhance patient stratication, make earlier and reasonable decisions.


Introduction
At the end of 2019, a strongly contagious viral pneumonia disease largely outbroke in Wuhan, Hubei province, China. [1] Within just three months, the new infection has spread throughout the country [2] and even global. [3,4] A novel coronavirus named severe acute respiratory syndrome-coronavirus-2 (SARS-CoV-2) was discovered and isolated from the bronchoalveolar lavage uid of patients with novel pneumonia. [5,6] Until April 20, 2020, a total of 2,319,066 patients were diagnosed with SARS-Cov-2 pneumonia and 157,990 cases died globally. The current mortality rate is around 6.8%. Presently most of the patients diagnosed with SARS-Cov-2 pneumonia are mild. Early clinical treatments have achieved gratifying results. Although the overall mortality rate is far lower than other members of coronaviridae, such as wellknown severe acute respiratory syndrome coronavirus (SARS-CoV) and Middle East respiratory syndrome coronavirus (MERS-CoV), [7,8] once the patients were diagnosed as critically illness, their risk of death increased sharply. A recent observational study showed that in a cohort of 52 patients with critical SARS-CoV-2 pneumonia, 32 (61.5%) patients died in 28 days. [9]This mortality rate was much higher than severe SARS in Toronto (34%) [10], Singapore (37%) [11], and Hong Kong (26%) [12]` reported in the same period.
Therefore, early discrimination of individuals at high risk of death is imperative for the patients with SARS-CoV-2 pneumonia. Our current study sought to uncover the clinical characteristics of non-survivors, develop and validate a nomogram that can predict mortality of SARS-CoV-2 pneumonia in an early stage.

Study Design and Participants
This two-center, retrospective study was done at Wuhan Pulmonary Hospital and Tongji Hospital a liated to Huazhong University of Science and Technology in Wuhan, which are both designated hospitals to treat patients with SARS-CoV-2 pneumonia. We retrospectively analyzed 1, 540 patients who were discharged or died from Jan 9, 2020 to March 24, 2020. All patients were diagnosed with SARS-CoV-2 pneumonia according to WHO interim guidance. The Ethics Commission of Tongji Hospital and Wuhan Pulmonary Hospital have all approved this study.

Data Collection
The data were collected from the hospital's electronic medical record system, which included demographic information, symptoms, comorbidities, complications, routine laboratory tests, immunological tests, computed tomography (CT) results, and clinical interventions. In order to early predict mortality, the data of the laboratory examinations and computed tomography (CT) ndings on admission or in the rst time during hospitalization was collected for analysis. CURB-65 score of each patient was calculated. The entry and calculation of all relevant data were veri ed by two experienced clinical researchers. The clinical outcomes of the patients included in this clinical study were observed until March 24, 2020. The outcome of this study was all-cause in-hospital mortality.

Statistical Analysis
All patients were divided into two groups: survival and non-survival group. The comparisons were performed between those two groups. Normally distributed continuous variables were tested using t-test and characterized by mean ± SD, while non-normally distributed continuous variables were compared by Mann-Whitney U test and characterized by median (interquartile range, IQR). The χ2 or Fisher exact tests were utilized for categorical variable analysis.
The percentages of missing values of variables in our cohort were lower than 50%. Multiple imputation (MI) method was used to impute missing data in our cohort with the guidance of previous study [13]. Conclusions of univariate logistic regression analyses with or without imputed data were unchanged. Continuous variables were categorized and retained for multivariate testing. Data of 1068 patients form Tongji Hospital was partitioned randomly into two complementary subsets: the training set of 749 (70%) was used to establish the model; the testing set of 319 (30%) was used for internal validation. Data of 472 patients from Wuhan Pulmonary Hospital was employed to validate the model externally. For mortality predictive model establishment, continuous variables were categorized by a cutoff point. The cutoff value was con rmed from Youden's index of receiver operator characteristic (ROC) curve. Variables with P < 0.05 were included in multivariate logistic regression model by a forward procedure. Then predictive nomogram was generated based on converting regression coe cient to a 0-to 100-point scale proportionally. Predictive performance of model was measured by validation, discrimination and decision analysis [14]. Calibration curve was generated with bootstrap samples to reduce the over t bias [15]. Hosmer-Lemeshow (HL) test implied good calibration when the test is insigni cant. Discriminative performance was assessed by area under the receiver operating characteristic (ROC) curve. In addition, clinical usefulness of the nomogram is evaluated by decision curve analysis (DCA) by evaluating net bene ts at different threshold probabilities. Survival analysis was also performed by univariate method with Kaplan-Meier analysis between low-risk and high-risk group according to the cut-off value of 50%.
Statistical analysis was conducted using Statistical Package for the Social Sciences (SPSS) 24.0 and R software 3.5.0. All tests were considered signi cant when two-sided P value was less than 0.05.

Clinical Characteristics
By March 24, 2020, a total of 1,540 patients with con rmed SARS-CoV-2 pneumonia from two Hospitals had been nally enrolled in the study. 1,068 patients were from Tongji Hospital; 472 from Wuhan Pulmonary Hospital. Of these patients, 268 patients failed to survive, with an in-hospital mortality around 16.1%. 749 patients with SARS-CoV-2 pneumonia were included in the training set, 319 patients in the internal testing set and 472 patients in the external validation set. Social-demography, comorbidities, symptoms, vital signs, laboratory ndings, and computed tomography results of different groups are described in Table 1 However, no signi cant difference in red blood cell count, hemoglobin, and activated partial thromboplastin time (APTT) was found between survivors and non-survivors (P > 0.05). Almost all nonsurvival patients (137/138[99.3%]) presented in ltrated bilateral lesions involving 5 lung lobules in CT images. But in the survival group, 556/611(91.0%) patients presented bilateral multi-lobular lesions. Only 56 patients (7.4%) in training set showed with unilateral lesions in CT scan. Therefore, whether in survivors or non-survivors, bilateral multi-lobular lesions can be found in vast majority of patients with SARS-CoV-2 pneumonia. The similar results can be found in internal testing set and external validation set, respectively (Table 1). . * underlying lung disease includes chronic obstructive pulmonary disease, asthma, bronchiectasis and tuberculosis etc .

Construction And Validation Of The Nomogram
On the basis of eight variables from multivariable logistic regression analysis in training set, a predictive model was successfully constructed and visualized as a form of a nomogram plot to utilize in clinical practice (shown in Fig. 1). Calibration curves derived from training set showed a good tting of the model with no statistical signi cance (P = 0.740) by Hosmer-Lemeshow test. The calibration curves indicated great consistence in training set, internal and external validating sets (Fig. 2). In Fig. 3 (Fig. 3). Each patient was divided into high-risk or low-risk group according to the cut-off value of 50% (death probability) predicted by nomogram. The three Kaplan-Meier survival curves derived from the data of training, internal, and external set were shown in Fig. 4. Kaplan-Meier survival curves all indicated that high-risk group had a much worse outcome than that of the low-risk group (P < 0.001). The Decision Curve Analysis (DCA) of predictive model showed a threshold probability of 5-95% (Fig. 5), in which the model had the ability to identify patients who might not survive superior to the "treat-all-patients" or "treat-none" schemes.

Discussion
Several previous studies have systematically summarized the features of SARS-Cov-2 pneumonia patients. [1,5,16,17] Yang et al [9] described the characteristics of critically ill patients with SARS-CoV-2 pneumonia. At present, some studies have investigated the risk factors affecting mortality in SARS-CoV-2 pneumonia [18], but papers constructing risk models to predict survival are very limited, which may be practical for doctors to make clinical decisions in the early stage.
In our training cohort, the mean age of non-survivors was older than survivors (68 years vs 58 years).
Most of elders were accompanied by underlying diseases, including hypertension, coronary heart disease, lung underlying diseases, and diabetes mellitus, which were documented in other viral pneumonia studies, such as SARS-CoV-2 [9,17], SARS [10], MERS [19]. As for vital signs, the levels of body temperature, heart rate, and respiratory rate were all higher in non-survivors than those of survivors.
Similar to a previous report [11], our univariate analysis had also shown a decrease in the count of lymphocyte and platelet, as well as an increase in LDH and d-dimer. The other mortality risk factors were increased neutrophil count, procalcitonin, BUN, hs-cTnI, AST and etc. As shown before, the lymphocyte count was much lower in non-survival patients. Previous study indicated that lymphocyte count < 0.8 × 10 9 /L was an independent mortality risk factor for viral pneumonia [20]. In our study, reduced lymphocyte count was con rmed to be a vital and independent mortality associated risk factor by using multivariate logistic regression analysis(OR = 4.853). It is presumed that lymphocytes are at an exhaustion state before the patients reached death. Nevertheless, WBC count of the non-survivors was signi cantly increased, especially the neutrophils in non-survivors were around 2.6-fold higher than survivors.
Although many patients in our study had neutrophil count below 7 × 10 9 /L and procalcitonin below 0.25 ng/ml, most of non-survival patients showed the increased neutrophil count and procalcitonin.
Elevated serum procalcitonin was regarded as one of most commonly used markers for bacterial infection [21]. Bacterial coinfection was known as a major cause of mortality H7N9 in uenza pneumonia [22], and the most common pathogen detected by sputum or blood culture was Acinetobacter baumannii [20,22]. Bacterial co-infection not only indicated higher mortality but also longer hospital stay time and much more hospital care cost compared with non-bacterial co-infection [20]. Furthermore, elevated levels of WBC and neutrophils are thought to be a major contributory factor for disease progression. [23] Univariate and multivariate analysis indicated that increased neutrophil count (≥ 7 × 10 9 /L) was risk factor for mortality of patients. This result was documented in other studies showing that the neutrophil count was elevated in non-survivors of novel coronavirus infected pneumonia compared with survivors. [23] In our current study, procalcitonin was identi ed to be a very important mortality associated factor, because of much higher weight(OR = 5.586) than other factors. Until now, there is rare report about procalcitonin contributing to the death of patients with SARS-CoV-2 pneumonia. Therefore, we concluded that lymphocyte count reduction and bacterial co-infection (increased neutrophil count and procalcitonin) are two leading risks for the mortality of patients with SARS-CoV-2 pneumonia.
In our study, we further evaluated the importance of LDH in SARS-CoV-2 pneumonia. The level of serum LDH was found to be signi cantly elevated in more than 70% of patients in our study. Moreover, LDH with a level ≥ 350 U/L was found to be associated with the risk of death in SARS-CoV-2 pneumonia patients by multivariate analysis. A previous study showed that LDH level was associated with the severity of various diseases including SARS [24]. It has been proved that the level of LDH was higher in ICU patients with SARS-CoV-2 pneumonia than that in non-ICU patients in Wuhan [25]. Other factors, such as the platelet count, APTT, PT, ALT, AST, CRP, TP, ALB, BUN, and Scr, were associated mortality of patients by univariate analysis, however, they were not identi ed as the risks of mortality by multivariate analysis in our study.
Cardiac complications (including heart failure, arrhythmia, or myocardial infarction) are common in patients with pneumonia, especially in critical illness [26]. Coronary heart disease was shown to be correlated with acute cardiac events and poor prognosis in in uenza and other viral pneumonia [27,28]. Chen C et al. showed that cTnI was increased in critically ill patients with COVID-19, which was an independent risk factor of clinical disease status [29]. In our study, hs-cTnI or cTnI on admission was found be increased in more than half of non-survival patients. Elevated hs-cTnI or cTnI level on admission indicated myocardial injury, which was con rmed to be signi cantly associated with mortality of patients.
This result was consistent with a recently published study [18]. Therefore, presence of myocardial injury on admission is another important risk factor for predicting the mortality of patients with SARS-CoV-2 pneumonia.
Higher coagulation activity was found in most hospitalized pneumonia patients (almost 90%), and its main marker was increased d-dimer concentration [30]. In this study, d-dimer greater than 1.5 µg/mL is associated with mortality of SARS-CoV-2 pneumonia, which was documented in a previous study [18].
The elevation of d-dimer might be due to hypercoagulable state of patients during SARS-CoV-2 infection.
Pathologists con rmed that hyaline thrombi were found in some microvessels of three cases with COVID-19 by minimally invasive autopsies [31]. Furthermore, anticoagulant therapy was found to be correlated with better prognosis in severe ill COVID-19 patients with increased level of D-dimer [32]. In addition, blood-borne tissue factor, which was found to be expressed on the cell surfaces of alveolar macrophages, neutrophils, and endothelial cells during lung infection, could highly induce systemic coagulopathy [33].
Therefore, it may predispose to thrombosis in the pulmonary micovessels of patients with SARS-CoV-2 pneumonia, presenting as the signi cant elevation of d-dimer.
Currently, CT scan is considered to be an effective modality for clinical diagnosis for viral pneumonia at early periods. [34] From our CT images in SARS-CoV-2 pneumonia, we concluded that in ltrated lesions in ve lung lobes were shown in the majority of the infected patients. Ground-glass opacities and consolidation are the main features of CT. These CT imaging manifestations may have some differences from that in SARS. In SARS, CT imaging normally presents in the lower lung lobes with multifocal airspace consolidation in a short period. [35] Therefore, CT has limited value in predicting death in the patients with SARS-CoV-2 pneumonia, mainly because most cases presented as mutiple lobular in ltration (usually 5 lobes) or diffuse lesions on CT images whether in survival or non-survival patients.
Although many predictive models are available to evaluate the prognosis of pneumonia, including IDSA/ATS minor criteria, CRB-65, CURB-65, Halm criteria, qSOFA, PSI, SCAP, SIRS-Score, SMART-COP, SOFA for community-acquired pneumonia (CAP) [36] and MuLBSTA score for viral infection [37], there is a lack of predictive tool especially for coronavirus pneumonia mortality. By multivariate logistic regression analysis, age(≥ 60 years), respiratory rate(≥ 30 breaths/minute), neutrophil count (≥ 7 × 10 9 /L), PCT(≥ 0.1 ng/mL), lymphocyte count (≤ 0.8 × 10 9 /L), d-dimer(≥ 1.5ug/mL), LDH(≥ 350U/L), and presence of myocardial injury were identi ed to be eight independent risks for mortality prediction for SARS-CoV-2 pneumonia patients. Then, we employed these eight risk factors and formulated a reliable mortality risk predictive model to stratify SARS-CoV-2 pneumonia patients. This model was also validated by both internal and external validation set, and the visual expression of the model was displayed by a nomogram. The good predictive performance of the nomogram is suggested by calibration, discrimination, survival curve analysis, and DCA. Furthermore, this nomogram had a better predictive ability than that of CURB-65 score both in training and validation sets.
The research has several limitations. First, this retrospective design may cause a potential and inherent selection bias. Secondly, most patients have not been tested for the levels of cytokines and the counts of T cells subgroup. Thus, we failed to consider these factors in multiple imputation and further analysis to avoid great bias from real situation. Through a simple comparison between groups, we found some interesting differences. We are planning to explore their signi cance in subsequent clinical studies. Third, all the patients included in our study are Chinese. The clinical features of patients might be different in other countries or areas. Then, there may be some inherent biases by using this study format. Our results should be further validated by the multiple-center, prospective study.

Conclusion
In summary, via this two-center retrospective study of patients diagnosed as SARS-CoV-2 pneumonia in Wuhan, we found some important mortality-related risks, including age(≥ 60 years), respiratory rate(≥ 30breaths/minute), neutrophil count (≥ 7 × 10 9 /L), PCT(≥ 0.1 ng/mL), lymphocyte count (≤ 0.8 × 10 9 /L), d-dimer(≥ 1.5ug/mL), LDH(≥ 350U/L), and presence of myocardial injury. We rstly establish a reliable predictive nomogram model, which can accurately stratify patients into risk categories and predict early mortality of patients with SARS-Cov-2 pneumonia. This nomogram can guide clinicians to improve their abilities to evaluate patient prognosis and make further reasonable decisions.       Decision curve analysis for early predicting in-hospital mortality in patients with SARS-CoV-2 pneumonia. The y-axis stands the net bene t. The red line stands the nomogram for predicting in-hospital mortality.
The black and thick line stands the CURB-65 for predicting in-hospital mortality. The grey line displays the assumption that all patients have died. The ne and black line represents the assumption that no patients have died. The decision curve showed that predicting the death risk applying this nomogram would be better than that of CURB-65, having all patients or none patients treated by this nomogram with a range of the threshold probability between >5% and <95%.