Development and validation of a prognostic model to predict clinical deterioration in hospitalized patients with COVID-19

Objectives We aimed to develop and validate a prognostic model to predict clinical deterioration dened as either death or intensive care unit admission of hospitalized COVID-19 patients. Methods This prospective, multicenter study investigated 172 consecutive hospitalized COVID-19 patients who underwent a chest computed tomography (CT) scan between March 20 and April 30, 2020 (development cohort), as well as an independent sample of 40 consecutive patients for external validation (validation cohort). The clinical, laboratory, and radiologic data were gathered, and logistic regression along with receiver operating characteristic (ROC) curve analysis was performed. The overall clinical deterioration rates of the development and validation cohorts were 28.4% (49 of 172) and 30% (12 of 40), respectively. Seven predictors were included in the scoring system with a total score of 15: CT severity score 15 (Odds Ratio (OR)=6.34, 4 points), pleural effusion (OR = 6.80, 2 points), symptom onset to admission ≤ 6 days (OR = 2.44, 2 points), age 70 years (OR = 2.44, 2 points), diabetes mellitus (OR = 2.24, 2 points), dyspnea (OR = 2.17, 1.5 points), and abnormal leukocyte count (OR = 1.89, 1.5 points). The area under the ROC curve for the scoring system in the development and validation cohorts was 0.823 (CI [0.751–0.895]) and 0.558 (CI [0.340–0.775]), respectively. This study provided a new easy-to-calculate scoring system with external validation for hospitalized COVID-19 patients to predict clinical deterioration based on a combination of seven clinical, laboratory, and radiologic parameters.


Introduction
The novel severe acute respiratory syndrome coronavirus (SARS-CoV-2) disease, also named COVID-19, was rst detected in December 2019 in Wuhan, China in a cluster of patients with pneumonia (1). As a result of its highly contagious character, soon it became a global threat, and the World Health Organization (WHO) declared COVID-19 a worldwide pandemic on March 11, 2020 (2). A wide spectrum of clinical features is described in COVID- 19 with the most common presenting symptoms being fever, cough, and dyspnea, however, a signi cant number of infected individuals are asymptomatic, and small percentage progress into severe pneumonia, acute respiratory distress syndrome (ARDS), multiorgan failure, and eventually death (3,4).
Many healthcare facilities across the world have anticipated a signi cant shortage of equipment and some already are struggling to provide care for the rising number of COVID-19 patients (5, 6). Lack of appropriate ≥ ≥ medical care delayed diagnosis, and insu cient hospital capacity increases the mortality (7). Therefore, due to overwhelmed healthcare systems during the current outbreak and limited resources, there is an urgent need for effective strati cation and accurate identi cation of patients who are at increased risk of mortality and other severe adverse outcomes.
Several studies have described older age, comorbidities e.g. hypertension, and multiple serum biomarkers as risk factors for disease progression and projected multivariable models to predict mortality and disease severity among COVID-19 patients (8-11). A systematic review evaluating these prediction models concluded that the recently proposed models were at high risk of bias because of methodological shortcomings and most of them were poorly reported (12). Moreover, the prognostic performance of pre-existing disease severity scores e.g.
CURB-65 and quick sequential organ failure assessment in the COVID-19 setting was veri ed to be generally suboptimal (13,14). Thus, the need for an objective measure to effectively triage COVID-19 patients has not met thoroughly up until now.
Herein, we aimed to develop and validate a comprehensive multivariable model consisting of various clinical, laboratory, and radiologic features to predict clinical deterioration de ned as either death or intensive care unit (ICU) admission of hospitalized patients with con rmed COVID-19. Through this study, we hope to shed light on the prediction of COVID-19 prognosis and assist in better strati cation of high-risk hospitalized patients to achieve better outcomes.

Methods And Materials
This prospective, multicenter cohort study was approved under a waiver of informed consent by the Ethics

Patient selection
We investigated consecutive hospitalized patients with COVID-19 who underwent a chest computed tomography (CT) scan between March 20 and April 30, 2020 (development cohort) in two early epicenters of the disease outbreak within the country (Tehran and Kashan university hospitals). Only those who had a positive reverse transcription-polymerase chain reaction test for SARS-CoV-2, using throat swabs, were included. Exclusion criteria were the presence of concomitant pulmonary diseases which interfered with the interpretation of chest CT scan (e.g. pulmonary edema due to other reasons, patients with positive blood or sputum culture caused by other infectious agents) and chest CT analysis confounders (e.g. blurred images, distorted images with beam hardening, and quantum mottle artifacts). Individuals with COVID-19 who were treated at home were not included. Between the above-mentioned dates, 184 patients were screened. After the exclusion of 12 patients because of CT scan violations, the total study sample for model development consisted of 172 patients.
An independent consecutive cohort of patients with similar inclusion and exclusion criteria to the development cohort, prospectively included in the study to externally validate the proposed model (validation cohort). This cohort consisted of 40 COVID-19 patients admitted to the Kashan university hospital between May 16 and June 2, 2020.
All patients were treated according to the interim guidance of WHO for COVID-19 (16). In case of respiratory failure requiring mechanical ventilation or other organ failures, the patients were admitted to the ICU. This criterion was similar to the previously published criterion for ICU admission of COVID-19 patients (17).

Data collection
The patients' medical history and physical examination data, along with all the available laboratory ndings were collected accordingly and double-checked by two independent authors to ensure accuracy.
All chest CT scans were performed with patient centered inside gantry in supine position, with raised arms at the rst day of admission. No contrast medium was administered. All chest CT scans were performed using one of cavitation, reticulation (intra-or interlobular septal thickening), parenchymal brotic bands, crazy paving pattern (inter and intralobular septal thickening within a GGO background), mosaic attenuation (alternative areas of differing attenuation with patchwork con guration), emphysematous changes (de ned as paucity of vascular structures within lungs parenchyma and prominent anterior junction line), pleural thickening (measurement of pleural thickness> 2mm), pleural effusion, cardiomegaly (cardiothoracic ratio more than 50%), pericardial thickening (pericardial stripe thicker than 4mm), and lymphadenopathy. Finally, based on the parenchymal involvement, the CT severity score was calculated. Semi-quantitative CT severity score was assessed according to the extent of GGO, consolidation, and crazy-paving pattern at thin-section CT scan based on volumetric measurements. Each lung lobe (according to the anatomical structure of lungs de ned by the Fleischner Society glossary of terms for thoracic imaging (18): left upper lobe, left lower lobe, right upper lobe, right middle lobe, and right lower lobe) was assigned a score based on a semi-quantitative criterion: score 0, no lobar involvement; score 1, <5% lobar involvement; score 2, 5-25% involvement; score 3, 25-50% involvement; score 4, 50-75% to involvement; and score 5, 75% or greater involvement. The summation of the scores was regarded as the CT severity score (scale of 0-25) ( Figure 1) (19,20).

Outcome de nition
All patients were followed up for their entire hospital stay to assess the clinical deterioration de ned as either inhospital mortality or ICU admission, the main outcome under the study. The patients who required ICU care but due to the shortage of ICU beds, did not admitted to the ICU were also regarded as of deterioration group. The patients who died or required ICU care during hospitalization were compared with the group of patients who required only general admission and discharged without ICU admission.

Statistical analysis
Categorical variables were described as frequencies and percentages and continuous variables were presented as means ± standard deviations (SD). Univariate logistic regression analyses were performed to select the best predictors for making the multivariate model for predicting the clinical deterioration of COVID-19. The signi cance level in univariate logistic regression was de ned at 0.2. The variables that were signi cantly associated with the clinical deterioration in univariate analysis were included in multivariate binary logistic regression, and their Z-Wald were also calculated. Then proportional to the Z scores, points were assigned to the predictor variables in the nal scoring system. Additionally, in order to make a simpli ed scoring system, we transformed our continuous variables into dichotomous ones based on the most discriminative cut-off points using the receiver operating characteristic (ROC) curve. Based on the developed model, the score of each patient was calculated, and the ROC analyses were performed for both the whole model and the scoring system. Finally, with the selection of a cut-off value, patients were divided into two low-and high-risk groups with clinical deterioration rate of 10% and 60%, respectively.
The calibration of the model, that is its ability to predict the clinical deterioration corresponding to the observed proportion of the COVID-19 patients who died during the hospital stay or admitted to the ICU, was assessed by conducting the Hosmer-Lemeshow (HL) goodness-of-t statistic. A P value of more than 0.05 indicated nonsigni cant inconsistency between observed and predicted deterioration. The discrimination abilities of the model and the scoring system in the development and validation cohorts were assessed using the ROC curve analysis and the area under the ROC curve (AUC) and its corresponding 95% con dence interval (CI) calculation.
Besides, the accuracy, sensitivity, speci city, and positive and negative likelihood ratio of the scoring system were calculated in both cohorts. All the analyses were performed in STATA v. 12.0 (STATA Corp., TX., USA).

Results
The general clinical characteristics of the development and validation cohorts as well as the patients with and without deterioration are summarized in Table 1. The overall clinical deterioration rates of the development and validation cohorts were 28.4% (49 of 172) and 30% (12 of 40), respectively.
Among the various clinical, laboratory, and radiologic parameters, 11 variables were found to be signi cantly associated with the clinical deterioration in univariate analysis ( Table 2): age, gender, symptom onset to admission, admission duration, fever, dyspnea, diabetes mellitus, hypertension, CT severity score, pleural effusion, and abnormal leukocyte count (below 4×10 9 /L or above 11×10 9 /L). The three continuous variables (age, CT severity score, and symptom onset to admission) were categorized into two groups (age 70 and 70 years; CT severity score 15 and 15; and symptom onset to admission ≤ 6 and 6 days) based on the ROC curve analysis (Figure 2).
Ultimately, with the exclusion of four insigni cant variables in the multivariate ordinal logistic regression, seven predictors were included in the nal model and scoring system with a total score of 15  Figure 3).

Discussion
In this study, we described an externally validated prognostic model consisted of seven clinical, laboratory, and radiologic features to predict the clinical deterioration de ned as in-hospital mortality or ICU admission of COVID-19 patients. This model allows physicians to categorize hospitalized COVID-19 patients into two low-and high-risk groups according to their deterioration risks. In the development cohort, the clinical deterioration rates were 11.7% and 61.1% for the low-and high-risk groups, respectively. Even though no proven effective therapy has been discovered for COVID-19 to date (21), adequate supportive measures most importantly ventilatory support is crucial for patient survival (22). Concerning this, the proposed model can alleviate the high burden of COVID-19 on health care systems by effectively detecting patients at high risk of severe complications who require more rapid initiation of supportive therapies.
Regarding the clinical credibility of the proposed model, we investigated more than 60 clinical, laboratory, and chest CT parameters and came up with a model involving all of these features. In our proposed model, older age with a cut-off value of 70 years, pre-existing diabetes mellitus, dyspnea at admission, symptom onset less than 6 days prior to admission, abnormal leukocyte counts either below 4×10 9 /L or above 11×10 9 /L, chest CT severity score more than 15, and the presence of pleural effusion had a prognostic impact on the deterioration of hospitalized COVID-19 patients.
Previously older age was identi ed as a major risk factor of mortality in COVID-19 patients (23). It was suggested that age-related alterations in B-cell and T-cell function as well as the excessive production of type 2 cytokines may contribute to de ciency in control of viral replication consequently leading to poorer outcomes in the elderly population (24). In a large meta-analysis, pre-existing diabetes has been veri ed to be associated with threefold in-hospital mortality among COVID-19 patients (25). Chronic in ammation, underlying metabolic changes, and innate or adaptive immune response attenuation are the main pathophysiological mechanisms hypothesized to play role in predisposing diabetic patients to severe infectious events namely COVID-19 (26).
Dyspnea, as an indicator of poor lung function and lack of oxygen, was previously veri ed as one of the predictors of criticality in COVID-19 cases (27). A shorter duration of symptoms until hospital admission implies a more severe disease course with rapid progression and undesirable prognosis.
Concerning laboratory abnormalities, both neutrophilia and lymphocytopenia have been described in critically ill COVID-19 patients which may result in either leucocytosis or leukopenia (28,29). The severity of lung damage and progression to ARDS is correlated mainly to the cytokine storm and neutrophil in ltration, as the main source of chemokines and cytokines, which may result in peripheral neutrophilia (29). Moreover, it is postulated that targeted invasion of cytoplasmic components of lymphocytes by viral particles as well as apoptosis induction in lymphocytes may contribute to lymphocytopenia in severe COVID-19 patients (28, 30).
In regard to radiologic aspects, CT severity score had been formerly veri ed as a rapid and objective method for evaluating the severity of pulmonary involvement with an excellent inter-reader agreement in COVID-19 patients (31). Additionally, the more prevalent presence of pleural effusions in severe COVID-19 patients was mainly linked to a more severe in ammatory process (32).
Regarding the validity of this model, the external validation step of the nal scoring system was performed in an independent cohort of COVID-19 patients derived from one of the hospitals involved in the model development process. While the accuracy of the scoring system in the validation cohort was lower than the development cohort (65.0% vs. 79.39%), it still provided important prognostic information in hospitalized COVID-19 patients.
Our proposed model has several crucial strengths over earlier reported ones. First, the study was conducted as a multicenter cohort study with a prospective design that let us assess numerous variables in the study participants without any signi cant missing data. Second, we evaluate all three aspects of the clinical, laboratory, and radiological aspects of the hospitalized COVID-19 patients and projected a model consisting of all of these aspects. Third, the described model score is easy to calculate with the data already available in hospitalized COVID-19 patients. This model consists of four clinical data that are easy to obtain from patients.
The only laboratory data needed for calculation of this model is the leukocyte count which is probably the most available laboratory data in hospitalized patients. The chest CT scan is the mainstay of severity assessment in COVID-19, and the two chest CT parameters de ned in this model are easy to obtain with the criteria de ned in the methods section. Fourth, the prede ned clinical deterioration as the main outcome is a clinically important consequence which our model is able to accurately predict it. Fifth, to overcome reporting issues, this study is conducted and reported with adherence to the TRIPOD statement. Sixth, we derived our model with the accepted statistical methods and validated it externally. This can result in a more accurate model with fewer concerns about over tting.
On the other hand, our study had some limitations too. First, the sample size used for model development and validation was limited. However, it was larger than many of the previous studies conducted on the same topic.
Second, we did not provide an explicit evaluation method for each of the parameters in the model except for the CT severity score, and we did not test inter-rater reproducibility. However, the predictors of the model are reasonably straightforward that this problem seems to be of little concern. Third, there are other clinical and laboratory features that have been demonstrated to predict outcomes in COVID-19 patients that were not included in our model. Nevertheless, our model with reasonable simplicity can precisely predict the clinical deterioration of COVID-19 patients.
In conclusion, this study provided a new scoring system for hospitalized COVID-19 patients to predict clinical deterioration de ned as mortality or ICU admission based on seven clinical, laboratory, and radiologic parameters. This prognostic model is clinically relevant and easy to calculate with sustained external validity.
Future studies should focus on testing the clinical usefulness of this model to predict the deterioration of COVID-

19.
Declarations FUNDING This research did not receive any speci c grant from funding agencies in the public, commercial, or not-for-pro t sectors.

DECLARATIONS OF INTEREST
None. The continuous variables as mean ± standard deviation and the categorical variables as frequency (percentage) are described.    Figure 1 A, B, and C. Low risk patient. A 72-year old man presenting with fever-chills, nausea-vomiting and without dyspnea for 8 days before admission. His WBC count was 6200 and fasting blood sugar was 106 on admission. Finally, he was discharged with signi cant clinical recovery after 8 days of in-ward admission with no need to respiratory support. Furthermore, his CT severity score was calculated as 12 and according to our model points, he obtained only 2 points out of 15. Black arrowheads indicate intra-lesional vascular engorgement within airspace opacities (ground glass opacity in A and C; crazy paving appearance in B). D, E, and F. High risk patient. An 82-year old diabetic man under insulin therapy, presenting with fever, malaise, nausea-vomiting, and dyspnea for 4 days before admission; WBC count was 14700 and fasting blood sugar about 431. Finally, he deceased after 5 days of ICU admission along with mechanical ventilation. His CT severity score was calculated as 21 and according to our model points, patient obtained 13 points. Short black arrows indicate air-bronchogram on a mixed ground glass/consolidation background (opacities are dominantly of consolidative density), and long blue arrows show lobular sparing.