Denition and Validation of a Risk-Stratication Model for Probable or Proven COVID-19 Patients in Emergency Departments: The Revised HOME-CoV Score.

The HOME-CoV rule is a list of clinical criteria dened by experts’ consensus, qualifying patients with probable or proven COVID-19 for home treatment when negative. The aims were to dene and validate a revised HOME-CoV score, optimizing the original rule. Denition of the revised HOME-CoV score using logistic regression in a prospective multicenter cohort and validation in another cohort of patients who presented to the emergency department with proven or probable COVID-19. The main outcome was non-invasive or invasive ventilation or all-cause death within the 7 days following inclusion. Two threshold values were dened using a sensitivity of > 0.9 and a specicity of > 0.9 to identify low-risk patients and high-risk patients, respectively. The revised HOME-CoV score included seven clinical criteria. In the denition cohort (n=1696), the AUC was 87.6 (95% CI 84.7 to 90.6). The cutoffs to dene low-risk and high-risk patients were <2 and >3, respectively. In the validation cohort (n=1304), the AUC was 85.8 (95% CI 80.6 to 91.0) and 85.5 (95% CI 76.8 to 94.1) in the subgroup of patients with positive RT-PCR for SARS-CoV2. A score of < 2 qualied 73% of patients as low risk with a sensitivity of 0.84 (0.66-0.95) and a negative predictive value of 0.99 (0.99-1.00). The revised HOME-CoV score compared favorably with the original rule and other models. The revised HOME-CoV score may allow accurate risk stratication and safely qualify for home treatment, a signicant proportion of patients with probable or proven COVID-19.


Introduction
The current pandemic of the novel 2019 coronavirus disease (COVID-19) has led to a substantial increase in the demands on conventional, acute and critical care services around the world and we continue to see a succession of waves 1 . The clinical spectrum of COVID-19 is broad, ranging from simple rhinitis to major pulmonary disease and death 2,3 . Mild or moderate infection corresponds to 80% of the COVID-19 population but approximately 2-4% of the patients require mechanical ventilation for Acute Respiratory Distress Syndrome (ARDS) [4][5][6] . For many of them, the deterioration is rapid after an initial period of relatively mild symptoms, emphasizing the need for early risk strati cation. A reliable risk assessment model is therefore required to standardize practices and guide frontline physicians in the decision to hospitalize or to manage at home mild or intermediate COVID-19 patients.

Importance
In this pandemic context, rationalizing hospital care is a major health issue to empower healthcare providers caring for patients at a high risk of adverse outcomes. Several clinical and biological features have been identi ed as mortality independent risk factors in COVID-19 [7][8][9] . Risk assessment models and triaging tools have been proposed, especially to refer COVID-19 patients to intensive care units. However, a recent living systematic review of COVID-19 conducted in collaboration with the Cochrane Prognosis Methods Group, found that all prognosis scores have a high risk of bias owing to poor reporting and poor methodological conduct enabling their recommendation for use in current practice 10 . The authors recommend following methodological guidance in order to avoid the use of unreliable and possibly harmful models in guiding clinical decision-making, and in particular, i) to build prediction models based on previous literature and expert consensus rather selecting predictors in a purely data driven way, ii) to develop and validate the model on a representative dataset of the target population in which it serves a clinical need, and iii) to currently focus on validating and updating promising available models.
Using a Delphi method with 51 experts, we de ned the "Hospitalization or Outpatient ManagEment of patients with SARS-CoV-2 infection" (HOME-CoV) rule. It contained 8 clinical criteria easily assessable at ED presentation (Table 1) 11 . The performances of the HOME-CoV rule in de ning a subgroup of patients with proven or probable COVID-19 who can be safely managed at home were recently con rmed in a prospective implementation before and after study: the HOME-CoV trial (NCT04338841, submitted).

Goals of this investigation
The global aim of the present study was to optimize the HOME-CoV rule in de ning and validating an updated version: the revised HOME-CoV score. The objective was to obtain a parsimonious and easy to apply score with 3 risk levels: low-risk patients who could be safely managed at home, intermediate-risk patients who may require hospitalization and high-risk patients who may bene t from intensive care.

Source of data
A total of 3000 patients with con rmed or probable SARS-CoV-2 were included, 1696 patients in the training cohort and 1304 in the validation cohort. Respectively, 65/1696 (3.8%) and 22/1304 (1.7%) patients had an adverse evolution in the training and the validation cohorts. Patients characteristics are presented in Table 2.

HOME-CoV Score Revision
The revised HOME-CoV score included 7 criteria (Table 3). The values of each criterion are mentioned in Table S2. The criterion "age over 65 years" was an independent factor of an adverse evolution within 7 days and its addition to the criteria of the original HOME-CoV score improved the performances of the score. Conversely, in the backward selection process, the criteria "Systolic blood pressure < 90 mmHg", "Heart rate ≥ 120 bpm" and "Severe comorbidity" were non-discriminatory and could be excluded without decreasing the revised score performances. The criteria "Ability to talk without breathing < 8 seconds" was a co-variable that positively impacting the model prediction. In the training cohort, the AUC of the revised HOME-CoV score was 87.6 (95% CI 84.7 to 90.6), a C-Index of 86.6 and the Brier score was of 0.045 (Table 4). The AUC of the original HOME-CoV score was 80.9 (95% CI 76.3 to 85.6), a C-Index of 80.5 and the Brier score was of 0.048.
The highest score value achieving a sensitivity of at least 0.90 was < 2 for the low-risk patients and for de ning high-risk patients, the lowest score value achieving a speci city of at least 0.90 was > 3 (Table 5).
In the subgroup of patients with con rmed SARS-CoV 2 infection (n = 364 patients), the revised HOME-CoV score had an AUC of 83.7 (95% CI 78.6 to 88.7), a C-Index of 83.2 and a Brier score of 0.11 (Table 4).

Revised-HOME-CoV Score Validation
In the validation cohort, the AUC of the revised HOME-CoV score was 85.8 (95% CI 80.6 to 90.9), the Brier score was 0.019 and the C-Index was 85.71 to predict an adverse evolution within the 7 days (Table 4). The calibration slope was 0.974 (close to 1) and the calibration intercept was − 0.76 suggesting an overestimation (Fig. 1).
Using a score value of < 2 as cut-off, 949/1304 (72.8%) were quali ed as low-risk patients and among them 5 had an adverse evolution (with a sensitivity of 0.84 (0.66-0.95) and a negative predictive value of 0.99 (0.99-1.00)) and 86/1304 (6.6%) patients had a score of > 3 and were quali ed as high-risk patients. Among them 12 had an adverse evolution with a speci city of 0.94 (0.93-0.96) and a positive predictive value of 0.14 (0.07-0.23)) ( Table 5).
In the subgroup of patients with con rmed SARS-CoV 2 infection (n = 183 patients), the revised HOME-CoV score had an AUC of 85.5 (95% CI 76.8 to 94.1) and a Brier score of 0.047 (Table 4). In this subgroup, 114/183 (62.3%) patients were quali ed as low-risk patients and among them 1 had an adverse evolution (with a sensitivity of 0.91 (0.59-1.00) and a negative predictive value of 0.99 (0.95-1.00)); 11/183 (6.0%) patients were quali ed as high-risk patients, and among them 3 had an adverse evolution (with a speci city of 0.96 (0.92-0.98) and a positive predictive value of 0.13 (0.05-0.32)).

Comparison of the clinical risk assessment models
The predictive performances within the 7 days of the different scores are summarized in Table 4 and Table S3. The revised HOME-CoV score exhibited the highest AUC in both cohorts. Its AUC was signi cantly higher than that of the qSOFA, CURB65 and SMART-COP scores.
In the validation cohort, the original HOME-CoV rule quali ed 627/1304 (48.1%) patients as low-risk (all criteria negative) and among them 1 had an adverse evolution (sensibility of 0.95 (0.92-0.98) and a negative predictive value of 0.99 (0.97-1.00).

Time-dependent prediction
The AUC of the revised HOME-CoV score slightly decreased from Day 2 to Day 7 and was stable after Day 7: 85.8 (80.1-91.0) at Day 28 (Table 4, Figure S1).

Prediction of patients requiring intubation or dying within the 7 days
Results were similar regarding the rate of patients requiring intubation or who died within the 7 days (WHO-OSCI ≥ 6) with an AUC of the revised HOME-CoV to 86.7 (95%CI 84.6 to 91.8) (Table S4).

Discussion
The revised HOME-CoV score included 7 clinical criteria easy to assess upon patient's presentation in the emergency department. It exhibited good and stable over-time performances in predicting an adverse evolution in patients with suspected COVID-19 and in patients with con rmed COVID-19, outperforming the other risk models for COVID-19. Three levels of risk have been established. Low-risk patients, eligible for home treatment, accounted for almost three quarters of the study population and less than 1% will require intensive care within the 28-day follow-up.
The de nition of the revised HOME-CoV score was made according to recent recommendations 10,11 . The criteria were rstly chosen by expert consensus and were completed by the clinical items present in the other scores 10,11 . A logistic regression model and the Akaïke Information Criterion were used to choose the most parsimonious model rather than to identify the criteria of interest. In this process, three criteria of the original rule were excluded (heart rate, blood pressure and severe comorbidity), and one additional criterion was included in the revised HOME-CoV score (age > 65 years). The age had not been retained by the experts in the original HOME-CoV rule because this rule was performed as a checklist, qualifying patients with probable or proven COVID-19 for home treatment when all criteria were negative. The experts participating to the Delphi consensus did not consider that advanced age, as a standalone criterion, preclude home treatment. The revised and somewhat simpli ed HOME-CoV score exhibits better performances in risk characterization of adverse evolution than the original version. This result emphasizes that the prognosis of COVID-19 is correlated to respiratory lesions rather than to systemic sepsis with hemodynamic instability 12 .To facilitate its implementation in ED current practice, biological variables and imaging data were not considered in the revised HOME-CoV score. The clinical variables are easy to assess upon presentation at the ED and enable rapid home discharge of low-risk patients.
Moreover, the limited number of items makes the score easily memorable.
Many other risk-assessment scores have been proposed for COVID-19 patients 9,[13][14][15][16][17][18][19] . Most of them are highly susceptible of bias, incorporate biological or imaging data and/or are complex requiring computational tools such as the recently proposed 4C mortality score and COVID-19 GRAM 15,16 . Furthermore, their performances do not appear to be better than those of the revised HOME-CoV score 15,16 . In our study, the AUC of the CURB65 (including uremia) and of the SMART-COP (including multilobar involvement on chest-ray or CT scan) were signi cantly lower than the AUC of the revised HOME-CoV score. Importantly, to our knowledge, all previous models were established in a population of hospitalized COVID-19 patients and could not serve in identifying ED patients who can be safely discharged home. The revised HOME-CoV rule was de ned and validated in two cohorts of consecutive ED patients with highly suspected or proven non-severe COVID-19. Therefore, these datasets represent the target population that may bene t from its implementation as a helpful decision-making tool 10 . Moreover, the same positive performances were observed in the overall population and in the subgroup of patients with a positive RT-PCR for SARS-CoV2. This reinforces the validity of the score and its usability in EDs where access to RT-PCR for SARS-CoV2 is limited and/or where obtaining the results requires several hours 20 . A revised HOME-CoV score below 2 allows rapid and safe quali cation by the emergency physicians of ¾ of patients with probable or con rmed COVID-19 for home treatment. As compared to the original rule, the revised HOME-CoV score may signi cantly increase the rate of patients managed at home without increasing their risk of death or to requiring intensive care. It could also be used by other frontline physicians such as general practitioners or geriatricians and e ciently participate in mitigating the burden on the healthcare system. This study has several strengths. Analyses and results are based on a multicenter study with a large number of participating centers and patients and prospective data collection. Two separate cohorts were used to review and validate the score. We used a clinical and consensual main judgment outcome: the ordinal scale for clinical improvement of COVID-19 from the WHO. This scale, and especially for the last stages, seems less dependent on physician's gestalt and hospital caseload than ICU admission or oxygen requirement. The methodology used follows the most recently published recommendations and the TRIPOD reporting guidelines 10 .
It also has some limitations. The rate of patients who had an adverse evolution was low, limiting the accuracy of the measurements. Both cohorts came from the same centers. The revised HOME-CoV score needs to be formally validated in a prospective implementation outcome study.
In conclusion, the revised HOME-CoV score allows for accurate risk-strati cation of patients presented to emergency departments with proven or probable COVID-19. Future studies may evaluate its usefulness to help frontline physicians in decision-making.

Study design
We performed a retrospective analysis of a prospective multicenter before and after implementation trial aimed at validating the original HOME-CoV rule (NCT04338841, submitted).
The rst step was the de nition of the revised HOME-CoV score to enhance the predictive performances of the original HOME-CoV rule through the inclusion of clinical predictors identi ed in other models and the exclusion of poor informative variables. The second step was the external validation of the revised HOME-CoV score in an independent cohort. The third step was the comparison of the performances of the revised and the original HOME-CoV score compared to those of other assessment models proposed for risk strati cation of COVID-19 patients: qCSI, qSOFA, CURB65, CRB65 and SMART-COP 13,21-23 (Table  1). We adhered to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) checklist (Appendix 1).

Source of data
We used two distinct and independent phases of a prospective study (n=3000 patients). This study was a quasi-experimental before and after prospective multicenter trial, performed in 34 Emergency Departments from in France (31), Principality of Monaco (1) and Belgium (2) from April 9 to May 11, 2020. Patients were eligible for inclusion if they provided informed consent, were at least 18 years old, had symptomatic COVID-19 con rmed by a positive SARS-CoV-2 Reverse Transcriptase-Polymerase Chain Reaction (RT-PCR), or had typical symptoms of COVID-19 and the COVID-19 was the most likely hypothesis for the physician in charge of the patient in the ED, and had provided informed consent. Only mild and moderate cases of COVID-19 were included. Patients were excluded if the main diagnostic hypothesis in the emergency room was not a SARS-CoV-2 infection but another differential diagnosis, if they required care in an ICU or a resuscitation unit, if a limitation decision of active therapies was made, and if the follow-up at 28 days was not possible. Patients characteristics and clinical data were collected prospectively at the ED presentation ( Table 2). Biological or imaging examinations were not required. Patients were followed-up at Day 7 and Day 28 by phone and their clinical status according to the Ordinal Scale for Clinical Improvement for COVID-19 patients from the World Health Organization was recorded (WHO-OSCI) (Table S1) 24 . The date of each change in WHO-OSCI status was also recorded to allow analysis over time.

Outcome
The main outcome was an adverse evolution within the 7 days after ED presentation. It was de ned, according to the World Health Organization, as a hospitalized patient with severe disease, under noninvasive ventilation or high-ow nasal oxygen therapy (stage 5), requiring intubation and invasive ventilation (stage 6), and/or additional organ support (stage 7) or who died whatever the cause (stage 8) (Table S1) 24 .

HOME-CoV Score Revision
To de ne the revised HOME-CoV score, we assessed if the inclusion of clinical variables included in other models suggested for risk strati cation of COVID-19 patients (Table 1), would improve the performances of the original HOME-CoV rule, and, conversely, if the exclusion of clinical variables of the original rule would decrease the performances of the revised HOME-CoV score. In order to facilitate the implementation of the revised HOME-CoV score in current practice of emergency and primary care physicians, we did not consider the inclusion of biological variables (uremia, pH, albuminemia). Only one clinical variable, an age of> 65 years, included in the CRB65 and CURB65 was found for this purpose 11 . A logistic regression model was performed to assess the discriminant value of each criterion of the original rule and of an age over 65 years regarding adverse evolution (WHO-OSCI ≥5). In order to choose the most parsimonious model, a selection of variables allowing the optimization of the Akaïke Information Criterion (AIC) was carried out in a manual backward selection process. All signi cant variables, as well as the non-signi cant variables that interact with the other items and provide the best AIC were retained. The absence of multicollinearity between predictors was checked using the variance in ation factor. We anticipated 3 levels of risk prognosis to identify a low-risk subgroup of patients who could be safely treated at home, an intermediate-risk subgroup who require hospitalization and a high-risk subgroup of patients who may require intensive care. For de ning low-risk patients, we chose the highest score value achieving a sensitivity of at least 0.90 and for de ning high-risk patients, we chose the lowest score value achieving a speci city of at least 0.90.

Revised-HOME-CoV Score Validation
The accuracy of the score was assessed by calculating the Receiver Operating Characteristic (ROC) curve and analysing the Area Under the Curve (AUC). The AUC con dence interval was computed with the Delong-Delong method. An AUC of ≥ 0.8 with a lower limit of the 95% con dence interval (95%CI) of ≥ 0.7 was considered as clinically relevant. The Brier score was also reported, summarizing the magnitude of error in the probability forecasts between 0.0 and 1.0, where a perfect model has a score of 0.0. We also calculated the C-Index corresponding to the model discrimination. Calibration was assessed with the slope and the intercept value. Using the two prede ned threshold values, the sensitivity, speci city, likelihood ratios and the negative and positive predictive values with their 95% con dence interval (95%CI) of the revised HOME-CoV score were calculated.

Comparison of the clinical risk assessment models
We used .632+ bootstrapped logistic regression to evaluate the scores' performances and their con dence intervals and to generate performance benchmarks of the different scores: original HOME-CoV, revised HOME-CoV, qCSI, qSOFA and CRB65. The comparison with the CURB65 was performed in the subgroup of patients with uremia measurement at baseline and the comparison with the SMART-COP in the subgroup of patients with pulmonary imaging. In order to calculate the SMART-COP, multi-lobar lesions either on chest X-ray or CT scan were considered as a positive criterion ("multi-lobar involvement on chest X-ray"). We report 95% con dence intervals derived from the percentiles of the bootstrapped distribution.

Complementary analyses
We performed a subgroup analysis in patients with con rmed SARS-CoV-2 infection by RT-PCR.
We also performed two sensitivity analysis with the following outcomes: (1) Adverse evolution as a function of time within the 28 days following ED presentation.
(2) Evolution to a severe disease within 7 days, de ned as requiring invasive ventilation or leading to death whatever the cause (WHO-OSCI ≥ 6).

Statistical analysis
Missing data were not imputed. Statistical analyses were performed using R software (version 3.5.1, R-Core Team) and the following R package: pec, timeROC, and survival 25-27 .

Declarations Acknowledgements
We wish to thank the members of the HOME-CoV study group*: -Andrianjafy H., Emergency Department, GH Nord-Essonne, Longjumeau, France.

Author Contribution
The rst, second, and last author have designed the study. The rst draft of the manuscript was written by the rst and the last authors. All the authors contributed to the nal manuscript and attest the accuracy of the data and the delity of the study to the protocol. The corresponding author a rms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.

Ethics approval and consent to participate
The HOME-CoV study obtained approval from the Comité de Protection des Personnes Ouest IV -Nantes