Comparison of different prognostic scores in estimating short- and long-term mortality in COVID-19 patients above 60 years old in a university hospital in Belgium

Determining among 6 different scores which one most accurately predicted short-and long-term mortality in hospitalized COVID-19 patients above 60 years old. Among 6 different prognostic scales, the 4C Mortality Score (4CMS) was the best to predict intrahospital mortality and mortality at 30 days and 6 months. To predict 12-month mortality, the Charlson Comorbidity Index (CCI) had the best performance. This study reflects the importance of considering comorbidities for short and long-term mortality after COVID-19. Multiple scoring systems were used for risk stratification in COVID-19 patients. The objective was to determine among 6 scores which performed the best in predicting short-and long-term mortality in hospitalized COVID-19 patients ≥ 60 years. An observational, retrospective cohort study conducted between 21/10/2020 and 20/01/2021. 6 scores were calculated (Clinical Frailty Scale (CFS), Charlson Comorbidity Index (CCI), 4C Mortality Score (4CMS), NEWS score (NEWS), quick-SOFA score (qSOFA), and Quick COVID-19 Severity Index (qCSI)). We included unvaccinated hospitalized patients with COVID-19 ≥ 60 years old in Brugmann hospital, detected by PCR and/or suggestive CT thorax images. Old and nosocomial infections, and patients admitted immediately at the intensive care unit were excluded. 199 patients were included, mean age was 76.2 years (60–99). 47.2% were female. 56 patients (28%) died within 1 year after the first day of hospitalization. The 4CMS predicted the best intrahospital, 30 days and 6 months mortality, with area under the ROC curve (AUROC) 0.695 (0.58–0.81), 0.76 (0.65–0.86) and 0.72 (0.63–0.82) respectively. The CCI came right after with respectively AUROC of 0.69 (0.59–0.79), 0.74 (0.65–0.83) and 0.71 (0.64–0.8). To predict mortality at 12 months after hospitalization, the CCI had the highest AUROC with 0.77 (0.69–0.85), before the 4CMS with 0.69 (0.60–0.79). Among 6 scores, the 4CMS was the best to predict intrahospital, 30-day and 6-month mortality. To predict mortality at 12 months, CCI had the best performance before 4CMS. This reflects the importance of considering comorbidities for short- and long-term mortality after COVID 19. This study was approved by the ethical committee of Brugmann University Hospital (reference CE 2020/228).


Introduction
In March 2020, a global pandemic caused by SARS-CoV-2 was declared by the World Health Organization (WHO) [1,2].SARS-CoV-2 infection has many faces and clinical presentation can vary from an asymptomatic infection to a severe respiratory infection with need of admission at an intensive care unit (ICU) [3,4].While fighting the virus during the first waves, hospitals suffered a shortage of human and technical resources [5].With this in mind, many hospitals were faced with an ethical question of how these resources could be most adequately allocated.In the Brugmann University hospital (Brussels, Belgium), several mortality scores (Clinical Frailty Scale (CFS), Charlson Comorbidity Index (CCI) and one COVID-19-specific score; the 4C Mortality Score (4CMS)) were performed.Based on these scores, scarce resources were allocated in the most efficient way, with respect for every patient's values and goals.
As it has been studied before, we know that frailty and comorbidities were associated with an important increase in mortality risk, also in COVID-19 patients [6].The use of the Canadian scale known as CFS was recommended by the Belgian Geriatric and Gerontologic Society during the pandemic, whereas this score was already used in our geriatrics department [7].Although less specific for older persons, the CCI was included in the evaluation for its ease of use, good reliability and reproducibility considering the impact of comorbidities on the risk of mortality.It calculates the 10-year mortality risk in different pathologies [8] and has proved its place in predicting mortality in COVID-19 patients [9].After validation of the 4CMS, it was immediately used by the team.This score considered comorbidities as well as physiological parameters and laboratory values at admission.Literature confirms its good predictive value for intrahospital mortality in critically ill COVID-19 patients [10].Since then, many more specific mortality risk assessment scales have been developed for the COVID-19 infection.With our data, some of them could be evaluated retrospectively such as the National Early Warning Score (NEWS), the quick-SOFA score (qSOFA) and the quick COVID-19 Severity Index (qCSI).These three scores take into account the vital parameters of the patient at the emergency room.The NEWS is an early warning score used in different kinds of pathology to identify patients at risk of developing critical illness.It uses basic clinical observations including heart rate, respiratory rate, blood pressure, oxygen saturation, and level of consciousness [11].It was also proved to be a sensitive predictor of 7-day admission at the ICU or death in SARS-CoV-2 infections [12,13].The qSOFA score had proven prognostic accuracy in predicting mortality in patients with suspected sepsis [14].And finally, the qCSI was a score developed specifically to evaluate COVID-19 patients to estimate which patient will progress to respiratory failure within 24 h [15].
The primary objective of our study was to evaluate which of these six scores (CFS, CCI, 4CMS, NEWS, qSOFA or qCSI) performed the best estimation of intrahospital, 30-day, 6-month and 1-year mortality after the start of the hospitalization with a COVID-19 infection for patients aged ≥ 60 years old, as we know age is an important risk factor in COVID-19 infections [16,17].We also described the demographic and clinical characteristics of the study cohort.

Study design
This is a single-center, observational, retrospective study, conducted in patients ≥ 60 years old hospitalized in the Brugmann University Hospital between October 21st, 2020, and January 20th, 2021, with SARS-CoV-2 infection.This study was approved by the ethical committee of Brugmann University Hospital (reference CE 2020/228).
Data collection were extracted from the electronic medical folder files: demographic (age, sex and living situation), clinical (respiratory rate, peripheral oxygen saturation, supplemental oxygen, temperature, systolic blood pressure, heart rate, Alert, Vocal, Pain, Unresponsive score (AVPU) and Glasgow Coma Scale (GCS)), and laboratory data (urea, creatinine, C-Reactive Protein (CRP) and albuminemia), as well as chest computer tomography protocols.Frailty was classified by the CFS [7].Comorbidities were defined by the CCI (myocardial infarction, congestive heart failure, peripheral vascular disease, cerebrovascular accident or transient ischemic attack, dementia, chronic pulmonary disease (including asthma), connective tissue disease, peptic ulcer disease, liver disease, diabetes mellitus, renal disease, hemiplegia, solid tumor, leukemia, lymphoma, AIDS).
Three scores were calculated directly at the admission (CFS, CCI and 4CMS) and three were calculated retrospectively (NEWS, qSOFA and qCSI) for every patient included in the study.The 4CMS, the NEWS, qCSI score and qSOFA, were calculated based on parameters and lab results taken at the urgency ward.
The length of stay was considered as the period of time between admission and discharge, and for the intrahospital non survivors, the date of death.
To establish the intrahospital, 30-day, 6-month and 1-year mortality after the start of hospital admission, the vital status was searched in the medical files (Xcare, Abrumet, National population register).

Participants
Patients included were all unvaccinated hospitalized patients with COVID-19 infection aged 60 and over, detected by positive PCR (nasal or pharyngeal swabs or lower respiratory tract aspirates) or suggestive CT thorax images, described as such by the radiologist.Patients excluded from the study are nosocomial cases, patients admitted immediately at the ICU, and old COVID-19 infections (PCR COVID-19 positive > 14 days before hospitalization or positive IgG antibodies against COVID-19 infection).

Test methods
Continuous variables were reported as mean and standard deviation.Parametric tests were done for variables with Gaussian distribution (QQ plot).Otherwise non-parametric tests (Mann-Whitney U tests) were used.Dichotomous variables were reported as absolute number (percentage) and are compared by chi-square test (with Fisher's test when required (n < 5).A value of P ≤ 0.05 was noted as statistically significant as we used a 95% confidence interval.The statistician did not have any clinical information about the patients included, other than the parameters he had to evaluate.

Analysis
All statistical analyses were done with SPSS version 28.Sensitivity, specificity, positive (PPV) and negative predictive value (NPV), and the area under the receiver operating characteristic (AUROC) curve of each score for predicting all-cause intrahospital mortality, mortality at 30 days, at 6 months and mortality at 1 year were calculated.Youden's index was calculated to find the optimal cut-off points for the different scores.The statistical significance between the ROC curves was calculated according to the Hanley method [18].We excluded scores with incomplete data.We included all patients who were fitting our inclusion and exclusion criteria between 21/10/2020 and 20/01/2021.Finally, we included 199 patients.

Participants
Among the 409 patients hospitalized with a COVID-19 infection at Brugmann University Hospital between October 21st, 2020, and January 20th, 2021, 139 were younger than 60 years old and were excluded from the study.Were also excluded: 62 patients with a nosocomial COVID-19 infection and 9 patients with an old COVID-19 infection (Fig. 1).Finally, a total number of 199 patients were included in the study.Mean age was 76.2 years old (minimum 60-maximum 99).105 patients were male (52.8%) and 28.3% of the patients were institutionalized before hospitalization.
Table 1 shows the demographic and clinical characteristics of the enrolled patients divided in different groups (all population, survivors and deceased at different time points).We highlight a significant difference between the survivors and the group who died at the different time points when comparing age (p-value significant in each group, but getting more significant in further time points), renal disease (p < 0.001), and urea level at admission (p < 0.001).The different vital parameters at admission (not included in Table 1) and other clinical characteristics did not highlight a significant difference in any group (see Appendix).

For intrahospital mortality
Figure 2a shows that in using the area under the Receiver Operating Characteristic (AUROC) curve to predict the intrahospital mortality, the 4CMS has the highest AUROC curve 0.695 (0.58-0.81), followed by CCI 0.69 (0.59-0.79),NEWS 0.67 (0.56-0.77), qSOFA 0.66 (0.55-0.77),CFS 0.64 (0.54-0.74) and qCSI 0.595 (0.48-0.71).But the best sensitivity was attributed to CCI with a value of 0.72 and a negative predictive value of 91.1% where the 4CMS had a better specificity with a value of 0.78 and the best positive predictive value of 36% (Table 2).The difference between the six scores were statistically non-significant when comparing AUROC curves.

For the 6-month mortality
Figure 2c shows that 4CMS has the highest AUROC curve 0.72 (0.63-0.82), followed by CCI 0.71 (0.66-0.82),CFS 0.68 (0.59-0.77), qSOFA 0.64 (0.54-0.74),NEWS 0.62 (0.52-0.72) and qCSI 0.56 (0.46-0.67).CFS had the best sensitivity with a value of 0.76 and a negative predictive value of 89.4%, followed by CCI with a sensitivity of 0.75 and a negative predictive value of 89.3%.The CCI had the highest specificity with a value of 0.66 and a positive predictive value of 41.1% (Table 4).The difference between the six scores were statistically non-significant when comparing the AUROC curves.

For the 1-year mortality
Figure 2d shows that the CCI has the highest AUROC curve 0.77 (0.69-0.85), followed by the 4CMS 0.69 (0.60-0.79),CFS 0.67 (0.58-0.75), qSOFA 0.60 (0.50-0.69),NEWS 0.58 (0.49-0.68), and qCSI 0.55 (0.45-0.64).CCI had the best sensitivity with a value of 0.73 and a negative predictive value of 86.6%.This is closely followed by the CFS with a value of 0.72 and a negative predictive value of 85.2%.The 4CMS had the highest specificity with a value of 0.81 and a positive predictive value of 51.7% (Table 5).We saw that the CCI had a significantly higher AUROC curve than NEWS and qCSI at this time point.

Discussion
Between the survivor and the deceased group of the enrolled patients, age, renal disease and urea level at admission were significantly different.Age is often reviewed as an important  risk factor for a higher mortality in COVID-19 [16,17], and even though we only looked at a population older than 60 years old, we still withheld a significant difference in age between the survivors and the deceased at all different time points.We saw that the significance got stronger in further time points, which may be due to a lower life expectancy at an older age.It is also well known that renal function and COVID-19 infection was associated with a higher mortality [19,20].These three criteria were not included in CFS, NEWS, qCSI or qSOFA.CCI uses age and moderate to severe chronic kidney disease (CKD) and 4CMS uses age, urea level and renal function with the glomerular filtration ≤ 30 ml/min as cut-off point, which could explain a better association with mortality in all different time points.We did not find a significant difference in sex between the deceased group and the survivor group.Although it has been described in literature that the male proportion is significantly higher in the deceased group [21].When we looked at our results, we did see a trend toward a higher male proportion in the deceased group.We also saw that more male patients were hospitalized than female patients.This may be because more male patients had a severe COVID-19 infection [22].Or maybe during this wave female patients were less commonly sent to the hospital.As this has been shown in literature [23].We also did not find a significant difference in proportion of institutionalized patients or patients dependent at home in the deceased group vs the survivor group.
Although in literature, it has been found that the case-fatality rate is 5 times higher in people who are institutionalized [24].When looking into the details; we found that only 80 of the 199 patients included had a CFS > 3. We know that during these first waves, people who were frail and institutionalized, were advised not to be hospitalized which led to avoidance or delay of urgent care [23].This may be why we did not withhold a significant difference between the institutionalized patients and the non-institutionalized patients, although more research is needed.We did not find a significant difference in physiological parameters at admission between the survivors and the groups of patients who died at different time points.This could possibly be attributed to the exclusion of acute patients who were immediately admitted at the ICU or patients with a nosocomial infection with severe altered vital parameters.We did include patients who were too weak to be transferred to the ICU and patients hospitalized with palliative care, with severely impaired physiological parameters at admission.
Diabetes, dementia and obesity were not associated with a higher mortality in our study, despite the fact it was often described as an important risk factor in other studies [25][26][27].It could be linked to the retrospective nature of the data received in acute and often isolated conditions.Also, to diagnose dementia you need physical examination, images of the brain and a neuropsychological examination and sometimes a lumbar puncture [28].This might have been misdiagnosed for some participants or underdiagnosed for others.
To predict intrahospital mortality, mortality at 30 days and mortality at 6 months after hospitalization, we saw that 4CMS has the highest AUROC curve (Tables 2, 3, 4 and Fig. 1a-c).But for the 1-year mortality, CCI had the highest AUROC curve (Table 5 and Fig. 1d).Although the difference between 4CMS and CCI was not statistically significant when comparing the AUROC curves, we did see an important trend.These results were also reflected in the way in which these scores are normally used: 4CMS is used to estimate intrahospital mortality [10], and CCI is used to estimate mortality after 10 years [8].CCI had also already shown a good predictivity in mortality at 1 year in older persons experiencing a first acute heart failure hospitalization [29], as well as at 1 year after having had a hip fracture [30].This suggests that we can use the CCI to estimate mortality 1 year after hospitalization, although more research is needed.
Following these results, it may be interesting to use the 4CMS until 6 months after hospitalization, and maybe also evaluate its use it in other diseases than COVID-19.To date, we did not find any articles that investigated its use in estimating more long-term mortality in other pathologies.
In our study, the CFS was not noted as the best way to estimate mortality, although it was widely considered a very important score in estimating intrahospital and 30-day mortality [31], as well as an important indicator of worse outcome in COVID-19 infections [32,33].It did come on the third place in estimating 30-day mortality, 6-month mortality and 1-year mortality, which is certainly not bad, if you consider the ease of use of this score.
NEWS, qSOFA and qCSI were scores that take into account the physiological parameters at hospitalization.Our study showed that they were not the best predictors of mortality in our population, neither on short term nor on long term.At the 1-year time point, we saw a significant difference between the NEWS and CCI, as well as qCSI and CCI, which shows that NEWS and qCSI were significantly less powerful to estimate long-term mortality.NEWS is known to identify patients at risk of developing critical illness [11], it is not a mortality score.Also, qCSI is a COVID-19-specific score to estimate which patient will progress to respiratory failure within 24 h [15].It only includes a limited number of parameters.Although these scores have proven their value in detecting acute deterioration in COVID-19 patients [34], we could not confirm a better estimation of mortality in our population.qSOFA is a mortality score in patients with suspected sepsis but has already been proven a poor indicator in estimating mortality in COVID-19 patients [35,36], possibly because of the limited number of parameters included and not taking into account oxygen saturation [34].
The 4CMS and CCI are both scores that take the comorbidities into consideration.The 4CMS is based on the CCI but also includes obesity.It focuses on the number of comorbidities.Once two comorbidities were included, you get the maximum score [10].The CCI takes into account more specific consideration of the impact on the gravity of each comorbidity on the total score [8].We saw a lower AUROC curve of the scores where only the vital parameters were taken into consideration.Comorbidities, therefore, had an important impact on mortality in our specific population.The CCI and the CFS had a good sensitivity in all groups and 4CMS for 30-day and 6-month mortality.All these scores had the highest negative predictive value, which means they can best exclude the risk of further deterioration, although not any score was perfect.
We did not find a significant difference between the AUROC curves in intrahospital, 30-day and 6-month mortality.To estimate 1-year mortality, we did see a significant difference between CCI and NEWS and also between CCI and qCSI.This means that we did not see a significant difference between CCI and 4CMS at any time point, although we still consider the difference of the AUROC curves.Larger studies might be necessary in the future to confirm statistical significance between 4CMS and CCI.
We observed an intrahospital mortality rate of 18% (N = 36) a 30-day mortality rate of 17% (N = 33), a 6-month mortality rate of 24% (N = 48) and a 1-year mortality rate of 28% (N = 56).In comparison, we saw a case-fatality ratio of 29% in Belgium in patients older than 85 years old, with proven COVID-19 infection and death within 3 weeks after confirmation of infection.
The limitations in this study were related to the retrospective characteristic of the study, the limited patients included and the exclusion criteria.Also, our findings may not be relevant for future old patients with severe COVID-19 due to variant changes or effects of vaccination against SARS-CoV-2.Detection by CT elevated the risk of false positives [37].Some positive PCR tests may be older COVID-19 infections that were non-detected in the 14 days before hospitalization.As this is a singlecenter observational study, information bias was possible if certain elements in different files were missing.We did not use the absolute value of the different physiological parameters, which may result in less powerful statistical analysis.To prevent selection bias, we determined the CFS and the parameters at admission.To have better results, we need more and larger studies.
In conclusion we saw that, when comparing the six different mortality risk scores, the 4CMS was the best predictor for intrahospital mortality, mortality at 30 days and 6 months after hospitalization.The CCI was the best predictor for the 12-month mortality.This reflected the importance of considering comorbidities for short-but particularly long-term mortality.Although we did not find a statistically significant difference between 4CMS and CCI, we did see difference between these two scores in estimating short-and long-term mortality.Larger studies may be necessary in the future to confirm the statistical significance.The 4CMS should be studied to evaluate mortality up to 6 months in patients over 60 years old and in other pathologies than COVID-19.

Fig. 1
Fig. 1 Flowchart of the study

Fig. 2
Fig. 2 Graphical representation of the receiver operating characteristic (ROC) curve of the evaluated score for a intrahospital mortality, b for mortality at 30 days after hospitalization, c for mortality at 6 months after hospitalization, d for mortality at 1 year after hospitalization

Table 1
Demographic and clinical characteristics of enrolled patients for all population, survivors and intrahospital, 30 days, 6 months and 1 year mortality

Table 4
AUROC, sensitivity and specificity, negative and positive predictive value and positive and negative likelihood ratio of the different prognostic scores for predicting 6-month mortality Optimal cut off values for sensitivity and specificity were chosen according to Youden index a The differences between the different AUROC curves were statistically non-significant according to the Hanley method