Evaluation of mortality in ICU-hospitalized COVID-19 patients by using REMS, APACHE-II, CCI, and SOFA

Introduction: Our study aimed to analyse the effectiveness of four scoring models in predicting mortality of intensive care unit (ICU) hospitalized COVID-19 patients. The models used in this regard were: Rapid Emergency Medicine Score (REMS), Charlson Comorbidity Index (CCI), Acute Physiology and Chronic Health Evaluation II (APACHE-II), and the Sequential Organ Failure Assessment (SOFA). Materials and Methods: A single-centre and retrospective analysis was carried out by considering denitive or probable COVID-19 patients hospitalized our hospital’s ICU unit. Patients who were admitted to our hospital’s ED between 11.03.2020 – 31.12.2020, and transferred directly to ICU from the ED due to being diagnosed with COVID-19 were included in our study. 411 patients above 18 years old were found appropriate for the study. Results: Among the patients, the mean age was 69 and 61.6% were male. Laboratory values such as creatinine, potassium(K), white blood cells(WBC), hematocrit(HTC), pH, and physiological ndings such as mean arterial pressure, systolic and diastolic blood pressure, FiO2 were found statistically signicant (p<0.05). Besides, comorbidities were observed in 368(89.5%) patients, and malignancy and dementia were statistically associated with death (p<0.001 and 0.019, respectively). All four of the scoring systems (REMS, CCI, APACHE-II, and SOFA) were statistically an indicator of in-hospital mortality (p<0.001). However, when ROC analysis was used to compare the discriminatory power of the scoring systems, no meaningful difference was detected (p>0.05). Conclusion: We investigated that REMS, CCI, APACHE-II, and SOFA were effective in determining the in-hospital mortality of critically ill COVID-19 patients; however, no remarkable superiority existed between each other. These models may be guiding for ED physicians in terms of risk classication.


Introduction
Coronavirus disease , which causes severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) was started in Wuhan, China, and spread very rapidly becoming a global pandemic affecting more than 180 countries. Patients of all ages with underlying diseases such as hypertension and diabetes mellitus showed a worse prognosis. [1] Middle-aged and elderly patients with COVID-19 are more susceptible to this disease, and it has been found that they are hospitalized in intensive care with high mortality rates. [2] In terms of evaluating the course of the disease and treatment possibilities, it is crucial to predict the risk factors associated with the mortality of the COVID-19 patients during the admission stage in the emergency department (ED). [3] Effective and simple methods/scoring systems to assess the severity and prognosis of COVID-19 patients are still new and challenging for clinicians. Among these scoring systems, the rapid emergency medicine score (REMS) is frequently used for ED patients. [4] REMS model is recommended at the initial stage in predicting the mortality of non-surgical patients admitted to the emergency department.
[5] Another model called Sequential Organ Failure Assessment (SOFA) was not initially designed as a score predicting mortality, but later on, it was detected that it predicts mortality and morbidity in patients with critical condition. [6] This method measures several parameters to describe the condition of multiple organ dysfunctions. [7] Novel studies managed to conclude that using SOFA during admission helped to classify the COVID-19 patients for in-hospital death. [6], [7] The 3 rd and 4 th scores included in this research are The Charlson Comorbidity Index (CCI), and Acute Physiology and Chronic Health Evaluation II (APACHE-II) scoring systems that can predict comorbid conditions and mortality in many disease groups. [8], [9] The Charlson Comorbidity Index (CCI) is a practical method widely applied to determine the mortality risk in ten years for the patients having comorbid conditions. [8] Several studies are aiming to better de ne its advantage and evaluate the relationship between CCI and poor outcomes in COVID-19 patients. [10]- [12] In a study conducted on patients with COVID-19 infection, a CCI score above 0 was associated with a 16% increase in the risk of death, and an increased mortality rate was found with each increasing point in CCI score. [12] Lastly, APACHE-II scoring system has been widely used in intensive care unit (ICU) patients, and demonstrated that it is an accurate measure of disease severity in various disease states and different clinical settings and correlates strongly with expected mortality in critically ill patients. [13] One study evaluated vital signs (respiratory rate, oxygen saturation), in ammation markers (WBC, neutrophil counts, CRP), and APACHE-II scores among the COVID-19 patients, and showed that these indications increase the discriminatory e cacy while predicting the severity of the outcome of patients. [14] The use of scoring systems in COVID-19 patients during their admission in ED may allow quick and accurate assessment of the disease severity and risk of mortality, yet it is still a new subject to literature. Therefore, we focused our research on evaluating the factors affecting in-hospital mortality by using REMS, APACHE-2, CCI, and SOFA scores in potential or de nite COVID-19 patients that were directly admitted to ICU from the ED. We aim to determine the effectiveness of the scores, and compare them with each other.

Study Design
This study has been designed as a single-centre, retrospective study in the XXX University Hospital, Emergency Medicine Clinic. The research unit is a third-level reference health centre in XXX (city), and about 190,000 patients are cared for annually. Study reporting was done per the Strobe Guidelines [15].
The research protocol was approved by the ethical committee of XXX University local ethics committee

Setting
Patients who were admitted to our hospital's ED between 11.03.2020 -31.12.2020, and transferred directly to ICU from the ED due to being diagnosed as a de nitive or probable COVID-19 case were included in our study. A total of 411 patients aged >18 were found eligible for the study. The diagnosis criteria, classi cation of cases and the treatments were based on the recommendations provided in the "COVID-19 General Information, Epidemiology and Diagnosis" report issued by the Ministry of Health. [16]

Data Collection
An archive search was conducted by an experienced data analyst from the hospital's IT department using the ICD (International Statistical Classi cation of Diseases and Related Health Problems) [17] code U07.3, which is the clinical code in XXX (country) for COVID-19. From the medical records, the data were extracted concerning the demographics, clinical characteristics, comorbidities, laboratory ndings, and outcomes. Then, the data were used for the calculation of REMS, SOFA, CCI, and APACHE-II scores.

Variables
There are four main variables in this study that are essentially the scoring models, namely, SOFA, REMS, CCI, and APACHE-II. In addition, except for CCI, all of these models include GCS as one of their parameters. Therefore, these ve variables are described in this section.
Glasgow Coma Scale (GCS): a practical method evaluating the patient's conscious state based on their ability to respond to the certain eye, verbal, and motor stimuli. The total score ranges from three to 15, and a point between three and eight represents a signi cant injury, while nine to 12 and 13 to 15 are assessed as medium and mild injuries, respectively. [18] Rapid Emergency Medicine Score (REMS): used for the non-surgical patients admitted to ED to predict the in-hospital mortality. This model contains 6 variables that are heart rate, blood pressure, respiratory rate, GCS, oxygen saturation level (SpO2), and age. The sum of the values assigned to these variables can be a maximum of 26, and a higher value indicates a worse prognosis. [4] Sequential Organ Failure Assessment (SOFA): a quantitative method to determine the extent of patients' organ dysfunction or failure over time. It evaluates variables such as oxygenation index (arterial oxygen pressure [PaO2] / inspiratory oxygen fraction [FiO2]), mean arterial pressure, Glasgow Coma Score (GCS), creatinine or urine volume, bilirubin and platelets. The SOFA score ranges from 0 to 24, and the higher the score, the more severe the situation. [19] Charlson Comorbidity Index (CCI): based on a point scoring system developed to assess the longterm prognosis and survival of the patient by classifying 17 comorbid conditions, such as cardiovascular disease, diabetes mellitus, liver disease, pulmonary disease, etc. A higher score indicates higher mortality within one year of hospital admission and the maximum score is 37. [8] Acute Physiology and Chronic Health Evaluation II (APACHE-II): a scoring system applied within 24 hours of admission to ICU, estimating mortality based on patient signs, various laboratory values, and acute and chronic diseases. The nal score ranges from 0 to 71, and a higher score corresponds to a higher risk of death. [9], [13]

Results
Initially, medical records of 786 patients were obtained who were admitted to ICU of the ED between 11 th of March and 31 st of December, 2020. The cases that were not COVID-19, without appropriate data, deceased, or not directly admitted to ICU from the ED were excluded. As a result, 411 patients were found suitable for the research, from which 122 were de nitive, 247 were probable, and 42 were low probability COVID-19 cases (Figure 1). The mean age of the participants was 69, and 61.6% were male.
Firstly, the categorical variables were analysed and 368 (89.5%) of the patients had comorbidities. The most frequently observed comorbidity was diabetes mellitus, and seen in 156 (37.9%) of the patients. Comparing categorical variables for survival (Table 1), malignancy and dementia were associated with mortality (p values <0.001 and 0.019, respectively). Additionally, 144 patients needed CPAP/MV and 116 of them did not survive. Inotrope treatment was required for the lesser amount of patients (n=65), however, 57 of them died. Both CPAP/MV and inotrope treatment were statistically signi cant in terms of mortality (p<0.001).
In comparisons of numerical variables, signi cant differences were detected in REMS, CCI, SOFA, and APACHE II scores concerning of survival ( Table 2). The physiological ndings such as MAP, systolic and diastolic BP during both ED and ICU were statistically different for the non-surviving group. Age was another factor associated with mortality (p<0.001). In addition, the laboratory ndings that are creatinine, K, WBC, HTC, and pH showed also statistically signi cant differences in terms of survival.
In Figure 2, receiver operating curve (ROC) analysis was used to determine the discriminatory power of the four scoring systems, REMS, CCI, SOFA, and APACHE-II. There was no statistically signi cant difference in the comparison of the sensitivity of each score (Table 3). This was concluded since the difference between the area under the curve was small and the p values were larger than 0.05.

Discussion
The COVID-19 pandemic has brought many managerial and clinical challenges to healthcare systems. Evaluating high-risk COVID-19 patients in advance is crucial to provide accurate medical monitoring and treatment for these patients, especially in the presence of shortages of medical sources following the outbreak. [7] A scoring system can assist emergency physicians to quickly evaluate the condition of the COVID-19 patient, and assess in terms of mortality, yet the effectiveness of such systems is still under study. In this respect, the scoring systems REMS, APACHE-2, CCI, and SOFA were used in this study.
For instance, SOFA is a practical tool to evaluate systematically and continuously the organ functionality of the patient during hospitalization. Some recent studies associate higher SOFA scores in COVID-19 patients with higher in-hospital mortality. [2], [6] In this study group, higher SOFA scores were also statistically an indicator of mortality (p<0.001). We investigated that four of the variables (GCS, MAP, creatinine, FiO2) of SOFA showed a statistical difference between non-survivors and survivors (p<0.05). A study performing mortality prediction for patients with COVID-19 pneumonia requiring mechanical ventilation concluded all the SOFA parameters except GCS were signi cantly different in the nonsurviving group. Also, this study estimated the median and the interquartile range as 6 and 4-8, respectively. [7] Similarly, the median of our study group was 5, and the IQR was 3-8. Accordingly, the SOFA score can be used to evaluate the mortality risk of critical COVID-19 patients independently.
To detect the high-risk patients and regulate the management of patients in the ED, many scoring systems were developed such as APACHE-II and REMS. [20] A study including 392 COVID-19 patients admitted to the ED was examined and it was reported that REMS could be guiding for physicians to estimate the one-month mortality of ED patients with COVID-19. [21] In another study, Hai Hu et al.
determined that the AUC value of REMS in predicting in-hospital mortality was 0.841, and among the REMS parameters; age, respiratory rate, SpO2, and GCS were strongly correlated with mortality. [20] Following these signi cant ndings, REMS was included in our research. The REMS score was higher in the non-surviving group (p<0.001), and the parameters involved; age, SpO2, and GCS showed a statistically meaningful difference. APACHE-II, on the other hand, is a more sophisticated predictive score, and reportedly has the highest accuracy in predicting mortality of ED patients, but less practical than REMS as its variables are di cult to attain in the early ED phase. [22] Xaojing Zou et al.
showed an APACHE-II score being 17 or greater is an alerting situation in terms of death, and could be guiding for physicians in making clinical decisions. [23] Parallel to that, we estimated the median value of non-surviving COVID-19 patients as 16, and higher scores were correlated with higher mortality (p<0.001). Olsson and Lind [24] reported in their study that REMS and APACHE-II were equally accurate in terms of determining mortality of non-surgical ED patients. Another study compared APACHE-II and SOFA scores for COVID-19 patients, and APACHE-II was a better predictor of mortality. [23] When we compared APACHE-II, SOFA, and REMS for COVID-19 cases in our study, we investigated that they cannot show superiority to each other in terms of mortality prediction.
Based on the current clinical ndings, comorbidities are frequently associated with severe outcomes in COVID-19 patients. A study in the USA investigated among their study group of 178 hospitalized COVID-19 patients, 89.3% had comorbidities, [25] and similarly, we found this ratio as 89.5% in our study.
A nationwide analysis in China resulted that patients with cancer tend to have worse outcomes. [26] Moreover, a pooled analysis considering a total of 18,650 patients with both COVID-19 and cancer also showed these patients have a higher probability of death. [27] Dementia was another frequently seen comorbidity in COVID-19 patients that was associated with higher mortality rates. [28], [29] We have also investigated that dementia and malignancy were linked signi cantly to increased fatality. Other studies correlate hypertension, chronic obstructive pulmonary disease (COPD), and diabetes to increased morbidity. [1][30] Nevertheless, comorbidities are a substantial factor in terms of predicting the risk of death in COVID-19 patients, thus a scoring system in this regard, namely CCI was included in this study. Several studies conclude the advantage of risk strati cation in COVID-19 patients when CCI is used, and poorer outcomes are frequently associated with relatively high CCI scores. [10]- [12] In our research, a signi cant relationship between mortality and higher CCI scores was concluded, and the median CCI value of the non-surviving group was 5, the same with a study including 383 COVID-19 patients. [11] Up to our research, there was no study comparing CCI with other scoring systems (SOFA, REMS, and APACHE-II) for COVID-19. Based on our analysis, there was no signi cant difference between CCI and those other scoring systems in predicting mortality of ICU hospitalized COVID-19 patients. It is advised that more studies should focus on comparatively investigating the accuracy of different models.

Study Limitations
This study should be interpreted in light of some limitations. First of all, it is a single-centre retrospective investigation; therefore the population of a study group is limited. In this respect, studies including a larger sample size, and also multicentre studies to prospectively validate the scoring systems are required. Secondly, at the beginning of the outbreak, the most frequently used COVID-19 medications for today were with limited availability, which may have worsened the patient outcomes.

Conclusion
There are limited studies concerning the effectiveness of mortality scoring systems used in routine procedures during emergency service admission and follow-up in intensive care in the COVID-19 pandemic. We investigated that the scoring systems SOFA, REMS, APACHE-II, and CCI were statistically indicators of mortality in ICU-hospitalized patients, although no remarkable superiority between each other was detected. These prognostic scoring models may guide ED physicians in evaluating the patient's condition quickly, and prioritizing the high-risk group.

Declarations
Competing interests: The authors declare no competing interests.