Predicting in-hospital mortality for patients admitted from the emergency department: An initial alarm score for inpatient care


 Background: The in-hospital mortality of patients admitted from the emergency department (ED) is high, but no appropriate initial alarm score is available. Methods: This prospective observational study enrolled ED-admitted patients in hospitalist-care wards and analyzed the predictors for seven-day in-hospital mortality from May 2010 to October 2016. Two-thirds were randomly assigned to a derivation cohort for development of the model and cross-validation was performed in the validation cohort. Results: During the study period, 8,649 patients were enrolled for analysis. The mean age was 71.05 years, and 51.91% were male. The most common admission diagnoses were pneumonia (36%) and urinary tract infection (20.05%). In the derivation cohort, multivariable Cox proportional hazard regression revealed that a low Barthel index score, triage level 1 at the ED, presence of cancer, metastasis, and admission diagnoses of pneumonia and sepsis were independently associated with seven-day in-hospital mortality. Based on the probability developed from the multivariable model, the area under the receiver operating characteristic curve in the derivation group was 0.81 [0.79–0.85]. The result in the validation cohort was comparable. The prediction score modified by the six independent factors had high sensitivity of 88.03% and a negative predictive value of 99.51% for a cutoff value of 4, whereas the specificity and positive predictive value were 89.61% and 10.55%, respectively, when the cutoff value was a score of 6. Conclusion: The seven-day in-hospital mortality in a hospitalist-care ward is 2.8%. The initial alarm score could help clinicians to prioritize or exclude patients who need urgent and intensive care.


Introduction
In-hospital mortality is a universally accepted indicator for the medical quality of inpatient care [1]. Although mortality is usually very low for elective admission, it is comparably higher for admissions from the emergency department (ED) [2,3]. The 30-day mortality of emergency admission has been reported to be 3.7-8.2% in a report from Ireland [4], and 4.7-5.8% and 6.93-7.04% in two UK reports, respectively [5,6]. Notably, the risk of death is relatively high in the initial days from emergency stay to hospitalization [4]. Therefore, identifying patients at risk of deterioration during their initial hospitalization days is a great challenge but important to clinicians.
One commonly used prediction score is the National Early Warning Score (NEWS), which is mainly composed of vital signs and consciousness level. However, changes in vital signs can indicate acute changes in the illness leading to a possibly irreversible late stage, and the NEWS is suggested for only very short prognosis, such as within 24 hours [7]. For inhospital prognosis before vital signs change, an initial predictive score that prioritizes the inpatient's care is needed for admission evaluation. It is particularly crucial to predict inhospital mortality at the initial admission because emergency admissions have been reported to have high in-hospital mortality of 6.2-12% in previous reports, and especially because early deterioration is not uncommon [8,9]. Therefore, in this study, we conducted a prospective observational study in a hospitalist model for emergencyadmitted patient care in Taiwan and developed a prediction model for seven-day inhospital mortality using available information at admission, including patient demographics, admission diagnosis, underlying co-morbidities, and performance.

Source of data
We prospectively conducted a six-year hospital-based cohort study in a tertiary referral medical center in Taipei, Taiwan. Participants were limited to adult (≥ 20 years old) non- other wards before discharge were excluded from the analysis of in-hospital outcomes.
The enrolled patients were divided into two groups by a computerized random process. Two-thirds of the patients were classified as a derivation cohort, and the remaining onethird were utilized as a validation cohort.

Study settings and participants
The tertiary medical center where our study was performed contained around 2,450 beds, and 9,500 patients visit the ED per month. Of these 9,500, nearly 20% are admitted. The inpatient wards had 36 beds each from Oct. 2009 to May 2013 and then 71 beds for every two wards from June 2013 to Oct. 2016, which were tended by hospitalists.

Outcome and clinical characteristics
The outcome of interest was seven-day in-hospital mortality. The in-hospital duration was defined as the time interval from ED admission to death or discharge of the patient. The demographic and clinical data, including age, sex, season of ED admission, triage level at ED arrival, underlying co-morbidities, Barthel index at ward admission, and admission diagnosis were coded in a standard case report form with default options by trained assistants.

Statistical analysis methods
After dividing the cases into derivation and validation cohorts, we used locally weighted least squares regression (LOWESS) curves to first find the inflection points of continuous predictors in the derivation cohort. We then transformed the continuous predictors into categorical predictors by using the inflection point on the curve. If the inflection point could not be found, we used threshold values adopted in either one or more of the existing EWSs or from clinical experience. Next, we performed univariable Cox proportional hazard regression for each predictor with the outcome. If the p-value of one predictor was less than 0.10 in the univariable analysis, it was put into the multivariable stepwise Cox proportional hazard regression. The area under the receiver operating characteristic curve (AUROC) was used to examine discrimination in the derivation and validation cohorts.
To make the prediction model easy to use for medical personnel, we modified the beta coefficient of each statistically significant predictor in multivariable stepwise Cox proportional hazard regression into an integer by multiplying the same but smallest number and adopting the nearest integer. The integer point value system was then built up, and the best cut-off value was chosen using Youden's J Index. We then used this value to calculate the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) when the model was applied separately to the derivation and validation cohorts. Then we tried to find the cutoff value with > 85% sensitivity and the highest possible specificity, and that with > 85% specificity and the highest possible sensitivity. Data were analyzed in SAS software (Version 9.4, SAS Institute Inc., Cary, NC) and R software (Version 3.3.2). Statistical significance was defined as p < 0.05.

Enrolled participants
During the study period, a total of 9,093 patients were admitted to the general ward. After excluding 444 (4.9%) patients with incomplete data (167 missed Barthel index, 275 missed triage level, and two patients missed survival status), 8,924 patients were included in the final analysis. Within seven days of hospitalization, 275 patients were transferred to other wards and exclude from the analysis. A flowchart of the cases is shown in Fig. 1.
The characteristics of the derivation cohort, the validation cohort, and all patients are shown in Table 1. The mean age of the patients was 71.05 years, and 51.91% were male.
Around 10.71% of the patients had triage levels of 1 at ED, and 24.94% had active cancer.
Barthel index scores at admission were totally dependent (scores of < 35) (30.67%) and severely dependent (scores of 35-60 score) (24.88%). The most common admission diagnosis was pneumonia (36%), followed by urinary tract infection (20.05%) and upper gastrointestinal bleeding (9.26%). Table 1 Characteristics of study participants.  After multiplication by 2 and adoption of the nearest whole number as the point value, the final model was developed (Table 3). The sensitivities and specificities of each score are shown on Figure S1. In brief, using Youden's J index, we determined the optimal cutoff values for the model. The sensitivity, specificity, PPV, and NPV for the derivation cohort were 88.03%, 58.81%, 4.91%, and 99.51%, respectively, when a score of 4 was determined as the optimal cutoff value (Table 4). Similarly, we found that a cutoff value of 4 had > 85% sensitivity and the highest possible specificity. On the other hand, a score of 6 had > 85% specificity and the highest possible sensitivity (Table S1 in the supplement file). The sensitivity, specificity, PPV and NPV were 50.69%, 89.61%, 10.55%, and 98.69% when the cutoff value was 6. Then we divided the derivation cohort into three groups according to score ranges of 0 ≤ score < 4, 4 ≤ score < 6, and ≥ 6. The Kaplan − Meier survival curve showed significant differences between any two of the three groups (all p < 0.001) (Fig. 3). Table 3 Derivation of the prediction score for 7-day in-hospital mortality (Data Source: Derivation cohort). In the validation cohort, the area under the ROC curve was 0.78 (95% CI, 0.73-0.82) (Fig. 2B). Using the optimal cutoff value of 4 according to Youden's index, the sensitivity, specificity, PPV, and NPV were 87.06%, 57.24%, 4.35% and 99.50%, respectively (Table S2 in the supplement file), which were similar to those for the derivation cohort.

Discussion
This present study found that the 7-day in-hospital mortality was 2.80% among medical patients admitted from the ED. We established a mortality prediction score at the initial admission using the Barthel index score, presence of cancer and metastasis, admission diagnoses of pneumonia and sepsis, and triage score at arrival at the ED. Scores of ≥ 4 had NPVs of up to 99% and could be used for exclusion screening. By contrast, scores of ≥ 6 had specificity of 89% and PPV of 10.55%, which could cover most of the frail patients.
To manage the mortality outcomes of patients admitted to general medical wards is never easy for hospitalists, although some mortality cannot be avoided. In particular, for patients admitted for unplanned causes leading to ED visits, it is difficult to stratify those who have a high risk for potential death at initial admission. Although prediction scores of NEWS [7] or clinical alert system [10] have been developed, the indications of the two scores are mostly regarding the change of vital signs and ensuing critical status that is not strange to initiate intensive care. However, to predict the targeted risk group before they exhibit unstable vital signs is important, for it would allow us to prepare further discussions regarding intensive care and prognosis explanations. By using the prediction score developed and validated in the present study with a large-scale sample, we could stratify the patients easily into three subgroups: low risk for prediction scores of < 4, intermediate risk for scores of 4-6, and high risk for scores of ≥ 6. Those with scores of < 4 can be excluded from the alarm status, and those with scores of ≥ 6 probably need intensive treatment.
The prediction score showed a high AUROC of 0.819 and a hazard ratio of 1.659 (95% C.I.: 1.55-1.76 per 1 point increment) in the derivation cohort to predict in-hospital mortality within 7 days; the results were similar in the validation group. This score is the first mortality prediction score developed from general medical hospitalized patients and could be applied broadly. However, because the score element is relatively non-specific to diseases, the sensitivity and specificity are not > 90%. Therefore, the score can be used in clinical practice to detect fragile patients at initial admission, but final judgement must be reserved for inpatient physicians.
Among the prediction model, the Barthel index, which measures performance in activities of daily living, can be used to represent general condition and disease severity, and it is one of the important factors in prognosis prediction. For patients with chronic illness, the Barthel index result can be affected by patients' frailty [11,12] and disability, which correlate with mortality [13]. On the other hand, it could be the severity of the acute illness which is responsible for the admission. Although we did not discriminate the influence proportions of the Barthel index by acute or chronic illness, it affected the 7-day mortality, with the highest beta coefficient of 2.38 in the multivariable analysis. It might be more easily applied generally at initial admission.
The presence of underlying active cancer and metastatic status are both important predictors for in-hospital mortality due to their immune-compromised status [14]. The effect of cancer on a patient's outcome has been proven in critical care [15,16]. However, whether patients have cancer with or without metastasis may provide little insight on the poor prognosis, and as high as 25% of terminal cancer patients receive vasopressors in the dying process [17]. Therefore, hospitalists need to hold family meetings to explain and discuss treatment plans for shared decision making on cancer patients with high prediction scores [18].
In contrast to a chronic illness such as cancer, triage on arrival to ED could be used as an initial summary index for acute status. The triage index is the five-level Taiwan Triage and Acuity Scale (TTAS) computerized system implemented nationally since 2010 [19]. The triage index includes changes in vital signs, organ failure and acute problems needing immediate treatment [20]. Triage level 1 at the ED has been classified as an impact factor in the prediction score in the present study. In addition, admission diagnoses including sepsis and pneumonia are responsible for acute illness and associated with 7-day inhospital mortality. Sepsis is a high-mortality syndrome caused by severe infection with or without organ failure. In-hospital mortality could be as high 17% for patients with sepsis and 26% for those with severe sepsis [21]. Both factors (triage at the ED and admission diagnosis) represent acute changes for admitted patients. However, age was statistically significant only in the univariable analysis and not significant in the multivariable Cox analysis, possibly because its effect was erased by other co-morbidity and performance statuses.
This study had several limitations. First, we did not record initial vital signs, laboratory results in the original study design of clinical analysis. In addition, pre-hospital changes in the medical condition or Barthel index were not recorded, so their roles in in-hospital mortality prediction require further study. In addition, patients enrolled in this study may have been more severely ill, and a higher proportion may have had cancer, because the present study was conducted in a tertiary referral center. Third, this study was performed in Taiwan, so whether the results can be generalized to other ethnic groups and areas should be validated.

Conclusion
The prediction score includes six factors: low Barthel index score, triage level 1 at the ED, admission diagnosis of sepsis, diagnosis of pneumonia, and presence of cancer and metastasis. It was developed to screen out the high-risk subgroup for 7-day mortality among patients admitted from the ED. Using the prediction score is important and convenient. A score of < 4 could exclude the alarm status, whereas a score of ≥ 6 represents a high potential for short-term in-hospital mortality. Further validation is required before generalization. Figure 1 Flow chart of participant recruitment and outcomes  Receiver operating characteristic (ROC) curves to predict seven-day in-hospital mortality by probability developed from the multivariate model. Receiver operating characteristic (ROC) curves to predict seven-day in-hospital mortality by probability developed from the multivariate model. Kaplan-Meier (KM) curves for seven-day in-hospital mortality for different ranges of the prediction score.