Dynamic APACHE II Score to Predict Outcome Among Intensive Care Unit Patients

Purpose: The Acute Physiology and Chronic Health Evaluation II (APACHE II) score is used to determine disease severity and predict outcomes in critically ill patients. However, there is no dynamic APACHE II score for predicting outcomes among ICU patients.The aim of this study is to explore the optimal timing to predict the outcomes of ICU patients by dynamically evaluating APACHE II score. Methods: Study data of demographics and comorbidities from the rst 24 h after ICU admission were retrospectively extracted from MIMIC-III, a multiparameter intensive care database. The primary outcome was hospital mortality. 90-day mortality was a secondary outcome. APACHE II scores on days 1, 2, 3, 5, 7, 14 and 28 were compared using area under the receiver operating characteristic (AUROC) analysis. Hospital survival was visualised using Kaplan-Meier Curves. Results:A total of 6374 eligible subjects were extracted from the MIMIC-III. Mean APACHE II score on day 1 were 18.4±6.3, hospital and 90-day mortality was 19.1% and 25.8%, respectively.The optimal timing where predicted hospital mortality was on day 3 with an area under the cure of 0.666 (0.607-0.726)(P (cid:0) 0.0001). The best tradeoff for preciction was found at 17 score, more than 17 score predicted mortality of non-survivors with a sensitivity of 92.8% and PPV of 23.1%. Hosmer-lemeshow goodness of t test showed that APACHE II 3 has a good predictive calibration ability (X 2 =6.198, P=0.625) and consistency of predicted death and actual death was 79.4%. The calibration of APACHE II 1 was poor (X 2 =294.898, P<0.001). Conclusions: APACHE II on 3 dayis the optimal prognostic marker and 17 score provided the best dignostic accuracy to predict outcomes for ICU patients. These nding will help medical make clinical judgment.


Introduction
Predicting ICU mortality plays an important role in patient care and resource allocation, Early identi cation and management are associated with lower mortality [1,2]. At present, there are many predictive scoring systems including Acute Physiology Chronic Health Evaluation II (APACHE II), Organ Dysfunction and Infection System, Sequential Organ Failure Assessment (SOFA) and Simpli ed Acute Physiology Score (SAPS) for hospital outcomes [3][4][5], especially for mortality [6,7], Among which APACHE II score is the most common and well-known effective prediction scoring method. At present, most studies use the APACHE II score within 24 h of admission to predict the outcomes of patients [2,8], However, for the rst 24 h of ICU admission, many patients have several comorbid conditions, and selecting only one principal diagnostic category may be very di cult, Moreover, the condition of patients was unstable and their physiological indexes uctuated greatly, All these factors had a great impact on APACHE II score within 24 h, which leads to inaccurate prognosis of hospital outcomes. In addition, Evaluation criteria are usually based on the area under receiver operating characteristic (ROC) curve, which may not be enough to guide the decision on which model is the best. Good of t testing is another factor that should be considered. Therefore, it is very signi cance to explore the optimal timing for APACHE score by dynamically evaluating the APECHE II score and verify its calibration.

Date Extraction And Management
Two authors independently extracted the relevant date and assessed the eligibility and quality of the study. Date on the following aspects were extracted: age, sex, survival status, admission type, date of ICU and hospital admission and discharge, date of birth and death. The hospital and ICU length of stay (LOS), Patients discharged from hospital or deceased before day 28 were censored to the last known APACHE II score. APACHE II scores was calculated as described in previous studies [12,13], The days of hospitalization were calculated by subtracting the time of admission from the time of discharge. To safeguard patient privacy, date were deidenti ed [14], therefore, The days of death after discharge were calculated by subtracting the date of discharge from the date of death of each patient, and the 90-day mortality was further calculated. Only the rst diagnosis was selected as the object of date analysis, and the disease classi cation was gradually re ned according to the International Classi cation of Diseases-9th (ICD-9) edition comorbidities [15], Serial APACHE II scores were calculated on days 1,2,3,5,7,14 and 28 after enrollment in ICU. The primary endpoint was hospital mortality. The secondary endpoints was 90day mortality, For patients younger (> 18 year old) readmitted to the ICU, only the rst ICU admissions were included. Date were extracted from the following tables(ADMISSIONS,ICUSTAYS, PATIENTS, CHARTEVENTS, LABARARY EVENTS and DIAGNOSIS_ICD). We excluded patients who were Individuals were exclued if (1)stayed in ICU less than on day; (2) Patients with missing date of APECH II score.

Statistical analysis
Quantitative variables and categorical date are expressed as mean ± SD and (number and percentage),respectively. Compared between groups using a t-test, if the data were normally distribted and the variance was homogeneous. If not, data were represented by the median and interquaritile range, and the wilcoxon rank-sum test was used for comparisons between groups.Discrimination and calibration were assessed using the area under the receiver operating characteristics curve (AUROC) and Hosemer-Lemeshow statistic, respectively. We used the Youden Index to estimate the optimal cut off values by nding values that returned the maximum speci city and sensitivity. P 0.05 was considered statistically signi cant. Statistical analysis was performed by IBM SPSS Statistics (IBM, Armonk, NY, USA).

Baseline patient characteristics and outcomes
The ow charts is shown in Fig. 1. A total of 46520 patients were including in MIMIC-III database. After including only patients' rst ICU admissions and exclueding admissions under the age of 18, less than 24 h in ICU and lack of Apache II scords, Finally, 6374 patients were included in this study (Fig. 1). The baseline characteristics and outcomes of patients are summarized in Table 1. The overall mean age was 64.1 ± 16.8 years (63.2 ± 16.7 in survivors Vs 68.4 ± 15.9 in non-survivors), 57.8% was male (n = 3683), The difference of demographics between survivors and non-survivors were statistically signi cant (P 0.05). APACHE II on days 1, 2, 3, 5, 7, 14 and 28 were 18.4 ± 6.3,15.4 ± 5.8, 15.9 ± 5.9,16.8 ± 5.8,17.5 ± 6.3, 17.6 ± 6.6 and 15.7 ± 6.9, respectively. The non-survivor group showed higher APACHE II score compared to survivor group at all test points (P < 0.001).The hospital mortality and ICU mortality were 19.1% and 15.7%, respectively, ICU and hospital LOS (Length of stay) were 8.62 ± 9.0 and 15.7 ± 13.7 days. Besides, the diseases of circulatory system was listed as the most common diagnosis among 1865 patients (29.3%).   Comparing the area under the ROC curve with the histogram, it is found that the area under the ROC curve on the third day is the earliest time point than the previous days to predict hospital mortality(P = 0.024) (Fig. 2b). To this end, Our subsequent analysis focuses on the development of APACHE II 3 as an optimal predictor of hospital mortality.
The third day Youden index was used to analyze the survival and non-survival group showed that in the APACHE II ≥ 17 group, 928(23.1%) patients were in the non-survival group, the sensitivity was 92.8%, PPV was 23.1%, 3081 patients (82.2%) were in the survival group, the sensitivity was 82.2%. In the group of APACHE II less than 17, 288 patients were non survival, the speci city was 90.1%, 2077 patients were survival, the speci city was 80.2%, NPV was 87.8% (Table.2). In the Kaplan-Meier survival curves, APACHE II less than 17 had higher survival rate(P 0.0001) (Fig. 3).
The predictive value of dynamic APACHE II score for 90-day mortality The APACHE II 3 day ROC curve for predicting 90-day mortality was 0.743( 95% CI : 0.729-0.756, P 0.001), Youden index of 17 was used to analysis of 90 day survival status,indicated that in the group of APACHE II less than 17, 435 patients were non survival, the speci city was 100%, 1930 patients were survival, the speci city was 80.2%, NPV was 81.6%. (Table.2)

Score Calibration
The results of Hosmer-lemeshow goodness of t test of APACHE II 3 day showed that there was no statistical difference between the predicted value and the actual value(X 2 = 6.198, P = 0.625), indicating that the model has a good predictive calibration ability (Table.3). The consistency of predicted death and actual death was 79.4%. For APACHE II on 1 day, the calibration is poor (X 2 = 294.898, P < 0.001)( Table.3).

Discussion
This is the rst article on the relationship between dynamic APACHE II score and outcomes of ICU patients, and found that APACHE II score on the third day was the optimal predictor of outcomes in ICU patients.
Many studies have con rmed that APACHE II score is a useful prognostic marker of mortality. A retrospective study of 200 Iranian patients admitted in ICU reported that an APACHE II score of 15 provides the best diagnositic accuracy to predict mortality of critically ill patients[6].Liu J have shown that initial APACHE II scores on day of ICU admission and correlated with survival outcomes [16]. A study included 109 cirrhotic MICU patients reported that APACHE II can be a predictor of mortality [17]. This help in categorizing patients and to facilitate early risk identi cation. These studies used APACHE II scores within 24 h after admission, although early identi cation of mortality helps to classify patients and early identify risks. However, many factors affecting the prognosis of ICU patients within 24 h were ignored. This study founded that the APACHE II scores on the rst day have poorly calibration on hospital mortality. This results is consistent with that of Kim[8], They founded that in the analysis of 826 Koreans patients, the APACHE II from the rst 24 h after admission to the ICU exhibits poor calibration for hospital mortality. In large representative study of 141,106 ICU patients in the UK also con rmed that APACHE II showed good discrimination but imperfect calibration for hospital mortality [18]. Therefore, a new model is needed to accuratey predict mortality in ICU patients. In recent years, although many new models have been developed to predict the mortality of ICU patients [19], there new models still need to be veri ed by multi-center and large sample studies. APACHE II score is still a high acceptance score, therefore, it is more meaningful and popular to nd an effective score based on APACHE II score to predict outcomes of ICU patients.
Just as the accuracy of dynamic SOFA score in predicting sepsis is higher than that of SOFA score [7,20,21]. Some scholars suggest that the APACHE II score should be proposed during the initial 7 days of ICU stay to reduce the error to the greatest extent [22]. However, due to the complexity of APACHE II score, daily assessment will bring huge workload, so it is more valuable to nd out the optimal time to predict outcome of ICU patienst. In this study, Serial APACHE II scores were calculated on day 1, 2, 3, 5, 7, 14 and 28 after admission to ICU, and found that APACHE II scores on day 3 is a useful predictor of hospital and 90-day mortality in ICU patients.
Bahtouee' study have reported that APACHE II with a score of 15 gave the best diagnostic accuracy to predict ICU mortality[6]. In a study by Hosseini et al and Fadaizadeh et al reported the best cut-off point chosen was 13.5 in APACHE II score predicted hospital mortality, These result is lower than the 17 points we got in this study. This difference may be related to the following factors: First, Based on the date from the rst 24 h after admision to ICU [23,24].There maybe ignore in uenced factors as recent and ongoing resuscitation and therapy. Second, The sample size of the previous study is small, and this study uses a larger sample size, so we can draw a more reliable conclusion. Third, The 17 score of this study were obtained from the third day of APACHE II, while the previous study was the rst, so different conclusions were drawn.
There are still limitations in this study. Firstly, this study is a retrospective study, but all the necessary data have been colledcted, so it limits the possible bias of this method. Second, all the date are from the MIMIC-III (2001-2012), the results may not re ect current practices, Besides, most of pateint included in MIMIC-III are from North American ICUs who descended from different populations. Third, our study only included date availavle online and more external validation is still required.

Conclusion
We explored the correlation between APACHE II score and outcomes of ICU patients, and found that APACHE II score on the third day was the optimal time to predict outcomes in ICU patients. These nding will help medical make clinical judgment and sound clinical sense.