Development and clinical empirical validation of the Prognosis Prediction Model of Chronic Critical Illness

Background A vast number of patients with chronic critical illness (CCI) have died of delayed organ failure in the intensive care unit (ICU). The weak organ function of patients needed appropriate tool to evaluate, which could provide reference for clinical decisions and communication with family members. The objective of this study was to develope and validate a prediction model for accurate, timely, simple, and objective identication of the critical degree of the patients' condition. Methods This study used a retrospective case–control and a prospective cohort study, with no interventions. Patients identied as CCI from a comprehensive ICU of a large metropolitan public hospital were selected. A total of 344 (case 172; control 172) patients were included to develop the Prognosis Prediction Model of Chronic Critical Illness (PPCCI Model) in this case-control study; 88 (case, 44; control 44) patients were included for the validation cohort in a prospective cohort study. The discrimination of the model was assessed by the area under the curve (AUC) of the receiver operating characteristic (ROC). Results The model comprised 9 predictors: age, prolonged mechanical ventilation (PMV), sepsis/other serious infections, Glasgow Coma Scale (GCS), mean artery pressure (MAP), heart rate (HR), respiratory rate (RR), oxygenation index (OI), and active bleeding.In both cohorts, the PPCCI Model could better identify the dead CCI patients (development cohort: AUC, 0.934; 95% CI, 0.908–0.960; validation cohort: AUC, 0.965; 95% CI, 0.931–0.999), and showed better discrimination than the Acute Physiology And Chronic Health Evaluation II (APACHE II), Modied Early Warning Score (MEWS), and Sequential Organ Failure Assessment (SOFA). The PPCCI Model can provide a standardized measurement tool for ICU medical staff to ward-monitoring

of family members about expectations of CCI, but possibly encounter challenges in communicating the expected disease outcomes, long-term rehabilitation care plan, or treatment cessation [16,17]. An effective evaluation tool can identify high-risk CCI patients at earlier timepoints to facilitate advanced interventions to protect patient organ function [3]. Similarly, the tool could accurately identify low-risk and medium-risk patients to facilitate early caregiver intervention in patient rehabilitation [18]. There are several critical illness assessment tools, such as the Acute Physiology And Chronic Health Evaluation II (APACHE II) [19], Modi ed Early Warning Score (MEWS) [20,21], and Sequential Organ Failure Assessment (SOFA) [22,23]. APACHE II is unsuitable to evaluate patient condition and mortality risk in long-term hospitalization [24]; MEWS is more appropriate for evaluating patients in the general ward [20]; and both APACHE II and SOFA require blood tests, which prolongs evaluation.
We sought to develop and validate a new model that can not only obtain data quickly and easily without increasing patient discomfort and economic burden, but also evaluate risk with high sensitivity, speci city, and accuracy.

Design
This observational study comprised two stages: rst, we collected and analyzed data from a development cohort and developed the proposed model based on a retrospective case-control study.
Second, we tested the model in the clinical setting of a prospective cohort study. Before study initiation, we uni ed the research purpose, disease de nition, inclusion and exclusion criteria, variable names, units, and judgments. We extracted data on demographic variables and potential independent predictors from medical records of the study center during hospitalization. Data collection and collation were undertaken by three investigators. In case of abnormal data, such as, variable had different data in the same period, or data inconsistent with the patient's progress, even might affect the research results, we would timely check and solve to avoid information bias. Patients or their legal guardians provided written informed consent for study participation.

Setting and patient selection
All study participants were from a comprehensive ICU of a large metropolitan public hospital. The inclusion criteria were: hospitalization for 8 or more critical care days with at least one clinical condition, including prolonged mechanical ventilation (PMV); tracheostomy; sepsis, and other severe infections [25]; wounds; multiorgan failure (at least two failures: heart, liver, kidney, respiratory), or brain hemorrhage/traumatic brain injury (TBI) [26]. The exclusion criteria were: poor prognosis, such as endstage cancer, end-stage multiorgan failure, etc., at the time of ICU admission; age below 18 years; family decision to abandon active treatment; poisoning or other diseases with unclear diagnosis; adverse events leading to changes in disease or death; and patients with missing data or refusal of consent for study participation.

Development cohort
In development cohort, we included CCI patients treated in the ICU between January 2012 and December 2017. A case-control study was undertaken with deceased patients as the case (death, n = 172) group.
The control group was randomly sampled at a ratio of 1: 1 according to the sample size of the case group. The random sampling method for the control group was as follows: we generated a corresponding random number for each research object; then ranked the random number in ascending order; and selected the smallest 172 numbers. Patients without exacerbations, who were transferred out of the ICU, or discharged were included in the control (survival, n = 172) group. Patients with persistent abnormalities in variables due to repeated illness were included in the control group upon eventual improvement and transfer out of the ICU.
The timepoint of data collection for predictive variables in the case group was the rst time that the variable appeared abnormal, showed a critical value, or worsened after 8 days in the ICU. Predictive variables included GCS less than 15 points, MAP greater than 109 or less than 70 mm Hg, HR greater than 100 or less than 60 times/min, RR less than 12 or more than 24 cycles/min, and OI less than 400 mm Hg, etc. For consistency of the control group data, the time of data collection was speci ed as 06:00 on the 9th day in the ICU. However, not all data points could be collected, and we needed to supplement these data with records of the same variable that rst appeared after the data-collection timepoint.

Validation cohort
The validation cohort included CCI patients treated at the hospital between January 2018 and March 2019. After ICU admission for more than 8 days, we ascertained whether patients were eligible for study inclusion. The prospective case-control (case, n = 44; control, n = 44) study followed the same inclusion and exclusion criteria as in the development cohort, with 1:1 random sampling and the data collection timepoint of predictive variables was consistent between the development and validation cohorts for both case and control groups.
Statistical analysis Data were validated by two researchers after independent entry. Descriptive statistics are presented using mean and SD for continuous variables with normal distribution or median and interquartile range (IQR) for variables with abnormal distribution. Categorical variables are presented as frequency and proportions. In the development cohort, Variables were evaluated for their association with hospital mortality using the chi-square, independent t-, or Mann-Whitney U test, as appropriate (p < 0.1). Furthermore, variables with clinical relevance were considered for further binary logistic regression analysis, where we used the Forward: Conditional method to develop a model with death or survival as the dependent variable, with an entry probability of 0.05. Similarly, we evaluated whether the model lacked a degree of t according to Hosmer-Lemeshow goodness-of-t statistics (p > 0.05) and chi square value. We used APACHE II, MEWS, and SOFA to score the previous development and subsequent validation cohorts, and evaluated the prediction e ciency and stability of the model with AUC [27].
All statistical analyses were conducted in IBM® SPSS® Statistics (Version 26, Property of IBM Corp. © Copyright IBM Corporation and its licensors 1989, 2019).

Results
We screened

Model development
A total of 344 patients were included in the development cohort, 172 in the case group and 172 in the control group. Table 1 presents baseline data and predictors for these patients, and shows the results of univariate analysis of their correlation with mortality.  A total of 88 patients (case, 44; control, 44) were included in the validation cohort. Before validation, we veri ed whether the development and validation cohorts were conditionally consistent through a univariate analysis of patient characteristics and risk variables, and results showed no signi cant between-group differences in patient characteristics and risk variables (Table 3). The PPCCI Model score ranges from 0 to 20.8. We used the quartile to grade the risk of the model: low < 4.7; medium 4.7-6.6; high 6.7-9.2, and extremely high ≥ 9.3. In both cohorts, Kruskal-Wallis H test showed signi cant difference in the mortality among risk strati cations (p < 0.001). According to the proportion of death cases in each risk strati cation, we found that the mortality rate of patients increased with higher risk strati cation. The higher the score, the higher the mortality.

Discussion
We developed and validated a clinical prediction model, which can predict outcomes of patients with CCI by scoring. The PPCCI Model has a good performance in terms of accuracy and stability of prediction in the development cohort cases, and similar prediction performance in the validation cohort. This model uses data that can be obtained accurately, timely, easily, and objectively in the clinic, without increased patient discomfort and costs.
The prediction e ciency of PPCCI Model is superior to APACHE II, MEWS, and SOFA in both the development and validation cohorts. Moreover, the PPCCI Model has its advantages. APACHE II and SOFA [29] require invasive procedures such as arterial blood gas analysis, renal function, electrolytes, and blood routine tests for each evaluation. However, the PPCCI Model shortens the time to assess the prognosis of CCI patients without increasing patient discomfort and medical costs. Except for the arterial blood, the predictive variables of the PPCCI Model are noninvasive and fast, which can be used to quantify the disease status of patients with CCI at any time. This advantage of the PPCCI Model makes it feasible to obtain all data in a much shorter time than APACHE II and SOFA. Moreover, it reduces the discomfort, medical cost, and workload of medical staff in patients who need long-term treatment.
There are some details about variables selection. First, RTI de nes PMV time as mechanical ventilation time of more than 96 hours [7]. Second, The diagnosis of "sepsis/other series infections" is actually a complex and time-consuming process. We just intercepted the diagnostic results and did not shorten the diagnosis time of this indicator in the true sense. When using the PPCCI Model to evaluate the patient's condition, the patient has already been diagnosed with "sepsis/other series infections".
The ultimate goal of CCI patient care is to return to social normalcy and regain self-care capabilities. However, patients are usually faced with the long-term isolation and the threat of cross-infection with drug-resistant bacteria [30]. In addition to screening high-risk and very high-risk patients, the PPCCI Model could be used for identifying low-risk patients, who are likely to recover their ability of self-care with the help of rehabilitation team, community medical staffs, and even family members in more active rehabilitation environments [31]. By transferring patients based on scoring, we can reduce the economic burden on the patients' families and the national medical burden, and improve the use e ciency of ICU beds. Moreover, through the model, primary medical institutions can judge whether the patient's condition is deteriorating or they need to be referred to the higher level medical institutions.
The risk strati cation of the PPCCI Model would be really helpful for medical staffs to discuss with families about patients' conditions and the ethical issues of treatment cessation. For example, patients with extremely high-risk in this study had a mortality rate of 100%, and the failure of organ function was usually irreversible. However, A study showed that less than 40% of the patients who need ICU care for more than 2 weeks had discussed with doctors about the prognosis or their idea of advanced life support [32]; more than half of the family members said they could not understand patient diagnoses, prognosis, or treatment.

Limitations
This study had several limitations. First, due to the limited time and energy of researchers, this was a single-center study, and the nal conclusion of the model may be biased by the sample size. Second, retrospective data were used in development cohort; therefore, the accuracy is lower than that of prospective research. Some cases lacked data and were excluded from the study, which might have led to biased results. However, we collected more than 1 year of prospective clinical case date in the validation cohort to verify the impact of the retrospective study. Even in prospective studies, incomplete data of some patients is inevitable. Third, the knowledge and experience of the surveyor affected the nal measurement results. Over the course of the study, we strengthened the training of surveyor. Fourth, although we put forward the risk strati cation of CCI, we did not propose evidence-based interventions, which need to be explored in the future research. Finally, this tool could not completely replace the clinician's judgment [33].

Conclusions
In summary, the PPCCI Model can provide a standardized measurement tool for medical staff to evaluate the condition of CCI patients. So that, we can attempt to rationally allocate ward-monitoring resources or discuss palliative care with family members [34,35]. The model can provide basis for clinical medical staff to undertake corresponding treatment and nursing based on patient condition [36]. There is a need to explore whether the PPCCI Model is suitable for the evaluation of CCI patients in primary hospitals, communities, and even at home to provide objective basis for the evaluation and referral of patients.