Predictive Risk Factors at Admission and a "Burning Point" During Hospitalization Serve as Sequential Alerts for Critical Illness in COVID-19 Patients

Background In critically ill COVID-19 patients, the crucial turning point before critical illness onset (CIO) remain largely unknown, and the combination of baseline risk factors with the turning point during hospitalization was rarely reported. Methods In this retrospective cohort study, 1150 consecutively admitted patients with conrmed COVID-19 were enrolled, including 296 critical and 854 non-critical patients. We compared the differences of all the clinically tested indicators and their dynamic changes between critical and non-critical patients. Three prediction models were established and validated based on the risk factors at admission, and an online baseline predictive tool was developed. Linear mixed model (LMM) was applied for longitudinal data analysis in 296 critical patients throughout the hospitalization, to predict the likelihood and possible time of critical illness in COVID-19 patients. A crucial turning point, where several indicators will experience a greater and signicantly continuous change before CIO, was dened as “burning point” in our study. This point indicates the deterioration of patient’s condition before CIO. Results We established a novel two-checkpoint system to predict critical illness for COVID-19 patients in which the rst checkpoint happened at patient admission was assessed by a baseline prediction model to project the likelihood of critical illness based on the variables selected from random forest and LASSO regression analysis, including age, SOFA score, neutrophil-to-lymphocyte ratio (NLR), D-dimer, lactate dehydrogenase (LDH), International Normalized Ratio (INR), and pneumonia area derived from CT images, which yields an AUC of 0.960 (95% condence interval, 0.941-0.972) and 0.958 (0.936-0.980) in the training and testing sets, respectively. This model has been translated into a public web-based risk calculator. Furthermore, the second checkpoint (designated as “burning point” in our study) could be identied as early as 5 days preceding the CIO, and 12 (IQR, 7-17) days after illness onset. Seven most signicant and representative “burning point” indicators were SOFA score, NLR, C-reactive protein (CRP), glucose, D-dimer, LDH, and blood urea nitrogen (BUN). Conclusions With this two-checkpoint prediction system, the deterioration of COVID-19 patients could be early identied and more intensive treatments could be started in advance to reduce the incidence of critical illness.

Introduction SARS-CoV-2 is known to cause severe acute respiratory illness in humans. Currently, the pandemic triggered by SARS-CoV-2 is still quickly unfolding in many countries. According to real-time statistics released by WHO, as of August 26, 2020, more than 24 million cases of COVID-19 were con rmed and over 800,000 patients died. Five to twenty percent of hospitalized patients with COVID-19 were admitted to the intensive care unit (ICU), with mortality rate reportedly standing between 26% and 61.5% [1][2][3] . The condition of critically ill patients tends to deteriorate over a very short period of time, frequently leading to acute respiratory distress syndrome (ARDS) or multiple-organ failure, and even death 4,5 .
Ongoing pandemic necessitates the discovery of reliable prognostic predictors and dynamic changes of certain laboratory variables to help guide clinical decision making tailored to the patient characteristics. Identifying patients' characteristics and dynamic changes associated with critical illness in patients diagnosed with COVID-19 can provide therapeutic targets as well as improve the design and analysis of future clinical trials. Similarly, the deciphering of prognostic predictors and their dynamic changes that have an adverse effect on the disease progression may provide new insights into the disease pathogenesis.
So far, several studies 6,7 have reported prediction models for critically ill patients with COVID-19. However, these models were solely-based on baseline characteristics, and therefore did not involve longitudinal analysis and were unable to predict disease progression during hospitalization. A recent report 8 was able to predict the mortality of patients more than 10 days in advance using laboratory indicators during hospitalization. Nevertheless, only blood samples from 485 patients were used for modeling and this model didn't involve the dynamic changes of all the indicators. Here, we introduced a novel twocheckpoint prediction system based on both baseline characteristics at patient admission and longitudinal data collected during hospitalization. A crucial turning point -"burning point" was found before patients deteriorated to a critical condition (such as ICU admission), which was incorporated into this warning system. The two-checkpoint prediction system is a workable early warning system, including the rst warning at admission and the second alert as early as ve days before critical illness onset (CIO), to predict the occurrence and possible time of critical illness in COVID-19 patients.

Study design and participants
A total of 1224 Laboratory-con rmed COVID-19 adult patients (≥ 18 years old) were consecutively admitted to Wuhan West Union Hospital between January 12 and February 25, 2020. Among which 74 patients were excluded including 57 patients transferred to other hospitals and 17 patients who died within 24 h after admission. The remaining 1150 participants were included in our study and they all had a de nite clinical outcome (death or discharge) as of early-May, 2020 (study owchart in Fig. 1A).

Criteria and de nitions
The diagnosis and discharge criteria for COVID-19 were consistent with previous reports 7,9 . According to the interim criteria of WHO 10 and the guidelines by the National Health Commission (trial version 7.0), critical COVID-19 illness was evaluated retrospectively and con rmed based on respiratory infection, plus one of the following: 1) acute respiratory distress syndrome (ARDS) needing mechanical ventilation; 2) sepsis leading to life-threatening organ dysfunction; 3) septic shock. All of these critical patients either was admitted to ICU or received invasive mechanical ventilation or died, which met the de nition of critical COVID-19 by Liang et al 7 . Otherwise, the patients were seen as non-critical patients. The critical illness onset (CIO) was recorded as the beginning time of moderate/severe ARDS requiring mechanical ventilation, or the time point at which sepsis caused the life-threatening multiple organ dysfunction or the septic shock developed or patient was admitted to ICU. We introduced a new concept -"burning point" and de ned it as a critical turning point at which the condition exacerbated before CIO and some indicators started to change signi cantly and continuously. The period from the burning point to CIO was deemed as the high-risk period of CIO. The rst alert comes from the baseline warning system at admission and the second alert comes from the "burning point" warning system during hospitalization.
ARDS was diagnosed according to the Berlin de nition 11 . Sepsis and septic shock were de ned based on the 2016 Third International Consensus De nition 12 . Sequential Organ Failure Assessment (SOFA) score was calculated as previously reported 13 . De nitions of various organ injuries were described in the additional le 1: supplementary notes.

Data Collection
A total of 87 baseline variables, covering demographics, comorbidities, symptoms, laboratory ndings, imaging features, SOFA score, and admission time, were collected from electronic medical documents.
The baseline CT images were interpreted independently by two senior radiologists experienced in chest radiology. For all participants, the SOFA score and all laboratory data (47 items in total) were recorded from admission to discharge or death. At least two experienced doctors carefully went through the medical records of each critical patient to determine the time of CIO. All of these data were summed up in a standardized form.

Descriptive analysis
Categorical variables were presented as frequencies (n) and percentages (%). The continuous variables with normal or non-normal distribution were expressed as mean ± standard deviation (SD) or median (interquartile range [IQR]). To compare the differences of baseline variables between critical and noncritical participants, we used the independent sample t-test or Mann-Whitney U test for continuous variables, x 2 test, Fisher's exact test, or Mann-Whitney U test were employed for categorical (binary or ordinal) variables wherever appropriate.

Variable selection and model construction
To ensure the data integrity and avoid potential selection bias, variables or patients with missing rate of less than 40% were all included. As a result, 81 variables and 1118 patients remained. The random forest machine learning method was employed to impute the missing values 14 . Principal component analysis (PCA) was then conducted by using the R package "factoextra" 15 to evaluate the distribution of patients and the most relevant variables for critical illness. No cases were labeled as outliers and excluded in this process. Thereafter, a total of 1118 remaining patients were randomized into training and testing sets at Three prediction models (i.e. the machine-learning based random forest, the Least Absolute Shrinkage and Selection Operator (LASSO) logistic regression, and the multivariable logistic regression models) were built to predict, at admission, the likelihood of progression to critical illness in COVID-19 patients. Brie y, we chose the predictors selected by both the random forest and LASSO regression models as candidate risk factors to conduct multivariable logistic regression analysis, and then developed a nomogram scoring system. Finally, the three models were further compared and validated. (See details in  additional le 1: supplementary methods, Table S1-6, Figure S1-4). The nomogram scoring system was nally transformed into an online predictive tool: https://hust-covid19.shinyapps.io/Critical-illness-Predictive-Tool/ ( Figure S4).
Longitudinal data analysis SOFA score and 46 laboratory markers (47 indicators in total) were recorded successively in all the 1150 hospitalized COVID-19 patients. To nd out the indicators that changed signi cantly during the period of critical illness development, the liner mixed model (LMM) implemented in the R package "lme4" 16 was used to explore the association between time and indicators by taking the age, sex and comorbidities as xed effects.
All tests were two-sided, and a P value less than .05 was considered statistically signi cant. R software (version 3.6.2, R Foundation) was used for all analyses.

Features and Outcomes of non-critical and critical COVID-19 Patients
In our study, we collected data from the 1150 consecutively admitted patients. All the participants were studied until discharge or death (Fig. 1A). Among them, 296 of 1150 patients (25.7%) were identi ed to be critically ill. As shown in Table 1, the overall mortality was 17.5% (201/1150), while up to 67.9% in critically-ill patients. All non-critical patients were discharged, and their hospital stay time was signi cantly shorter than critical patients discharged (23.0 vs. 43.0, P < 0.0001). The median age of noncritical and critical patients were 59.0 (IQR, 48.0-68.0) and 68.0 (61.0-76.0) years respectively. And there were more male patients in critical group than in non-critical group (64.2% vs. 46.6%, P < 0.0001). Over half of the patients had fever (81.4%) and cough (68.3%) at admission. 778 (68.7%) patients had at least one comorbidity, including hypertension (33.6%), diabetes (20.4%), and coronary heart disease (10.9%) as the top three comorbidities. Sepsis (48.1%) was the most frequent complication, followed by acute liver injury (31.4%), ARDS (31.1%), acute cardiac injury (13.5%), and acute kidney injury (13.1%). The frequencies of complications were signi cantly higher in critical patients (all P < 0.0001). Both the SOFA score at admission (3.00 vs. 1.00, P < 0 .0001) and highest SOFA score during hospitalization (6.00 vs. 1.00, P < 0.0001) were signi cantly higher in critical patients. The baseline CT features and laboratory ndings among critical and non-critical patients were also summarized in Table 1. The time from illness onset to admission, "burning point", critical illness onset (CIO), death or discharge was listed in Fig. 1B.   Baseline predictor Selection in the Training Set The random forest and LASSO regression analysis were conducted in the training set respectively, with top 20 important variables remaining after random forest analysis and 19 variables selected by the latter (Table S3-S4, Figure S2 in additional le 1). The nine variables selected by both random forest and LASSO regression models were used in the subsequent multivariable logistic regression analysis, with two variables (neutrophils and CRP) excluded for their high correlation, respectively, with NLR and LDH and relatively lower AUC value ( Fig. 2A) (Fig. 2C). The non-parametric bootstrap test in the validation dataset showed that there was no statistically signi cant differences in AUCs among the three models (all P > 0.05) (additional le 1: Table S5). In addition, the calibration curve of this nomogram model indicated that the predictive probability for critical illness tted very well with the actual probability, in both the training and the testing set (Fig. 2D). In the testing set, the H-L test further con rmed the good performance of this model (P = 0.863) (additional le 1: Table S6, Figure S3). Importantly, we performed a sensitivity analysis for this nomogram model based on the variables without missing values, yielding an AUC of 0.948 (P = 0.43) and 0.929 (P = 0.26) respectively in the training and testing set (Fig. 2C, additional le 1: Table S5). As shown in Fig. 2E-F, the Decision curve analysis (DCA) and clinical impact curves proved that this nomogram worked well in supporting clinical decision-making, not much different from the other two prediction models.
Differences in dynamic changes of SOFA score and laboratory markers between critical and non-critical patients We compared the change patterns of SOFA score and 46 laboratory variables in 296 critical and 854 noncritical patients from illness onset to 26 days later by plotting line charts (Fig. 3, additional le 1: Figure  S5-S7). Most of the indicators were substantially higher in critical patients than in non-critical patients during the whole observation period, including a sustained high level of SOFA score, in ammatory Moreover, indicators, including PT, INR, and ALP, were constantly within the reference value range but also began to change persistently on the 5th day before CIO (all in Fig. 3, additional le 1: Figure S5-S7).
Based on the above facts, the "burning point" was identi ed to be at the 5th day before CIO, a critical turning point indicating that CIO was only ve days away, at which several indicators would experience further clear and continuous changes. This "burning point" appeared 12 (IQR, 7-17) days after illness onset (Fig. 1B). As shown in  (Fig. 3, Table 2). The dynamic changes of all theses 47 indicators after the critical illness onset (CIO) have been shown in additional le 1: Table S7.

Discussion
In this study, on the basis of the analysis of 1150 COVID-19 consecutive patients who were admitted to Wuhan West Union Hospital from January 12 to February 25, 2020, we established a reliable baseline prediction model and developed an online tool to predict, at admission, the risk for the development to critical illness, which can be used as the rst warning sign (the rst alert). Moreover, in critical patients, we retrospectively identi ed a "burning point", a warning sign that CIO was only ve days away and several indicators would experience signi cant and continuous changes. The "burning point" can serve as a second warning sign (the second alert) which can give clinicians precious time to take proactive measures before CIO. The two-checkpoint system can tell us "who" and "when" the critical illness will be developed in COVID-19 patients.
The predictors incorporated into the baseline prediction model were selected based on the random forest and LASSO regression analysis, which can provide a double guarantee for the selected predictors, ensuring the accuracy of the baseline model. Meanwhile, the model was translated into a nomogram system. Actually, the differentiating power of this nomogram scoring system was comparable to that of the aforementioned two models, yielding an AUC of 0. easily obtained since they are included in the essential examinations at admission. Several studies 5,[17][18][19] have demonstrated that advanced age was an independent risk factor for death in COVID-19 patients.
Higher SOFA score at admission was associated with increased odds of in-hospital death for COVID-19 patients 9 . Previous studies 9,20,21 showed that NLR, D-dimer, and LDH were risk factors for the fatal outcome related to COVID-19. INR was reportedly higher in deceased patients than in convalescent patients with COVID-19 22 . Pneumonia area was larger in patients who died from COVID-19 9 . Overall, the risk-factors based nomgram model is simple, effective and amenable to clinical application, especially when transformed into a web-risk calculator, which can serve as the rst alert for predicting critical illness in COVID-19 patients.
In addition, the longitudinal data analysis of critical and non-critical patients with COVID-19 demonstrated that almost all indicators showed conspicuous differences between those two groups and several laboratory markers started to rise or drop on the 8th (7th-9th) day after illness onset in critical patients, supporting the hypothesis that the acute phase starts from the 7th-10th day after illness onset of COVID-19, as proposed by a previous study 23 . Collectively, differences in the aforementioned laboratory markers between critical and non-critical populations suggested that critical patients experienced a long-term of coagulopathy, in ammatory activation, lymphocyte exhaustion, malnutrition, metabolic disorders, myocardial injury, liver dysfunction, and kidney injury. These ndings can help us gain insight into the pathogenesis of COVID-19 and distinguish between critical and non-critical patients.
What's more, we further looked into the dynamic changes in these 47 indicators before and after the CIO in 296 critical patients. The median time from illness onset to CIO was 17.0 (IQR, 12.0-22.0) days. We found that, prior to CIO, critical patients also suffered from severe coagulopathy (elevated D-dimer and declined PLT), in ammatory activation (elevated neutrophils), lymphocyte exhaustion, myocardial damage (ascendant LDH and BNP), impaired liver function (elevated TBIL, AST, GGT, and ALT), kidney injury (ascendant BUN and Cys-C), malnutrition (reduced TP, albumin and hemoglobin) and metabolic disorders (elevated glucose). Most importantly, we noticed that many laboratory markers started to have further and continuous changes on the 5th day before CIO. It indicates a turning point, at which the patient's condition began to deteriorate before the CIO, appeared. We designated this point as the "burning point", which occurred 12 (IQR, 7-17) days after illness onset. This "burning point" corresponded exactly to a point in the early acute phase of COVID-19 proposed by Lin et al 23 .
Furthermore, results of LMM revealed that 26 out of 47 indicators changed signi cantly and continuously within the ve days before CIO, covering almost all the aspects of abnormities mentioned above. For clinical application, we selected seven most signi cant and representative indicators as reference indicators and calculated their median values at the "burning point" (at the 5th day before CIO) and their average daily increments from "burning point" to CIO. These indicators were SOFA score, LDH, BUN (two organ-dysfunction indicators), CRP (in ammatory biomarkers), NLR (immune indicator), glucose (metabolism index), and D-dimer (coagulation indicator). In practice, we can judge whether a patient has passed the "burning point" on the basis of the time after illness onset, value of each indicator at the "burning point" and its daily change increment. The appearance of "burning point" indicates that CIO is only ve days away, which can serve as the second alert before critical illness developed in COVID-19 patients.
Until now, although the vaccine against COVID-19 is in full swing [24][25][26] , there are still no special and effective treatments 27,28 . Intensifying multidisciplinary treatments, such as enhanced nutritional support, anticoagulation (low molecular weight heparin [LMWH]), anti-in ammatory (γ-globulin, etc.), respiratory support (mechanical ventilation), and replacement therapy (continuous renal replace therapy [CRRT]) are adopted to save lives of critical COVID-19 patients 1,29,30 . But the implementation of the above-mentioned intensive treatments usually started after the occurrence of critical illness. A recent study 23 about COVID-19 proposed that early initiation of intravenous γ-globulin and LMWH anticoagulant therapy was effective in improving the prognosis of COVID-19 patients. Since the "burning point" in this study represented the starting point at which the patient's condition began to deteriorate before CIO, the highrisk period between the "burning point" and CIO might provide a precious time window for earlier intensive care and multidisciplinary interventions, thereby avoiding the aggravation to critical illness and improving survival.
Our study had several limitations. First, it was a single-center study. However, the participants in our study were consecutively enrolled from the beginning of the outbreak to the near end in the epicenter and very few patients were excluded (74/1224). All 1150 patients were observed until death or discharge and the data during hospitalization were collected continuously. Second, data generation was clinically driven and not prospective. Third, since all data were from China, the conclusion should be further validated in other countries.

Conclusions
In conclusion, the baseline risk factors based nomogram (the rst alert) can be employed at admission to identify the high-risk patients who might progress to critical illness. During hospitalization, the "burning point" (the second alert) could be identi ed in COVID-19 patients based on the time after illness onset, value of each indicator at "burning point", and their daily change increments. The appearance of "burning point" indicates that CIO was only ve days away. The two sequential alerts allow early identi cation of deterioration of patients' condition, which is critical in optimizing medical intervention and reducing the mortality rate of COVID-19 patients.