Development of an Early Prediction Model for Postoperative Delirium in Neurosurgical Patients Admitted to the ICU after Elective Craniotomy (E-PREPOD-NS): A Secondary Analysis of A Prospective Cohort Study

Background: Postoperative delirium (POD) is a signicant clinical problem in neurosurgical patients after intracranial surgery. Identication of high-risk patients may optimise individual perioperative management, but an adequate and simple risk model for use at super early phase after operation has not been developed. Methods: Adult patients were admitted to the ICU after elective intracranial surgery under general anaesthesia. The POD was diagnosed as Confusion Assessment Method for the ICU positive on postoperative day 1 to 3. Multivariate logistic regression analysis was used to develop the early prediction model (E-PREPOD-NS) and the nal model was validated with 200 bootstrap samples. Results: Among 800 patients included in the study, POD occurred in 157 cases (19.6%). We identied nine variables independently associated with POD in the nal E-PREPOD-NS model: age > 65 years [odds ratio (OR) = 3.336, 95% condence interval (CI) = 1.765-6.305, 1 risk score point], education level < 9 years (OR = 2.528, 95% CI = 1.446-4.419, 1 point), history of smoking (OR = 2.582, 95% CI = 1.611-4.140, 1 point), history of diabetes (OR = 2.541, 95% CI = 1.201-5.377, 1 point), supra-tentorial lesions (OR = 3.424, 95% CI = 2.021-5.802, 1 point), anesthesia duration > 360 min (OR = 1.686, 95% CI = 1.062-2.674, 0.5 point), GCS <9 at ICU admission (OR = 6.059, 95% CI = 3.789-9.690, 1.5 points), metabolic acidosis (OR = 13.903, 95% CI = 6.248-30.938, 2.5 points), and positioning of neurosurgical drainage tube (OR = 1.924, 95% CI = 1.132-3.269, 0.5 point). The area under the receiver operator curve (AUROC) of the risk score for prediction of POD was 0.865 (95% CI = 0.835-0.895). After internal validation by bootstrap, the AUROC was 0.851 (95% CI = 0.791-0.912). The model showed good calibration (Hosmer-Lemeshow P = 0.593). Conclusions: The E-PREPOD-NS model based on nine perioperative risk factors can predict POD in patients admitted to the ICU after elective intracranial surgery with fairly good accuracy. External validation is needed before use in clinical practice.


Background
Postoperative delirium (POD) is a common postoperative complication which occurs in 11-51% in patients after major surgery (Inouye et al. 2014). Investigations have shown that POD is associated with signi cantly increased morbidity, mortality, intensive care unit (ICU) and hospital length of stays, healthcare costs and decreased postoperative neurocognitive dysfunctions (pNCD) (Franco et al. 2001; Abelha et al. 2013; Gleason et al. 2015;Evered et al. 2018). In a multicenter observational study, Van den Boogaard et al. found that neurosurgical patient was an independent risk factor for the development of ICU delirium (van den Boogaard et al. 2012). Therefore, there has been increased attention to the postoperative management of neurosurgical patients to improve long-term neurocognitive, and reducing POD has been identi ed as an important target for surgical quality improvement (Berian et al. 2018).
Around 30-40% of delirium cases are thought to be preventable (Siddiqi et al. 2006). Therefore, identifying high-risk patients for POD and early intervention for high-risk subjects may reduce the occurrence of delirium. Several prediction models for POD have been developed in different types of surgical patients (Kim et al. 2016;Kim et al. 2020;Bohner et al. 2003), but a risk model for POD speci c to neurosurgical patients may provide unique insights in this vulnerable population. Recently, there are a few studies developed risk models to predict POD in neurosurgical patients, but each of these models has its own limitation, including small sample size, only involving single disease, or predictors unavailable at intensive care unit (ICU) admission (Wang et al. 2020b; Zhan et al. 2020; Harasawa et al. 2014;Flanigan et al. 2018). In this study, we developed, internally validated, and tested a risk score model for POD in neurosurgical patients using data from our previous prospective cohort study of adult patients after elective craniotomy (Wang et al. 2020a).

Study Design and Data Source
This was secondary analysis of our previous study (Wang et  We obtained written informed consent from all participants or their surrogates, who allowed for data abstraction from the electronic health record, and the study protocol was approved by the Institutional Review Board of Beijing Tiantan Hospital, Capital Medical University, Beijing, China (KY 2017-018-02).

Participants
Inclusion criteria for the present study were adult (at least 18 yrs old) patients who underwent elective craniotomy under general anaesthesia admitted to ICU. Exclusion criteria were non-Chinese speaker, transsphenoidal surgery, cerebrospinal uid shunt and drainage surgery, deep brain stimulation surgery, pre-operative coma (GCS ≤ 8), pre-operative mental retardation due to Parkinson's disease or dementia, history of psychosis, could not be assessed for delirium, or unlikely to survive 24 hours.

Outcome
The primary outcome was the present of POD within 3 days after surgery. Delirium was determined by four researchers who have been trained to perform the delirium assessment. Delirium was evaluated by the Chinese version of the Confusion Assessment Method for the ICU (CAM-ICU) (Wang et al. 2013). The delirium assessment was performed in two steps. Firstly, the arousal level was assessed by RASS (Sessler et al. 2002). If RASS score ≤-4, the patient was recorded as comatose, the remaining delirium assessment was aborted. When the RASS score was greater than or equal to -3, delirium was evaluated by the CAM-ICU. The CAM-ICU consists of four features: (1) acute onset of a mental status change or a uctuating level of consciousness, (2) inattention, (3) disorganized thinking and (4) an altered level of consciousness. Delirium was diagnosed if both the rst and second features were present, and either the third or fourth was present. During the study, delirium was assessed in each patient twice a day, at 08:00 to 10:00 and 20:00 to 22:00 at postoperative day 1 to 3. The patients were assigned to the POD group if any assessment was diagnosed with delirium, otherwise they were assigned to the non-POD group.

Predictors
Candidate predictors of the primary outcome were obtained from the dataset of our previous prospective cohort study including demographic information, medical comorbidities, preoperative-related data, anaesthesia and surgery-related data and data at the time of ICU admission. Preoperative risk factors included preoperative of hydrocephalus, history of antiepileptic, lesion location (supra-tentorial lesion), and present of metabolic acidosis. Hemoglobin level less than 120 g/L for women and less than 130 g/L for men was de ned as anemia. Serum sodium < 135 mmol/L was de ned as hyponatremia, serum albumin level < 3.5 g/dL was de ned as hypoalbuminemia and pH < 7.35 with bicarbonate < 24 mmol/L was de ned as metabolic acidosis (Wassenaar et al. 2015).

Statistical analysis
Continuous variables were presented as mean ± SD or median and interquartile range (IQR); categorical variables were expressed as frequency and percentage. Univariate analyses between the POD and the non-POD group were performed. The continuous variables used the Student t-test or Mann-Whitney Utest, while categorical variables were analyzed by the Pearson chi-square test or Fisher's exact test.
Factors with P values < 0.1 in the univariate analyses were entered into the multivariate analysis with a stepwise backward logistic regression to identify independent risk factors for POD and developed a risk prediction model. The results of the nal multivariate model are expressed as odds ratios (ORs) with their con dence intervals (CIs) and p values. The absence of multicollinearity was veri ed by a variance in ation factor below 10 and interaction terms were not considered.
With the variables selected for the model, the probability of POD was derived from a logistic model with relative weights (regression coe cients) from each variable: logit = B 0 + B 1 x 1 + B 2 x 2 +… and p(probability of postoperative delirium) = 1/(1 + e −(B ).
The accuracy of the model was veri ed and compared in terms of calibration and discrimination.
Calibration was evaluated using the Hosmer-Lemeshow goodness-of-t test, p values > 0.05 indicated no signi cant difference between the observed and predicted outcomes. Discrimination was evaluated using the area under the receiver operator curve (AUROC) curve, and a value close to 1 indicates a greater probability that the outcome was predicted correctly in randomly chosen pairs of patients ( For ease of application, we turned the risk model into a risk prediction score. The coe cient was rounded to the closest whole number while conserving proportionality (Omar 2010). The sum of the scores obtained based on the presence or absence of the respective variables in each patient determined the probability of POD. Furthermore, to evaluate the ability of the model to predict increasing rates of POD (discrimination of model), patients were strati ed by the prediction scores over the tertiles and divided into three ranges re ecting low, medium, and high risk for POD. Last, for analysis of diagnostic performance, information for sensitivity and speci city, positivity and negative predictive values, as well as positive and negative likelihood ratio, are presented.
Data analysis was performed using R v 3.6.3 (http:// www. R-project.org). For all analyses, two-side P value of less than 0.05 was considered statistically signi cant.

Participant Characteristics
All 800 patients who were admitted to the ICU after elective craniotomy were eligible for this cohort and were available for analysis. Of the analyzable cases, 157 (19.6%) were POD patients and 643 (80.4%) were non-POD patients. All cases were included in the development cohort to create the clinical score (E-PREPOD-NS) for predicting POD.

Development Cohort Analysis
In the development cohort, univariate analysis for comparison of clinical and laboratory characteristic between POD and non-POD at the time of ICU admission was shown in Table 1. Compared with non-POD group, POD was more likely to occur in patients who were female, older than 65 years, and education level less than 9 years, had history of alcohol abuse and smoking, had history of hypertension, diabetes and stroke, ASA physical status III -IV, supra-tentorial lesions, intraoperative use of steroid, intraoperative transfusion, and anesthesia duration > 6 hours, as well as had GCS < 9, metabolic acidosis, use of PCIA, and positioning of neurosurgical drainage tube at ICU admission (Table 1).   Table S1). The comparison table of total score and probability of POD is showed in Additional le: Table S2.

Score Performance And Internal Validation
The nal model showed good calibration (Fig. 2C) using the Hosmer-Lemeshow goodness-of-t test (χ2 = 6.910, degrees of freedom = 7, p = 0.546) and good discrimination (AUROC = 0.865, 95% CI 0.835-0.895; Fig. 2A) without collinearity (variance in ation factor = 1.362). The sensitivity of the model was 0.78, and the speci city was 0.82. Combined with its sensitivity and speci city, 0.59 was de ned as the cut-off point of the model. On performing bootstrapping with 200 repetitions for internal validation, no signi cant differences were found regarding the model, the variables, or their regression coe cients. The performance of the prediction model was further con rmed by bootstrap internal validation, in which the AUROC was 0.851 (95% CI 0.791-0.912) (Fig. 2B), and calibration plot showed pretty tting effect (Fig. 2D).
A cut-off value of 2 was identi ed to distinguish the high or low risk of POD. In patients with E-PREPOD-NS score ≤ 2, the POD rate was 6%. However, in those with E-PREPOD-NS score > 2, the POD rate was 46.1% (Additional le: Table S3). In addition, to evaluate the ability of the model to predict increasing rates of POD, categorisation using cutoffs of 3 and 6 produced three groups with clearly different incidences of POD in neurosurgical patients (Fig. 3).

Discussion
In the present study, we developed and internally validated a practical prediction model for POD in neurosurgical patients admitted to the ICU after elective craniotomy, consisting of 9 clinical factors: age, education level, history of smoking, history of diabetes, supra-tentorial lesions, anaesthesia duration, GCS at ICU admission, metabolic acidosis, and positioning of neurosurgical drainage tubes. For ease of application, we turned the risk prediction model into a risk prediction score with a total score of 10 points.
E-PREPOD-NS score is the rst simpli ed clinical score for early predicting POD in neurosurgical patients in ICU that can be used at the time of ICU admission and with good discriminative power for identifying patients at risk of POD.
Clinical studies have shown that POD is associated with high morbidity and mortality (Franco et (Siddiqi et al. 2006). When preventive measures are restricted to high-risk patients, the number of patients who will be unnecessarily exposed to potential harmful side effects are likely less. Therefore, developing and routine use of the risk prediction model for POD may facilitate early recognition and identi cation of those patients at high risk of POD who may bene t the most from POD preventive measures (Siddiqi 2016). More importantly, the use of delirium risk strati cation to target high-risk patients makes wider based on data available at ICU admission. More importantly, the score is easy to calculate and can be applied to almost all ICUs, even in resource-limited settings.
According to previous reviews of POD prediction (Aldecoa et al. 2017), our study identi ed advanced age, education level, history of smoking, history of diabetes, anesthesia duration, GCS < 9 at ICU admission and metabolic acidosis as leading risk factors. In our analysis, some variables were previously reported and had high signi cance in univariate analysis (history of alcohol abuse, history of hypertension, history of stroke, ASA physical status and intraoperative transfusion), but were not selected into the nal model. This means there may be some indirect associations between these variables and independent predictors. We still need more researches about these variables. Based on the accuracy of ROC risk strati cation, patients with total scores less than or equal to 2 was associated with low risk of POD, otherwise with high risk. ROC analysis further revealed total scores greater than 5.5 yielded a hundred percent speci city, which indicated patients with total scores greater than or equal to 6 were major concerns in the high risk group. It was noteworthy that there was have around 50% probability of POD occurred in patients with ranging 2.5 to 5.5 according to the corresponding probabilities calculation of the model. Therefore, the use of the E-PREPOD-NS model to identify and subsequently preventively treat moderate and high risk patients could offer a great contribution to perioperative intensive care practice and ensure e cient use of medical resources to provide to the only patients at risk.
There are several strengths of our study. Firstly, we included very comprehensive candidate risk factors in our study, including those risk factors reported in previous studies and those unreported neurosurgeryspeci c factors. All of them are easily to obtain from medical history and routine clinical examinations, and only 9 variables in the nal model. Therefore, our risk scores might be convenient for clinicians to apply and simple to calculate. Secondly, the patients who were enrolled in our cohort had been admitted after elective craniotomy with various types of disease that is similar to real-life situation. Delirium occurs in patients with traumatic brain injury (TBI) and stroke might be more closely to critical illness rather than surgical stress. Therefore, TBI and stroke patients were excluded in our cohort. Thirdly, multicollinearity was not found among the 9 predictors which were tested by variance in ation factors.
However, our study also has several limitations. The primary limitation is that this is a single-center study.
We only conducted internally veri cation and did not perform externally veri cation. Therefore, our model might be limited for widely generalizing in other races and regions. Secondly, recent studies emphasized the importance of biomarkers (Khan et al. 2020) and intra-operative electroencephalogram signatures (Fritz et al. 2020) to predict POD. However, for ease of application, we did not include biomarkers and intra-operative electroencephalogram signatures because they are not yet widely used clinically. Thirdly, the latest de nition of POD is that delirium occurs in hospital up to 1 week post-operation or until discharge (whichever occurs rst) (Evered et al. 2018). Nevertheless, only POD on postoperative 1 to 3 were available in our previous dataset. Finally, in order to facilitate statistics, we represent our variables as binary variables, and the severity of each variable is not taken into account.

Conclusions
In conclusion, we built a practical model with 9 clinical factors to predict POD for neurosurgical patients admitted to the ICU after elective craniotomy. It will be necessary to externally validate our results in future studies to investigate how this score model affects the prevention of POD in clinical practice.   Column plot demonstrating probabilities of delirium after neurosurgical interventions according to the predictive scores of the POD over the tertiles.