Development of machine learning models for predicting unfavorable functional outcomes in patients with chronic subdural hematomas

doi:10.21203/rs.3.rs-2242345/v1

Download PDF

Article

Development of machine learning models for predicting unfavorable functional outcomes in patients with chronic subdural hematomas

https://doi.org/10.21203/rs.3.rs-2242345/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Chronic subdural hematoma (CSDH) often causes neurological deterioration and is treated with hematoma evacuation. This study aimed to assess the feasibility of various machine learning models to preoperatively predict the functional outcome of patients with CSDH. Data were retrospectively collected from patients who underwent CSDH surgery at two institutions: one for internal validation and the other for external validation. The poor functional outcome was defined as a modified Rankin scale score of 3–6 upon hospital discharge. The unfavorable outcome was predicted using four machine learning algorithms on an internal held-out cohort (n = 188): logistic regression, support vector machine (SVM), random forest, and light gradient boosting machine. The prediction performance of these models was also validated in an external cohort (n = 99). The area under the curve of the receiver operating characteristic curve (ROC-AUC) of each machine learning-based model was found to be high in both validations (internal: 0.906–0.925, external: 0.833–0.860). In external validation, the SVM model demonstrated the highest ROC-AUC of 0.860 and accuracy of 0.919. This study revealed the potential of machine learning algorithms in predicting unfavorable outcomes at discharge among patients with CSDH undergoing burr hole surgery.

Health sciences/Medical research/Outcomes research

Biological sciences/Computational biology and bioinformatics/Predictive medicine

Health sciences/Neurology/Neurological disorders/Brain injuries

Chronic subdural hematoma (CSDH) is one of the most common neurosurgical conditions and is characterized by the collection of blood and fluid between the brain and the dura. CSDH is prevalent among elderly people, and its incidence is growing with the rapidly increasing aging population worldwide [1, 2]. The symptoms of CSDH include headache, vomiting, speech difficulty, paralysis, confusion, and even death. Operative hematoma evacuation is the mainstay treatment of symptomatic CSDH. Generally, this condition has been considered to have a good postoperative prognosis. However, as much as 30% of patients undergoing surgery for CSDH require assistance after hospital discharge [3].

Various factors are correlated with poor outcomes among patients with CSDH. These include age [4, 5, 6], obesity [5], preoperative low Glasgow coma scale (GCS) score [4, 6, 7], bilateral involvement [5, 8], and radiological factors, such as brain atrophy on head computed tomography (CT) [9, 10]. To date, a couple of studies have reported multivariate regression models used for predicting unfavorable CSDH outcomes [10, 11]. These prediction models were based on patient demographics, physical examination results upon admission, and radiological findings. Although a logistic regression-based scoring model by Kwon et al. reported high discrimination ability, they evaluated the model on the same dataset on which it was built [11]. This evaluation process would have been affected by data leakage. Another logistic regression-based model by Abouzari et al. did not have an accuracy high enough for clinical application [10]. Recently, several possible outcome predictors were additionally identified—activated partial thromboplastin time, prothrombin time, and international normalized ratio (INR) upon hospital admission [12]. These blood test results are yet to be incorporated into prediction models for functional outcome in patients with CSDH. Therefore, training prediction models with an increased number of possible predictors derived from clinical findings, including blood tests, have the potential to improve their prediction performance.

Recently, machine learning has been gaining attention in the medical field. This computer-based technique incorporates variables and learns from them. Because machine learning is excellent in managing a large amount of data, it is suitable for clinical settings that require or handle different complex variables [13]. To date, machine learning techniques have increasingly been used for predicting conditions, such as stroke and brain tumors, in the field of neurosurgery [14, 15]. The current study aimed to develop and validate various machine learning models with preoperative blood test results and clinical findings for predicting the functional outcome of patients with CSDH who underwent burr hole surgery. Moreover, the created predictive models were validated with external data to determine the accuracy of the models.

Patient selection

We retrospectively collected patient data from two institutions. The patients who underwent their first burr hole surgery between June 2017 and March 2021 at Handa City Hospital were included into the internal cohort. Patients data from Japanese Red Cross Aichi Medical Center Nagoya Daini Hospital was used for external validation dataset. All patients had a preoperative diagnosis of CSDH based on a plain head CT scan or brain magnetic resonance imaging. The exclusion criteria were patients with a previous history of craniotomy, CSDH surgery, shunt surgery, and those with subdural abscess.

Operative procedure and perioperative management

All patients underwent hematoma evacuation via single burr hole craniostomy under local anesthesia. There are three types of surgical procedures; irrigation, simple drainage, and combination. During an irrigation procedure, a flexible rubber tube was inserted into the hematoma area to drain the fluid and consequently irrigate the subdural space with normal saline solution. Simple drainage procedure requires placing a drainage tube in the subdural space. Whether to place a drainage tube after irrigation depended on the operators’ discretion. Postoperatively, patients with good postoperative neurological status were discharged from the hospital on postoperative days 3–7. Patients with speech or physical difficulties were enrolled in an in-hospital rehabilitation program. Those who could not return to home due to physical difficulties or comorbidities were transferred to a rehabilitation hospital or a long-term nursing facility.

Data collection

Clinical findings, including age, sex, medical history, neurological symptoms, preoperative CT scan findings, drainage tube placement, and surgical information, were collected using a retrospective chart review. Baseline blood samples were routinely collected from the patients upon hospital admission for performing blood tests. The type of collected laboratory data varied at the attending neurosurgeon’s discretion. The following characteristics of hematomas were obtained from the preoperative CT: volume, thickness, and internal architecture. Hematoma volume was calculated using the method described by Won S-Y[16]. Hematoma thickness was measured on axial images. When hematomas were seen bilaterally, their volumes were summed, as were their thickness. The internal architecture of hematomas was divided into two groups based on previous reports on preoperative hematoma characteristics and recurrence [17]. One group consisted of hyperdensity, separated, laminar, gradation, or trabecular types of hematoma density on CT scan. The other was hypodensity or isodensity types. Surgeons were classified as residents (with <5 years of training) and others (with >5 years of experience).

Functional outcomes of patients were assessed upon discharge using the modified Rankin scale (mRS) [18]. This scale was originally developed for patients with stroke. It is also used for the evaluation of difficulties in performing daily activities in patients with CSDH [19,20]. The mRS is an ordered scale that runs from 0 (no symptoms) to 5 (severe disability) and 6 (death). The scores between 0 and 5 represent no significant disability (1), slight disability (2), moderate disability (3), and moderately severe disability (4). This study defined the favorable functional outcome as an mRS score of 0–2 based on previous studies [3,21]. In contrast, an unfavorable functional outcome was defined as an mRS score of 3–6, indicating the inability to manage daily activities without assistance.

Statistical analyses

Statistical analyses were performed to examine the association between each patient parameter and postoperative functional outcome. Univariate analysis of the dataset was conducted using R version 4.1.2 (R Foundation for Statistical Computing, Vienna, Austria) and RStudio version 2021.09.0+35 (Boston, Massachusetts: RStudio, Inc. 2021). The Mann–Whitney U test was used to identify perioperative variables associated with an unfavorable outcome. Fisher’s exact test was used to assess the categorical variables. P values of <0.05 were considered significant. To prevent data contamination, the statistical analysis results were not used in constructing the machine learning prediction models described below.

Dataset creation

Data cleaning was conducted using the dataset with R and RStudio. Demographic characteristics and laboratory data were extracted from the patients in the internal cohort as input variables for the internal dataset. Variables with missing values above 8% of all subjects were excluded as input features. Information about operative procedure and postoperative Goreisan usage were excluded from predictors because it was unavailable before surgery. Categorical variables were converted to binary features with one-hot encoding. For the external validation dataset, the same variables were collected from the patients in the external cohort.

Machine learning and prediction model development

Data splitting and normalization

The internal dataset was randomly split into the training and test datasets with a split ratio of 3:1 [22]. The process was performed in a stratified fashion on Python version 3.9.2. Subsequently, the variables were normalized with StandardScaler (default setting) in the Sklearn library.

Supervised machine learning approach

The machine learning framework was written on Python version 3.9.2 using standard machine learning libraries. The framework comprised three steps (feature selection, data balancing, and training of classifiers) using the training dataset.

Variables were selected using two methods: the filter and wrapper methods. As one of the standard filter methods, univariate feature selection with SelectPercentile was conducted using sklearn.feature_selection package. SelectPercentile is a method used to rank each variable and to extract variables according to the percentile of the highest scores. Percentiles of 5, 10, 15, 20, 50, and 100 were applied. Conversely, recursive feature elimination, one of the popular wrapper-type feature selection methods, was used. It fits a model (Random forest) and eliminates the least important variables until the specified number of features is reached [23]. The recursive feature elimination process was cross-validated using the training dataset to identify the optimal number of features utilized to build a machine learning model.

The outcome variable of the dataset was imbalanced (favorable, 67%: unfavorable, 33%); therefore, two data balancing techniques were applied. One is undersampling of most data under the condensed nearest neighbor rule. The other is the upsampling of the minority of the data. A commonly used approach—the Synthetic Minority Over-sampling Technique (SMOTE)—was applied [24].

Four machine learning classifiers (logistic regression, support vector machine (SVM), random forest, and light gradient boosting machine (light GBM)) were trained to predict unfavorable outcomes after CSDH surgery. Their hyperparameters were optimized to give the largest area under the curve of the receiver operating characteristic curve (ROC-AUC) via grid search with 5-fold cross-validation. These algorithms were selected to compare various machine learning methods based on different learning styles (logistic regression: regression, SVM: instance-based, random forest: bagged decision trees, light GBM: boosted decision trees).

Logistic regression is a traditional statistical method used for dichotomous classification, and it is a technique used to apply regression algorithms to classification tasks using an S curve to provide input values between 0 and 1. Logistic regression is now considered as one of the basic machine learning algorithms.

SVM is another machine learning method used for binary classification. Its name is derived from support vectors, which are the points closest to the line or hyperplane dividing the dataset. The distance between the support vectors and line/hyperplane is called the margin. SVM creates a boundary that maximizes the margin, thereby separating the two classes.

Random forest, a combination of decision trees, is a robust machine learning method invented in 2001 [25]. A large number of decision trees are created via random sampling using datasets. Subsequently, the majority vote of the predicted results for each tree is taken to determine the final predictive value. Random forest has a lower risk of overfitting than a single decision tree because it uses multiple trees to prevent the influence of variance or bias.

Light GBM is a relatively novel type of gradient boosting decision tree algorithm developed in 2017 [26]. A gradient-boosting decision tree builds decision trees (weak learners) one-by-one to minimize the error of the previous model. Light GBM is a modified version of the original gradient-boosting decision tree model by reducing computation time while maintaining prediction accuracy.

Evaluation of the machine learning models

After hyperparameters were fixed, the machine learning models were tested using the test data of the internal dataset. This evaluation was conducted independently of the training process. ROC curve analysis was conducted to evaluate the discrimination ability of the models. The accuracy, sensitivity, specificity, and f1 score of each model were compared at the optimal cutoff point as set by Youden’s index [27]. In terms of the interpretation of algorithm predictions, values of standardized beta coefficients of the logistic regression were calculated. The coefficients describe the size and direction of the association between the predictor and outcome. Moreover, the feature importance of the random forest and light GBMs were evaluated. Feature importance is expressed as a score assigned to features based on their contribution to the prediction model. The score was computed based on the total impurity reduction of splits (Gini importance). The best-performing model was validated using the external validation dataset for each type of the four machine learning algorithms.

Ethical statement

This retrospective study is approved by the Ethics Review Committee of Nagoya University Graduate School of Medicine (2021-0442). Since this study was noninvasive, the Ethics Review Committee of Nagoya University Graduate School of Medicine approved that the requirement for written informed consent from patients was waived, but the opt-out method was adopted in accordance with the Japanese ethics guidelines. A public notice regarding this study was given on the websites of Handa City Hospital and Japanese Red Cross Aichi Medical Center Nagoya Daini Hospital. This research was completed in accordance with the Declaration of Helsinki as revised in 2013.

Overview of the development of the internal dataset

In total, 241 patients with CSDH who experienced their first burr hole surgery were eligible for the internal cohort of this study. Exclusion criteria were met in eight patients: prior shunt for idiopathic normal pressure hydrocephalus (n = 2), prior craniotomy (n = 5), and subdural abscess (n = 1). Data from the remaining 233 patients were analyzed to determine input variables for the creation of the prediction model. A total of 52 variables comprising 19 categorical and 33 continuous features were extracted as predictors (Tables 1, 2). Two of the 233 patients were excluded for the lack of preoperative CT scans. Moreover, ten patients were excluded because of unavailable accurate patient background, and 33 were excluded for missing variables of the laboratory results. The remaining 188 patients were finally included in this study to build the internal dataset. The training dataset comprised 141 patients, whereas the test dataset comprised 47 patients. The flowchart of this procedure is depicted in Fig. 1 (left).

Baseline characteristics of the included patients

The baseline characteristics of patients with the evaluation via univariate analyses are shown in Table 1. The mean age of the patients upon hospital admission was 79 years. As previously reported [4,5,7], the unfavorable outcome group was significantly older than the favorable outcome group (p < 0.001; mean age is 85 or 77 years, respectively). Moreover, patients in the unfavorable outcome group were more likely to experience low GCS scores (≤13) upon admission (53%) than those in the favorable outcome group (29%; p < 0.001). The ratio of mRS 0–2 before the onset was 91% in the favorable outcome group and 48% in the unfavorable outcome group (p < 0.001). Other significantly different characteristics between the two groups were BMI (p = 0.003) and speech difficulty upon admission (p = 0.005).

Preoperative laboratory data of the study patients

Preoperative laboratory data on admission with the evaluation via univariate analyses are presented in Table 2. The unfavorable outcome group had a significantly lower serum albumin level of 3.65 g/dL, compared to the favorable outcome group (4.00 g/dL; p < 0.001). Similarly, the albumin/globulin ratio was lower in the unfavorable outcome group (unfavorable: 1.17, favorable: 1.37; p < 0.001). Moreover, hemoglobin level was significantly lower in the unfavorable outcome group (unfavorable: 11.80 g/dL, favorable: 13.00 g/dL; p < 0.001). A hematocrit test showed that the unfavorable outcome group had a lower percentage of red blood cells (34.9%) than the other group (38.0%; p < 0.001). Other components of laboratory data significantly lower in the unfavorable outcome group were red blood cell counts (p < 0.001), mean corpuscular hemoglobin concentration (MCHC; p < 0.001), mean corpuscular hemoglobin (MCH; p = 0.015), the percentage that basophils account for white blood cell count (p < 0.001), alanine aminotransferase (p = 0.027), total bilirubin (p = 0.030), and total protein levels (p = 0.035). Meanwhile, the unfavorable outcome group had a significantly higher INR than the favorable outcome group (p < 0.001).

Optimal features selected for each machine learning model

The optimal number of features required for developing each machine learning model was determined using the SelectPercentile and recursive feature elimination methods (Supplementary Table S1). The optimal model for logistic regression analysis was developed using the features in the top 20th percentile according to the SelectPercentile method. Overall, 11 features (four from clinical findings and seven from laboratory data) were identified. Notably, the optimal light GBM model was developed using the identical 11 features. In contrast, the best SVM model was developed using the features in the top 10th percentile, which included six features (two from clinical findings and four from laboratory data). The features in the top 15th percentile yielded the best model in random forest.

Comparison of the performances of the models

The accuracy, sensitivity, specificity, F1 score, and ROC-AUC of the four prediction models that were trained on the optimally selected features are shown in Table 3. The performance metrics of the four machine learning algorithms trained on all features without selection are presented in Supplementary Table S2. The prediction accuracy scores of all four models with optimal feature selection were higher than the scores of the models trained for all features. The highest accuracy score of 0.894 was observed in the logistic regression and SVM models (Table 3). The accuracies in the random forest and light GBM models were 0.830 and 0.851, respectively. Among the prediction models, the logistic regression model demonstrated the highest specificity of 0.875. In contrast, the other three models demonstrated the highest sensitivity of 1.000. Among the evaluated machine learning models, F1 score was the highest in the SVM model. ROC curves of the four machine learning algorithms are illustrated in Fig. 2. The largest ROC-AUC of 0.925 was demonstrated in the logistic regression model. ROC-AUC values of the SVM, random forest, and light GBM models were 0.919, 0.906, and 0.906, respectively.

Evaluation of input variable importance for machine learning model development

The value of the standardized beta coefficient of the logistic regression model is illustrated in Fig. 3A. The highest value of the beta coefficient was 0.894 in the feature “mRS before onset,” followed by 0.870 in “ Age”. In contrast, all the selected features of the laboratory data had negative coefficients. MCHC (−0.572) and albumin/globulin ratio (−0.255) were the top two among the seven variables. The feature importances of the random forest and light GBM models are depicted in Fig. 3B and 3C, respectively. Both machine learning models ranked albumin/globulin ratio, mRS before onset, age, and MCHC among the top four essential features.

External validation of the machine learning models

A total of 121 consecutive patients with CSDH who had not experienced prior burr hole surgery were eligible for the external validation cohort. Six patients were excluded due to prior craniotomy or shunt surgery. Sixteen patients were excluded due to missing laboratory data, and the remaining 99 patients were included in the analysis (Fig. 1, right). A favorable outcome was observed in 84 patients (85%), whereas an unfavorable outcome was noted in 15% of patients. All input variables from demographic and laboratory data are illustrated with univariate analysis in Supplementary Table S3. Of the baseline characteristics, low GCS scores upon admission (favorable, 13%, vs. unfavorable, 40%; p = 0.021), history of ischemic stroke (11% vs. 33%; p = 0.036), and ratio of mRS 0–2 before the onset (96% vs. 40%; p < 0.001) were statistically different between the favorable and unfavorable outcome groups. Unlike the internal cohort, BMI and the presence of speech difficulty during onset were not significantly different (p = 0.6 and p = 0.3, respectively). Preoperative laboratory data on admission shows that the serum albumin level (favorable, 4.02, vs. unfavorable, 3.45 g/dL; p = 0.002) and albumin/globulin ratio (1.32 vs. 0.89; p < 0.001) were significantly different between the two groups. Serum glucose level was significantly higher in the unfavorable outcome group (favorable, 114 mg/dL; unfavorable, 141 mg/dL; p = 0.044), which was inconsistent with the internal cohort.

The performance metrics of each machine learning model on the external validation dataset are shown in Table 4. Among the prediction models, the SVM model demonstrated the highest accuracy of 0.919, followed by the logistic regression (0.889), light GBM (0.788), and random forest (0.768) models. The highest ROC-AUC value of 0.860 was observed in the SVM model. The ROC-AUC values of the logistic regression, random forest and light GBM models were 0.856, 0.835, and 0.833, respectively. The ROC curves of the proposed machine learning models are summarized in Fig. 4.

The current study aimed to assess the prediction performance of various machine learning models for functional outcome after burr hole surgery for CSDH. This study compared multiple machine learning algorithms, including logistic regression, SVM, random forest, and light GBM. The ROC-AUCs of all machine learning approaches in the internal cohort dataset were high (0.906-0.925). Adding blood test results into candidate predictors and selecting essential ones from them contributed to the increase in the prediction performance of the machine learning models. The cross-validation method was used to identify the optimal number of input features and the best hyperparameters of each machine learning model. To strengthen its reproductivity and reliability, the prediction models were validated in an external cohort. The ROC-AUCs of these models showed satisfactory performance, ranging from 0.833 to 0.860 on the external cohort.

This study showed that postoperative functional outcome at hospital discharge can be efficiently predicted in patients with CSDH. Most patients with CSDH are generally treated with surgical intervention. Even after successful evacuation of hematomas, some patients require prolonged rehabilitation, are discharged to nursing-care facilities, and even die. The frequency of CSDH patients with these unfavorable outcomes is increasing [3]. Thus, an early indication of postoperative functional status is helpful for health workers in deciding treatment, rehabilitation programs, and the most appropriate discharge destinations for patients with CSDH. In this regard, the use of machine learning models with high accuracy for predicting prognosis has clinical significance.

In accordance with significant improvements in the calculation speed of computers, various machine learning algorithms have been developed thus far. These techniques have been applied in the medical field. For the CSDH outcome prediction, two machine learning approaches (artificial neural network (ANN) and classification and regression tree (CART)) have been reported [10,28]. ANN is a network of connected neurons, and its architecture is based on at least three layers: input, hidden, and output. Abouzari et al. showed that ANN had a better score than logistic regression in predicting discharge outcomes in patients with CSDH [10]. However, in our study, ANN was not applied because it generally requires a large number of cases to produce good outcomes [29]. Moreover, ANN decimates the interpretability of the input variables because of its black-box nature, making it difficult to use the model in clinical settings. CART is a machine learning method used to build a decision tree based on input variables. Rovlias et al. used CART to predict outcome 2 months after CSDH surgery [28]. Although the model had a relatively good accuracy, postoperative and preoperative patient information was applied to build the model. Since early identification of patients unable to be discharged home after surgery is the purpose of this study, information only available before surgery was treated as a predictor in our study. We used random forest instead of CART as a decision tree-based method. The random forest algorithm is designed to produce better accuracy with less overfitting than a single tree by creating multiple decision trees.

Inclusion of essential predictors by data collection and eliminating irrelevant ones are critical in developing a robust machine learning model. This process reduces overfitting and improves the generalization of models. Preoperative patient information, including laboratory data, was broadly collected to ensure that essential predictors were not missed. Furthermore, less important variables were eliminated to improve the model performance. The feature elimination technique effectively improved the predictive performance in all four machine learning models. Notably, feature selection improved the accuracy score in the logistic regression and SVM models by 7.7%, contributing to the best-scored accuracy of 0.894 among the proposed machine learning models.

Accurate interpretation of machine learning models is vital to make use of them in clinical settings. Logistic regression is one of the algorithms with high interpretability because the coefficient of each feature shows its relation to the output variable. Our results indicated that poor mRS score before onset, old age, speech difficulty, and GCS score upon admission were associated with unfavorable functional outcome. These findings were consistent with the previous studies [4,5] and our clinical experience. Several predictors derived from laboratory results were also found to be relevant factors for an unfavorable outcome, with serum MCHC levels representing the highest absolute value of the standardized beta coefficient. Moreover, the feature importance in random forest and light GBM models showed that preoperative blood test parameters, such as albumin/globulin ratio and serum MCHC levels, were considered important in establishing both models. In summary, our study revealed that patient background, clinical findings, and blood test results were useful predictors of postoperative functional prognosis in patients with CSDH. Although machine learning models can occasionally reveal unexpected factors that correlate with the outcome, they do not tell the causal association between these variables. Further investigation on their causality is warranted.

Previous studies have suggested that the volume and type of CSDH and midline shift are associated with postoperative recurrence [17,30]; however, they were not significantly related to unfavorable outcomes at discharge in this study. Anticoagulant and antiplatelet use have also been associated with postoperative recurrence [31], though they were not selected as essential features in the presented prediction models. Thus, the present study suggests that unfavorable outcomes at discharge and postoperative recurrence for CSDH are distinct predictors. These results indicate that the crucial characteristics vary depending on the value desired to be predicted. Therefore, as we have done in this study, providing a large number of candidate features and selecting the important features are essential in improving the predictive performance of machine learning models.

The current study had several limitations. First, it was a retrospective study, which inevitably inherited relatively significant variance and biases; therefore, cross-validation was conducted during feature selection and hyperparameter tuning to improve the robustness of machine learning models. In addition, to eliminate the possibility of data contamination, validation processes using the test dataset and external cohort were conducted independently from the training process. Second, there might be other types of machine learning models that could perform better than our models. Algorithms are improving rapidly as the world of machine learning is expanding. This study uses one of the state-of-the-art machine learning algorithms such as light GBM. In future studies, the application of newly invented techniques can help improve model performances.

This study investigated the feasibility of four different machine learning algorithms in predicting postoperative functional outcomes in patients with CSDH. The proposed machine learning models that were developed using preoperative data accurately predicted an unfavorable outcome at discharge among patients with CSDH undergoing burr hole surgery.

Acknowledgments

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. The authors would like to thank Enago (www.enago.jp) for the English language review.

Author contributions

Y.F.: Conceptualization (lead), formal analysis (lead), investigation (lead), writing-original draft (lead), and writing-review and editing (equal). Y.N.: Conceptualization (equal) and writing-review and editing (lead). H.N.: Formal analysis (equal). F.O.: Conceptualization (equal) and writing-review and editing (equal). Y.M.: Resources (lead). Y.A.: Writing-review and editing (supporting). Y.N.: Writing-review and editing (supporting). J.I.: Resources (lead). T.N.: Resources (equal). Y.S.: Resources (equal). K.W.: Resources (equal). K.O.: Supervision (supporting) and formal analysis (supporting). R.S.: Supervision (lead). All authors reviewed the manuscript.

Data availability statement

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request. The code of the prediction models in this study is available at https://github.com/yutarofuse/CSDH_mRS_prediction.

Competing interests

The authors declare no competing interests.

Kolias, A. G., Chari, A., Santarius, T. & Hutchinson, P. J. Chronic subdural haematoma: modern management and emerging therapies. Nat. Rev. Neurol. 10, 570–578 (2014).
Beard, J. R. et al. The World report on ageing and health: a policy framework for healthy ageing. Lancet 387, 2145–2154 (2016).
Toi, H. et al. Present epidemiology of chronic subdural hematoma in Japan: Analysis of 63,358 cases recorded in a national administrative database. J. Neurosurg. 128, 222–228 (2018).
Ramachandran, R. & Hegde, T. Chronic subdural hematomas-causes of morbidity and mortality. Surg. Neurol. 67, 367–372 (2007).
Han, M. H. et al. Predictive factors for recurrence and clinical outcomes in patients with chronic subdural hematoma. J. Neurosurg. 127, 1117–1125 (2017).
Pilitsis, J. et al. Outcomes in octogenarians with subdural hematomas. Clin. Neurol. Neurosurg. 115, 1429–1432 (2013).
Amirjamshidi, A., Abouzari, M. & Rashidi, A. Glasgow Coma Scale on admission is correlated with postoperative Glasgow Outcome Scale in chronic subdural hematoma. J. Clin. Neurosci. 14, 1240–1241 (2007).
Agawa, Y. et al. Bilateral chronic subdural hematoma is associated with rapid progression and poor clinical outcome. Neurol. Med. Chir. (Tokyo). 56, 198–203 (2016).
Markwalder, T. M. Chronic subdural hematomas: a review. J. Neurosurg. 54, 637–645 (1981).
Abouzari, M., Rashidi, A., Zandi-Toghani, M., Behzadi, M. & Asadollahi, M. Chronic subdural hematoma outcome prediction using logistic regression and an artificial neural network. Neurosurg. Rev. 32, 479–484 (2009).
Kwon, C. S., Al-Awar, O., Richards, O., Izu, A. & Lengvenis, G. Predicting Prognosis of Patients with Chronic Subdural Hematoma: A New Scoring System. World Neurosurg. 109, e707–e714 (2018).
Idowu, O. E., Oyeleke, S. O. & Vitowanu, J. M. Impact of inflammatory cell ratio, biomarkers, activated partial thromboplastin time and prothrombin time on chronic subdural haematoma severity and outcome. Eur. J. Trauma Emerg. Surg. 48, 1085–1092 (2022).
Obermeyer, Z. & Emanuel, E. J. Predicting the Future — Big Data, Machine Learning, and Clinical Medicine. N. Engl. J. Med. 375, 1216–1219 (2016).
Heo, J. N. et al. Machine Learning-Based Model for Prediction of Outcomes in Acute Stroke. Stroke 50, 1263–1265 (2019).
Senders, J. T. et al. An Online Calculator for the Prediction of Survival in Glioblastoma Patients Using Classical Statistics and Machine Learning. Neurosurgery 86, E184–E192 (2020).
Won, S. Y. et al. Excellent accuracy of ABC/2 volume formula compared to computer-assisted volumetric analysis of subdural hematomas. PLoS One 13, 1–6 (2018).
Miah, I. P. et al. Radiological prognostic factors of chronic subdural hematoma recurrence: a systematic review and meta-analysis. Neuroradiology 63, 27–40 (2021).
Van Swieten, J. C., Koudstaal, P. J., Visser, M. C., Schouten, H. & Van Gijn, J. Interobserver agreement for the assessment of handicap in stroke patients. Stroke 19, 604–607 (1988).
Kan, P. et al. Middle Meningeal Artery Embolization for Chronic Subdural Hematoma: A Multi-Center Experience of 154 Consecutive Embolizations. Neurosurgery 88, 268–277 (2021).
Chari, A. et al. Core Outcomes and Common Data Elements in Chronic Subdural Hematoma: A Systematic Review of the Literature Focusing on Baseline and Peri-Operative Care Data Elements. J. Neurotrauma 33, 1569–1575 (2016).
Uyttenboogaart, M., Stewart, R. E., Vroomen, P. C. A. J., De Keyser, J. & Luijckx, G. J. Optimizing cutoff scores for the Barthel Index and the modified Rankin Scale for defining outcome in acute stroke trials. Stroke 36, 1984–1987 (2005).
Lai, Q. et al. Proposal and validation of a liver graft discard score for liver transplantation from deceased donors: a multicenter Italian study. Updates Surg. 74, 491–500 (2022).
Guyon I, Weston J, Barnhill S, Vapnik V. Gene selection for cancer classification using support vector machines. Mach Learn. 46, 389-422 (2002)
Chawla, N. V., Bowyer, K. W., Hall, L. O. & Kegelmeyer, W. P. SMOTE: Synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002).
Breiman, L. Random forests. Mach. Learn. 45, 5–32 (2001).
Ke, G. et al. LightGBM: A highly efficient gradient boosting decision tree. Adv. Neural Inf. Process. Syst. 2017-Decem, 3147–3155 (2017).
Youden, W. J. Index for rating diagnostic tests. Cancer 3, 32–35 (1950).
Rovlias, A., Theodoropoulos, S. & Papoutsakis, D. Chronic subdural hematoma: Surgical management and outcome in 986 cases: A classification and regression tree approach. Surg. Neurol. Int. 6, 127–131 (2015).
Alwosheel, A., van Cranenburgh, S. & Chorus, C. G. Is your dataset big enough? Sample size requirements when using artificial neural networks for discrete choice analysis. J. Choice Model. 28, 167–182 (2018).
Stanišíc, M. & Pripp, A. H. A reliable grading system for prediction of chronic subdural hematoma recurrence requiring reoperation after initial burr-hole surgery. Neurosurgery 81, 752–760 (2017).
Poon, M. T. C. & Al-Shahi Salman, R. Association between antithrombotic drug use before chronic subdural haematoma and outcome after drainage: a systematic review and meta-analysis. Neurosurg. Rev. 41, 439–445 (2018).

Table 1. Characteristics and Treatment of Patients Assessed via Univariate Analyses
		functional outcome at discharge
Baseline characteristics and treatment	Overall N = 188¹	Favorable N = 126¹	Unfavorable N = 62¹	p-value²
Age	79 (11)	77 (12)	85 (6)	<0.001
Sex				0.2
Female	60 (32%)	36 (29%)	24 (39%)
Male	128 (68%)	90 (71%)	38 (61%)
Body mass index	21.5 (3.3)	22.0 (3.2)	20.4 (3.2)	0.003
GCS on admission				0.001
14-15	119 (63%)	90 (71%)	29 (47%)
3-13	69 (37%)	36 (29%)	33 (53%)
Hemiparesis	123 (65%)	79 (63%)	44 (71%)	0.3
Speech difficulty	21 (11%)	8 (6.3%)	13 (21%)	0.005
Preceding trauma events	109 (58%)	76 (60%)	33 (53%)	0.4
Hemisphere				0.8
Bilateral	35 (19%)	25 (20%)	10 (16%)
Left	84 (45%)	56 (44%)	28 (45%)
Right	69 (37%)	45 (36%)	24 (39%)
Hypertension	101 (54%)	62 (49%)	39 (63%)	0.088
Diabetes mellitus	39 (21%)	28 (22%)	11 (18%)	0.6
Dyslipidemia	50 (27%)	35 (28%)	15 (24%)	0.7
Chronic kidney disease	54 (29%)	36 (29%)	18 (29%)	>0.9
Ischemic stroke	19 (10%)	10 (7.9%)	9 (15%)	0.2
Cardiovascular disease	13 (6.9%)	7 (5.6%)	6 (9.7%)	0.4
Liver disease, alcohol abuse	8 (4.3%)	7 (5.6%)	1 (1.6%)	0.3
Anticoagulant use	12 (6.4%)	6 (4.8%)	6 (9.7%)	0.2
Antiplatelet use	35 (19%)	19 (15%)	16 (26%)	0.11
mRS before onset				<0.001
0-2	145 (77%)	115 (91%)	30 (48%)
3-6	43 (23%)	11 (8.7%)	32 (52%)
Hematoma density on preoperative CT				0.3
Hyperdensity, Separated, Laminar, Gradation, Trabecular	147 (78%)	95 (75%)	52 (84%)
Hypodensity, Isodensity	41 (22%)	31 (25%)	10 (16%)
Preoperative hematoma volume	154 (121, 194)	148 (118, 194)	163 (130, 194)	0.13
Preoperative hematoma thickness	2.50 (2.10, 3.00)	2.50 (2.10, 2.98)	2.45 (2.00, 3.10)	0.9
Midline shift	8.1 (5.1, 11.2)	8.0 (4.9, 11.1)	8.4 (5.2, 11.1)	0.7
Operative procedure				0.12
Irrigation and drainage	98 (52%)	71 (56%)	27 (44%)
Irrigation only	90 (48%)	55 (44%)	35 (56%)
Surgeon				0.11
More experienced	34 (18%)	27 (21%)	7 (11%)
Resident	154 (82%)	99 (79%)	55 (89%)
Postoperative Goreisan use	140 (74%)	99 (79%)	41 (66%)	0.076
GCS, Glasgow coma scale; mRS, modified Rankin scale; CT, computed tomography. ¹Data are presented as the mean ±standard deviation (SD), the median (interquartile range: IQR), or the number (%). ²Fisher's exact test; Mann-Whitney U test

Table 2. Preoperative laboratory data evaluated via univariate analyses
		functional outcome at discharge
Laboratory data on admission	Overall, N = 188¹	Favorable, N = 126¹	Unfavorable, N = 62¹	p-value²
Hemoglobin (g/dL)	12.50 (11.50, 13.90)	13.00 (12.03, 14.47)	11.80 (10.50, 12.50)	<0.001
RBC (*10^6/μL)	406 (368, 444)	416 (379, 454)	376 (350, 417)	<0.001
WBC (*10^3/μL)	6.80 (5.68, 7.82)	6.80 (5.90, 7.68)	6.40 (5.00, 8.17)	0.6
Platelet (*10^3/μL)	21 (17, 26)	20 (17, 25)	21 (17, 28)	0.5
MCHC (g/dL)	34.05 (33.30, 34.80)	34.45 (33.50, 34.98)	33.60 (32.50, 34.10)	<0.001
MCH (pg)	31.30 (29.80, 32.62)	31.40 (30.30, 32.88)	30.80 (29.15, 32.08)	0.015
MCV (fL)	91.8 (87.9, 94.8)	91.4 (88.0, 95.0)	92.0 (87.7, 94.5)	0.8
Hematocrit (%)	37.0 (34.0, 40.5)	38.0 (35.3, 41.5)	34.9 (31.6, 37.1)	<0.001
Basophil (%)	0.30 (0.20, 0.50)	0.30 (0.20, 0.50)	0.20 (0.12, 0.30)	<0.001
Eosinophil (%)	1.50 (0.50, 2.90)	1.50 (0.52, 2.85)	1.65 (0.50, 2.85)	>0.9
Monocyte (%)	5.60 (4.60, 6.50)	5.45 (4.62, 6.40)	5.95 (4.53, 6.90)	0.13
Lymphocyte (%)	20 (13, 25)	20 (13, 25)	19 (12, 24)	0.14
Neutrophil (%)	72 (65, 80)	72 (65, 79)	72 (65, 81)	0.5
PT-INR	1.02 (0.97, 1.09)	1.00 (0.96, 1.08)	1.05 (1.01, 1.14)	<0.001
APTT (sec)	31.0 (29.3, 33.9)	31.0 (29.3, 34.1)	30.9 (29.3, 33.5)	0.7
BUN (mg/dL)	18 (13, 23)	18 (13, 21)	18 (14, 28)	0.2
Creatinine (mg/dL)	0.77 (0.62, 1.01)	0.78 (0.62, 1.01)	0.77 (0.60, 1.04)	>0.9
eGFR (ml/min/1.7)	68 (49, 84)	68 (52, 86)	66 (47, 84)	0.3
ALT (U/L)	15 (11, 22)	16 (12, 23)	13 (10, 21)	0.027
AST (U/L)	22 (18, 28)	22 (18, 28)	22 (18, 29)	0.8
Total bilirubin (mg/dL)	0.70 (0.50, 0.90)	0.70 (0.50, 1.00)	0.60 (0.40, 0.78)	0.030
Total protein (g/dL)	6.90 (6.50, 7.30)	6.90 (6.70, 7.30)	6.80 (6.43, 7.10)	0.035
Albumin (g/dL)	3.90 (3.58, 4.20)	4.00 (3.73, 4.30)	3.65 (3.20, 3.90)	<0.001
Albumin/globulin ratio	1.30 (1.15, 1.46)	1.37 (1.21, 1.56)	1.17 (1.03, 1.31)	<0.001
Glucose (mg/dL)	118 (102, 144)	118 (102, 142)	117 (100, 146)	0.8
CK (U/L)	106 (65, 204)	114 (74, 175)	101 (54, 254)	0.5
Na (mmol/L)	140.0 (137.0, 142.0)	140.0 (138.0, 142.0)	140.4 (137.0, 142.0)	0.8
K (mmol/L)	4.00 (3.80, 4.30)	4.00 (3.80, 4.20)	4.00 (3.62, 4.38)	>0.9
Cl (mmol/L)	104.0 (101.0, 106.0)	104.0 (102.2, 106.0)	105.0 (100.2, 107.0)	0.9
RBC, red blood cell; WBC, white blood cell; MCHC, mean corpuscular hemoglobin concentration; MCH, mean corpuscular hemoglobin; MCV, mean corpuscular volume; PT-INR, prothrombin time-international normalized ratio; APTT, activated partial thromboplastin time; BUN, blood urea nitrogen; eGFR, estimated glomerular filtration rate; ALT, alanine aminotransferase; AST, aspartate aminotransferase; CK, creatine kinase; mRS, modified Rankin scale. ¹Median (Interquartile range) ²Mann-Whitney U test

Table 3. Comparison of the predictive abilities of the four machine learning algorithms on the test dataset
models	accuracy	sensitivity	specificity	f1 score	ROC-AUC
logistic regression	0.894	0.933	0.875	0.848	0.925
SVM	0.894	1.000	0.844	0.857	0.919
random forest	0.830	1.000	0.750	0.789	0.906
light GBM	0.851	1.000	0.781	0.811	0.906
SVM, support vector machine; light GBM, light gradient boosting machine; ROC-AUC, area under the curve of the receiver operating characteristic curve.

Table 4. The predictive performance of the four machine learning algorithms in external cohort
models	accuracy	sensitivity	specificity	f1 score	ROC-AUC
logistic regression	0.889	0.667	0.929	0.645	0.856
SVM	0.919	0.667	0.964	0.714	0.860
random forest	0.768	0.800	0.762	0.511	0.835
light GBM	0.788	0.867	0.774	0.553	0.833
SVM, support vector machine; light GBM, light gradient boosting machine; ROC-AUC, area under the curve of the receiver operating characteristic curve.

No competing interests reported.

supplementarymaterial.docx

Download PDF

Version 1

posted

You are reading this latest preprint version

Development of machine learning models for predicting unfavorable functional outcomes in patients with chronic subdural hematomas

Status:

Version 1

Abstract

Figures

Introduction

Methods

Results

Discussion

Declarations

References

Tables

Additional Declarations

Supplementary Files

Status:

Version 1