Comparison of different machine learning classification models for predicting deep vein thrombosis in lower extremity fractures

doi:10.21203/rs.3.rs-3330955/v1

Download PDF

Article

Comparison of different machine learning classification models for predicting deep vein thrombosis in lower extremity fractures

https://doi.org/10.21203/rs.3.rs-3330955/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 22 Mar, 2024

Read the published version in Scientific Reports →

You are reading this latest preprint version

Deep vein thrombosis (DVT) is a common complication in patients with lower extremity fractures. Once it occurs, it will seriously affect the quality of life and postoperative recovery of patients. Therefore, early prediction and prevention of DVT can effectively improve the prognosis of patients. Based on the predictive factors of DVT in patients with lower limb fractures, this study constructed a DVT prediction model with the help of different machine learning classification models to explore the effectiveness of different models in predicting DVT. The researchers conducted a retrospective analysis of DVT-related factors in patients with lower limb fractures from the Second Affiliated Hospital of Nanchang University from July 2017 to July 2023, and then calculated the incidence of DVT. Five prediction models were applied to the experiment, including Extreme Gradient Boosting (XGBoost) model, Logistic Regression (LR) model, RandomForest (RF) model, Multilayer Perceptron (MLP) model and Support Vector Machine(SVM) model. Afterwards, the performance of the obtained prediction models were evaluated by area under the curve (AUC), accuracy, sensitivity, specificity and F1 score. A total of 4,424 patients were included in this study, of which 207 patients had DVT. Theincidence rate of DVT was 4.68%. The prediction performance of the model based on machine learning: XGBoost model (AUC=0.730, accuracy=0.951), LR model (AUC =0.740, accuracy=0.712), RF model (AUC=0.703, accuracy=0.952), MLP model (AUC=0.571, accuracy=0.704), SVM model (AUC=0.488, accuracy=0.826). Although the LR model has the largest AUC, its accuracy is not as good as that of the XGBoost model. By comparing the AUC and accuracy, the XGBoost model performed the best. The DVT prediction model constructed by the XGB has high reproducibility, universality and feasibility. However, the model still needs external verification research before clinical application.

Health sciences/Diseases

Health sciences/Risk factors

Deep vein thrombosis

Lower extremity fractures

Achine learning

Prediction model

XGBoost model

LR model

RF model

MLP model

SVM model

DVT is a common perioperative complication in fracture patients and the incidence of postoperative DVT is significantly higher than that before the operation[1]. It is the third leading cause of cardiovascular death worldwide and a major complication limiting recovery in patients with lower extremity fractures[2]. Moreover, it can cause symptoms such as limb swelling, pain and limitions in motor function[3]. As the disease progresses, DVT can lead to the occurrence of pulmonary embolism, increasing the risk of death in patients[4].

More and more studies have tried to find the relevant factors for the occurrence and development of DVT in patients with lower extremity fractures during hospitalization, including age, underlying diseases, fracture types, operation time, bed rest time, coagulation function, etc[5–7]. These studies help us better understand the factors associated with increased risk of DVT. However, whether to take measures to prevent DVT is usually based on the doctor's experience and the patient's symptoms. This approach, although clinically feasible, may delay the prevention and treatment of DVT. Therefore, identification of high-risk patients with DVT and timely prevention can avoid its occurrence.

Determining the important predictors of DVT is the key to accurately identify high-risk patients. However, the occurrence and development of DVT is influenced by many factors[8]. The occurrence of DVT is characterized by diversity and instability, and its influencing factors are still in the stage of continuous exploration. As a quantitative tool for disease risk assessment, the disease prediction model is an important auxiliary tool for identifying high-risk groups in the current clinical diagnosis, treatment and nursing process[9]. In-depth analysis of the interaction mechanism of disease-related factors is the core to accurate disease prediction.

Machine learning is a scientific discipline that studies how computers learn from data. Through the computer algorithm, it automatically learns knowledge and rules from the data, so as to simulate human intelligent behavior and then achieve a certain purpose of technology and methods. Although there are currently scoring scales for predicting the risk of venous thrombosis in clinical practice, some studies have found that machine learning is superior to traditional scoring methods in the accuracy of prediction[10]. Although there are clinical scoring scales for predicting the risk of venous thrombosis, some studies have found that machine learning is better than traditional scoring methods in terms of prediction accuracy. Shohat et al.[11] demonstrated that machine learning can predict venous thrombosis risk in hospitalized patients more accurately than scoring models developed in the past. Danilatou et al. [12] also considered machine learning as a method suitable for VTE risk prediction and believed that it has great advantages in finding the best predictors of VTE. At present, machine learning, as a new risk assessment and prediction tool, has been used by many researchers in clinical research and practice.

Ferroni et al.[13] selected 1179 outpatient cancer patients who received chemotherapy but did not receive thrombus prophylaxis, and used SVM model and randomized optimized multi-core learning model to predict DVT risk factors in these patients. The AUCPR of the model = 0.212. This study confirmed the possibility and effectiveness of machine learning method for DVT risk prediction. Ferroni et al.[14] compared machine learning with Khorana score (KS) risk assessment scale, and the results showed that the predictive model was better than KS. The study conducted by Spencer et al.[15] aimed to measure the accuracy of machine learning classifier for predicting DVT in cancer inpatients, and found that the random forest model could correctly classify DVT and non-DVT patients with an accuracy of 74%. The study suggests that machine learning classifiers could be incorporated into electronic medical record systems that could provide an effective blood clot risk assessment for patients.

Although there are several prediction models for DVT, these prediction models have limitations[16, 17]. Firstly, the inclusion and exclusion criteria of patients in the model were strict, and the population representativeness needed to be improved. Secondly, there are problems of bias in the selection of predictors and overfitting of models. Therefore, in this study, we compared the DVT prediction performance with several machine learning algorithms, such as XGB, LR, RF, MLP and SVM model. Finally, we obtained the optimal model based on the comprehensive score.

Study Population

The retrospective study was conducted according to STROBE guidelines and all methods were performed in accordance with the relevant guidelines and regulations. The Ethics Committee of the Second Affiliated Hospital of Nanchang University approved the study, without patient informed consent due to the anonymous nature of the data. Two specialists used the hospital digital medical record system to retrieve information of patients after lower limb fracture surgery in the Second Affiliated Hospital of Nanchang University from July 1, 2017 to July 1, 2023. Eligible patients were screened by checking medical records, doctor orders and nursing records in the hospital digital medical record system. The results of patients with deep venous thrombosis of lower limbs were obtained through imaging examination reports in the hospital's digital medical record system. All the obtained data were recorded into Excel software for sorting.

In this study, patients with lower extremity fractures who met the following criteria were included as eligible subjects: patients aged 18 years and older, with less than 20% missing items. Exclusion criteria are: pathological fractures, fractures in other parts (such as sternum, and vertebrae), history of venous thromboembolism, oral contraceptives (nearly one month), pregnancy, and blood system diseases.

Clinical Outcomes and Data Preprocessing

With the occurrence or absence of DVT as the outcome index, all patients underwent color Doppler ultrasonography of both lower limbs. The sensitivity and specificity of color Doppler ultrasonography for DVT is more than 90%, which can determine the location and type of thrombus, determine the degree of embolization and collateral circulation, as well as evaluate the therapeutic effect[18]. DVT ultrasound diagnosis DVT includes the following signs: enlarged lumen below the thrombus obstruction site, thickened vessel wall, solid echo in the lumen, filling defect of blood flow signal in the lumen, loss of phasic changes in blood flow spectrum, extrusion of the distal end increased blood flow to extremities disappears or weakens.

For individual missing values of variables, continuous variables were filled by the imputation method and categorical variables were filled by the random forest method. For outliers, they are reconfirmed by the data source and treated as null if they are still outliers after reconfirmation. The StandardScaler is used for standardization processing. The mean value of the data features processed by StandardScaler is 0, while the standard deviation is 1.

Selection of Predictors

Based on a search of relevant literature and clinical judgment, we compiled a list of potential predictors of DVT. There are 36 predictors including sex, rh blood type, ABO blood type, smoking, alcohol consumption, operation time, hypoalbuminemia, kidney disease, cerebrovascular disease, atrial fibrillation, heart disease, cancer, chronic obstructive pulmonary disease(COPD), osteoporosis, hypertension, diabetes, lung infection, fracture type, surgical grade, age, total cholesterol (mmol/L), T triacylglycerol (mmol/L), free fatty acid (mmol/L), albumin (g/L), Globulin ratio, calcium (mmol/L), potassium (mmol/L), platelets (10^12/L), red blood cells (10^12/L), white blood cells (10^9/L), fibrinogen (g/L), D-dimer (mg/lFEU), international normalized ratio(INR), plasma prothrombin time (s), hospital stay (days) and C-reactive protein (mg/L). The fracture site was divided into hip, femur, tibia, and multiple fractures. Besides, the operation grade was divided into first-level operation, second-level operation, third-level operation, and fourth-level operation. The operation time was divided into two groups according to 3 hours.

Model Building

The XGBoost model is also called reinforcement learning. Its working principle is to re-adjust the training samples of the decision tree obtained from the initial training set, especially after further adjustment of the training samples that the decision tree got wrong, and then use them to train the next decision[19]. Tree, that is to say, as the learning depth increases, the decision tree constantly correcting the errors of the previous decision tree, and finally improves the prediction effect of the XGBoost model and obtains a stronger learner[20].

The LR model is a generalized linear regression analysis model that can be extended to assess the correlation between various types of observed data and certain factors[21]. The LR model follows the Bernoulli distribution and parameters are solved using gradient descent by maximizing the likelihood function[22].

The RF model is an algorithm that combines multiple decision trees through ensemble learning[23]. Its fundamental element is the decision tree, and the multiple decision trees form a random forest generated, and the prediction result of the model is equal to a certain "weighted average" of the results of each decision tree. In classification problems, multiple tree classifiers vote (the principle of minority obeying the majority) to determine the final classification result. For regression problems, the average of all decision tree predictions determines the final results[24].

The construction principle of the MLP model is to imitate the human brain, which consists of an input layer, an output layer and multiple hidden layers[25]. Each layer contain several neurons. The feature vector is received by the neurons in the input layer and then passed to a hidden layer. The hidden layer calculates the final result through the weight and activation function, and then passes it to the next hidden layer. Eventually, the vector reaches the output layer to obtain the final result[26].

The SVM model maps the original data to a high-latitude space through a nonlinear function, in which the original data can be separated by a line called a separating hyperplane, and the distance from the data point to this line represents the prediction result of the model Confidence (the farther the distance, the more confident the model is about the accuracy of the prediction)[27]. The SVM model maximizes the sum of the distances from the straight line to the two categories of data points (making it the farthest from the straight line), and the interval is the sum of the distances[28].

Before building the prediction model, we used the XGBoost model to calculate the average contribution value of each variable in the XGBoost model, and listed the top 10 important factors. We then divided all the data into five equal parts and chose one of them as the validation set. The remaining 4 parts were used as training sets and repeated 5 times. Five machine learning algorithms, including XGBoost model, LR model, RF model, MLP model and SVM model, were used for model construction by using 5-fold cross-validation.

Model Evaluation

The confusion matrix is a cross-tabulation of real values through which multiple evaluation indexes of the model can be constructed. True Positive (TP) means that the true classification of the sample is positive, and the identification result of the model is also positive; False Negative (FN) means that the true classification of the sample is positive, but the model serves as a negative class. False Positives (FP) means that the true classification of the sample is a negative class, but the model will regard it as a positive class. True Negative (TN) means that the true classification of the sample is negative, and the identification result of the model is also negative.

The size of the area under the receiver operating characteristic (ROC) curve is AUC. AUC reflects the overall predictive performance of the model. The closer the AUC is to 1, the better the predictive performance of the model; Accuracy is used to describe the accuracy of the model, that is, the accurate number/total number of samples that the model can identify. The higher the accuracy of the model, the better the effectiveness of the model. Sensitivity represents the ratio of the number of positive patients correctly identified in the model to the total number of positive patients. Specificity refers to the proportion of the number of correctly predicted negative patients in the number of true negative cases. F1 score is the harmonic mean of accuracy and sensitivity and the value of F1 score ranges from 0 to 1. The higher the F1 value, the better the efficiency.

Statistical Analysis

Using DVT as the outcome indicator, the data were divided into two groups. The Mann-Whitney U test was used for continuous variables such as age, expressed as median and interquartile range (IQR). For gender and other variables, the chi-square test was used, expressed as percentage (%). We obtained the AUC, accuracy, sensitivity, specificity and F1 score of five machine learning algorithms on the test set. Finally, the optimal model was selected by comprehensive score. All Statistical analyses were performed using R version 3.6.3 and python version 3.7. P < 0.05 was considered to be statistically significant. This work was supported by Extreme Smart Analysis platform(https://www.xsmartanalysis.com/).

Patient Demographic Features

A total of 4424 patients were included in this study, including 4217 patients without DVT and 207 patients with DVT, the incidence rate of DVT was 4.68%. The average age was 61 years old, mainly between 45 and 76 years old. There were 2353 males accounting for 53.187% of the total included patients and 2071 females accounting for 46.813% of the total included patients. The incidence of DVT in patients with different parts of lower limb fractures was different (P < 0.001), that the incidence of DVT in patients with femoral fractures and multiple fractures was significantly higher than that of tibial and pelvic fractures. Since the number of first-level surgery patients in the non-DCT group was 0, the statistical result of the final surgery level was nan. Between the DVT group and the non-DVT group, the gender (P < 0.001), age (P < 0.001), ABO blood type (P < 0.001), hypoalbuminemia (P < 0.001), atrial fibrillation (P < 0.001), Total cholesterol (P < 0.001), T triacylglycerol (P = 0.002), albumin (P < 0.001), white blood cell ratio (P < 0.001), calcium ion (P < 0.001), platelets (P = 0.003), red blood cells (P < 0.001), fibrinogen (P < 0.001), D-dimer (P = 0.027), International Normalized Ratio (P = 0.036), C-reactive protein (P = 0.007) were statistically significant. There were no statistically significant differences in the remaining variables, as shown in Table 1.

Table 1

Baseline characteristics of lower extremity fractures patients with and without DVT.
Predictors	Nonthrombosis group (n = 4217)	Thrombosis group (n = 207)	p
Sex [n(%)]			< 0.001
Male	2269(53.806)	84(40.580)
Female	1948(46.194)	123(59.420)
Rh blood type [n(%)]			0.940
Negative	22(0.522)	1(0.483)
Positive	4195(99.478)	206(99.517)
ABO blood type [n(%)]			0.007
O Type Blood	1360(32.250)	51(24.638)
A Type Blood	1647(39.056)	91(43.961)
B Type Blood	929(22.030)	41(19.807)
AB Type Blood	281(6.664)	24(11.594)
Smoking [n(%)]			0.610
No	4089(96.965)	202(97.585)
Yes	128(3.035)	5(2.415)
Alcohol consumption [n(%)]			0.501
No	4103(97.297)	203(98.068)
Yes	114(2.703)	4(1.932)
Operation time [n(%)]			0.611
≤ 3h	3848(91.250)	191(92.271)
>3h	369(8.750)	16(7.729)
Hypoalbuminemia [n(%)]			< 0.001
No	4057(96.206)	180(86.957)
Yes	160(3.794)	27(13.043)
Kidney disease [n(%)]			0.672
No	4133(98.008)	202(97.585)
Yes	84(1.992)	5(2.415)
Cerebrovascular disease [n(%)]			0.168
No	3738(88.641)	177(85.507)
Yes	479(11.359)	30(14.493)
Atrial fibrillation [n(%)]			< 0.001
No	4046(95.945)	188(90.821)
Yes	171(4.055)	19(9.179)
Heart disease [n(%)]			0.658
No	4021(95.352)	196(94.686)
Yes	196(4.648)	11(5.314)
Cancer [n(%)]			0.097
No	3952(93.716)	188(90.821)
Yes	265(6.284)	19(9.179)
COPD [n(%)]			0.112
No	4079(96.728)	196(94.686)
Yes	138(3.272)	11(5.314)
Osteoporosis [n(%)]			0.072
No	3891(92.269)	198(95.652)
Yes	326(7.731)	9(4.348)
Hypertension [n(%)]			0.703
No	3066(72.706)	148(71.498)
Yes	1151(27.294)	59(28.502)
Diabetes [n(%)]			0.073
No	3671(87.052)	189(91.304)
Yes	546(12.948)	18(8.696)
Lung infection [n(%)]			0.288
No	3949(93.645)	190(91.787)
Yes	268(6.355)	17(8.213)
Fracture type [n(%)]			< 0.001
Hip	177(4.197)	13(6.280)
Femur	2551(60.493)	132(63.768)
Tibial	1283(30.424)	25(12.077)
Multiple	206(4.885)	37(17.874)
Surgical grade [n(%)]			nan
I	24(0.569)	0(0.000)
II	155(3.676)	4(1.932)
III	3191(75.670)	170(82.126)
IV	847(20.085)	33(15.942)
Age median[IQR]	60.000[45.000,76.000]	70.000[59.000,79.000]	< 0.001
Total cholesterol median[IQR]	3.742[3.156,4.340]	3.853[3.230,4.663]	0.012
T triacylglycerol median[IQR]	1.085[0.856,1.464]	1.204[0.927,1.550]	0.002
Free fatty acid median[IQR]	0.505[0.350,0.660]	0.510[0.340,0.670]	0.723
Albumin median[IQR]	37.060[33.700,40.230]	34.670[30.900,37.710]	< 0.001
Globulin ratio median[IQR]	1.420[1.240,1.620]	1.300[1.120,1.485]	< 0.001
Calcium median[IQR]	2.240[2.150,2.320]	2.210[2.080,2.300]	< 0.001
Potassium median[IQR]	3.880[3.620,4.140]	3.850[3.630,4.140]	0.998
Platelets median[IQR]	185.000[143.000,234.000]	202.000[153.000,269.000]	0.003
Red blood cells median[IQR]	3.840[3.280,4.340]	3.470[2.890,3.990]	< 0.001
White blood cells median[IQR]	8.090[6.390,10.180]	7.940[6.220,9.370]	0.067
Fibrinogen median[IQR]	3.320[2.600,4.240]	3.720[3.040,4.680]	< 0.001
D dimer median[IQR]	5.860[2.700,13.100]	6.980[3.800,14.080]	0.027
INR median[IQR]	1.020[0.970,1.090]	1.040[0.970,1.110]	0.036
Prothrombin time median[IQR]	11.800[11.200,12.500]	12.000[11.300,12.700]	0.051
Hospital stay median[IQR]	12.000[8.333,17.000]	12.000[7.000,17.000]	0.252
C-reactive protein median[IQR]	41.900[15.830,85.360]	53.500[20.600,97.410]	0.007

Ranking of important features

The XGBoost model is used to analyze the importance of variables. The model parameters are: learning_rate = 0.1, maximum tree depth = 8, minimum fork weight sum) = 4, L2 regularization coefficient = 0.5. The 10 most important variables in the XGBoost model (from high to low) are age, fracture type, platelets, free fatty acids, total cholesterol, D-dimer, white blood cell ratio, C-reactive protein, white blood cells, albumin. See Table 2 for details.

Table 2

Ranking of important features of the XGBoost model.
Variable	Total Gain	Total Cover	Gain	Cover	Weight Importance
Age	264.345	17053.337	5.084	327.949	52
Fracture type	177.934	11986.418	7.117	479.457	25
Platelets	167.881	9819.958	3.65	213.477	46
Free fatty acid	125.724	6074.618	2.372	114.615	53
Total cholesterol	125.184	8760.246	2.504	175.205	50
D-dimer	94.389	1238.932	3.776	49.557	25
Globulin ratio	89.83	3655.635	2.428	98.801	37
C-reactive protein	89.079	1349.453	2.62	39.69	34
White blood cells	84.362	3548.009	2.41	101.372	35
Albumin	83.355	5231.839	1.938	121.671	43

Predictive Model Results and Evaluation

In the training set, the AUC, accuracy, sensitivity, specificity, and F1 score of the XGBoost and RF models are all 1.000, showing perfect performance, while the AUC, accuracy, sensitivity, specificity and F1 score of the LR model are 0.737, 0.717, 0.664, 0.720, 0.183; the AUC, accuracy, sensitivity, specificity and F1 scores of the MLP model are 0.615, 0.706, 0.503, 0.717, 0.129, respectively. In the SVM model, AUC, accuracy, sensitivity, and specificity are 0.444, 0.836, 0.287, and 0.862, respectively.

In the verification set, the AUC, accuracy, sensitivity, and specificity of the XGBoost model are 0.730, 0.951, 0.757, and 0.624. The AUC, accuracy, sensitivity, and specificity of the SVM model are 0.488, 0.826, 0.275, and 0.847. The AUC, accuracy, sensitivity, specificity and F1 scores of the LR model are 0.740, 0.712, 0.742, 0.671, 0.175 respectively. The AUC, accuracy, sensitivity, specificity and F1 scores of the RF model are 0.703, 0.952, 0.748, 0.579, 0.228. The AUC, accuracy, sensitivity, specificity and F1 scores of the MLP model are 0.571, 0.704, 0.643, 0.589, 0.091, respectively. In the verification set, although the LR model has the largest AUC, its accuracy is not as good as that of the XGBoost model. By comparing AUC and accuracy, the XGBoost model performs best, as shown in Table 3.

Table 3

Evaluation indicators of 5 prediction model training sets and verification sets.
	Model	AUC	Accuracy	Sensitivity	Specificity	F1 score
Training set	XGBoost	1.000	1.000	1.000	1.000	1.000
	LR	0.737	0.717	0.664	0.720	0.183
	RF	1.000	1.000	1.000	1.000	1.000
	MLP	0.615	0.706	0.503	0.717	0.129
	SVM	0.444	0.836	0.287	0.862	NaN
Validation set	XGBoost	0.730	0.951	0.757	0.624	NaN
	LR	0.740	0.712	0.742	0.671	0.175
	RF	0.703	0.952	0.748	0.579	0.228
	MLP	0.571	0.704	0.643	0.589	0.091
	SVM	0.488	0.826	0.275	0.847	NaN

Figure 1 shows the training set ROC curves of these five models. The AUC values of the ROC curves of different models are XGBoost model (1.000), LR model (0.737), RF model (1.000), MLP model (0.615) and SVM model (0.444). XGBoost model and RF model are better than the other models on the training set.

Figure 2 shows the validation set ROC curves and PR curves of these five models. The AUC values of the ROC curves of different models are XGBoost model (0.730), LR model (0.740), RF model (0.703), MLP model (0.571) and SVM model (0.488). Although the XGBoost and RF models performed better than other models on the training set, the LR model had the largest AUC value.

By comparing the characteristic factors between the two groups for different thrombotic outcomes in patients with lower extremity fractures, we found sex, age, ABO blood type, hypoalbuminemia, atrial fibrillation, total cholesterol, T triacylglycerol, albumin, white blood cell ratio, calcium ion, platelets, red blood cells, fibrinogen, D dimer, international normalized ratio, C-reactive protein and fracture types were significantly different between the two groups (P < 0.05). In the XGBoost model, the importance ranking (from high to low) of 10 factors are age, fracture type, platelets, free fatty acids, total cholesterol, D dimer, white blood cell ratio, C-reactive protein, white blood cells and albumin. Among the prediction models built by five machine learning algorithms, although the LR model has the highest AUC value (0.740), its accuracy is not as good as that of the XGBoost model. By comparing multiple indicators such as AUC, accuracy, sensitivity, specificity, and F1 score, the XGBoost model have the best performance.

DVT is a common complication in patients with lower extremity fractures and may lead to fatal pulmonary embolism. The incidence of DVT in this study was 4.67%. Surgery is an independent risk factor for DVT in previous studies[29]. This is mainly because, during the operation, the long-term inability of the body to actively move and the trauma caused by the operation will lead to slow blood flow and damage to the blood vessel wall resulting in th formation of venous thrombosis. In addition, advanced age is also an independent risk factor for DVT[30]. For the elderly, not only the aging of vascular endothelial cells is accelerated, but also the vascular tension, permeability and the regulation ability of the vascular wall are reduced, making the vascular homeostasis difficult to maintain and more prone to thrombosis[31, 32].

Blood hypercoagulability is one of the mechanisms of thrombus formation, so coagulation function tests are closely related to thrombus formation. Previous studies have found a correlation between fibrinogen, INR, D-dimer and thrombus[33, 34], which is consistent with our research results. High blood lipids and cholesterol will keep the body in a low-inflammatory state for a long time and increase blood viscosity[35]. Serum albumin, which has the function of inhibiting platelet aggregation and anticoagulation, has been found to be closely related to nutritional status and inflammatory response[36, 37]. Studies found that serum albumin level was a predictor of left atrial thrombus in elderly patients with nonvalvular atrial fibrillation[38, 39]. However, no research has found its correlation with DVT so far, and the relationship between them needs further verification by higher-quality research.

Blood type is an important characteristic of DVT in patients with lower extremity fractures. Li et al.[40] found that patients with blood type B had a higher risk of DVT. Differently, Haddad et al. claimed that O blood type does not increase the chance of DVT. However, a higher-quality systematic review suggests that ABO blood type is closely related to the occurrence of DVT in cancer patients. The reason may be that ABO blood type is closely related to vWF level, which act as a protective carrier of coagulation factor VIII and can promote the formation of thrombus[41].

At present, more and more evidence has shown that there is a close relationship between inflammatory immune response and DVT[42, 43]. In addition, the C-reactive protein, platelets and neutrophils play an important role in the development of DVT[44]. However, many literatures have shown that severe trauma, stress sepsis and blood loss can all lead to systemic inflammatory response syndrome, which increases the reactivity of neutrophils and platelets in the body[45–47]. Neutrophils provide the original stimulation of DVT and recruit other cells in the coagulation process. Platelets produce circulating microparticles that accelerate blood coagulation[48]. Over the past few years, many scholars have investigated the association between inflammation and venous thrombosis. Kyril et al.[49] analyzed all thrombosis patients in their Thrombosis Research Center and found that a high platelet-to-lymphocyte ratio would increase the risk of inducing DVT by 3 times.

Overall, we analyzed various possible predictors of DVT and compared the predictive performance of various machine-learning methods. AUC is a performance measure of the ROC curve: the higher the AUC, the higher the predictive power of the model. Although the AUC of the LR model is the largest, its accuracy is not as good as that of the XGBoost model. The XGBoost model performs the best in terms of AUC, accuracy, sensitivity, specificity and F1 score. The XGBoost model is an optimized distributed gradient boosting library designed to be efficient, flexible and portable. The XGBoost model uses the second-order Taylor expansion of the loss function as its replacement function and solves its minimization to determine the best segmentation point and leaf node output value of the regression tree. In addition, the XGBoost model fully considers the regularization problem by introducing the number of subtrees and the value of subtree leaf nodes in the loss function. In terms of efficiency, the XGBoost model has greatly improved the modeling efficiency by using the unique approximate regression tree bifurcation point estimation and sub-node parallelization, coupled with the characteristics of second-order convergence.

The occurrence and development of DVT in the actual clinical environment is the result of the long-term interaction of multiple risk factors. The progress of disease is complex as well. The impact of predictors on the occurrence of DVT is judged solely by whether there is a statistical difference, which is different from the clinical situation and the pathophysiology of DVT. The machine learning algorithm can present the degree of influence of all predictors and the degree of influence of each predictor on the occurrence of model outcome events can be reflected by ranking the importance[50, 51]. Therefore, the prediction model based on machine learning can fully reflect the impact of all predictors on DVT, and its results are closer to the actual clinical diagnosis and treatment.

As shown in Table 1, we included variables possibly associated with DVT in the current algorithm to obtain accurate predictions. However, some limitations need to be considered when interpreting the findings of this study. First, there is a lack of detailed description of the variables regarding medication and lifestyle during hospitalization, such as antiplatelet, anticoagulant, antihypertensive, hypoglycemic, chemotherapy, antibiotics, diet and physical activity, which may lead to some confounding effects. Second, we are unable to obtain symptoms and signs such as pain, cramping, heaviness, pruritus and varicose veins at the site of DVT, which can hinder a more comprehensive analysis. Although vascular ultrasonography has gradually replaced venous angiography and is widely used, this method is not the "gold standard". As a result, the possibility of a false positive or false negative in DVT diagnoses can not be ruled out in this study. In addition, it was difficult for us to include all variables that may affect the statistical results. The relationship between variables and DVT is correlation, not causation. Therefore, the findings should be interpreted in the context of the clinical situation. Finally, no external datasets were used for validation in this study and all models were validated only with cross-validation and a test set during training.

Among all the models, the XGBoost model is the most effective in predicting the occurrence of DVT in patients with lower extremity fractures during hospitalization. The DVT prediction model constructed by the XGBoost model has high reproducibility, universality and feasibility, while the model still needs external verification research before clinical application.

Author contributions

Lingling Yu and Pengfei Yu conceived and designed the experiments;Jialiang Wang and Conghui Wei performed the experiments; Ang Li and Ziying Xiong analyzed the data; Jun Luo and Zhen Yuan contributed reagents and materials; Conghui Wei wrote the paper.

Ethics approval

The present research was approved by the Biomedical Research Ethics Committee of the Second Affiliated Hospital of Nanchang University (BR/AFISG-04/1.0). The informed consent was waived by the approving ethics committee due to the retrospective nature of the study.

Acknowledgment

Funding supported from Project of the Science and Technology Department of Jiangxi Province, China (20224ABC03A02) and Science and technology support project of Jiangxi Provincial Health Commission (202130466) are greatly appreciated.

Funding

This research was supported by Project of the Science and Technology Department of Jiangxi

Province, China (20224ABC03A02) and Science and technology support project of Jiangxi Provincial Health Commission (202130466) .

Conflict of interest

The authors declare no conflict of interest.

Data availability

The supporting information was supplied regarding data availability:S1 Dataset. Patients’ clinical dataset.xlxs

Bartlett, M.A., et al., Perioperative Venous Thromboembolism Prophylaxis. Mayo Clinic Proceedings, 2020. 95(12): p. 2775-2798.http://doi.org/10.1016/j.mayocp.2020.06.015.
Lutsey, P.L. and N.A. Zakai, Epidemiology and prevention of venous thromboembolism. Nature Reviews Cardiology, 2022. 20(4): p. 248-262.http://doi.org/10.1038/s41569-022-00787-6.
Jaff, M.R., et al., Management of Massive and Submassive Pulmonary Embolism, Iliofemoral Deep Vein Thrombosis, and Chronic Thromboembolic Pulmonary Hypertension. Circulation, 2011. 123(16): p. 1788-1830.http://doi.org/10.1161/CIR.0b013e318214914f.
Di Nisio, M., N. van Es, and H.R. Büller, Deep vein thrombosis and pulmonary embolism. The Lancet, 2016. 388(10063): p. 3060-3073.http://doi.org/10.1016/s0140-6736(16)30514-1.
Xing, F., et al., Admission prevalence of deep vein thrombosis in elderly Chinese patients with hip fracture and a new predictor based on risk factors for thrombosis screening. BMC Musculoskeletal Disorders, 2018. 19(1).http://doi.org/10.1186/s12891-018-2371-5.
Luksameearunothai, K., et al., Usefulness of clinical predictors for preoperative screening of deep vein thrombosis in hip fractures. BMC Musculoskeletal Disorders, 2017. 18(1).http://doi.org/10.1186/s12891-017-1582-5.
Silveira, P.C., et al., Performance of Wells Score for Deep Vein Thrombosis in the Inpatient Setting. JAMA Internal Medicine, 2015. 175(7).http://doi.org/10.1001/jamainternmed.2015.1687.
Tøndel, B.G., et al., Risk factors and predictors for venous thromboembolism in people with ischemic stroke: A systematic review. Journal of Thrombosis and Haemostasis, 2022. 20(10): p. 2173-2186.http://doi.org/10.1111/jth.15813.
Price, E.L. and T. Minichiello, The Wells Deep Vein Thrombosis Score for Inpatients. JAMA Internal Medicine, 2015. 175(7).http://doi.org/10.1001/jamainternmed.2015.1699.
Mooney, S.J. and V. Pejaver, Big Data in Public Health: Terminology, Machine Learning, and Privacy. Annual Review of Public Health, 2018. 39(1): p. 95-112.http://doi.org/10.1146/annurev-publhealth-040617-014208.
Shohat, N., et al., Using machine learning to predict venous thromboembolism and major bleeding events following total joint arthroplasty. Scientific Reports, 2023. 13(1).http://doi.org/10.1038/s41598-022-26032-1.
Danilatou, V., et al., Outcome Prediction in Critically-Ill Patients with Venous Thromboembolism and/or Cancer Using Machine Learning Algorithms: External Validation and Comparison with Scoring Systems. International Journal of Molecular Sciences, 2022. 23(13).http://doi.org/10.3390/ijms23137132.
Ferroni, P., et al., Risk Assessment for Venous Thromboembolism in Chemotherapy-Treated Ambulatory Cancer Patients. Medical Decision Making, 2016. 37(2): p. 234-242.http://doi.org/10.1177/0272989x16662654.
Ferroni, P., et al., Validation of a Machine Learning Approach for Venous Thromboembolism Risk Prediction in Oncology. Disease Markers, 2017. 2017: p. 1-7.http://doi.org/10.1155/2017/8781379.
James, S.L., K. Mody, and J.J. Shatzel, Novel algorithms to predict the occurrence of in-hospital venous thromboembolism in cancer patients: Machine learning classifiers developed from the 2012 national inpatient sample. Journal of Clinical Oncology, 2015. 33(15).http://doi.org/DOI 10.1200/jco.2015.33.15_suppl.1582.
González, J., et al., Limited diagnostic workup for deep vein thrombosis after major joint surgery. Thrombosis and Haemostasis, 2017. 99(06): p. 1112-1115.http://doi.org/10.1160/th08-02-0115.
Rogers, M.A.M., et al., Triggers of Hospitalization for Venous Thromboembolism. Circulation, 2012. 125(17): p. 2092-2099.http://doi.org/10.1161/circulationaha.111.084467.
Rose, S.C., et al., Symptomatic Lower-Extremity Deep Venous Thrombosis - Accuracy, Limitations, and Role of Color Duplex Flow Imaging in Diagnosis. Radiology, 1990. 175(3): p. 639-644.http://doi.org/DOI 10.1148/radiology.175.3.2188293.
Mitchell, R. and E. Frank, Accelerating the XGBoost algorithm using GPU computing. PeerJ Computer Science, 2017. 3.http://doi.org/10.7717/peerj-cs.127.
Dhaliwal, S., A.-A. Nahid, and R. Abbas, Effective Intrusion Detection System Using XGBoost. Information, 2018. 9(7).http://doi.org/10.3390/info9070149.
Liu, Y. and J. Hannig, Generalized Fiducial Inference for Logistic Graded Response Models. Psychometrika, 2017. 82(4): p. 1097-1125.http://doi.org/10.1007/s11336-017-9554-0.
Stoltzfus, J.C., Logistic Regression: A Brief Primer. Academic Emergency Medicine, 2011. 18(10): p. 1099-1104.http://doi.org/10.1111/j.1553-2712.2011.01185.x.
Amaratunga, D., J. Cabrera, and Y.-S. Lee, Enriched random forests. Bioinformatics, 2008. 24(18): p. 2010-2014.http://doi.org/10.1093/bioinformatics/btn356.
Fu, G., X. Dai, and Y. Liang, Functional random forests for curve response. Scientific Reports, 2021. 11(1).http://doi.org/10.1038/s41598-021-02265-4.
Tran, D.T., et al., Heterogeneous Multilayer Generalized Operational Perceptron. IEEE Transactions on Neural Networks and Learning Systems, 2020. 31(3): p. 710-724.http://doi.org/10.1109/tnnls.2019.2914082.
Tang, J., C. Deng, and G.-B. Huang, Extreme Learning Machine for Multilayer Perceptron. IEEE Transactions on Neural Networks and Learning Systems, 2016. 27(4): p. 809-821.http://doi.org/10.1109/tnnls.2015.2424995.
Wang, H., et al., Support Vector Machine Classifier via Soft-Margin Loss. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022. 44(10): p. 7253-7265.http://doi.org/10.1109/tpami.2021.3092177.
Liu, Z., D. Elashoff, and S. Piantadosi, Sparse support vector machines with L0 approximation for ultra-high dimensional omics data. Artificial Intelligence in Medicine, 2019. 96: p. 134-141.http://doi.org/10.1016/j.artmed.2019.04.004.
Chopra, V., et al., Risk of venous thromboembolism associated with peripherally inserted central catheters: a systematic review and meta-analysis. The Lancet, 2013. 382(9889): p. 311-325.http://doi.org/10.1016/s0140-6736(13)60592-9.
Tick, L.W., et al., Risk factors for post‐thrombotic syndrome in patients with a first deep venous thrombosis. Journal of Thrombosis and Haemostasis, 2008. 6(12): p. 2075-2081.http://doi.org/10.1111/j.1538-7836.2008.03180.x.
Strijkers, R.H.W., M.A.F. de Wolf, and C.H.A. Wittens, Risk factors of postthrombotic syndrome before and after deep venous thrombosis treatment. Phlebology: The Journal of Venous Disease, 2016. 32(6): p. 384-389.http://doi.org/10.1177/0268355516652010.
Stain, M., et al., The post-thrombotic syndrome: risk factors and impact on the course of thrombotic disease. Journal of Thrombosis and Haemostasis, 2005. 3(12): p. 2671-2676.http://doi.org/DOI 10.1111/j.1538-7836.2005.01648.x.
Menéndez, J.J., et al., Incidence and risk factors of superficial and deep vein thrombosis associated with peripherally inserted central catheters in children. Journal of Thrombosis and Haemostasis, 2016. 14(11): p. 2158-2168.http://doi.org/10.1111/jth.13478.
Jeraj, L., M.K. Jezovnik, and P. Poredos, Insufficient Recanalization of Thrombotic Venous Occlusion—Risk for Postthrombotic Syndrome. Journal of Vascular and Interventional Radiology, 2017. 28(7): p. 941-944.http://doi.org/10.1016/j.jvir.2017.03.031.
Ye, C., et al., Prediction of Incident Hypertension Within the Next Year: Prospective Study Using Statewide Electronic Health Records and Machine Learning. Journal of Medical Internet Research, 2018. 20(1).http://doi.org/10.2196/jmir.9268.
Soaita, I., W. Yin, and D.A. Rubenstein, Glycated albumin modifies platelet adhesion and aggregation responses. Platelets, 2017. 28(7): p. 682-690.http://doi.org/10.1080/09537104.2016.1260703.
Lam, F.W., et al., Histone induced platelet aggregation is inhibited by normal albumin. Thrombosis Research, 2013. 132(1): p. 69-76.http://doi.org/10.1016/j.thromres.2013.04.018.
Napolitano, M., et al., Optimal Duration of Low Molecular Weight Heparin for the Treatment of Cancer-Related Deep Vein Thrombosis: The Cancer-DACUS Study. Journal of Clinical Oncology, 2014. 32(32): p. 3607-3612.http://doi.org/10.1200/jco.2013.51.7433.
Rana, P. and M.N. Levine, How Long to Treat Acute Venous Thrombosis in Cancer: Can Treatment Be Personalized? Journal of Clinical Oncology, 2014. 32(32): p. 3586-3587.http://doi.org/10.1200/jco.2014.55.6977.
Li, D., et al., ABO non‐O type as a risk factor for thrombosis in patients with pancreatic cancer. Cancer Medicine, 2015. 4(11): p. 1651-1658.http://doi.org/10.1002/cam4.513.
Swystun, L.L., et al., Genetic determinants of VWF clearance and FVIII binding modify FVIII pharmacokinetics in pediatric hemophilia A patients. Blood, 2019. 134(11): p. 880-891.http://doi.org/10.1182/blood.2019000190.
Budnik, I. and A. Brill, Immune Factors in Deep Vein Thrombosis Initiation. Trends in Immunology, 2018. 39(8): p. 610-623.http://doi.org/10.1016/j.it.2018.04.010.
Salemi, R., et al., Overactivation of IL6 cis‑signaling in leukocytes is an inflammatory hallmark of deep vein thrombosis. Molecular Medicine Reports, 2022. 25(4).http://doi.org/10.3892/mmr.2022.12652.
Navarrete, S., et al., Pathophysiology of deep vein thrombosis. Clinical and Experimental Medicine, 2022. 23(3): p. 645-654.http://doi.org/10.1007/s10238-022-00829-w.
von Brühl, M.-L., et al., Monocytes, neutrophils, and platelets cooperate to initiate and propagate venous thrombosis in mice in vivo. Journal of Experimental Medicine, 2012. 209(4): p. 819-835.http://doi.org/10.1084/jem.20112322.
Zhao, Y., et al., Preoperative systemic inflammatory response index predicts long-term outcomes in type B aortic dissection after endovascular repair. Frontiers in Immunology, 2022. 13.http://doi.org/10.3389/fimmu.2022.992463.
Yang, M., et al., STING activation in platelets aggravates septic thrombosis by enhancing platelet activation and granule secretion. Immunity, 2023. 56(5): p. 1013-1026.e6.http://doi.org/10.1016/j.immuni.2023.02.015.
Yan, Y.Y., et al., Kindlin-3 in platelets and myeloid cells differentially regulates deep vein thrombosis in mice. Aging-Us, 2019. 11(17): p. 6951-6959.http://doi.org/10.18632/aging.102229.
Cole, K.L., et al., Factors Associated with Venous Thromboembolism Development in Patients with Traumatic Brain Injury. Neurocritical Care, 2023.http://doi.org/10.1007/s12028-023-01780-8.
Liu, H., et al., Prediction of venous thromboembolism with machine learning techniques in young-middle-aged inpatients. Scientific Reports, 2021. 11(1).http://doi.org/10.1038/s41598-021-92287-9.
Rinaldo, L., et al., Venous thromboembolic events in patients undergoing craniotomy for tumor resection: incidence, predictors, and review of literature. Journal of Neurosurgery, 2020. 132(1): p. 10-21.http://doi.org/10.3171/2018.7.Jns181175.

No competing interests reported.

S1Dataset.Patientsclinicaldataset.xlsx

Download PDF

Journal Publication

published 22 Mar, 2024

Read the published version in Scientific Reports →

Editorial decision: Revision requested
21 Dec, 2023
Reviews received at journal
04 Dec, 2023
Reviewers agreed at journal
20 Nov, 2023
Reviews received at journal
11 Oct, 2023
Reviewers agreed at journal
09 Oct, 2023
Reviewers invited by journal
06 Oct, 2023
Editor assigned by journal
28 Sep, 2023
Editor invited by journal
21 Sep, 2023
Submission checks completed at journal
21 Sep, 2023
First submitted to journal
06 Sep, 2023

You are reading this latest preprint version

Comparison of different machine learning classification models for predicting deep vein thrombosis in lower extremity fractures

Status:

Journal Publication

Version 1

Abstract

Figures

Introduction

Materials and Methods

Study Population

Clinical Outcomes and Data Preprocessing

Selection of Predictors

Model Building

Model Evaluation

Statistical Analysis

Results

Patient Demographic Features

Ranking of important features

Predictive Model Results and Evaluation

Discussion

Conclusions

Declarations

References

Additional Declarations

Supplementary Files

Status:

Journal Publication

Version 1