Characteristics of the participating population
The overall participants included 351 hypertensive patients (55.3% male patients), and the mean age was 51.79 ± 12.84 ys. Supplementary data online, Tables S1 and S2 show the clinical and echocardiographic features of the study participants by LASI and LAVI groups. Regarding LA stiffness, all patients were divided into two groups based on the median of LASI. Regarding LA enlargement, the cut-off value of LAVI was 28 mL/m2, and LA enlargement was present in 154 (43.9%) patients.
Feature selection
After the missing data were filled, all data were split into training set and test set. 90% patients (315 individuals) were included in training set, and 10% patients (36 individuals) were in test set. RF, GBDT, LightGBM and XGBoost models were established for all clinical and echocardiographic features, aiming to find the best model according to the performance on test set, and we could get the candidate features based on the feature importance of such models.
Figure 1 shows the ML classifiers workflow. Table 1 (all) shows AUC/ROC and AUC/PR metric of ML models for predicting LA stiffness and LA enlargement based on all features. For predicting LA stiffness, both default and optimized RF model presented the highest AUC/ROC (0.82 and 0.87), and the highest AUC/PR (0.83 and 0.87) (shown in Fig. 2A and Supplemental Fig. S1). For LAVI, both default and optimized RF model also presented the highest AUC/ROC (0.67 and 0.76), and the highest AUC/PR (0.54 and 0.63) (shown in Fig. 2B and Supplemental Fig. S2), but lower than that for LASI.
Table 1
AUC of ROC and PR results of LASI and LAVI detection combined over repeats and folds on test set
| AUC/ROC (95% CI) | | AUC/PR (95% CI) |
Optimized | Default | Optimized | Default |
LASI (all) | | | | | |
RF | 0.87 (0.74, 0.96) | 0.82 (0.70, 0.92) | | 0.87 (0.66, 0.96) | 0.83 (0.64, 0.92) |
GBDT | 0.74 (0.58, 0.87) | 0.72 (0.63, 0.88) | | 0.76 (0.57, 0.88) | 0.81 (0.64, 0.91) |
XGBoost | 0.81 (0.63, 0.91) | 0.77 (0.58, 0.89) | | 0.84 (0.68, 0.92) | 0.80 (0.62, 0.90) |
LightGBM | 0.83 (0.64, 0.93) | 0.82 (0.65, 0.93) | | 0.85 (0.69, 0.94) | 0.84 (0.67, 0.94) |
LAVI (all) | | | | | |
RF | 0.76 (0.56, 0.92) | 0.67 (0.45, 0.91) | | 0.63 (0.21, 0.84) | 0.54 (0.17, 0.84) |
GBDT | 0.66 (0.42, 0.87) | 0.63 (0.37, 0.85) | | 0.53 (0.16, 0.78) | 0.37 (0.16, 0.66) |
XGBoost | 0.58 (0.35, 0.78) | 0.54 (0.28, 0.72) | | 0.31 (0.15, 0.50) | 0.28 (0.14, 0.44) |
LightGBM | 0.65 (0.38, 0.83) | 0.63 (0.42, 0.80) | | 0.47 (0.16, 0.75) | 0.40 (0.17, 0.70) |
LASI (part) | | | | | |
RF | 0.80 (0.64, 0.93) | 0.77 (0.61, 0.87) | | 0.80 (0.62, 0.92) | 0.79 (0.62, 0.90) |
GBDT | 0.85 (0.70 0.94) | 0.82 (0.70, 0.92) | | 0.86 (0.73, 0.94) | 0.81 (0.64, 0.93) |
XGBoost | 0.84 (0.67, 0.95) | 0.80 (0.62, 0.91) | | 0.86 (0.72, 0.95) | 0.84 (0.68, 0.93) |
LightGBM | 0.83 (0.68, 0.93) | 0.80 (0.65, 0.91) | | 0.85 (0.71, 0.94) | 0.83 (0.69, 0.93) |
LAVI (part) | | | | | |
RF | 0.75 (0.57, 0.92) | 0.74 (0.55, 0.91) | | 0.60 (0.22, 0.88) | 0.54 (0.20, 0.80) |
GBDT | 0.69 (0.40, 0.88) | 0.58 (0.35, 0.83) | | 0.54 (0.17, 0.82) | 0.33 (0.15, 0.68) |
XGBoost | 0.60 (0.30, 0.80) | 0.54 (0.33, 0.77) | | 0.33 (0.16, 0.52) | 0.31 (0.14, 0.63) |
LightGBM | 0.66 (0.45, 0.84) | 0.60 (0.37, 0.83) | | 0.49 (0.19, 0.73) | 0.38 (0.15, 0.69) |
Table 2 shows the other evaluation metrics for predicting LA stiffness and enlargement based on all features, including F1, Sensitivity, Precision, Specificity and NPV. For LASI, we selected RF as the ML method for feature selection. As shown in Fig. 3A, the top 20 features included CC, age, BNP, NLR, HbAIc, FPG, RDW_SD, HsCRP, eGFR, apoB, HT-duration, ARRL, TG, BMI, RenL, apoA, AldL, LPa, UA, and LDL-C. For LAVI, we also selected RF for feature selection. As shown in Fig. 3B, the top 20 features included BNP, ARRL, RenL, FT4, age, eGFR, HT-duration, CC, AldL, HsCRP, TSH, PLR, FT3, UA, NLR, 24h urinary potassium, AngIIL, LPa, RDW_SD, FPG.
Table 2
F1, sensitivity, precision, specificity, and negative predictive value of LASI and LAVI on test set prediction from optimized models based on all features, combined over repeats, and folds
| F1 | Sensitivity (TPR) | Precision (PPV) | Specificity (TNR) | Negative predictive value (NPV) |
LASI | | | | | |
RF | 0.71 (0.52, 0.88) | 0.68 (0.40, 0.94) | 0.78 (0.47, 0.92) | 0.81 (0.44, 0.96) | 0.74 (0.54, 0.93) |
GBDT | 0.65 (0.38, 0.82) | 0.66 (0.24, 0.88) | 0.69 (0.45, 0.93) | 0.73 (0.48, 0.95) | 0.71 (0.44, 0.90) |
XGBoost | 0.66 (0.30, 0.84) | 0.57 (0.20, 0.77) | 0.80 (0.54, 0.93) | 0.86 (0.56, 0.96) | 0.70 (0.57, 0.85) |
LightGBM | 0.71 (0.52, 0.86) | 0.69 (0.47, 0.89) | 0.74 (0.53, 0.93) | 0.78 (0.57, 0.95) | 0.75 (0.56, 0.89) |
LAVI | | | | | |
RF | 0.47(0.22, 0.68) | 0.55 (0.18, 0.85) | 0.51 (0.16, 0.86) | 0.71 (0.39, 0.96) | 0.80 (0.56, 0.90) |
GBDT | 0.42(0.18, 0.67) | 0.49 (0.13, 0.78) | 0.42 (0.17, 0.80) | 0.70 (0.37, 0.96) | 0.77 (0.59, 0.92) |
XGBoost | 0.44(0.16, 0.67) | 0.60 (0.18, 0.88) | 0.37 (0.17, 0.63) | 0.58 (0.12, 0.84) | 0.77 (0.38, 0.94) |
LightGBM | 0.47 (0.21, 0.70) | 0.58 (0.25, 0.86) | 0.41 (0.15, 0.67) | 0.66 (0.26, 0.85) | 0.80 (0.60, 0.95) |
ML performance and Model selection
The top 20 selected variables were used as inputs for four tree-based models (RF, GBDT, XGBoost and LightGBM). After the hyperparameters tuning based on 10-fold cross validation on the training set, we could get the performance of all models on test set. As shown in Table 1 (part), for predicting LA stiffness, GBDT exhibited the best AUC/ROC (0.85, 95% CI 0.70–0.94) (Fig. 4A), and AUC/PR (0.86, 95% CI 0.73–0.94) (shown in Supplemental Fig. S3). For predicting LA enlargement, similar as the model which included all features, RF showed the best AUC/ROC (0.75, CI 0.57–0.92) (shown in Fig. 4B), and AUC/PR (0.60, CI 0.22–0.88) (shown in Supplemental Fig. S4). Table 3 shows the other major evaluation metrics of the test set for all ML models including the top 20 selected features for LASI and LAVI.
Table 3
F1, sensitivity, precision, specificity, and negative predictive value of LASI and LAVI on test set prediction from optimized models, based on top selected features, averaged over repeats, and folds
| F1 | Sensitivity (TPR) | Precision (PPV) | Specificity (TNR) | Negative predictive value (NPV) |
LASI | | | | | |
RF | 0.65 (0.38, 0.81) | 0.60 (0.31, 0.87) | 0.74 (0.50, 0.90) | 0.81 (0.65, 0.96) | 0.70 (0.48, 0.88) |
GBDT | 0.73 (0.57, 0.85) | 0.70 (0.50, 0.88) | 0.78 (0.50, 0.93) | 0.81 (0.57, 0.96) | 0.75 (0.58, 0.91) |
XGBoost | 0.71 (0.52, 0.88) | 0.66 (0.44, 0.89) | 0.80 (0.60, 0.93) | 0.85 (0.59, 0.96) | 0.74 (0.59, 0.91) |
LightGBM | 0.70 (0.50, 0.84) | 0.67 (0.40, 0.90) | 0.77 (0.47, 0.93) | 0.80 (0.50, 0.96) | 0.74 (0.47, 0.92) |
LAVI | | | | | |
RF | 0.44 (0.15, 0.73) | 0.51 (0.13, 0.91) | 0.50 (0.19, 0.83) | 0.73 (0.32, 0.96) | 0.79 (0.60, 0.94) |
GBDT | 0.44 (0.20, 0.64) | 0.50 (0.20, 0.73) | 0.42 (0.17, 0.67) | 0.70 (0.33, 0.93) | 0.77 (0.50, 0.92) |
XGBoost | 0.42 (0.11, 0.64) | 0.49 (0.13, 0.83) | 0.40 (0.10, 0.67) | 0.70 (0.04, 0.88) | 0.76 (0.50, 0.92) |
LightGBM | 0.44 (0.18, 0.67) | 0.47 (0.17, 0.82) | 0.45 (0.17, 0.75) | 0.77 (0.44, 0.93) | 0.79 (0.62, 0.90) |
In summary, for LASI, GBDT model exhibited the best AUC/ROC, AUC/PR, F1, Sensitivity, and NPV, but lower Precision and Specificity than XGBoost. Therefore, the GBDT model was taken to predict LA stiffness. The RF model was taken for predicting LA enlargement. The RF model showed the best AUC/ROC, AUC/PR and Sensitivity, Precision and NPV, but slightly lower F1and Specificity than LightGBM. Overall, the power of ML models for predicting LA enlargement was not as good as predicting LA stiffness.
Supplementary Table S3 shows the optimized hyperparameters GBDT for LASI and RF for LAVI.
Model explanation based on SHAP
SHAP summary plot was applied on GBDT and RF model to identify feature contribution to LA stiffness and LA enlargement, and the top 20 features in the ensemble model were displayed in Fig. 5. In both figures, each feature was analyzed independently, and each point represented one patient. The position in the x-axis demonstrated risk factors (> 0) or protective factors (< 0). The point color corresponded to the value of each variable, from blue to red representing low to high value. As shown in Fig. 5A, age, CC, BNP, FPG, TG, ARR, BMI and HsCRP have more significant impacts on LA stiffness. As shown in Fig. 5B, BNP, ARR, FT4, the plasma concentration of Rennin, CC, age, eGFR, and TSH have more significant impacts on LA enlargement.
SHAP values also revealed the interactions between variables (shown in Supplementary Fig. S5 and Fig. S6). Age played the most important role in LASI with the cutoff value around 52 years old, and the older the patient, the stiffer the left atrium. Furthermore, age and NLR interacted, and the increased NLR seemed to partly offset the beneficial effect of the younger age on LA stiffness. There were also the important interaction effects between CC and UA, BNP and CC, as well as FPG and age. Furthermore, as shown in supplementary S5 B and C, the interaction effect of CC and UA between 0.8-1.0 of CC, and the interaction effect of BNP and CC between 20–50 of BNP, were minimal (close to 0) on LASI. As shown in Supplementary Figure S6, BNP played the most important role in LAVI with the cutoff value around 36.3. Furthermore, there were significant interaction effects between the BNP and the concentration of aldosterone, ARR and FT4, FT4 and age, as well as the concentration of renin and TSH.