General characteristics. From January 2018 to September 2019, a total of 1473 patients with AMI who received phase I cardiac rehabilitation during hospitalization were enrolled in this study. A total of 539 patients remained eligible for analysis. The process of patient recruitment is provided in Supplementary Fig. 1. After random allocation, 379 (70%) patients were assigned to the training cohort, while 160 (30%) patients composed the validation cohort. Among the 539 patients, the median age was 50.0 (45.0–54.0) years, 34 were females (6.3%), 98 had a higher education level (18.2%), and 516 were married (95.7%). Among the patients followed up for twelve months, 431 (80.0%) returned to work, and 344 (63.8%) received phase II cardiac rehabilitation. Table 1 presents the baseline characteristics used for model construction, with the complete baseline characteristics provided in Supplementary Table S1. Apart from income, variables selected for modeling showed no significant differences between the training and validation cohorts.
Table 1
Comparison of Baseline Selected Characteristics and Rework Outcome Between the Training and Validation Cohorts
Characteristic | Overall, n = 539 | Training, n = 379 | Validation, n = 160 | p-value |
Rework, n (%) | 431 (80.0%) | 303 (79.9%) | 128 (80.0%) | > 0.999 |
Age | 50.00 (45.00, 54.00) | 50.00 (45.00, 54.00) | 50.00 (45.00, 54.00) | 0.846 |
Married, n (%) | 516 (95.7%) | 365 (96.3%) | 151 (94.4%) | 0.352 |
BMI, kg/m2 | 26.00 (23.90, 28.40) | 26.00 (23.90, 28.40) | 26.00 (23.98, 28.40) | 0.904 |
Occupation, n (%) | | | | 0.470 |
Blue-collar workers | 270 (50.0%) | 194 (51.2%) | 76 (47.5%) | |
White-collar workers | 118 (21.9%) | 76 (20.1%) | 42 (26.3%) | |
Self-employed | 54 (10.0%) | 39 (10.3%) | 15 (9.4%) | |
Other | 97 (18.0%) | 70 (18.5%) | 27 (16.9%) | |
Income, n (%) | | | | 0.001 |
< 2500 | 262 (48.6%) | 199 (52.5%) | 63 (39.4%) | |
2500–5000 | 184 (34.1%) | 110 (29.0%) | 74 (46.3%) | |
5000 − 1000 | 74 (13.7%) | 54 (14.2%) | 20 (12.5%) | |
> 10000 | 19 (3.5%) | 16 (4.2%) | 3 (1.9%) | |
Hypertension, n (%) | 231 (42.9%) | 161 (42.5%) | 70 (43.8%) | 0.860 |
Anterior wall AMI, n (%) | 197 (36.5%) | 139 (36.7%) | 58 (36.3%) | > 0.999 |
Beta-blocker, n (%) | 13 (2.4%) | 10 (2.6%) | 3 (1.9%) | 0.825 |
FPG, mmol/L | 7.61 (6.42, 10.22) | 7.49 (6.41, 9.86) | 7.73 (6.44, 11.02) | 0.318 |
TG, mmol/L | 1.52 (1.04, 2.24) | 1.54 (1.02, 2.24) | 1.49 (1.08, 2.17) | 0.886 |
AST, U/L | 35.00 (22.00, 81.00) | 36.00 (22.00, 84.50) | 34.50 (22.00, 72.50) | 0.605 |
CR, n (%) | 344 (63.8%) | 239 (63.1%) | 105 (65.6%) | 0.640 |
FPG, Fasting plasma glucose; CR, Phase II cardiac rehabilitation; AST, Aspartate transaminase; TG, Triglyceride; BMI, Body mass index; Beta-blocker, Medication history of Beta-blocker. |
Feature selection. Through the Lasso for feature selection, a Spearman correlation test was conducted on the factors identified by the best lambda value, as shown in Supplementary Figs. 2 and 3. Finally, 12 features, including age, occupation, income, anterior wall AMI, hypertension, fasting plasma glucose (FPG), beta-blocker, married, aspartate transaminase (AST), body mass index (BMI), TG (triglyceride) and phase II cardiac rehabilitation (CR), were identified to construct ML models (Supplementary Fig. 4).
Model performance. These factors were then combined with the LR, RF, XGBoost, SVM, and ANN algorithms, with the algorithm demonstrating the best performance being selected. Each model was optimized by 5 repeats of 10-fold cross-validation or tuned to the best parameters. Table 2 summarizes the performance of each model. In the validation cohorts, the accuracies of LR, XGBoost, RF, ANN, and SVM were 0.719, 0.831, 0.750, 0.669, and 0.763, respectively; the sensitivities were 0.703, 0.875, 0.766, 0.625, and 0.766, respectively; the F1-scores were 0.800, 0.893, 0.831, 0.751, and 0.838, respectively; and the ROC-AUC values were 0.793, 0.791, 0.793, 0.783, and 0.773, respectively. The AUCs for the five models ranged from 0.773 to 0.793, and the receiver operating characteristic (ROC) curves are shown in Fig. 1 (A, B). The LR model performed best, achieving an AUC of 0.793 (95% CI, 0.712–0.874), with 0.719 (95% CI, 0.642–0.787) accuracy, 0.703 sensitivity, 0.781 specificity, 0.928 PPV, and 0.800 F1 score on the validation cohort. Additionally, in the validation part, the XGBoost model achieved the highest accuracy, sensitivity, and F1 score. According to further model evaluation, the Brier score for the ability of the LR model to predict RTW was 0.135 in the validation cohort. Five repeats of Ten-fold cross-validation was also performed on the LR model, and its average accuracy were 0.813, indicating that our model was reliable.
Table 2
Model performance in predicting return to work in the training and validation cohorts.
ML Models | AUC | Accuracy | Sensitivity | Specificity | PPV | NPV | F1 | Brier |
Training cohort | | | | | | | | |
Logistic regression | 0.784 (0.726–0.841) | 0.826 (0.784–0.863) | 0.908 | 0.500 | 0.879 | 0.576 | 0.893 | 0.124 |
XGBoost | 0.814 (0.755–0.872) | 0.797 (0.753–0.836) | 0.819 | 0.711 | 0.919 | 0.495 | 0.866 | 0.116 |
Random Forest | 0.807 (0.754–0.860) | 0.678 (0.629–0.725) | 0.640 | 0.823 | 0.937 | 0.366 | 0.761 | 0.160 |
ANN | 0.836 (0.780–0.892) | 0.807 (0.764–0.846) | 0.822 | 0.750 | 0.929 | 0.514 | 0.872 | 0.111 |
SVM | 0.815 (0.758–0.873) | 0.847 (0.807–0.881) | 0.911 | 0.592 | 0.899 | 0.625 | 0.905 | 0.159 |
Validation cohort | | | | | | | | |
Logistic regression | 0.793 (0.712–0.874) | 0.719 (0.642–0.787) | 0.703 | 0.781 | 0.928 | 0.397 | 0.800 | 0.135 |
XGBoost | 0.791 (0.701–0.881) | 0.831 (0.764–0.886) | 0.875 | 0.656 | 0.911 | 0.568 | 0.893 | 0.130 |
Random Forest | 0.793 (0.711–0.875) | 0.750 (0.676–0.815) | 0.766 | 0.688 | 0.907 | 0.423 | 0.831 | 0.158 |
ANN | 0.783 (0.696–0.869) | 0.669 (0.590–0.741) | 0.625 | 0.844 | 0.941 | 0.360 | 0.751 | 0.135 |
SVM | 0.773 (0.684–0.861) | 0.763 (0.689–0.826) | 0.766 | 0.750 | 0.925 | 0.444 | 0.838 | 0.160 |
ML, Machine learning; AUC, Area under the curve; XGBoost, Extreme gradient boosting; ANN,Artificial neural network; SVM, Support vector machine. |
To determine the major predictors of return to work in our cohort, the importance of each permutation feature was measured from the LR, RF, XGBoost, ANN and SVM models. In the XGBoost model, Supplementary Fig. 5 shows that the 5 most important variables were occupation, age, income, TG, and FPG. In the LR model, Supplementary Table S2 shows that the 5 most important variables were occupation (coef = -1.785, OR = 0.168), marital status (coef = 1.484, OR = 4.409), beta-blocker history (coef = -1.134, OR = 0.322), phase II cardiac rehabilitation (coef = 0.667, OR = 1.948), and income (coef = 0.637, OR = 1.890). In the RF model, Supplementary Fig. 6 shows that the 5 most important variables were occupation, age, AST, FPG, and income. In the ANN model, Supplementary Fig. 7 shows that the 5 most important variables were occupation, beta-blockers, anterior wall AMI, income, and marital status.
Nomogram. Since the LR model performed best, with an AUC of 0.793, it was transformed into a dynamic nomogram for easier clinical application. From the 12 candidate factors identified after LASSO feature selection, a stepwise approach based on the Akaike information criterion (AIC) was used to select the optimal model (Supplementary Table S3), which eventually included 6 factors (age, occupation, anterior wall acute myocardial infarction, TG, AST, and phase II cardiac rehabilitation) that formed the dynamic nomogram (Fig. 2A). Points were assigned to each predictor by drawing a vertical line to the “points” axis, and then the total points were calculated as the sum of them. The dynamic nomogram with an intuitive web-based interface was also developed so as to facilitate the use for clinicians in clinical practices (Fig. 2B) (Dynamic Nomogram: https://returntowork.shinyapps.io/AMIRTWApp/)