Development and Validation of a Risk Score for Predicting 1-Year Mortality Risk of STEMI Based on a Nationwide Cohort in China

before variable selection. All of 12611 were included for PSM. We performed one-to-ten matching by propensity score calculated using cox regression with a 0.02 caliper width. The PSM results represented 1-year mortality risk to each individual given a set of these covariates (gender, age and comorbidities including atrial brillation, heart failure, COPD, cardiac valvular surgery, peripheral artery disease, Hypertension, lipid metabolism disorders, Diabetes, chronic kidney disease and cerebrovascular disease). We considered it reasonable when standardized mean differences in percent balance improvement less than 0.1.

patients, which is the serious type of ACS. Study reported that the cumulative mortality by 2 years postdischarge of ACS was 5.2%, meanwhile STEMI patients had a mortality of 7.7% in China [4] .
Multiple factors, including a situation compounded by multifarious demographic factors, as well as type and severity of ACS, timely diagnosis and treatment, and evidence based secondary and tertiary prevention strategies, have in uences on prognosis of ACS patients [5,6] . Considering combined in uences of multiple risk factors, risk assessment is a useful tool for assessing prognosis and guiding management of patients with ACS. The Global Registry of Acute Coronary Events (GRACE) risk model [7] is the most common strati cation tool with highest validity for risk strati cation among ACS patients. It was developed by population mostly from North America, South America and Europe [8] . Recently, the EPICOR (long-tErm follow uP of antithrombotic management patterns In acute CORonary syndrome patients) and EPICOR Asia studies used a national wide population including 35.5% Chinese patients to establish a risk-scoring model for 2-year post discharge survival in ACS patients [9,10] . Although prior studies have constructed several risk scoring system for ACS, those risk scoring systems are lack of validation among Chinese STEMI patients. Besides, due to different economic and medical development would lead to different major impact factors of prognosis in STEMI patients, a risk scoring system for Chinese STEMI patients with a comparative performance of long-term outcomes remained to study in detail.
In this study, we aimed to explore relationships between patient demographics, medical history and management, thereby, facilitating the development of a reliable and user-friendly risk-scoring system for individual Chinese STEMI patient one-year survival in a large Chinese population. The subjects analyzed in our study were collected from China STEMI Care Project Phase 2 (CSCAP-2) which has involved more than 20,000 patients hospitalized and discharged following a STEMI event [11] .

Study Population
This study was based on a national healthcare improvement project named China STEMI Care Project Phase 2 (CSCAP-2) [11] . It is a large national prospective cohort study of STEMI patients initiated in 2015.
We collected patients between 2015 and 2016 from 236 hospitals among 23 districts in China. More detail about the project has been described previously. 13018 patients with STEMI were eligible for the study if they were aged 18 years or over; where multiple admissions for acute myocardial infarction (AMI) were recorded per person, only the rst admission was included; admission within 30 days of onset. 407 patients were excluded for missing outcome value. Finally, 12611 subjects were included in the analysis as the development cohort (Fig. 1).

Data collection procedures
All enrolled subjects would answer a structured baseline questionnaire which was administered by a trained interviewer. The questionnaire collected information of age, gender, adverse life habits, demographic characteristics, medical history, clinical symptoms and auxiliary examination results, diagnosis and treatment process, and secondary preventive drugs. History of smoking was identi ed positive as smoked within 1 year. Diagnosis of diabetes, hypertension, cerebrovascular disease and chronic renal failure were based on clinical judgmental. Physical examination, including height, weight, heart rate (bmp) and blood pressure (Systolic blood pressure, SBP; Diastolic blood pressure, DBP) were measured according to a standard protocol. Body mass index (BMI) was calculated as weight (Kg)/ [height (m)] 2 . Information of treatment strategy, and secondary preventive drugs were collected from the medical records.

Outcomes
Patients were followed up from the admission to 1 year after discharge from the hospital. The outcomes during the hospitalization period were collected at the time of discharge from hospital according to the inpatient medical record. After discharge from the hospital, patients were followed up for mortality status in 1 month, 3 months, 6 months and 1 year by telephone. Follow-up outcome was the incidence of allcause death. All-cause death analyzed in this study was the mortality of in-hospital, 6-month follow-up and 1-year follow-up.

Data validation
The validation cohort was enrolled from CSCAP-2 collected between 2017 and 2018 from 236 hospitals among 23 districts. After excluding 33 participants for missing outcome value, a total of 7342 patients were included in the validation analysis ( Fig. 1). We compared the discrimination and calibration between new model and Fox model (one of the GRACE scores) to identify the validation of our scoring system.
The GRACE score provides an estimate of the probability of death within 1 year of hospital discharge in patients with ACS. It was calculated according to the information documented in inpatient medical records.

Statistics analysis
All data represented as mean ± standard deviation (SD) for continuous variables and counts (percentages) for categorical variables. Missing value was imputed by multiple imputation.
As our study was based on real-world data, propensity score matching(PSM) was used to remove confounding bias before variable selection. All of 12611 patients were included for PSM. We performed one-to-ten matching by propensity score calculated using cox regression with a 0.02 caliper width. The PSM results represented 1-year mortality risk to each individual given a set of these covariates (gender, age and comorbidities including atrial brillation, heart failure, COPD, cardiac valvular surgery, peripheral artery disease, Hypertension, lipid metabolism disorders, Diabetes, chronic kidney disease and cerebrovascular disease). We considered it reasonable when standardized mean differences in percent balance improvement less than 0.1.
After PSM, 5676 participants (non-death: death of 5043:633) were matched successful and ready for variables selection through Extreme gradient boosting (XGboost) model. The parameters of XGBoost model was brought into GridSearchCV for traversal tuning. The optimal model was de ned by the following metrics: accuracy(ACC) and area under curve (AUC) of the receiver operating characteristic curve (ROC), value of precision, recall, and F1-Measure. Interpretation of the XGBoost is carried out as a separate process after the model is trained. The parameter that measures the extent to which features contribute to the predictive ability of the model is called feature importance. Our study mainly applied two kinds of feature importance: gain metric of substitution importance (Permutance Importance) and SHapley Additive explanation (SHAP) value in XGBoost feature importance. The number of variables for risk scoring system was chosen by traversal procedure, according to ACC. Finally, the top 8 variables from the ranking were chosen to develop the Cox regression model and the risk scoring system.
Participants were divided into a training set (70%, 8827 patients) and a test set (30%, 3784 patients) by random sampling. In order to construct the risk scoring system, we developed the Cox regression model with rth penalized maximum likelihood estimation, which was particularly suitable for large number of events and the number of covariates [10] . The performance of the model was de ned by and C-statistics and ROC curve. In ROC analysis, Youden index was the value presenting the best sensitivity and speci city for prediction of a given threshold. If the predicting ability is considered reasonable when the AUC is higher than 0.7 and strong when the AUC exceeds 0.8. Expectation (E)/Observation (O) ratios were used to assess model calibration. Usually, E/O ratio between 0.8-1.2 means a good calibration.
According to the nal Cox regression model and the critical mortality rate, we accessed the risk scoring system by constructing a nomogram to show the correspondence table between the total score and the risk prediction probability. 7342 patients were included in the validation analysis. ROC curve, C-index and E/O ratio was examined in the validation cohort for the scoring system external validation. We further compared the discrimination and calibration between new model and Fox model (one of the GRACE scores). We applied the STEMI risk score for risk strati cation as 3 level according to mortality risk for < 1%, 1-3% and ≥ 3%. A Kaplan-Meier survival curve of 1-year all-cause death strati ed by the risk level was explored to validate the practicability of the STEMI risk score.

Baseline Characteristics
A total of 12611 STEMI patients collected between 2015 and 2016 based on CSCAP-2 were included in the analysis. The total mortality was 5.19%.
Overall, the mean age(± SD) of participants were 61.3(± 12.6), and 79.11% were male. The diagnosis of hypertension, diabetes and cerebrovascular disease were the highest prevalence comorbidity as 48.5%,20.66% 10.49%, respectively. Chest pain was the most common symptom (95.85%). Most patients have low Killip class (76.88% of level 1). More details were showed in Table 1. All variables listed in Table 1 were used for variable selection procedure in XGBoost model analysis.

Risk scoring system for 1-year mortality on STEMI patients
The development cohort was divided into training set (70%) and test set (30%). The results of Cox regression analysis using training set are presented in Table 2. Finally, 8827 participates were analyzed in Cox regression model as the training set. independently associated with 1-year mortality ( Table 2).
The 8 risk factors selected by XGBoost selection procedure were used to form mortality risk estimation nomogram (Fig. 3)  Validation of the STEMI risk scoring system A total of 7342 patients with similar distribution with the development cohort were included in the external validation analysis regarded as the validation cohort (Supplement Table 1). Overall, the mortality for 1-year after discharged in the validation cohort is 95.03%. The mean age(± SD) of participants were 61.6(± 12.7), and 77.67% were male. The risk scoring system demonstrated good accuracy in estimating the 1-year mortality, with a c-statistics of 0.89(95%CI, 0.82-0.95) and an E/O ratio of 1.00(95%CI, 0.81-1.23). The performance of this risk score predicting 1-year mortality was satisfactory with accuracy based on AUC of 0.84(95%CI, 0.82-0.86), meanwhile, the sensitivity and speci city were 83.0% and 71.8%.
To further validate the scoring system, we compared the performance of the STEMI risk scoring system with the GRACE score. We developed the GRACE scoring model in the validation cohort. The performance of GRACE score predicting 1-year mortality have a similar performance with the STEMI risk score with an AUC (95%CI) of 0.85(95%CI, 0.85-0.86), meanwhile, the sensitivity and speci city of the GRACE risk score were 83.8% and 69.9 % ( Table 4). Risk strati cation by STEMI risk scoring system We divided patients into 3 groups for low risk, median risk and high risk according to the mortality risk calculated by the risk scoring system for < 1%(score between 0-43), 1-3%(scored between 44-70) and ≥ 3% (scored over 71), respectively. We explored a Kaplan-Meier survival curve of 1-year all-cause death according to the strati cation mentioned above using the validation cohort. The risk strati cation was de ned to have signi cant differences (< 0.001) (Fig. 4).

Discussion
This study presents a risk scoring system which was constructed by nomogram of COX regression including 8 variables selected by XGBoost that predicts 1-year mortality risk of STEMI patients from a nationwide cohort in China, including 244 hospitals. The risk scoring system was comprised of 8 variables: level of Killip class, early reperfusion strategy, use of non-PCI intraoperative anticoagulants, heart rate, gender, age, diagnosis of AWMI and IWMI. It was derived from patients hospitalized from 2015 to 2016 and was validated in patients hospitalized in 2017. The performance of this risk score was satisfactory with discriminatory accuracy based on c statistics in the training set, test set and validation cohort of 0.87, 0.88, and 0.89, respectively.
Previous studies have established several risk scoring systems for estimating the prognosis of STEMI, such as the GRACE score and thrombolysis in myocardial infarction (TIMI) risk score. Those risk scores are derived from studies mainly enrolling patients from Western countries, and both were developed initially only to assess short-term prognoses. Few risk score was developed by Chinese population with STEMI.
Although several studies validated the present risk scores in Chinese STEMI patients, those studies were limited by single-center study [11] . Recently, the EPICOR and EPICOR Asia studies used a national wide population to establish a risk-scoring model for 2-year post discharge survival in ACS patients [9,10] . But the risk score presented a limited clinical value with a c statistics of 0.712 (95% CI, 0.650-0.772) among STEMI patients in China [12] . A risk assessment tool for STEMI patients with a comparative performance of long-term outcomes in Chinese populations still remained to study in detail.
Therefore, in our study, we constructed a scoring system based on a national wide population in China to predict 1-year mortality risk for STEMI.
We compared our risk scoring system with the GRACE score among follow-up outcomes at discharged and 6-month, 1-year long follow-up duration (Table 4.). The GRACE score is the most notable risk score for STEMI with highest vitality [13,14] . It consists of 8 indicators: age, heart rate, SBP, creatinine, level of Killip class and symptoms of cardiac arrest on admission, ECG ST-segment changes and elevated myocardial necrosis markers. Limited study has validated the GRACE score among Chinses patients.
Only few single-center study [11] estimated that the accuracy of the GRACE score in predicting all-cause mortality which was ranged 0.766 to 0.789 for c-statistics. In our study, the results showed that the GRACE score have a similar discriminatory accuracy and lower sensitivity than the STEMI risk score.
More high-risk STEMI patients would be recognized by the STEMI risk score. It is helpful for health management among Chinese STEMI patients by accessing preventive intervention or more frequently follow-up to decrease death risk at the early stage.
The features selected by XGBoost are mostly coherent with previous reports [15] . Older age, male sex and worse Killip class have been constantly emphasized as major risk factors for adverse outcomes in patients with STEMI. A prior study showed that every additional year of age of the patient with AMI will lead to an increasing risk of 9.3% higher on death [16] . Another study using large population-based cohort observed that female has an higher in-hospital mortality risk than male among patients with STEMI (odds ratio, 1.42; 95% CI, 1.24-1.64) [17] . As reported in previous studies, admission HR are predictors of inhospital and long-term mortality. Every increase of 5 bpm. in heart rate was associated with a 29% increased risk of cardiovascular mortality at 1-year follow-up [18] . AWMI and IWMI were pointed to have different prognosis. With AWMI, signi cant reduction in left ventricular function is responsible for the hemodynamic compromise. The myocardial mass supplied by the right coronary artery or left circum ex is usually smaller than that of the left atrial dimension, but with proximal right coronary artery associated IWMI, there may be additional hemodynamic compromise due to right ventricular infarction [19][20][21] . Since the myocardium at risk is greater in patients presenting with AWMI than IWMI, the reported incidence of cardiogenic shock is higher in patients with AWMI compared with those with IWMI [21] .And in-hospital mortality was modestly lower in patients with cardiac shock complicating IWMI vs. AWMI (30.3% vs. 31.9%; OR, 0.80; 95% CI, 0.75-0.86) [19] .
In addition to previous reports, we take an added concern of treatment indicators which have more value in the risk strati cation for long-term prognosis prediction, especially in China [22,23] . Poverty, limited healthcare infrastructure for PCI, and poor accessibility to acute emergency medical services are most important system-level limitations for STEMI care in middle-income countries [24,25] . In 2013, only 36% of AMI patients could receive reperfusion therapy and visited at hospitals equipped with catheterization labs within 12 hours after onset in China. The proportion would be lower among primary hospitals for lacking medical resources [26] . It has been proved that PCI can decrease the 1-year mortality in patients with STEMI [27] . When PCI could not access within 90 minutes, STEMI patients are suggested to have anticoagulants [28] . Under this condition, the factors of early reperfusion strategy and non-PCI intraoperative anticoagulants have import predictive value in China.
Our risk scoring system based on simple patient demographics and subjective symptoms using machine learning and nomogram can be useful for the health management of STEMI patients after discharged.
First, our risk scoring system is easy-to-use. It uses information that can be easily provided, such as age, sex, and pre-hospitalize diagnosis. Previous scoring systems [4,11,15] , including a recently reported deep learning model [29] , require laboratory or radiographic ndings as the main variables. Although such models can be helpful in fully equipped medical facilities, they initially consume a certain amount of medical resources and time. Besides, we used nomogram to present the risk scoring system which can calculate the risk directly using graphs, so that the results are more readable and have higher practical value than conventional models. Second, because the variables were selected by a data-driven method, it can recognize predictive factors of the risk scoring system with su cient accuracy and discrimination.
XGBoost is a newly integrated machine learning algorithm based on decision tree. It is more effective than previous methods by using clever penalization of trees and implementation on single, distributed systems and out-of-core computation. With regularization to prevent over tting and extra parameter de ning loss function, XGBoost is expected to be useful in the eld of real-world data based study.
Previous study [30] demonstrated that XGBoost algorithm showed an improvement in the prediction ability compared with other machine classi ers and conventional risk scores. As our study was drawn from a real-world observation cohort, XGBoost is optimal for our study to identify the independent risk factors of 1-year mortality. The variables included in the risk scoring system were selected by traversal procedure of XGBoost model, according to ACC. This selection method was a data-driven method and can select variables from a large scale of potential factors and can powerfully reduce variance and reduce bias caused by subjective experiences. Then the nomogram was developed based on the selected variables for practical value in clinical application. However, small number of outcomes were observed in the analysis which may cause over matched during the PSM procedure. Extending follow-up time of the cohort or expanding the population scale may improve the results by increasing the observed number of outcomes. We may further validate the risk scoring system in larger scale of population in the future.

Conclusion
We developed a risk scoring system for predicting 1-year mortality risk of STEMI in a large Chinese population. The new score is easy-to-use and demonstrating a good discriminatory accuracy in predicting both short-term and long-term mortality risk in Chinese patients with STEMI. And it has a better sensitivity than the GRACE score.

Abbreviate
Anterior-wall Myocardial Infarction, AWMI Acute coronary syndromes, ACS Available of data and materials  Variable importance ranking according to XGBoost and SHAP value