Development and validation of a prediction rule of major adverse cardiac and cerebrovascular event for high-risk STEMI patients after primary percutaneous coronary intervention

Background and Aims We aimed to develop a clinical prediction tool to improve the prognostication of major adverse cardiac and cerebrovascular events (MACCE) among high-risk myocardial infarction (MI) patients undergoing primary PCI. Methods Among 4151 consecutive MI patients who underwent primary percutaneous coronary intervention (PPCI) from FuWai Hospital in BeiJing, China (January 2010 and June 2017), and a prediction rule was derived from derivation and internally validation cohort to predict MACCE after PPCI. Subject must have met at least one clinical criterion and at least one angiographic criterion to be eligible for treatment in the study. The predictive values of markers and clinical variables were assessed with least absolute shrinkage and selection operator (LASSO) regression. The most important variables were included in the score with weights proportional to the model coe ﬃ cients. Results The full model included 7 variables, and the risk score was total 160 points. The full model had similar discriminatory value across pre-specied subgroups and was well calibrated. Derivation cohort models predicting MACCE events had C statistics of 0.695 and 0.673, respectively. The areas under curve (AUC) of the survival receiver operator characteristic curve (ROC) were 0.991 and 0.883 in derivation and validation cohort among 3-year follow-up for predicting the MACCE events. The relative high risk group was observed to have signicantly greater likelihood of occurrence of all-caused death, recurrence MI, heart failure, ischemic stroke, hemorrhagic stroke and revascularization compared with the low risk group (p<0.05 respectively). Conclusion The predicted model was internally validated and calibrated in large cohorts of patients with high risk MI receiving primary PCI therapy to predict the MACCE event and showed modest accuracy in derivation and validation cohorts.


Introduction
Early primary percutaneous coronary intervention (PPCI) has now been set up as the rst-line treatment for subjects who has acute myocardial infarction (MI) [1] . A randomized trial of moderate size [2][3][4][5][6][7] showed there is a signi cant increase in major adverse cardio-cerebral events (MACCE) after undergoing the PPCI.
Indeed, in high-risk patients and lesion subsets, including those older than 65 years old, with renal dysfunction, diabetes mellitus (DM), thrombotic target lesion and multi-vessel disease residual atherothrombotic risk remains substantial. Framingham Heart Study investigators have developed various cardiovascular disease risk prediction project which identi ed high-risk patients more precise than the conventional classi cation. It is bene cial and effective that pretreatment risk factors to reduce the risk of cardiovascular disease within patients who are evaluated as high-risk with multivariable prediction equations than treating patients with high levels of single risk factors [8,9] . However, few tools are provided to assess the incidence of MACCE events among high-risk MI subjects undergoing primary PCI to guide long-term risk management. Using these speci c data elements, a new risk score project was created, with which we sought to: 1) de ne major independent predictors of MACCE among MI patients with high-risk after undergoing PPCI; and 2) develop and validate a full pre-procedure risk prediction model which adapted to individuals based on precision medicine, healthcare decisions. We present the following article in accordance with the TRIPOD reporting checklist (Appendix le) [1]. [1] The authors have completed the TRIPOD reporting checklist Material And Methods

Study population -enrollment and randomization
This observational, retrospective cohort study analyzed data from Fuwai Hospital, National Center for Cardiovascular Diseases, Peking Union Medical College & Chinese Academy of Medical Sciences from a total of 4151 consecutive MI patients who underwent primary PCI at FuWai Hospital in BeiJing, China, between January 2010 and June 2017, were enrolled. 48 patients without follow-up data were excluded from the study. Enrollment into the study will require meeting at least one of the clinical inclusion criteria and one of the angiographic inclusion criteria but none of the exclusion criteria, as shown in gure 1.
Clinical criteria included: 1) adult patients ≥65 years of age; 2) female gender; 3) documented PAD or CAD/PAD revascularization; 4) diabetes mellitus; 5) chronic kidney disease;6) troponin positive. The angiographic criteria included: 1) multi-vessel coronary artery disease; 2) target lesion requiring total stent length>30 mm; 3) thrombotic target lesion; 4) bifurcation lesions; 5) left main (≥50%) or proximal LAD (≥70%) lesion; 6) calci ed target lesion. These pre-speci ed criteria were chosen to enroll STEMI patients at high risk for MACCE complications after PCI. Elements are included in validated risks score-projects for either ischemic, bleeding or both types of complications after PCI [10][11][12][13][14][15][16][17][18] . The R software was used to divide the derivation cohort and the validation cohort randomly and proportionally (70%:30%). All patients were referred to the coronary catheterization center with the diagnosis of MI ful lling the criteria for PPCI according to the guidelines [19,20] . The study was approved by the Ethics Committee of Fuwai Hospital, and all patients enrolled will require providing written informed consent for coronary angiography and PPCI. Patient records, including demographics, medical history, physical examination, blood test results, electrocardiography (ECG), echocardiography data, and discharge medication regimen was reviewed.
Blood testing was performed at the clinical laboratory in Fuwai Hospital. Experimental protocols and the process for obtaining informed consent were approved by the appropriate by Fuwai hospital institutional review committee. This investigation conformed to the principles outlined in the Declaration of Helsinki.
We stated that informed written consent was given prior to the inclusion of subjects in the study.

De nitions and primary outcome
The primary outcome for this analysis was MACCE which de ned as the composite of all-cause death, recurrence myocardial infarction, stroke (including ischemic stroke and hemorrhagic stroke), heart failure or target-vessel revascularization. Hypertension was de ned as blood pressure ≥140/90 mmHg in three occasions at rest or previous diagnosis of hypertension and current use of antihypertensive drugs.
Diabetes mellitus (DM) was de ned according to the 75-g oral glucose tolerance test (OGTT), that is, patients were diagnosed with DM if they met one of the following criteria: (i) fasting plasma glucose level of ≥7.0 mmol/L, (ii) 2-h value of ≥11.1 mmol/L in 75-g OGTT, and (iii) casual plasma glucose level of ≥ 11.1 mmol/L. Dyslipidemia was de ned by any of the following parameters: total cholesterol (TC) 5.0 mmol/L, low-density lipoprotein cholesterol (LDL-C) ≥3.0 mmol/L, triglycerides (TG) ≥1.7 mmol/L, highdensity lipoprotein cholesterol (HDL-C) ≥ 1.2 mmol/L (women) or ≥ 1.0 mmol/L (men), or statin treatments. Height and weight were measured by trained medical staff; body mass index was calculated by weight (kg)/height squared (m 2 ). No-re ow phenomenon was de ned as thrombolysis in myocardial infarction (TIMI) ow grade <3 after PPCI. Stroke is de ned by the World Health Organization (WHO) Multinational Monitoring of Trends and Determinants in Cardiovascular diseases (MONICA) standard. Stroke is de ned as a rapidly developing focal or general brain dysfunction which lasts for more than 24 hours or causes death excluding non-vascular causes (such as trauma, metabolic disorders, tumors and any neurological abnormalities caused by CNS infection). According to the imaging examination in the rst week of onset, the neurologist diagnosed stroke included subarachnoid hemorrhage, intracranial hemorrhage, cerebral thrombosis and cerebral embolism. Hemorrhagic stroke includes subarachnoid hemorrhage and intracranial hemorrhage. Furthermore, ischemic stroke includes cerebral thrombosis and cerebral embolism. Transient ischemic attack (TIA) and chronic cerebrovascular disease are not included. The outcome of this study included only initial stroke. Investigators collected data including head CT, head MRI, hospital records from patients during their hospitalization. The STROBE checklist has been provided as a supplementary gure.

Statistical analysis
The normal distribution of outcome variables was con rmed by Kolmogorov-Smirnov tests. Categorical variables are summarized as frequencies (percentages) and compared with person chi-square tests. Continuous variables are presented as the median and compared using independent t-test. Characteristic of the derivation cohort and validation cohort are showed in table 1. The study population was randomly split into a development sample consisting of 70% of admissions and a validation sample consisting of the remaining 30% of admissions. Baseline patient characteristics and variables from coronary angiography and diagnostic catheterization were considered candidate variables and all prespeci ed. Candidate variables had <1.8% missing data except for the use of IABP (27%), ApoA (29.2%) and uric acid (12.6%). The variables included in the least absolute shrinkage and selection operator (LASSO) regression were showed in the appendix table 1 and it was used to screen the independent variables to draw the corresponding nomogram model.
We developed a predicting model using all potential predictive variables selected by LASSO regression. We also developed a risk prediction score by taking the regression coe cients from the pre-procedure model and assigning them an integer weighted associated with the risk factors. The corresponding nomogram model is drawn according to the regression coe cient of the selected independent variables. For the variables selected in the nomogram model, the values of different variables can correspond to different scores on the integral line at the top of the nomogram (the score range is 0-160 points) through the projection of the vertical line, and the total score can be obtained by adding up the scores corresponding to the values of each variable. The cumulative occurrence probability of MACCE events in 3 and 5 years can be obtained from the total score on the prediction line at the bottom of the nomogram.
In order to reduce the over-tting bias, the self -sampling method is used to verify the nomogram model. The Harrell's C-statistic was used to compare discrimination between derivation cohort and validation cohort including 3-year and 5-year. Calibration plots were used to access goodness of t. We draw the survival receiver operator characteristic curve (survival ROC curve) by R language. Survival ROC curves export the best cut-off values and divided into low risk group and high risk group by R language. We conducted the K-M survival analysis between two groups and export the discrepancy result of the analysis. The subgroup of the K-M curves included the all-caused death, recurrence MI and stroke during 3 and 5 years. LASSO method adopts glmnet package of R language for variable selection, and RMS package of the R language for drawing and internal veri cation of nomogram (c-index and calibration chart). The cox regression analysis was performed using the survival package. The main statistical analysis software used in this study is the R language version I 386 3.6.2. Other analyses were performed with SPSS version 20.0 statistical software (SPSS, Inc., Chicago, IL.). A p value <0.05 was considered statistically signi cant. All statistical tests were 2 sided.

Performance and internal validation of new risk prediction equations
The baseline survival probabilities of each model were obtained by the R language version I 386 3.6.2 commands that were utilized to t the models. Calibration performance was assessed graphically to predict 3-year and 5-year MACCE events risk and to plot 3-year and 5-year predicted risk against observed 3-year and 5-year risk. A diagonal line with a slope of 1 represents perfect calibration. Observed 3-year and 5-year risk was obtained by the Kaplan-Meier method, and the slopes of regression lines comparing predicted versus observed 5-year risk were calculated. Standard statistical metrics of model and discrimination performance (R², Harrell's C statistic) were calculated. The calibration and discrimination performance of equations developed in the derivation sub-cohort was assessed in the validation subcohort and compared with the performance of models developed in the entire cohort; baseline survival functions and hazard ratios were also compared. Indicators of internal veri cation include c-index and calibration degree, which respectively represents the prediction accuracy and prediction consistency of the nomogram prediction model. The degree of calibration is represented by a calibration graph. ROC plotting was used for the survival roc package.

Patient demographics of derivation and validation cohort
Between Jan 1, 2010, and Jun 30, 2017, the study population included 4151 men and women and excluded 48 people without following up. After applying inclusion criteria, 3404 high-risk MI subjects remained. The average follow up time was 3 year. Subject must have met at least one clinical and at least one angiographic criteria to be eligible for treatment in the study. 2384 people constituted the derivation cohort and 1020 cases consisted of validation cohort used in these analyses by random allocation ( gure 1). Candidate variables had <1.8% missing data except for the use of IABP (27%), ApoA (29.2%) and uric acid (12.6%). Table 1 displays the baseline patient, procedure, and hospital characteristics of the development and validation samples. There were 578 high-risk MI subjects that had MACCE events in derivation cohort after undergoing PPCI procedures, yielding a MACCE event rate of 24.24%. Of these events, 25.09% were all-caused death; whereas 14.09% were detected due to recurrence MI, 60.07% by revascularization, 3.99% by heart failure, 8.15% by ischemic stroke and 1.39% were hemorrhagic stroke.

Screening risk factors for MACCE by LASSO method
Baseline patient characteristics and variables from coronary angiography and diagnostic catheterization were considered candidate variables and all prespeci ed (appendix table 1). These variables were ltered by the method of LASSO regression. The ltering and cross-validation processes of independent variables are shown in gure 2A1 and 2A2 respectively. Lambda.1se is the lambda value of the optimal e ciency model in the standard error range which gives a model with excellent performance.
The establishment of risk prediction model At this time, a total of 7 independent variables (the subgroup of age, Killip classi cation, ejection fraction, history of CABG, type of lesions, complete revascularization at admission and multi-vessel disease of coronary artery) were included in the predictive model. The forest plot of the variables which conducted by the multivariate cox regression was shown in the gure 2B and the binary decision diagram of the variables was shown in gure 2C. It is necessary to make the classi cation variables into factorization and then use the as.matrix() function to convert the data from the non-matri x format to the matrix format before the R language "glmet" package can call the data. According to the nomogram model ( gure 2D), the score predicting project included 7 variables as the variables of predictive factors.

Clinical Prediction Score
A simpli ed risk score was generated to predict MACCE events. The score, ranging from 0 to 160, assigned points as follows: for patients younger than 40 years, 100 points; for age 40 to younger than 50 years, 80 points; for age 50 to younger than 60 years, 60 points; for age 60 to younger than 70 years, 40 points; for age 70 to younger than 80 years, 20 points; for patients 80 years or older,0; for Killip II, 7 The performance of risk score project The MACCE predicting risk model had good discrimination in both the development and validation samples (c-index, development sample 0.695; validation sample 0.673). The model calibration plot for the full model is shown in Figure 3A-D. There was high concordance between the risk predicted by the models and the observed MACCE events. Calibration is indicated by the estimated risk against survival from Kaplan-Meier analysis. Gray line represents perfect calibration. Figure 3E Survival ROC curves export the best cut-off values and divided into relative low risk group and high risk group by R language. We conducted the K-M survival analysis ( Figure 4A-P) and export the discrepancy result of the analysis. In the group of predicting MACCE events, the two groups displayed signi cant difference in both derivation cohort (p<0.001) and validation cohort (p<0.001) shown in gure 4A-D. In the subgroup of predicting all caused death, it is remarkable difference (p<0.001) between the high risk group and relative low risk group in both development and validation group ( gure 4E-H). Furthermore, when the endpoint was recurrence MI, the logrank p value was less than 0.02 in the 3-year derivation cohort and p less than 0.01 in the 5-year K-M curve in derivation cohort ( gure 4I-L). Finally, we also found distinct discrepancy in predicting the stroke (p<0.05) ( gure4 M-P) events in 3-year and 5-year development and validation cohort.

Discussion
This study developed a clinical prediction score based on clinical and coronary angiology index to help predicting the incidence of long-term MACCE events among MI patients with greater risk factors underwent primary PCI. The MACCE predicting risk model had good discrimination in both the development and validation samples (c-index, validation sample 0.673; development sample 0.695). For patients who divided into relative low risk group and high risk group by best cut-off values in the prediction model study (derivation cohort), the relative high risk group was observed to have signi cantly greater likelihood of occurrence of all-caused death, recurrence MI, heart failure, ischemic stroke, hemorrhagic stroke and revascularization compared with the low risk group. These results suggest that it may be possible to identify individual patients with discordant the incidence of MACCE. Although prediction score project is expected to be applied to the subjects represented by enrollment criteria, inconsistent in setting up treatment risks and bene ts, adjusting treatment according to personal data, provides opportunities for further optimization results in order to maximize bene ts and reduce harm. Yet few equivalent scores are available for use in high-risk patients with acute myocardial infarction undergoing primary PCI to predict the MACCE events. For these patients, cardiac imaging, coronary angiography and advanced biomarkers are routinely available at admission period, so it is convenient to include them in a score for this setting for long-term management.
The thing that matters, a lot of patient characteristics were correlated with long-term of the incidence of MACCE. Many of the predictive elements which we have identi ed have been shown in many other studies to be predictive of MACCE events. For instance, age is consistently associated with an increased risk of the incidence of MACCE [21] , as are other variable like killip classi cation, EF at admission, history of CABG and multi-vessels lesion of coronary artery [22] . In addition to these factors, we also identi ed unique variables not present in other predicting models, such as the complete revascularization and the type of lesion detected by the coronary angiography. The addition of such forecasting factors is a noteworthy superiority where the acuity of clinical presentation is generally not as severe compared with previous models that merely included clinical characteristics. It is conspicuous to minimize the risk of the inappropriate care and management for MI patients with high-risk characteristics who undergoing primary PCI by including these factors. Bleeding complications including hemorrhagic stroke after primary PCI are not rare and it is correlated with an increased short-term and long-term risk of mortality [23,24] . It has been proposed to drop the hemorrhage among higher-risk patient subjects by the using of vascular closure devices, bivalirudin and radial approach which called bleeding avoidance strategies [25][26][27][28] . Therefore, this model can be used to predict the long-term incidence of MACCE events for the high-risk MI subjects' post-PCI, identify leaders and laggards, and ultimately improve the long-term prognostic of primary PCI by making healthcare decisions in the follow-up and encouraging the adoption of taking corresponding measures at admission.
According to the present model, the score predicting project included 7 variables (age, Killip classi cation, ejection fraction, history of CABG, type of lesions, complete revascularization at admission and multivessel disease of coronary artery) as the variables of predictive factors. Previous analyses [29,30] have also identified that the risk factors including age, atrial brillation, female sex, killip classi cation, as well as chronic disease could predict the incidence of stroke within 12-months of PCI which are generally accordance with classic risk variables in the general population. Stroke including ischemic and hemorrhagic are devastating complications with high MACE rates and mortality following PCI. Similarly database derived from the British Cardiovascular Intervention Society (BCIS) has reported ischemic stroke was independently associated with both 30-day mortality and in-hospital MACE by following adjustment for baseline clinical and procedural demographics [31] . Previously, Luke P et al [32] demonstrated that the incidence of stroke among outpatients following percutaneous coronary intervention are higher for younger instead of older comparing to the general population. It is coincident with our score projection that age less than 40 years old contributed the greater weight compared with other groups of age. Furthermore, during the beginning period of cardiac catheterization (1970-1980s), the incidence of cerebrovascular events was ranged from 0.03% to 0.06% [33] comparing to 0.18%-0.44% during the following years. The increasing in the incidence of stroke-complicating PCI might account for extended use of PCI and coronary angiography especially among subjects with severe vascular calci cation [34] . The time of risk assessment post-event and cardiovascular disease is both the main factors to evaluate the performance of the risk score for secondary prevention. The research of CALIBER [35] enrolled 102 023 stable CAD subjects and developed a risk score for people with stable CAD to identify patients at high risk and stand by a management decision.

Study Limitation
Several limitations of our study should be considered in interpreting these results. On the basis of the clinical and angiography inclusion and exclusion criteria of the trial, as well as the single-center and retrospective study design, score project of model should be interpreted with the understanding that patients enrolled in clinical trials may not be completely representative of those cared for in routine practice of primary PCI. The analysis ought to be regarded as exploratory despite the predetermination of the score variables. Therefore, the predictive score should be used with circumspection until further external validation is carried on. Optimal and suitable long-term management of procedural and care should be administered independent of the patient's score to reduce overall MACCE events. Furthermore, the extent and severity of granular measures of atherosclerosis were not available and the situation of receiving ticagrelor or other antiplatelet combinations may in part make a difference to the discrimination of the cohort and have a different risk bene t relationship [36] . Finally, cerebrovascular events are determined by contacting the subjects followed by validation through medical records. In spite of it is probably to cover almost all hemorrhagic stroke and ischemic strokes, it may undervalue stroke incidence if the patients were asymptomatic and not admitted to hospital.

Conclusion
In summary, we developed a risk predicted model for estimating long-term (3-year and 5-year) incidence of MACCE based on clinical parameters and indexes of coronary angiography which suit for high-risk subjects with MI who underwent primary PCI. The score project can be implemented alongside further medical investigations to support therapeutic decision making. This project requires further prospective assessment to evaluate potential impacts on subjects' management, as well as external validation in other cohorts.