A total of 3536 patients with acute nontraumatic chest pain and initial cTn tests presented in the participating EDs from August 24, 2015 to September 30, 2017. Some patients were excluded due to denial of informed consent (77) and diagnosis of STEMI (472). There were 88 patients with insufficient information to calculate the GRACE scores, including 74 due to no initial ECG and 14 due to no SBP values. For 13 surviving patients, follow-up contacts were unsuccessful. Eventually, 2886 patients remained for analysis (Figure 1). Baseline characteristics and initial evaluation between patients with and without 30-day MACEs are compared in Table 2. Patients with 30-day MACEs tended to be older, be male, have a higher burden of risk factors and have significantly higher GRACE scores than those without 30-day MACEs (p<0.001).
There were 590 (20.4%) chest pain patients with adjudicated MACEs in 30 days after presentation, including 52 patients (1.8%) who died from all causes, 549 (19.0%) with index AMI, 24 (0.8%) with subsequent AMI, 10 (0.3%) who underwent emergency revascularization, 32 (1.1%) who experienced cardiac arrest and 32 (1.1%) who experienced cardiogenic shock.
Correlation between GRACE scores and actual event rates
All six GRACE models showed good positive linear correlation with the actual MACE rates in patients with undifferentiated chest pain (Figure 2). The GRACE (IHDthMI) and GRACE (IH6mDthMI) exhibited very strong relationships, with r values of 0.913 and 0.920, respectively (p<0.001).
Agreement between observed and predicted probabilities of an event
As shown in Figure 3, the predicted probabilities of an event were much close to the observed event rates across deciles of five GRACE models. And the HLT P values for the GRACE (IH6mDthMI), GRACE (IHDth), GRACE (IH6mDth), GRACE (OH6mDth) and GRACE (OH6mDthMI) were 0.113, 0.446, 0.608, 0.312 and 0.073, respectively. However, the P value of GRACE (IHDthMI) was <0.001.
The ROC curves of all the GRACE models, HEART and TIMI scores are depicted in Figure 4. The AUCs of GRACE (IHDthMI), GRACE (IH6mDthMI), GRACE (IHDth), GRACE (IH6mDth), GRACE (OH6mDth) and GRACE (OH6mDthMI) were 0.83 (0.81, 0.84), 0.82 (0.81, 0.83), 0.75 (0.73, 0.76), 0.73 (0.72, 0.75), 0.72 (0.70, 0.73) and 0.70 (0.68, 0.71), respectively. The AUCs of GRACE (IHDthMI) and GRACE (IH6mDthMI) were equal to the C-statistic of HEART score at 0.82 (0.80, 0.83) and superior to the other GRACE models and the TIMI score (0.71, 0.69-0.73) (p<0.001). The C-statistics of the GRACE models and the HEART and TIMI scores in each participating hospital are presented in Supplementary Table 2.
As shown in Table 3, the GRACE (IHDthMI) improved risk classifications of chest pain patients with positive NRI and IDI over the other GRACE models and the TIMI score (p<0.001), while it presented comparable ability to the HEART score. The same trend was seen in the GRACE (IH6mDthMI) model.
Rule-out and rule-in of events
For discharging low-risk patients safely with sensitivity ≥95% (as shown in Table 4), GRACE (IHDthMI) ≤81 and GRACE (IH6mDthMI) ≤79 could identify 868 (30%) and 821 (28%) patients as low risk, respectively, which was significantly better than other GRACEs and a HEART score ≤3 (22%). Additionally, GRACE (IHDthMI) ≤81 and GRACE (IH6mDthMI) ≤79 ruled out MACEs with an NPV of 0.967 (0.955, 0.979) and 0.965 (0.952, 0.977), respectively, exceeding other GRACEs and comparable to a HEART score ≤3 with an NPV of 0.970 (0.957, 0.983). If the sensitivity was set at ≥98%, the proportion of patients identified as low risk would decrease to 14% for GRACE (IHDthMI) ≤64 and GRACE (IH6mDthMI) ≤61 with no change in the superiority to HEART ≤2 (11%) and TIMI =0 (12%) (Supplementary Table 3). If the sensitivity was improved up to ≥99%, the proportions of low-risk patients would drop below 10% for GRACEs, which were still significantly greater than a HEART score ≤1 (3%) (Supplementary Table 3).
Regarding ruling in MACEs with specificity ≥95% (Table 4), GRACE (IHDthMI) >186 and GRACE (IH6mDthMI) >161 could recognize 12% and 11% of patients as high risk, respectively, which were more than a HEART ≥8 (9%) and a TIMI ≥5 (8%). The sensitivities of GRACE (IHDthMI) >186 and GRACE (IH6mDthMI) >161 were even better than those of HEART ≥8 and TIMI ≥5 without compromising specificity. As shown in Supplementary Table 4, GRACE (IHDthMI) >168 and GRACE (IH6mDthMI) >146 recognized 19% and 18% of patients as high risk, which were smaller than the proportion (22%) identified by the HEART score (7–10). However, the GRACE (IHDthMI) >168 had greater specificity (0.904, 0.892-0.916) and PPV (0.591, 0.550-0.633) than HEART ≥7, with a specificity of 0.870 (0.856, 0.884) and a PPV of 0.534 (0.496, 0.573).
Performance of the mini-GRACE models
The mini-GRACE (IHDthMI), mini-GRACE (IH6mDthMI), mini-GRACE (IHDth), mini-GRACE (IH6mDth) and mini-GRACE (OH6mDth) showed positive linear correlations with the actual MACE rates (r≥0.793, p<0.001). A very strong relationship remained in the mini-GRACE (IH6mDthMI) (r=0.917). The mini-GRACE (IHDth), mini-GRACE (IH6mDth) and mini-GRACE (OH6mDth) had good calibration (p≥0.517) while the other two did not (Supplementary Table 5). The mini-GRACE (IHDthMI) and mini-GRACE (IH6mDthMI) models, with AUCs of 0.82 (0.80, 0.83) and 0.81 (0.79, 0.82), respectively, were still superior to other models in discrimination and reclassification (Supplementary Figure 1) (Supplementary Table 6).