A Deep-learning Algorithm With the Real World Validation for Detecting Acute Myocardial Infarction

The initial detection and diagnosis of ST-segment or non-ST-segment elevation myocardial infarction (STEMI or NSTEMI) denitely rely on a 12-lead electrocardiogram (ECG). Delay or misdiagnosis is not unusual by subjective interpretation. Our aim is to develop a DLM as a diagnostic support tool to detect MI based on a 12-lead ECG and to evaluate the performance of this model. Methods This study included 1,051 ECGs from 737 coronary angiography (CAG)-validated STEMI patients, 697 ECGs from 287 CAG-validated NSTEMI patients, and 140,336 not-MI ECGs from 76,775 patients at emergency departments. DLM was trained and validated for the performance using 80% and 20% of the ECGs, respectively. A human-machine competition was conducted. The area under the receiver operating characteristic curve (AUC), sensitivity, and specicity were used to evaluate the performance of DLM and experts. STEMI versus not-STEMI, and MI versus not-MI were evaluated by DLM. 3 The standard input format of the DLM is a length of 1,024 numeric sequences, but the original length of our 12-lead ECG signals is 5,000. In the training process, we randomly cropped a length of 1,024 sequences as input. For the inference stage, 9 overlapping lengths of 1,024 sequences based on interval sampling were used to generate a prediction and averaged as the nal prediction. Due to the scarcity of MI cases in our study, an oversampling process was implemented to ensure that rare samples were adequately recognized. The settings for the training model were as follows: (1) Adam optimizer with standard (β1 and a batch size of 36 (2) rate of and (3) validation by DLM were better than those of the participating experts. ECG is the timeliest tool among all objective detection methods for MI. However, the low sensitivity and disagreement in interpreting ECGs between physicians are issues for detecting STEMI and NSTEMI. The sensitivity of manual interpretation for MI detection using a 12-lead ECG is only from 61 to 74% with the specicity ranging from 72 to 89.0%.[32-35] In contrast to previously prehospital computer algorithm interpretation for STEMI with the sensitivity of approximately 69%.[36-38] Our DLM provides extraordinary performance, which supports decision-making systems in clinical practice.


Abstract Background
The initial detection and diagnosis of ST-segment or non-ST-segment elevation myocardial infarction (STEMI or NSTEMI) de nitely rely on a 12lead electrocardiogram (ECG). Delay or misdiagnosis is not unusual by subjective interpretation. Our aim is to develop a DLM as a diagnostic support tool to detect MI based on a 12-lead ECG and to evaluate the performance of this model.

Methods
This study included 1,051 ECGs from 737 coronary angiography (CAG)-validated STEMI patients, 697 ECGs from 287 CAG-validated NSTEMI patients, and 140,336 not-MI ECGs from 76,775 patients at emergency departments. DLM was trained and validated for the performance using 80% and 20% of the ECGs, respectively. A human-machine competition was conducted. The area under the receiver operating characteristic curve (AUC), sensitivity, and speci city were used to evaluate the performance of DLM and experts. STEMI versus not-STEMI, and MI versus not-MI were evaluated by DLM.

Results
The AUCs of DLM for identifying STEMI and MI were 0.976 and 0.944 in the human-machine competition, respectively, which were signi cantly better than those of our best clinicians. In the real world setting, DLM presented with AUC of 0.995/0.916 with corresponding sensitivities of 96.9%/77.0%, and speci cities of 96.2%/92.9% in the identi cation of STEMI and MI, respectively. Furthermore, DLM demonstrated su cient diagnostic capacity for STEMI without the aid of troponin I (TnI) (AUC= 0.996) with corresponding sensitivity and speci city of 98.4% and 96.9%. The AUC of combined DLM and the rst recorded TnI for the detection of NSTEMI were increased to 0.978 with corresponding sensitivity and speci city of 91.6% and 96.7%, which was better than that of DLM (0.877) or TnI (0.949) alone.
Conclusions DLM may serve as a diagnostic decision tool to assist intensive or emergency medical system-based networks and frontline physicians in identifying STEMI and NSTEMI in a timely and precise manner to prevent delay or misdiagnosis, and thereby to facilitate subsequent reperfusion therapy.

Background
Acute myocardial infarction (AMI) remains a major public health issue despite advances in diagnosis and management globally. [1] AMI refers to an abrupt cause of an unmet need of coronary blood supply to the myocardium. Based on the presentations of electrocardiogram (ECG), it is categorized mainly into two distinct populations: ST-segment elevation myocardial infarction (STEMI) and non-ST-segment elevation myocardial infarction (NSTEMI). [2] STEMI with ECG presentations of ST-segment elevation over infarcted areas indicates acute complete coronary occlusion that warrants prompt aggressive therapeutic strategies for coronary reperfusion of an occluded infarct-related artery (IRA) to prevent a cardiac disaster. [3] A delay in reperfusion therapy is signi cantly associated with an increase in subsequent mortality. [4][5][6] Similarly, as for NSTEMI with a high risk pro le of unstable condition, an invasive reperfusion strategy should be adopted to prevent a worse outcome. [2,7] However, prompt management depends on rapid recognition and precise diagnosis. The diagnosis of AMI requires a syndrome indicative of myocardial ischemia with some extents of myocardial necrosis detected by ECG and cardiac biomarkers. Even though the established criteria for the diagnosis of AMI, it is still a critical challenge for emergent physicians to rapidly recognize. [8] Previous studies have reported a missed rate of diagnosis of AMI at rst medical contact that ranges from 2 to 30%. [9][10][11][12] The failure to identify high-risk ECG ndings in patients with AMI results in lower quality care and higher adverse prognosis. One of the most leading causes in the diagnostic process was incorrect interpretation of a diagnostic test. [13,14] Systematic processes to improve ECG interpretation may have important implications for treatment and outcomes. Since the principal diagnostic tool for patients with suspected AMI is a 12-lead ECG, a more detailed analysis of the ECG could speed up this process signi cantly.
The current arti cial intelligence revolution started by deep learning model (DLM) has provided us with an unprecedented opportunity to improve the health care system, and DLMs have been proven to be effective in medical applications. [15][16][17][18] DLM have been con rmed to surpass the cardiologist level on ECG interpretation when they are trained by large annotated ECG datasets. [19][20][21] To our knowledge, the available ECG databases of AMI are relative small. [22] Our study aimed to develop a DLM to timely, objectively and precisely diagnose AMI by ECG. DLM which learned more than 100,000 ECGs associated MI exhibited excellent diagnosis power in the detection of MI by ECG. Facilitated by the system's powerful computing ability, the performance of the trained model was compared with that of different levels of participants including cardiologists, emergency physicians, residents and medical students. We also evaluate the diagnostic power for STEMI and NSTEMI by DLM and conventional cardiac troponin I (TnI).

Study design and setting
This was a single center, retrospective and case-control study. The data were provided from the Tri-Service General Hospital, Taipei, Taiwan, and the retrospective research was ethically approved by the institutional review board (IRB NO. 2-107-05-168). Our hospital has built an electronic health system for collecting ECGs and the records from January 1, 2012, to December 31, 2018.

Study population
The MI patients were collected from records at the cardiac catheterization lab; they had received coronary angiography (CAG) to rule in type I MI and to con rm the IRA in STEMI. [23] There were 1051 ECGs before primary percutaneous coronary intervention from 737 STEMI cases. For NSTEMI cases, ECG records were collected before CAG, and 697 ECGs from 287 NSTEMI cases were included in this study. Right side or posterior ECG records were excluded. Not-MI ECGs were collected from patients in the ED during the same period. Patients with a history of AMI or elevated TnI were excluded in the not-MI population. A total of 76,775 patients with 140,336 ECGs were de ned as not-MI in this study. We divided these cases into development (80%) and validation (20%) cohorts by date. The ECGs in the development cohort were excluded in the validation cohorts. There were no overlapping patients between these two cohorts.
Data collection ECG recordings were collected using a Philips 12-lead ECG machine (PH080A). The ECG signal was recorded in a digital format. The sampling frequency was 500 Hz with 10 seconds recorded in each lead. Patient characteristics and laboratory tests were collected from our electronic medical records. The timely nearest laboratory data were assigned for each ECG record. Because the ECG records were sometimes conducted in a related short time period, some ECGs from the same patients shared the same patient characteristics and laboratory data.

Implementation of the DLM
We have developed a DLM with 82-layer convolutional layers and an attention mechanism. The technology details, such as the model architecture, data augmentation, and model visualization, were described previously. [21] We used the same architecture to train two new deep learning models for MI detection and location analysis of STEMI. The rst deep learning model was trained via full samples with 3 categories, including STEMI, NSTEMI, and not-MI, and the output of this model was a 3-class softmax output. The second deep learning model was trained via STEMI ECGs, and the output of this model was a 4-class softmax output for location analysis.
The standard input format of the DLM is a length of 1,024 numeric sequences, but the original length of our 12-lead ECG signals is 5,000. In the training process, we randomly cropped a length of 1,024 sequences as input. For the inference stage, 9 overlapping lengths of 1,024 sequences based on interval sampling were used to generate a prediction and averaged as the nal prediction. Due to the scarcity of MI cases in our study, an oversampling process was implemented to ensure that rare samples were adequately recognized. The settings for the training model were as follows: (1) Adam optimizer with standard parameters (β1 = 0.9 and β2 = 0.999) and a batch size of 36 for optimization; (2) a learning rate of 0.001; and (3) a weight decay of 10−4. The 100th epoch model was used as the nal model, and the presented performance in the validation set was only evaluated once.

Human-machine competition
We evaluated the performance of participant physicians using a sub-validation set. This sub-dataset included 174 STEMI, 138 NSTEMI, and 138 not-MI ECGs. In STEMI, based on the IRA, it was further classi ed into the left main coronary artery (LMCA), left anterior descending artery (LAD), left circum ex artery (LCx), or right coronary artery (RCA). There were six visiting staff, ve residents, and six medical students who participated in the competition. The physicians had no possible access to patient information for further diagnosis. The responses they provided were entered into an online standardized data entry program. We calculated their sensitivities, speci cities, and kappa values to compare their results with those of the DLM.

Statistical analysis
We presented their characteristics as the means and standard deviations, numbers of patients, or percentages, where appropriate. They were compared using either Student's t-test or the chi-square test, as appropriate. The statistical analysis was carried out using the software environment R version 3.4.4.
All analyses were based on ECGs but not patients. The described statistical analyses are shown in the Supplement, and we used a signi cance level of p < 0.05 throughout the analysis. The primary analysis was to evaluate the performance of the DLM and clinicians in MI and STEMI identi cation in a human-machine competition. Receiver operating characteristic (ROC) curve analysis and the area under curve (AUC) were applied to evaluate the competition results. Because the proportions of STEMI, NSTEMI, and not-MI cases were distorted in the competition set, we reweighted the samples via the proportions in the hypothetical real world (0.1%, 0.2%, and 99.7% of STEMI, NSTEMI, and not-MI cases, respectively). [24][25][26] The secondary analyses were performed on the whole validation cohort. We tried to include more clinical information, such as patient characteristics and laboratory tests, to improve the model performance. A multivariable logistic regression model was used to integrate the DLM and clinical information. A series of logistic regression models identi ed the effects of different clinical information on the performance in STEMI and MI identi cation. The AUC values based on the ROC curve were applied to evaluate the changes in model performance.

Results
The baseline characteristics of cohorts.
The development and validation cohorts included records from 58,056 and 19,743 patients, respectively, and the characteristics and laboratory results are shown in the Table 1. Patients in the validation cohort were signi cantly older, had more comorbidities, had an impaired estimated glomerular ltration rate, impaired alanine aminotransferase, lower TnI, higher glucose and low-density lipoprotein cholesterol than those in the development cohort. The development/validation cohorts consisted of 860/191, 559/138, and 109,904/30,432 ECGs from STEMI, NSTEMI, and not-MI, respectively. The LAD and RCA were the most commonly identi ed IRA in STEMI. Patients with STEMI were more likely to be male, more overweight, had more prior coronary artery disease (CAD), had higher TnI, and more impaired lipid pro les than those in the not-MI group. Patients with NSTEMI were more likely to be male, older, had more prior CAD, and comorbidities, higher cardiac biomarkers, and more impaired lipid pro les than those in the not-MI group.

Prediction of STEMI, MI and not-MI
The results of the human-machine competition were summarized in Figure 1. The AUC of DLM in the human-machine competition involving 450 ECGs were 0.976 and 0.944 for the detection of STEMI and the discrimination of MI and not-MI, respectively. The corresponding sensitivities and speci cities for STEMI/MI detection were 89.7%/83.7% and 94.6%/95.7%, respectively. By contrast, the sensitivities and speci cities for STEMI/MI detection among human experts ranged from 57.7-93.1%/46.5-99.4% and 42.9-95.6%/5.8-97.8%, respectively, which were lower than those of DLM. We further reweighted the samples via the proportion of the hypothetical real-world settings, and the AUC of DLM in the detection of STEMI and MI were 0.995 and 0.916, respectively in the hypothetical real world. There were also no experts who were better than DLM in this setting. The precision-recall ROC (PRROC) curve analysis demonstrated the feasibility in an automatic ECG screening system, which revealed that the AUC of DLM for STEMI and MI detection were 0.586 and 0.300, respectively, in the hypothetical real world. DLM achieved 63.2% precision and 50.3% recall using the appropriate cutoff point; these values were signi cantly better than those of all participants in the discrimination of STEMI and not-STEMI.
Consistency analysis of the human experts and DLM and their performance rankings in the human-machine competition were conducted ( Figure  2). DLM achieved the best global performance (kappa = 0.645) (Figure 2A). Intriguingly, among 6 medical students, one (M6) had the best performance (kappa = 0.438) owing to superior not-MI interpretation. Among 5 residents, one (resident in the ED) had the poorest performance (kappa = 0.258) owing to overdiagnosis of not-MI as MI ECGs. Most visiting staff had relatively good detection of STEMI but poor discrimination of NSTEMI and not-MI. The capacities of MI detection were divided into three clusters as shown in the heat map: visiting staff, residents and medical students ( Figure 2B).
The analysis of infarct related artery of STEMI DLM also achieved the best global performance (kappa = 0.629) in the IRA of STEMI (Additional le 1: Figure S1). DLM achieved the best global performance (kappa = 0.629) for the IRA detection of STEMI. Both the LAD and RCA were easily detected by the DLM and clinicians. The LCx had troublesome interpretation. The LMCA was only correctly detected by medical students.

Consistency assessments of MI ECGs
Selected STEMI ECGs in the human-machine competition were shown in Figure 3. A typical ECG of STEMI was consistently detected as STEMI with an IRA of the LAD by both DLM and the clinicians (Figure 3 Case A). A total of 10 ECGs were detected as not-STEMI by DLM. Five of ten misdiagnosed by DLM were correctly recognized as STEMI by the best cardiologists ( Figure 3 Case B), and the remainder were misdiagnosed by both DLM and the best cardiologists (Figure 3 Case C). DLM could identify ECGs as STEMI that expert cardiologists had misdiagnosed (Figure 3 Case D). Among 138 NSTEMI ECGs in the human-machine competition, 58 cases were detected as not-MI by DLM, with an accuracy of 58.0%, which was worse than the 75.4% accuracy of the best cardiologists. This was due to a more conservative MI diagnostic strategy by DLM. The speci city of 96.4% of DLM in 138 not-MI cases was much better than that of 82.6% and 64.5% of the two best cardiologists. After adjustment of the speci city, the misdiagnosis of NSTEMI cases by DLM was obviously less than that by cardiologists (Table 2). Nevertheless, DLM still offered the best performance in the detection of MI by ECG under the standardization of the best cardiologists.
ECG lead-speci c analysis ECG leads were speci cally analyzed for the detection of STEMI and MI in the hypothetical real world (Additional le 1: Figure S2). Lead III, V2, aVL, and V3 demonstrated better performance than other leads for the detection of STEMI, with the AUC of 0.913, 0.913, 0.911, and 0.908, respectively. For the detection of MI, V4, Lead I, and V3 demonstrated better performance, with the AUC of 0.841, 0.825 and 0.825, respectively. Lead-speci c PRROC curve on the detection of MI and STEMI in the hypothetical real world (Additional le 1: Figure S3), and on the IRA of STEMI (Additional le 1: Figure S4) were analyzed. Lead-speci c PRROC curve analysis demonstrated the best performance for the detection STEMI with the AUC of 0.300 on aVL. Moreover, lead-speci c PRROC curve analysis on the IRA of STEMI demonstrated the best performance for the LAD with the AUC of 0.970, 0.955, and 0.953 on V4, V2, and V3, respectively; that for the RCA yielded the AUC of 0.995, 0.978, and 0.966 on aVL, Lead III, and aVF, respectively.
Logistic regression analysis of MI, STEMI, and NSTEMI The univariate and multivariate logistic regression analyses in the development cohort revealed that male, prior CAD, troponin I, hemoglobin, total cholesterol and low density lipoprotein were independent risk factors for the detection of MI, STEMI and NSTEMI (Additional le 1: Figure S5).

Diagnostic value analysis
We evaluated the algorithm performance after adjusting for signi cant patient characteristics, disease histories, and laboratory results to ensure consistency across a wide range of putative confounding variables in the validation cohort. DLM had signi cantly better performance than the use of troponin I alone to detect STEMI with the AUC of 0.996. The corresponding sensitivity and speci city are 98.4% and 96.9%, respectively. However, the use of troponin I alone had signi cantly better performance than DLM to detect NSTEMI. The AUC of combined DLM and the rst recorded TnI for the detection of NSTEMI were increased to 0.978, with the corresponding sensitivity and speci city are 91.6% and 96.7%, respectively, which was better than that of DLM (0.877) or TnI (0.949) alone ( Figure 4). It is enough to detect STEMI using the DLM alone, and the addition of patient characteristics did not signi cantly improve the performance. However, troponin I was found to improve the diagnostic accuracy for NSTEMI, and the improvement was better than the combination of all additional characteristics (Additional le 1: Figure S6).

Discussion
In this study, we established a DLM to precisely detect STEMI and MI through ECG analysis, which applied a deep convolutional network to extract notable ECG features with a development cohort of more than 110,000 ECGs. All MI cases were validated by coronary angiography with the identi cation of the corresponding IRA in patients with STEMI. Most importantly, our DLM demonstrated better performance than that of clinicians for STEMI and MI detection with high sensitivities of 89.7%/83.7% and speci cities of 94.6%/95.7%.
The application of deep learning technology in the cardiovascular eld for arrhythmias, dyskalemia, and valvular heart disease had become popularized recently. [19][20][21][27][28][29] However, no large scale study has been designed to apply deep learning technology for MI detection. Previous DLMs for MI detection by ECG were analyzed mainly from the Physikalisch-Technische Bundesanstalt (PTB) diagnostic ECG Database. [30,31] These studies may be limited because they did not have further validation. Moreover, the comparison between DLM and human experts was lacking. In comparison with previous studies, we enrolled the largest clinically validated ECG records for the development and validation processes. Additionally, we further con rmed the role of TnI in assisting with NSTEMI detection by our DLM. All these results point out the strengths of the current study.
The sensitivities and speci cities for STEMI/MI detection by DLM were better than those of the participating experts. ECG is the timeliest tool among all objective detection methods for MI. However, the low sensitivity and disagreement in interpreting ECGs between physicians are issues for detecting STEMI and NSTEMI. The sensitivity of manual interpretation for MI detection using a 12-lead ECG is only from 61 to 74% with the speci city ranging from 72 to 89.0%. [32][33][34][35] In contrast to previously prehospital computer algorithm interpretation for STEMI with the sensitivity of approximately 69%. [36][37][38] Our DLM provides extraordinary performance, which supports decision-making systems in clinical practice.
With the aid of the rst recorded TnI, DLM exhibited excellent diagnostic yield with an AUC of 0.978 for NSTEMI detection, which was signi cantly better than those of DLM or TnI alone, with the AUC of 0.877 and 0.949, respectively. The universal diagnosis of NSTEMI is derived from the clinical presentation, 12-lead ECG, and cardiac troponin. [2] To date, biomarker measurement for cardiomyocyte injury, preferably high-sensitivity cardiac troponin (hsTnI), is mandatory in all patients with suspected NSTEMI due to the high sensitivity and speci city. [2,39,40] However, several concerns should be considered in current practice. First, the guidelines suggest the second cardiac troponin assessment to be performed 1-3 hours after the rst blood test in uncon rmed cases. Repeated time-costly laboratory tests might delay the diagnosis. Second, cardiac troponin might be perturbed in some clinical conditions other than MI. Combined with the information of the rst recorded TnI, DLM allows rapid and powerful NSTEMI detection in high-risk patients.
DLM can objectively conclude highly suspected STEMI based on analyzing and learning a large amount of ECG data. Moreover, subtle changes in the ECG presentation in the acute and early phases of STEMI that were easily missed by clinicians could be correctly recognized. Interestingly, there were two main characteristics of 10 STEMI ECGs that were unrecognized by DLM, including an infarct Q wave with ST elevation in indicated leads and an atypical ST-T change in reciprocal leads related to old MI. Thus, information regarding previously available ECGs and the history of old MI may be needed to further strengthen the capacity of DLM in STEMI detection.
Regarding NSTEMI detection, DLM showed less sensitivity than cardiologists. Several points should be clari ed. Among 58 NSTEMI ECGs unrecognized by DLM, there were several atypical ECG presentations, including intraventricular conduction disorders, ventricular hypertrophy, poor R wave progression, or baseline variant. [41] Even experienced cardiologists could not identify some of these ECGs. Moreover, overdiagnosis of NSTEMI by ECG is commonplace in clinical practice, which may partly explain the high sensitivity and low speci city of the performance of the physicians in this study. [42,43] With the aid of DLM with high speci city in the detection of NSTEMI, clinicians could easily exclude NSTEMI, which reduces subsequent lab tests and ED observation time and guides clinicians to differentiate it from other diagnosis with clinical presentations at that time. As a result, it is worthwhile to increase the ECG training data along with the rst-record cardiac biomarkers to enhance the capacity of DLM in NSTEMI detection in the future.
Our novel DLM has several potential clinical and educational applications. First, DLM could be incorporated into ECG machines in ambulances or remote areas to facilitate telemedicine and shorten the decision time for reperfusion therapies. Second, the developed model can be applied to a wearable device for MI detection, especially for patients with an extremely high risk of atherosclerotic cardiovascular disease. Third, DLM provides decision support and a high-risk alarm system for MI and will help to reduce medical errors in the ICU or ED resulting from intense time pressure or heavy workload and harried staff during the busy working hours. Finally, the application of a DLM in medical education is probably a future trend. [44] Young physicians and medical students could be trained and tested for the detection of MI with currently developed explainable DLM.
Accordingly, our DLM exhibits diagnostic and educational bene ts and promotes healthcare for cardiovascular disease in the near future.

Limitations
Some limitations of this study should be mentioned. First, the human-machine competition was based on a well-design retrospective study. A realworld prospective study should be conducted to verify the clinical impact of DLM. Moreover, only eleven clinicians participated in the competition with DLM. [45] Although their performance in MI detection was relative consistent with that of the previous studies, comparisons should be made with more experts to con rm the superiority of DLM. Second, the studied patients were only enrolled from one academic medical center although the diagnosis and management of MI was followed up according to the guidelines. Multicenter validation is needed to con rm the value and application of this study. Third, the number of NSTEMI cases was not as large as that of STEMI cases, which may limit the capacity for NSTEMI detection with our DLM. Finally, patients only in the ED with both an ECG and a diagnosis of MI were enrolled in this study, which may have led to selection bias and constrained the generalizability of the results.

Conclusion
We established an optimal DLM to detect STEMI and discriminate between MI and not-MI based on 12 lead ECG with an accuracy better than that of clinicians. Integration of a DLM may assist frontline physicians to recognize MI, especially STEMI, in a timely and precise manner to prevent delay or misdiagnosis and thereby provide prompt reperfusion therapy. Further prospective validation with prehospital and in-hospital ECG tests are needed to con rm the performance of our DLM.

Declarations
Ethics approval and consent to participate The institutional review boards of the Tri-Service General Hospital, Taipei, Taiwan granted a waiver of consent for this study (IRB NO. 2-107-05-168).

Consent for publication
Not applicable.

Availability of data and materials
The data that support the ndings of this study are available on request from the corresponding author. The data are not publicly available because they contain information that could compromise research participant privacy.

Competing interests
The authors declare that they have no competing interests.

Authors' contributions
All authors made substantial contributions to the study, were involved in critically revising it for intellectual content and accuracy, and approved the nal version of the article submitted for publication. WCL, CSL CST and TPT conceived and designed the study and drafted the article. CCC, JTL, WSL and SMC acquired and analyzed results. YSL and CCL was responsible for the statistical analyses. CL takes responsibility for the paper as a whole.   The revised item is used to modify the probability of non-MI given by DLM. For example, if an original probability of STEMI/NSTEMI/non-MI is 0.220/0.310/0.470, then the prediction is de ned as not-MI according to the largest probability. However, the revised item is used to let DLM become more sensitive, which is used to modify the probability of not-MI as 0.470 -0.450 = 0.020 as the rst situation. Therefore, the new prediction of this case is de ned as NSTEMI according to the largest revised probability (0.220/0.310/0.020).
b : The sensitivity of STEMI is de ned as the percentage of STEMI cases that are correctly identi ed as STEMI.
c : The sensitivity of NSTEMI is de ned as the percentage of NSTEMI cases that are correctly identi ed as STEMI/NSTEMI.   Selected STEMI ECGs of consistent and inconsistent assessments given by DLM and human experts. Case A shows a typical ECG of STEMI with ST segment elevation and reciprocal lead depression that was consistently detected as anterior wall STEMI with an IRA related to the LAD by both DLM and the clinicians; Case B was misdiagnosed by DLM but was correctly detected as STEMI by cardiologists; Case C revealed an old anterior wall MI that caused misdetection by both DLM and clinicians; Case D was an early-phase STEMI misdiagnosed by cardiologists but correctly detected by DLM.

Figure 4
Comparison of the diagnostic value between DLM based on ECGs and cardiac enzymes in the validation cohort. The ROC curves were generated from the logistic regression analysis using the validation cohort. The p values were the comparison between each AUC and the AUC of DLM.

Supplementary Files
This is a list of supplementary les associated with this preprint. Click to download.