Using Machine Learning to Predict the Requirement for Revascularization in Patients with Chest Pain in the Emergency Department

doi:10.21203/rs.3.rs-690276/v1

Download PDF

Research Article

Using Machine Learning to Predict the Requirement for Revascularization in Patients with Chest Pain in the Emergency Department

https://doi.org/10.21203/rs.3.rs-690276/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 14 Apr, 2022

Read the published version in Journal of Healthcare Engineering →

Version 1

posted

You are reading this latest preprint version

Background: The study aimed to use machine learning algorithms to predict the need for revascularization in patients presenting with chest pain to the emergency department.

Methods: We obtained data from 581 patients with chest pain, 264 who underwent revascularization, and the other 317 were treated with medication alone at 3 months. Using standard algorithms, linear discriminant analysis, and standard algorithms, we analyzed 41 features relevant to coronary artery disease (CAD).

Results: We identified seven robust predictive features. The combination of these predictors gave an area under the curve (AUC) of 0.830 to predict the need for revascularization. By contrast, the GRACE score gave an AUC of 0.68.

Conclusions: This machine learning-based approach predicts the need for revascularization in patients with CAD.

Medical Informatics

coronary artery disease

revascularization

machine learning

Chest pain is among the most common complaints of patients in the emergency or cardiology outpatient department. Because of the severity of the consequences of many etiologies of chest pain, rapid evaluation is critical. Many patients undergo coronary computed tomography angiography or coronary angiography (CAG) because chest pain may signal coronary artery disease (CAD).

Routine use of risk scores might improve decision-making. The Global Registry of Acute Coronary Events (GRACE) score evaluates the outcomes in patients with CAD, particularly those with acute coronary syndrome^[1]. Current guidelines call for aggressive management, including revascularization in at-risk patiens^[2]. Nevertheless, there are data suggesting that patients at low risk of developing ischemic complications are treated overly aggressively, generating the so-called ‘treatment-risk” paradox’ ^[3–5]. For this reason, there is a need for prediction models in addition to the GRACE risk score^[6].

In recent years, we have developed software applications such as data engineering, data architecture, and machine learning (ML). The latter are algorithms that identify patterns embedded in large datasets containing large numbers of variables. ML enables the generation of predictive models and disease classification models^[7]. These technologies already showed great success in electrocardiography^[8] and image analysis^[9].

In the present study, we used an ML algorithm to generate a predictive model to identify patients with chest pain at high risk for cardiovascular events. Such patients would benefit from immediate revascularization either in the cardiology outpatient or emergency department.

Datasets

We used a derivation cohort of 585 adult patients (≥ 18 years) with chest pain who underwent invasive CAG to develop the ML models. These patients were admitted to the emergency department of Anzhen Hospital, Beijing, China, between 1 May 2017 and 31 January 2018. CAG was performed if aortic dissection and pulmonary embolism were tentatively excluded and if the patient exhibited symptoms consistent with CAD or if tests suggested cardiac ischemia. Excluding four patients with missing critical data, a total of 581 patients were finally enrolled. Two interventional cardiologists performed coronary angiographies. All patients provided informed written consent. The institutional review board approved this study.

To assess the performance of the models, we used an external validation cohort including 172 adult patients admitted with CAD from the China Rehabilitation Research Center, Beijing, China, who underwent CAG to assess coronary lesions.

Outcomes

We used the classification ML model to predict the occurrence of each of two outcomes: revascularization and medication alone. Patients undergoing revascularization were defined as having significant coronary artery stenosis; these patients underwent percutaneous coronary intervention or coronary bypass grafting. Patients with no evidence of significant stenotic lesion were treated with medication alone for 3 months. A total of 264 patients underwent revascularization treatment, and the remaining 317 patients received only drug therapy. We trained the classification model with data from existing patients, and then we used the trained model to make predictions based on the data of external patients.

ML method

The ML was run on Python 3.7.3 (www.python.org). The entire process of ML prediction included data preprocessing, classification using support vector machines as the classifier, and verification using a 10-fold cross-validation method. Data preprocessing included discretization of data classified by continuous values according to the range, feature selection by chi-square test, and feature dimension reduction using linear discriminant analysis. The ML method used the sci-kit learn open source library.

Feature selection and data preprocessing

The structured dataset included 41 variables (Table 1). In the training set, there were 581 instances, among which 264 instances were labeled class 1 (revascularization treatment), and 317 instances were labeled as class 0 (medication treatment). In the validation dataset, there are 172 instances with 41 attributes. Of these, 39 instances were class 1 (revascularization treatment), and 133 instances were class 0 (medication treatment).

Table 1

Features for analysis
		Features
Demographic data	1	Gender
	2	Age
Clinical data at emergency or outpatient department	3	SBP
	4	DBP
	5	HR
	6	Arrhythmia
	7	ST-segment changes
	8	Killip classification
History	9	CAD
	10	MI
	11	PCI
	12	CABG
	13	Chest pain
	14	Diabetes
	15	Hypertension
	16	Stroke
	17	Hyperlipidemia
	18	PAD
	19	Smoking
	20	Drinking
	21	Family history of CHD
Laboratory data at emergency or outpatient department	22	WBC
	23	Monocyte
	24	Lymphocyte
	25	RBC
	26	HBG
	27	HCT
	28	PLT
	29	FBG
	30	Hs-CRP
	31	HCY
	32	Uric acid
	33	CRE
	34	BUN
	35	TC
	36	TG
	37	LDL-C
	38	HDL-C
	39	LDH
	40	Cardiac markers change
Calculation data	41	NLR
SBP, systolic blood pressure; DBP, diastolic blood pressure; HR, heart rate; CHD, coronary heart disease; MI, myocardial infarction; PCI, percutaneous coronary intervention; CABG, coronary artery bypass grafting; PAD, peripheral arterial disease; WBC, white blood cell; RBC, red blood cell; HGB, hemoglobin; HCT, hematocrit; PLT, platelet; FBG, fasting Blood Glucose; HCY, homocysteine; CRE, creatinine; BUN, Blood Urea Nitrogen; TC, total cholesterol; TG, triglyceride; LDL, low-density lipoprotein cholesterol; HDL-C, high-density lipoprotein cholesterol; NLR, neutrophil to lymphocyte ratio.

By calculating the chi-squared value of each attribute and the classification result of the training dataset, the chi-squared value of each attribute was sorted (Fig. 1).

To simplify the explanation of the model from the clinical perspective, we set the upper limit of the number of selected features to 20. Of these, seven features were strongly correlated with the outcome event. The seven features selected were neutrophil-to-lymphocyte ratio, ST-segment changes, cardiac markers, lymphocyte count, lactate dehydrogenase, gender, and history of hyperlipidemia.

Dimensionality reduction

After feature selection, we performed dimensionality reduction to reduce the multi-dimensional input data to one dimension. To be specific, we used the using the linear discriminant analysis feature dimension reduction method.

To perform receiver operating characteristic (ROC) curve analysis, we used the decision function of the linear support-vector classifiers to generate probabilistic outputs of the predictions; then, probabilities were used to plot the ROC curves.

Validation

To evaluate the accuracy of the trained classification model, we used 10-fold cross-validation. To determine whether the classifier was overfitted, we prepared a new validation dataset. The new validation dataset underwent the same data preprocessing process (i.e., data discretization, feature selection, and dimension reduction) as the training data. Finally, the preprocessed validation dataset was fed into the classification model for prediction.

The area under the curve (AUC) of ML for the training set showed the best performance, with a value of 0.83; by contrast, the GRACE score AUC was 0.68 (P < 0.05 for the comparison). The precision value for class 0 was 0.88 in the training set and 0.86 in the validation set. The recall was also high for class 0 (medication treatment) in the validation set. The accuracy of the trained model on the training set was 75 (Fig. 2). The AUC for the validation set was 0.79 (Fig. 3). The classification report is shown in Table 2.

Table 2

ML results on the training set and the validation set
SET		Precision	Recall	F1 score	Accuracy	ROC AUC
Training	0	0.88	0.66	0.75	0.75	0.83
Training	1	0.66	0.88	0.76	0.75	0.83
Validation	0	0.86	0.81	0.84	0.76	0.79
Validation	1	0.47	0.56	0.51	0.76	0.79

Calibration

We performed model calibration to calculate the certainty of new observations in either of the established classes. The Brier score for ML was 0.212 before and 0.162 after calibration. This finding suggests a slight difference between the predicted and observed probability of treatment strategies and a good overall fit for the model (Fig. 4).

We used ML algorithms to predict the requirement for revascularization in patients with CAD using only basic clinical information at the time of admission. The AUC for the predictive value was 0.83 in the training set and 0.79 in the external validation set. These encouraging results suggest that our ML algorithm can help develop treatment strategies for individual patients with CAD.

We found that (1) ML was 75% accurate in predicting strategies with an AUC of 0.83 in the training set; in the external data validation, ML reached 76% accuracy with an AUC of 0.79; (2) ML had 88% precision for predicting treatment strategies, especially for medication-treated patients in the training set; and (3) ML reached 86% precision and 81% recall for predicting medication-treated patients in the external data validation.

In the training set, ML's predictive accuracy for patients treated with medication was higher at 0.88; however, the recall was lower at 0.66. For the patients with revascularization treatment, accuracy was 0.66, the recall was 0.88, and overall accuracy was 0.83. When validated with external data, ML's prediction model performed well for patients treated with medication, with accuracy set recalls of 0.86 and 0.81, respectively; however, the prediction for revascularization therapy was poor and performed less well than the prediction for medication therapy. The prediction of overall patient outcome, with a ROC value of 0.83 for ML, was better than the GRACE score of 0.68. We also found that, with proper calibration, the prediction of outcome events can be enhanced. Implementation of ML models in clinical settings can automate selecting candidates who might benefit most from additional diagnostic testing while avoiding the need for time-consuming and unnecessary routine clinical steps.

Correctly identifying patients at high risk who may benefit from appropriate treatment will improve patient clinical outcomes. The GRACE risk score is a validated predictor of adverse outcomes in CAD patients, and recent studies showed that the GRACE score could assess the severity of coronary artery stenosis in patients with CAD^{[10, 11]}. Current guidelines recommend the GRACE risk score to perform risk stratification in CAD, especially for patients with acute coronary syndrome^[14]. Even though the GRACE score is easy to implement, the score in isolation was associated with significant over- and under-treatment, suggesting the need for more accurate assessments using a wider range of clinical variables^{[3, 4]}.

Integrating a patient's various clinical information for risk scoring is a challenge for cardiovascular physicians. The complexity of assessment is increasing as additional clinical variables need to be considered. In general, it is challenging for cardiovascular physicians to predict risk in individual patients. In the present study, we showed that our ML overcame these challenges, providing deep integration of comprehensive clinical data.

There are some differences between our study and previous studies. Most of the latter were designed to predict clinical outcomes after coronary artery revascularization; most relied on data from non-invasive (coronary computed tomography angiography) or invasive (coronary angiography, CAG) coronary angiography, and assistive technologies such as cardiac magnetic resonance, intravascular ultrasound, or fractional flow reserve^{[12, 13]}. In the present study, by contrast, we used an ML to predict whether patients with CAD could be treated with immediate revascularization based only on clinical data, history, and laboratory findings in the emergency department.

We used an ML approach, an artificial intelligence that differs from traditional prognostic methods in that it makes no a priori assumptions regarding the cause of disease. This characteristic permits agnostic explorations of available data that may predict the risk to individuals (i.e., precise risk stratification). This approach diverges from the ‘hypothesis-driven approach in standard prognostic risk assessment^{[15, 16]}.

We found that the precision value for class 0 (medication treatment) and the recall value for class 1 (revascularization treatment) of these two subsets were both high, especially in the training set. The recall was also high for class 0 (medication treatment) in the validation set. The high precision value of class 0 suggests that the actual class 0 instances account for a high proportion of all predicted class 0 instances, further suggesting that it is rare for the model to misjudge class 1 as class 0. The high recall value of class 1 suggests that the instances correctly identified as class 1 have a high percentage of all instances of class 1, further suggesting that the model has a high recognition accuracy for class 1. This finding was the same for the high recall value of class 0.

The neutrophil-to-lymphocyte ratio showed the highest predictive weight for the outcome. The ML avoided ignoring important but unexpected predictor variables or interactions by not making the necessary prior assumptions between cause and outcome and allowed us to identify clinically essential risks in patients with multiple marginal risk factors. Machines can quickly and seamlessly integrate new data to continuously update and optimize their algorithms, thereby continuously improving their predictive performance over time.

In general, our ML approach provided incremental gains in prognostic performance while managing 40 variables and numerous patient-specific variable-variable interactions. This process permits individualized risk assessment and circumvents several of the limitations inherent in the standard statistical approach.

Our findings have considerable clinical importance. ML may help generate more accurate cardiovascular risk stratification for individual patients.

Classical statistical methods hand-pick features based entirely on medical domain knowledge. Statistical methods are then used to calculate the importance of each feature and construct prediction models. ML methods start from the data and do not refer to traditional risk factors or weighted factors. Furthermore, they do they pay attention to the interpretability of the model. It remains a challenge to fuse medical domain knowledge and ML methods to build highly interpretable predictive models.

ML uses extraction methods and feature representation to extract features from enormous data sets to build models without reference to known weights and risk factors. Therefore, the models are less interpretable than traditional disease prediction methods. Furthermore, ML identifies risk factors different from those generated by traditional methods, allowing for more in-depth prospective studies to determine etiology and interactions. These advantages may eventually lead to new therapeutic targets^{[15, 17]}.

We established an ML method for predicting revascularization in patients presenting to the emergency department with chest pain. The comparable performance with traditional models suggests the potential value of ML approaches for evaluating chest pain, which is a complex, multifactorial symptom. The specific mechanisms of the seven clinical predictors in this ML model require further study. Longer follow-up and accumulation of multicenter data may improve ML models' sensitivity and specificity. When combined with a large and growing dataset, the ML models can be dynamically and automatically improved to achieve better performance.

AUC: Area Under Curve; BUN: Blood Urea Nitrogen; CABG: Coronary Artery Bypass Grafting; CAD: Coronary Artery Disease; CAG: Coronary Angiography; CHD: Coronary Heart Disease; CRE: Creatinine; DBP: Diastolic Blood Pressure; FBG: Fasting Blood Glucose; GRACE: The Global Registry of Acute Coronary Events; HCT: Hematocrit; HCY: Homocysteine; HDL-C: High Density Lipoprotein Cholesterol; HGB: Hemoglobin; HR: Heart Rate; LDL-C: Low Density Lipoprotein Cholesterol; MI: Myocardial Infarction; ML: Machine Learning; NLR: Neutrophil to Lymphocyte Ratio; PAD: Peripheral Arterial Disease; PCI: Percutaneous Coronary Intervention; PLT: Platelet; RBC: Red Blood Cell; ROC: Receiver Operating Characteristic Curve; SBP: Systolic Blood Pressure; TC: Total Cholesterol; TG: Triglyceride; WBC: White Blood Cell

Acknowledgements

We would like to thank the physicians at Emergency & Critical Care Center in Beijing Anzhen Hospital affiliated of Capital Medical University for their assistance in subject recruitment (with special thanks to Prof. Mei Wang, Chunmei Wang, and Qingxiang Li).

Authors’ Contributions

SPN, XW and ZCZ contributed to study design. NW and ZCZ analyzed and interpreted the data. CPM and HA provided their technical expertise for coronary angiography. ZCZ drafted the initial manuscript. XW and BJ provided critical advice in the data analysis, interpretation and critically assessed the manuscript. All authors read and approved the final manuscript.

Funding

This study was supported in part by grants from the Beijing Nova Program (Z201100006820087) and the Capital Health Research and Development of Special Fund (2018-1-2061).

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Ethical approval and consent to participate

The Ethics Committee Research Beijing Anzhen Hospital affiliated of Capital Medical University approved this study (No：2018055X). Informed consent was obtained from all individual participants included in the study.

All procedures performed in studies involving human participants followed the ethical standards of the institutional or national research committee and the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

FOX K A, DABBOUS O H, GOLDBERG R J, et al. Prediction of risk of death and myocardial infarction in the six months after presentation with acute coronary syndrome: prospective multinational observational study (GRACE) [J]. BMJ, 2006, 333(7578): 1091.
ROFFI M, PATRONO C, COLLET J P, et al. 2015 ESC Guidelines for the management of acute coronary syndromes in patients presenting without persistent ST-segment elevation: Task Force for the Management of Acute Coronary Syndromes in Patients Presenting without Persistent ST-Segment Elevation of the European Society of Cardiology (ESC) [J]. Eur Heart J, 2016, 37(3): 267-315.
AKHTER Z, RIND I, AIJAZ S, et al. Management, hospital outcome and revascularization trends in non-ST elevation myocardial infarction with high GRACE risk score [J]. Journal of the Pakistan Medical Association, 2019, 0):
SAAR A, MARANDI T, AINLA T, et al. The risk-treatment paradox in non-ST-elevation myocardial infarction patients according to their estimated GRACE risk [J]. Int J Cardiol, 2018, 272(26-32.
FOX K A, FITZGERALD G, PUYMIRAT E, et al. Should patients with acute coronary disease be stratified for management according to their risk? Derivation, external validation and outcomes using the updated GRACE risk score [J]. BMJ Open, 2014, 4(2): e004425.
BAWAMIA B, MEHRAN R, QIU W, et al. Risk scores in acute coronary syndrome and percutaneous coronary intervention: a review [J]. Am Heart J, 2013, 165(4): 441-450.
KRITTANAWONG C, ZHANG H, WANG Z, et al. Artificial Intelligence in Precision Cardiovascular Medicine [J]. J Am Coll Cardiol, 2017, 69(21): 2657-2664.
MINCHOLE A, CAMPS J, LYON A, et al. Machine learning in the electrocardiogram [J]. J Electrocardiol, 2019,
BAE Y, KANG S J, KIM G, et al. Prediction of coronary thin-cap fibroatheroma by intravascular ultrasound-based machine learning [J]. Atherosclerosis, 2019, 288(168-174.
CAKAR M A, SAHINKUS S, AYDIN E, et al. Relation between the GRACE score and severity of atherosclerosis in acute coronary syndrome [J]. J Cardiol, 2014, 63(1): 24-28.
RAHMANI R, MAJIDI B, ARIANNEJAD H, et al. The Value of the GRACE Score for Predicting the SYNTAX Score in Patients with Unstable Angina/Non-ST Elevation Myocardial Infarction [J]. Cardiovasc Revasc Med, 2020, 21(4): 514-517.
DEBSKI M, KRUK M, BUJAK S, et al. Coronary computed tomography angiography equals invasive angiography for the prediction of coronary revascularization [J]. Postepy Kardiol Interwencyjnej, 2019, 15(3): 308-313.
RONNOW SAND N P, NISSEN L, WINTHER S, et al. Prediction of Coronary Revascularization in Stable Angina: Comparison of FFRCT With CMR Stress Perfusion Imaging [J]. JACC Cardiovasc Imaging, 2019,
NEUMANN F-J, SOUSA-UVA M, AHLSSON A, et al. 2018 ESC/EACTS Guidelines on myocardial revascularization [J]. European Heart Journal, 2019, 40(2): 87-165.
LEE I, SHIN Y J. Machine learning for enterprises: Applications, algorithm selection, and challenges [J]. Business Horizons, 2020, 63(2): 157-170.
MURDOCH W J, SINGH C, KUMBIER K, et al. Definitions, methods, and applications in interpretable machine learning [J]. Proc Natl Acad Sci U S A, 2019, 116(44): 22071-22080.
FOSTER K R, KOPROWSKI R, SKUFCA J D. Machine learning, medical diagnosis, and biomedical engineering research - commentary [J]. Biomed Eng Online, 2014, 13(94.

No competing interests reported.

Download PDF

Journal Publication

published 14 Apr, 2022

Read the published version in Journal of Healthcare Engineering →

Version 1

posted

You are reading this latest preprint version

Using Machine Learning to Predict the Requirement for Revascularization in Patients with Chest Pain in the Emergency Department

Status:

Journal Publication

Version 1

Abstract

Figures

Background

Methods

Datasets

Outcomes

ML method

Feature selection and data preprocessing

Dimensionality reduction

Validation

Results

Calibration

Discussion

Conclusion

Abbreviations

Declarations

References

Additional Declarations

Status:

Journal Publication

Version 1