Assessment of sex and racial biases in electronic health records of emergency department patients with acute coronary syndrome

doi:10.21203/rs.3.rs-1735655/v1

Background

Electronic health records (EHRs) contain individualized patient data that can be used to develop diagnostic and risk prediction models with artificial intelligence (AI) algorithms. Explicit and implicit sources of bias embedded in EHRs may hinder generalizable model performance and perpetuate bias.

Objective

This study explores the question of how existing sex and racial disparities in the clinical assessment and management of acute coronary syndrome (ACS) are reflected in patients’ EHRs. We outlined recommendations on how these sources of bias should be examined and mitigated within the framework of development and validation of AI-based diagnosis and risk stratification algorithms.

Methods

This retrospective study examined several previously unrecognized EHR-embedded biases in a multisite emergency departments (ED) setting. We assessed sex and race differences in EHR data missingness and timeliness of several key ED procedures following ACS suspicion. We additionally conducted a data-driven clustering analysis to detect latent groups of ACS patients.

Results

10,043 ACS-associated ED visits were included. We identified sex and race differences in the prevalence of ACS symptoms, data missingness (vitals and labs), and waiting time for troponin order and treatment initiation within 24 hours post-admission. Our cluster analysis discovered four groups of ACS patients corresponding to common symptoms. Additionally, we identified differences in clinical management and patient demographics across these clusters.

Conclusions

We discovered several sources of bias inherent in the clinical practice of ACS diagnosis and treatment that are also present in EHR data. Our study supports the inclusion of a wider range of symptoms into AI-based diagnosis and risk stratification tools. These models should be also validated in ED admissions without chest pain and in patients who experience a longer waiting time for ED procedures.

Artificial Intelligence and Machine Learning

electronic health records

acute coronary syndrome

bias

sex

race

The application of electronic health records (EHRs) in risk prediction models has yielded significant insights into complex diseases. However, because EHR data reflect multi-level clinical decision-making, there is potential for biases, which may impact the value of EHR data for research and perpetuate disparities in health care.^1–5 As real-time, patient-centered databases, EHRs are vital for providing personalized information representing a large number of acute coronary syndrome (ACS) patients. Machine learning (ML) models have delivered promising results for use of EHRs to support clinical decisions in ACS patients, most notably for predicting mortality and other negative outcomes ^6–8 and as emergency department (ED) triage support tool to expedite treatment initiations.⁹ Current artificial intelligence (AI) based models do not explicitly account for sex- and race-related biases that are embedded in EHR, potentially leading to a biased performance.

A significant body of research has demonstrated the prevalence of sex- and race-related disparities in the clinical diagnosis, treatment, and outcomes of ED patients with ACS.^10–13 In the U.S., minorities and females with ACS experience worse outcomes due to a broad range of issues, many of which have psychosocial and socioeconomic origins.^14,15 Other sources that contribute to these disparities include systemic bias and the presentation of lesser known ACS symptoms that are often designated as “atypical”.^15–17 Females have a higher frequency of atypical symptoms and higher rates of comorbidities, which may contribute to delayed diagnosis, misdiagnosis, and disproportionate mortality.^10–13 Black non-Hispanic patients experience similar disparities, including mortality rates disproportionate to those in white non-Hispanic patients.¹⁸ Compounding these issues are race and sex differences in the accuracy and completeness of EHR data entry, as some hospitals with disproportionate care for underprivileged patient groups may lack the required resources for EHR quality optimization.¹⁹

This study begins to explore the question of how existing bias in clinical diagnosis and treatment of ACS is reflected in patients’ EHRs and how this may impact the performance of AI-based diagnostic tools. Here, we utilized a multisite EHR dataset to illuminate explicit and implicit sources of bias that should be considered in the development and testing of AI models.

Data Source

This multicenter retrospective study spanned 30 sites across the U.S. and utilized EHR data from ED admissions with confirmed ACS diagnosis. EHR data were extracted from 01-01-2007 to 06-3-2020.In compliance with the Health Insurance Portability and Accountability Act, all data were de-identified prior to extraction and retrospective analysis. We did not require Institutional Review Board approval, as this project constituted non-human subjects research per 45 Code of Federal Regulations 46.102.

Inclusion/Exclusion Criteria

We included ED patients who were 18 years and older. Sex was determined via patient’s self-identification within the health record. ED admissions with a confirmed ACS diagnosis during the hospital stay were identified using international classification of disease (ICD) codes to indicate unstable angina and myocardial infarction, including STEMI and NSTEMI (Supplementary Table 1). Figure 1.A presents the attrition chart.

Patient EHR Features

A total of 12 symptoms were identified and used to examine their prevalences across different populations. These symptoms included chest pain, dyspnea, dizziness, nausea, diaphoresis, epigastric pain, unexplained fatigue, jaw pain, palpitation, shoulder pain, abdominal pain, and coughing.^20–23 Other relevant features included were vital signs, basic metabolic panel (BMP), and age. Vital signs and BMP are routinely collected patient data that are often added as input features in ML-based models.^24,25 Supplementary Table 2 presents the list of EHR features.

Troponin Measurement & Pharmacological Treatments

The American College of Cardiology/American Heart Association guidelines recommend stratifying ACS risk in ED admissions using cardiac changes indicated by electrocardiogram, troponin levels, myocardium imaging, and symptoms.²⁶ Combined with demographics and medical history, this information can be used to generate risk scores that inform subsequent treatment steps and timing. We examined the timing of the troponin lab order (identified by current procedural terminology, CPT, codes 84512 and 84484) as an indicator for ED processing time. ED management of ACS suspected admissions also includes the administration of several types of medications, such as antiplatelets/ADP receptor antagonists, glycoprotein IIb/IIIa antagonists, anticoagulation agents, including direct thrombin inhibitors and factor Xa inhibitors, anti-ischemic/antianginal agents, ACE inhibitors, beta-blockers, and calcium channel blockers. ^27–29 The initial treatment was defined as the first administration of any of these medications within 24 hours post-admission. The timing of the initial treatment was examined in different subpopulations.

Data Analysis

Several potential sources of bias were examined. These included sex and race differences in 1) symptom presentation, 2) troponin order time within 24 hours after ED admission, 3) time of administration of first medication in suspected ACS within 24 hours post-admission, and 4) EHR data missingness within vital signs and lab measures. We additionally identified subgroups of patients (clusters) based on specific characteristics, such as common symptoms, using an unsupervised ML technique. The analysis included several EHR features in patient data ( Supplementary Table 2). We first projected the data with uniform manifold approximation and projection (UMAP) for dimensional reduction and visualization.³⁰ Distance between visits was measured with a weighted L1 metric, such that contributions from symptoms were weighted with twice the weight of contributions from these normalized clinical features. Two parameters, n-neighbors and minimum distance, were used to control the UMAP performance. DBSCAN was used as the clustering algorithm.³¹

Statistical Analysis

All statistical analyses were conducted in Python software. Group differences were calculated using Mann-Whitney U-test and Z-test. We also used a series of weighted marginal structural models³² to explore associations between sex/race and symptoms while adjusting for confounders, such as age.³³

A total of ACS 10,043 visits from 9,118 patients were included in this study. Figures 1.B-D present race/sex characteristics of the study participants.

Sex/Race Differences in Symptom Presentation

Males had a higher prevalence of chest pain (47.53%) than females (43.67%) (Table 1). Caucasian ACS patients had the highest prevalence of chest pain (47.13%) and dyspnea (41.45%), followed by African Americans (42.11% and 31.86%) and Asians with 37.54% and 31.72% (Table 2). The prevalence of dyspnea was similar for both sexes (females: 39.29%; males: 39.49%). We observed significant group differences in the prevalence of several symptoms (Tables 1 and 2). Females had a higher prevalence of abdominal pain (6.45%) and nausea (5.96%) than men (abdominal pain: 3.78%; p<0.0001; z-statistics: 122.96 and nausea: 3.58%; p<0.0001; z-statistics: 60.15). We also conducted a series of marginal structural models to determine associations between sex/race (predictors) and the two primary symptoms (chest pain and dyspnea) as outcomes while controlling for the effect of confounding variables. When predicting symptoms with race, the model was adjusted for age and sex. When predicting symptoms with sex, the model was adjusted for age and race. The causal odds ratios and associated 95% confidence intervals (CIs) for all pairwise comparisons are summarized in Supplementary Table 3. We found that Caucasians were more likely to report chest pain (odds ratio: 1.53; p < 0.0001; 95% CI: 1.43 - 1.63) and dyspnea (odds ratio: 1.44; p < 0.0001; 95% CI: 1.35 - 1.54), whereas African Americans were less likely to report these symptoms (chest pain: 0.64, p < 0.0001; 95% CI: 0.55 - 0.75; dyspnea: 0.65, 95% CI: 0.56 - 0.76). In terms of the average number of symptoms per visit from 12 examined symptoms, Caucasians had a significantly higher number of average symptoms per visit than Asians (p = 0.0009; t-statistics: 3.1; Cohen’s d = 0.180) and African Americans (p < 0.0001; t-statistics: 3.9; Cohen’s d = 0.115).

Troponin Measurement & Pharmacological Treatment

Initial troponin levels were ordered within 24 hours after admission for 4,384 ED visits. The time to the first troponin order was shorter in male ACS patients than in female patients (p-value = 0.0167). However, the difference did not reach the Bonferroni-adjusted level of significance of p = 0.00357 for 14 comparisons across sexes, races, and tests. Antiplatelet administration, including glycoprotein IIb/IIIa antagonists, was the most common treatment within 24 hours post-admission, experienced in 6,699 ED visits (66.70% of total ED visits). This was followed by anticoagulation (56.90%), beta-blockers (42.17%), opiates (25.38%), ACE inhibitors (18.32%), calcium channel blockers (15.98%), and anti-ischemic/antianginal agents (0.11%). For most patients, the initial medication was administered within the first five hours. However, there were individuals who received their medications after 10 hours (see the outliers in Figures 2.B and 2.C). The remaining 1,774 patient visits did not receive pharmacological interventions within the first 24 hours. Figure 2.A illustrates the sex and race proportion of ACS patients who did not receive any medications within 24 hours. Compared to males, females were more likely to receive no medication (p-value = 0.033; z-statistics = 1.84; Cohen’s d = 0.037). However, the difference was only marginally significant. The proportion of visits receiving no treatment within the first 24 hours was higher in Caucasians (Figure 2.A) than African Americans (p-value < 0.0001; z-statistics = 4.25; Cohen’s d = 0.123) and Asians (p-value < 0.0001; z-statistics = 3.73; Cohen’s d = 0.216 ). Sex and race differences in the post-admission treatment timing were also observed. Female patients experienced a longer door-to-treatment time within the first 24 hours than male patients (Figure 2.B) (p-value <0.0001; z-statistics = 6.18; Cohen’s d = 0.142). Caucasian patients experienced a shorter time than African American patients (p < 0.0001, z-statistics = -4.39; Cohen’s d = -0.14).

Sex/race Differences in EHR Data Missingness

Vital signs, creatinine, blood urea nitrogen, and glucose levels were used to examine sex/race differences associated with the EHR data missingness among ACS patients (Supplementary Figure 1). Caucasian ACS patients had the highest mean feature missingness (2.02) compared to Asians (mean feature missingness = 0.83; p-value < 0.0001; t-statistics = 7.92; Cohen’s d = 0.46), and African Americans (mean feature missingness = 1.54; p-value < 0.0001; t-statistics = 6.42; Cohen’s d = 0.19). African Americans had a higher number of feature missingness than Asians (p-value < 0.0001; t-statistics = 5.15; Cohen’s d = 0.32). We did not observe a significant difference in data missingness between males and females.

Cluster Analysis

Four clusters were identified in our study population (Figure 3), each corresponding to ACS patients with common clinical features (vital signs and lab measures) and symptom presentations. A total of 6,408 ED visits were included in the cluster analysis. ED visits that did not have all required vital signs and lab measures were excluded. A total of 6,127 visits were assigned to one of the four non-noise clusters. The largest cluster encompassed 3,125 ED visits, accounting for 51% of ED visits in the analysis. The clusters largely grouped around ACS symptom presentations, with clusters roughly corresponding to chest pain in all patients and no dyspnea (cluster 0 with 1,104 visits), no dominant symptom and almost no chest pain and dyspnea (cluster 1, the largest cluster with 3,125 visits), chest pain and dyspnea in nearly all patients (cluster 2 with 1,390 visits), dyspnea in all patients and almost no chest pain (cluster 3 with 506 visits). The prevalence of male patients was 62.6% in cluster 0, 59.4% in cluster 1, 59.9% in cluster 2, and 52.0% in cluster 3. The prevalence of female patients was 37.4% in cluster 0, 40.6% in cluster 1, 40.1% in cluster 2, and 48.0% in cluster 3. Using logistic regression, we identified several significant associations between patient demographics and clusters. Age was negatively associated with cluster 0 (p <1e-20) and positively associated with cluster 3 (p <1e-20). The prevalence of Caucasians was negatively associated with cluster 1 (p<6e-4). We identified significant differences in door-to-treatment time across the clusters (Figure 4.B), with clusters 0 (all chest pain and no dyspnea) and 2 (nearly all chest pain and dyspnea) having the shortest mean time-to-treatment than the two other groups with lower chest pain prevalence (Supplementary Table 4). The proportion of visits receiving no pharmacological treatment also varied across the four clusters (Figure 4.A). Cluster 1 (no dominant symptom, almost no chest pain and dyspnea) had the largest proportion of visits receiving no pharmacological administration within 24 hours post-admission and cluster 0 (chest pain in all patients and no dyspnea) had the smallest proportion of visits with no pharmacological administration within 24 hours after ED admission (Supplementary Table 5). Cluster 2 (chest pain and dyspnea) experienced a significantly shorter time to troponin order with 24 hours than cluster 0 (p <0.0001; z-statistics = -4.96; Cohen’s d = -0.27), cluster 1 (p <0.0001; z-statistics = -6.72; Cohen’s d = -0.32), and cluster 3 (p <0.0001, z-statistics = -3.82, Cohen’s d = -0.27).

Summary

Although the existence of systemic sex and race-based bias in the healthcare system is highly documented ^34–40, this study is one of the largest to examine several areas of EHR-embedded biases that have the potential to impact the performance of AI diagnosis tools. To our knowledge, this is also the first study to examine sex and race differences in EHR data missingness, time to troponin order, and time to treatment initiation in a multisite ED setting.

Background & Comparison to Previous Research

In the U.S., only 10% of ED admissions with acute chest pain are ultimately diagnosed as ACS.⁴¹ Despite recent advances in rapid ACS diagnosis, current approaches still lack adequate sensitivity and specificity.⁴² Over 80% of ACS patients have multiple symptoms,⁴³ and up to 30% of patients do not experience chest pain.⁴⁴ There is a shift in healthcare towards leveraging EHR data to develop automated clinical risk prediction tools for ACS and other acute conditions. With this comes the responsibility to create balanced datasets that are representative of diverse populations.^45–47 Although previous research has highlighted potential bias in AI-based technologies in medicine,^3,48,49 very few studies have analyzed racial and gender biases that stem from EHR data itself, and even fewer have developed AI tools for ACS that account for bias.

Finding Implications: Symptom Presentation

Hospital admissions without chest pain are frequently misdiagnosed and undertreated.⁵⁰ We found that chest pain dominant clusters experienced a shorter door-to-treatment time and were less likely to receive no early pharmacological treatment than other clusters. These findings support previous research showing that patients without chest pain presentation are not initially recognized as having ACS and are less likely to receive early cardiac medications.⁵⁰ Further research is needed to determine the value of these clusters in the context of diagnosis and management of suspected ACS. In our study population, females had a lower prevalence of chest pain than males. After accounting for age, the odds ratio of having chest pain was insignificantly lower in females, indicating potential variations in symptoms due to aging and comorbidities.⁵¹ In line with several previous studies,⁵² females with ACS in our study presented a more diverse spectrum of symptoms. Our study supports the inclusion of a wider range of symptoms into ACS models and their validation in ED admissions without chest pain.

Finding Implications: Timeliness of ED Procedures

Cardiac troponin is a specific marker of myocardial damage.^53,54 Rapid testing of troponin is a critical part of ED practice indicating how quickly ED patients are stratified according to ACS risk.⁵⁵ The shortest waiting time to treatment was observed in male and Caucasian ACS patients. These findings suggest that the performance of EHR-based ACS models should be evaluated across implicit subpopulations with varying waiting times for diagnostic and management procedures.

Finding Implications: Data Missingness

This is the first study that aimed to explore sex and race differences in data missingness of key EHR features that are commonly used as input variables in ML models.^47,56–58 Although different approaches, such as imputation methods, have been implemented to treat missing data,⁵⁹ little is known about how race and sex differences in data missingness can perpetuate biases in ML performance. Data quality can be impacted by the data collection process or other factors. For example, there may be bias on the part of the physician while treating patients;⁶⁰ or a single EHR database may not capture the complete medical history of low income patients, as this population tends to seek care across multiple healthcare sites.⁴⁹ Despite this, just over half of ML models designed to make predictions in healthcare take these shortfalls into account.⁴⁹ One potential solution is to ensure that ACS models are trained and validated on targeted populations for whom they are intended.⁴⁹

Future Directions And Study Limitations

This study will open new avenues for evaluating bias in ACS models with ML algorithms. This can be achieved by testing the model’s performance in different subpopulations that are stratified by symptoms, clusters, or implicit attributes. There are several limitations in this study to be noted. We used ICD codes to identify ACS patients. Although it is common practice in research to rely on ICD codes to determine patients diagnosed with cardiovascular diseases,^61–66 there may be human errors and biases in ICD coding. Racial breakdown in our study was limited to African American, Asian, and Caucasian populations. Other research found significant disparities in ED treatment for other minority groups, such as American Indian ^67,68 and Hispanic/Latino ⁶⁹ patients. Our database did not contain more nuanced racial and ethnic information.

Acknowledgment

This study was supported by the National Institute On Minority Health And Health Disparities under Award Number R43MD016363. The authors would like to thank Dr. Wendy Ying, M.D. at Kaiser Permanente Santa Clara Homestead Medical Center for valuable discussions.

Ethics

The data acquisition and subsequent study were determined to not constitute human subjects research due to the use of de-identified data. The use of de-identified retrospective data is classified as a non-human subject study and exempt from Institutional Review Board approval, consistent with 45 Code of Federal Regulations 46.102.

Disclosures

All authors are employees or contractors of Dascena (Houston, Texas, U.S.A).

1. Chen, I. Y. et al. Ethical Machine Learning in Healthcare. Annu. Rev. Biomed. Data Sci. 4 , 123–144 (2021).

2. Ferryman, K. & Pitcan, M. Fairness in Precision Medicine. 54.

3. Agniel, D., Kohane, I. S. & Weber, G. M. Biases in electronic health record data due to processes within the healthcare system: retrospective observational study. BMJ 361, k1479 (2018).

4. Verheij, R. A., Curcin, V., Delaney, B. C. & McGilchrist, M. M. Possible Sources of Bias in Primary Care Electronic Health Record Data Use and Reuse. J. Med. Internet Res. 20 , e185 (2018).

5. Bower, J. K., Patel, S., Rudy, J. E. & Felix, A. S. Addressing Bias in Electronic Health Record-Based Surveillance of Cardiovascular Disease Risk: Finding the Signal Through the Noise. Curr. Epidemiol. Rep. 4 , 346–352 (2017).

6. Austin, P. C., Lee, D. S., Steyerberg, E. W. & Tu, J. V. Regression trees for predicting mortality in patients with cardiovascular disease: what improvement is achieved by using ensemble-based methods? Biom. J. Biom. Z. 54 , 657–673 (2012).

7. Sherazi, S. W. A., Jeong, Y. J., Jae, M. H., Bae, J.-W. & Lee, J. Y. A machine learning-based 1-year mortality prediction model after hospital discharge for clinical patients with acute coronary syndrome. Health Informatics J. 26 , 1289–1304 (2020).

8. D’Ascenzo, F. et al. Machine learning-based prediction of adverse events following an acute coronary syndrome (PRAISE): a modelling study of pooled datasets. Lancet Lond. Engl. 397, 199–207 (2021).

9. Iannattone, P. A., Zhao, X., VanHouten, J., Garg, A. & Huynh, T. Artificial Intelligence for Diagnosis of Acute Coronary Syndromes: A Meta-analysis of Machine Learning Approaches. Can. J. Cardiol. 36 , 577–583 (2020).

10. Eastwood, J.-A. et al. Anginal Symptoms, Coronary Artery Disease, and Adverse Outcomes in Black and White Women: The NHLBI-Sponsored Women’s Ischemia Syndrome Evaluation (WISE) Study. J. Womens Health 22 , 724–732 (2013).

11. Sonel, A. F. et al. Racial Variations in Treatment and Outcomes of Black and White Patients With High-Risk Non–ST-Elevation Acute Coronary Syndromes. Circulation 111, 1225–1232 (2005).

12. Lee, C. Y. et al. Sex and gender differences in presentation, treatment and outcomes in acute coronary syndrome, a 10 year study from a multi-ethnic Asian population: The Malaysian National Cardiovascular Disease Database—Acute Coronary Syndrome (NCVD-ACS) registry. PLOS ONE 16 , e0246474 (2021).

13. Elsaesser, A. & Hamm, C. W. Acute coronary syndrome: the risk of being female. Circulation 109, 565–567 (2004).

14. Sanchis-Gomar, F., Perez-Quilis, C., Leischik, R. & Lucia, A. Epidemiology of coronary heart disease and acute coronary syndrome. Ann. Transl. Med. 4 , 256 (2016).

15. Muncan, B. Cardiovascular disease in racial/ethnic minority populations: illness burden and overview of community-based interventions. Public Health Rev. 39 , 32 (2018).

16. Beery, T. A. Gender bias in the diagnosis and treatment of coronary artery disease. Heart Lung J. Cardiopulm. Acute Care 24 , 427–435 (1995).

17. Record, N. B. et al. Community-Wide Cardiovascular Disease Prevention Programs and Health Outcomes in a Rural County, 1970–2010. JAMA 313, 147 (2015).

18. DeVon, H. A., Burke, L. A., Nelson, H., Zerwic, J. J. & Riley, B. Disparities in Patients Presenting to the Emergency Department with Potential Acute Coronary Syndrome: It Matters if You Are Black or White. Heart Lung J. Crit. Care 43 , 270–277 (2014).

19. Elnahal, S. M., Joynt, K. E., Bristol, S. J. & Jha, A. K. Electronic Health Record Functions Differ Between Best and Worst Hospitals. Am. J. Manag. Care 17 , e121–e147 (2011).

20. Araújo, C. et al. Sex differences in presenting symptoms of acute coronary syndrome: the EPIHeart cohort study. BMJ Open 8 , e018798 (2018).

21. Acute coronary syndrome - Symptoms and causes. Mayo Clinic https://www.mayoclinic.org/diseases-conditions/acute-coronary-syndrome/symptoms-causes/syc-20352136.

22. Vitulano, N. et al. Atypical Presentation of Acute Coronary Syndrome-Not ST Elevation: A Case Report. Case Rep. Med. 2012, 182379 (2012).

23. DeVon, H. A., Ryan, C. J., Ochs, A. L. & Shapiro, M. Symptoms Across the Continuum of Acute Coronary Syndromes: Differences Between Women and Men. Am. J. Crit. Care Off. Publ. Am. Assoc. Crit.-Care Nurses 17 , 14–25 (2008).

24. Noy, O. et al. A machine learning model for predicting deterioration of COVID-19 inpatients. Sci. Rep. 12 , 2630 (2022).

25. Heilbroner, S. P. et al. Predicting cardiac adverse events in patients receiving immune checkpoint inhibitors: a machine learning approach. J. Immunother. Cancer 9 , e002545 (2021).

26. Risk Stratification of the ACS Patient in the Emergency Department and Initial Medical Therapy. The Cardiology Advisor https://www.thecardiologyadvisor.com/home/decision-support-in-medicine/cardiology/risk-stratification-of-the-acs-patient-in-the-emergency-department-and-initial-medical-therapy/ (2019).

27. Anderson, J. L. et al. 2011 ACCF/AHA Focused Update Incorporated Into the ACC/AHA 2007 Guidelines for the Management of Patients With Unstable Angina/Non–ST-Elevation Myocardial Infarction. Circulation 123, e426–e579 (2011).

28. Amsterdam, E. A. et al. Testing of Low-Risk Patients Presenting to the Emergency Department With Chest Pain. Circulation 122, 1756–1776 (2010).

29. Kushner, F. G. et al. 2009 focused updates: ACC/AHA guidelines for the management of patients with ST-elevation myocardial infarction (updating the 2004 guideline and 2007 focused update) and ACC/AHA/SCAI guidelines on percutaneous coronary intervention (updating the 2005 guideline and 2007 focused update) a report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines. J. Am. Coll. Cardiol. 54 , 2205–2241 (2009).

30. McInnes, L., Healy, J. & Melville, J. UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. ArXiv180203426 Cs Stat (2020).

31. Ester, M., Kriegel, H.-P. & Xu, X. A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise. 6.

32. Robins, J. M., Hernán, M. Á. & Brumback, B. Marginal Structural Models and Causal Inference in Epidemiology. Epidemiology 11 , 550–560 (2000).

33. Williamson, T. & Ravani, P. Marginal structural models in clinical research: when and how to use them? Nephrol. Dial. Transplant. 32 , ii84–ii90 (2017).

34. Lee, S. K. Sex as an important biological variable in biomedical research. BMB Rep. 51 , 167–173 (2018).

35. Hall, W. J. et al. Implicit Racial/Ethnic Bias Among Health Care Professionals and Its Influence on Health Care Outcomes: A Systematic Review. Am. J. Public Health 105, e60–e76 (2015).

36. Chinn, J. J., Martin, I. K. & Redmond, N. Health Equity Among Black Women in the United States. J. Womens Health 30 , 212–219 (2021).

37. Sex/gender bias in the management of chest pain in ambulatory care - Christelle Clerc Liaudat, Paul Vaucher, Tommaso De Francesco, Nicole Jaunin-Stalder, Lilli Herzig, François Verdon, Bernard Favrat, Isabella Locatelli, Carole Clair, 2018. https://journals.sagepub.com/doi/full/10.1177/1745506518805641.

38. Norori, N., Hu, Q., Aellen, F. M., Faraci, F. D. & Tzovara, A. Addressing bias in big data and AI for health care: A call for open science. Patterns 2 , 100347 (2021).

39. 2021 National Healthcare Quality and Disparities Report: Executive Summary. 7 (2021).

40. Introduction: CDC Health Disparities and Inequalities Report — United States, 2013. https://www.cdc.gov/mmwr/preview/mmwrhtml/su6203a2.htm?s_cid=su6203a2_w.

41. Fanaroff, A. C., Rymer, J. A., Goldstein, S. A., Simel, D. L. & Newby, L. K. Does This Patient With Chest Pain Have Acute Coronary Syndrome?: The Rational Clinical Examination Systematic Review. JAMA 314, 1955–1965 (2015).

42. Shah, A. S. V. et al. High sensitivity cardiac troponin and the under-diagnosis of myocardial infarction in women: prospective cohort study. BMJ 350, g7873 (2015).

43. Richards, S., Funk, M. & Milner, K. Differences Between Blacks and Whites With Coronary Heart Disease in Initial Symptoms and in Delay in Seeking Care. Nurs. Fac. Publ. (2000).

44. DeVon, H. A., Mirzaei, S. & Zègre‐Hemsey, J. Typical and Atypical Symptoms of Acute Coronary Syndrome: Time to Retire the Terms? J. Am. Heart Assoc. 9 , (2020).

45. Huang, Z., Dong, W., Duan, H. & Liu, J. A Regularized Deep Learning Approach for Clinical Risk Prediction of Acute Coronary Syndrome Using Electronic Health Records. IEEE Trans. Biomed. Eng. 65 , 956–968 (2018).

46. Gibson, W. J. et al. Machine learning versus traditional risk stratification methods in acute coronary syndrome: a pooled randomized clinical trial analysis. J. Thromb. Thrombolysis 49 , 1–9 (2020).

47. VanHouten, J. P., Starmer, J. M., Lorenzi, N. M., Maron, D. J. & Lasko, T. A. Machine Learning for Risk Prediction of Acute Coronary Syndrome. AMIA. Annu. Symp. Proc. 2014, 1940–1949 (2014).

48. Parikh, R. B., Teeple, S. & Navathe, A. S. Addressing Bias in Artificial Intelligence in Health Care. JAMA 322, 2377–2378 (2019).

49. Gianfrancesco, M. A., Tamang, S., Yazdany, J. & Schmajuk, G. Potential Biases in Machine Learning Algorithms Using Electronic Health Record Data. JAMA Intern. Med. 178, 1544 (2018).

50. Brieger, D. et al. Acute coronary syndromes without chest pain, an underdiagnosed and undertreated high-risk group: insights from the Global Registry of Acute Coronary Events. Chest 126, 461–469 (2004).

51. Canto, J. G. et al. Symptom Presentation of Women With Acute Coronary Syndromes: Myth vs Reality. Arch. Intern. Med. 167, 2405–2413 (2007).

52. Dey, S. et al. Sex-related differences in the presentation, treatment and outcomes among patients with acute coronary syndromes: the Global Registry of Acute Coronary Events. Heart 95 , 20–26 (2009).

53. Wu, A. H. B. & Jaffe, A. S. The clinical need for high-sensitivity cardiac troponin assays for acute coronary syndromes and the role for serial testing. Am. Heart J. 155, 208–214 (2008).

54. Garg, P. et al. Cardiac biomarkers of acute coronary syndrome: from history to high-sensitivity cardiac troponin. Intern. Emerg. Med. 12 , 147 (2017).

55. Hamm, C. W. et al. Emergency Room Triage of Patients with Acute Chest Pain by Means of Rapid Testing for Cardiac Troponin T or Troponin I. N. Engl. J. Med. 337, 1648–1653 (1997).

56. Panchavati, S. et al. Retrospective validation of a machine learning clinical decision support tool for myocardial infarction risk stratification. Healthc. Technol. Lett. n/a,.

57. Frontiers | Enriching the Study Population for Ischemic Stroke Therapeutic Trials Using a Machine Learning Algorithm | Neurology. https://www.frontiersin.org/articles/10.3389/fneur.2021.784250/full?&utm_source=Email_to_authors_&utm_medium=Email&utm_content=T1_11.5e1_author&utm_campaign=Email_publication&field=&journalName=Frontiers_in_Neurology&id=784250#h1.

58. Lindholm, D. & Holzmann, M. Machine learning for improved detection of myocardial infarction in patients presenting with chest pain in the emergency department. J. Am. Coll. Cardiol. 71 , A225–A225 (2018).

59. Batista, G. E. A. P. A. & Monard, M. C. An analysis of four missing data treatment methods for supervised learning. Appl. Artif. Intell. 17 , 519–533 (2003).

60. Beach, M. C. et al. Testimonial Injustice: Linguistic Bias in the Medical Records of Black Patients and Women. J. Gen. Intern. Med. 36 , 1708–1714 (2021).

61. Patel, A. B. et al. Validity and utility of ICD-10 administrative health data for identifying ST- and non-ST-elevation myocardial infarction based on physician chart review. CMAJ Open 3 , E413-418 (2015).

62. S, S. et al. Development and validation of techniques for phenotyping ST-elevation myocardial infarction encounters from electronic health records. JAMIA Open 4 , (2021).

63. Cozzolino, F. et al. A diagnostic accuracy study validating cardiovascular ICD-9-CM codes in healthcare administrative databases. The Umbria Data-Value Project. PLOS ONE 14 , e0218919 (2019).

64. Floyd, J. S., Blondon, M., Moore, K. P., Boyko, E. J. & Smith, N. L. Validation of methods for assessing cardiovascular disease using electronic health data in a cohort of Veterans with diabetes. Pharmacoepidemiol. Drug Saf. 25 , 467–471 (2016).

65. Alaa, A. M., Bolton, T., Angelantonio, E. D., Rudd, J. H. F. & Schaar, M. van der. Cardiovascular disease risk prediction using automated machine learning: A prospective study of 423,604 UK Biobank participants. PLOS ONE 14 , e0213653 (2019).

66. Rubbo, B. et al. Use of electronic health records to ascertain, validate and phenotype acute myocardial infarction: A systematic review and recommendations. Int. J. Cardiol. 187, 705–711 (2015).

67. Puumala, S. E. et al. The Role of Bias by Emergency Department Providers in Care for American Indian Children. Med. Care 54 , 562–569 (2016).

68. Weber, T. L. et al. Leaving the emergency department without complete care: disparities in American Indian children. BMC Health Serv. Res. 18 , 267 (2018).

69. Sarkar, R. et al. Performance of intensive care unit severity scoring systems across different ethnicities in the USA: a retrospective observational study. Lancet Digit. Health 3 , e241–e249 (2021).

Table 1: Prevalence and statistical comparison of symptoms between males and females.

Symptoms	Male	Female	p-value	z-statistic
Chest pain	47.53	43.67	<0.0001*	-13.19
Dyspnea	39.49	39.29	0.2821	-1.08
Unexplained fatigue	1.92	2.44	0.0004*	13.93
Jaw pain	0.22	0.17	0.0609	-1.87
Palpitation	1.1	1.7	<0.0001*	14.42
Abdominal pain	3.78	6.45	<0.0001*	122.96
Coughing	0.55	1.28	<0.0001*	5.25
Dizziness	2.24	3.1	<0.0001*	24.85
Nausea	3.58	5.96	<0.0001*	60.15
Hyperhidrosis	0.64	0.59	<0.0001*	-10.78
Epigastric pain	1.14	1.97	<0.0001*	25.93
Shoulder pain	1.09	1.45	<0.0001*	-11.74

‘∗’ or ‘·’ associated with numbers in the table indicating that these values are either significant ‘∗’ or marginally significant ‘·’. Significance codes (Bonferroni-adjusted p-value 0.05/14 = 0.00375): 0; “*” = 0.00375; “·” = 0.05; Abbreviations used: CI - Confidence Interval; P>|z| - two-tail probability value

Table 2: Prevalence and pairwise statistical comparisons of symptoms between races.

	Prevalence %			African American vs. Asian		Caucasian vs. African American		Caucasian vs. Asian
Symptoms	African American	Asian	Caucasian	p-value	z-statistic	p-value	z-statistic	p-value	z-statistic
Chest pain	42.11	37.54	47.13	0.1574	1.41	<0.0001*	38.39	<0.0001*	14.09
Dyspnea	31.86	31.72	41.45	<0.0001*	8.74	<0.0001*	64.73	<0.0001*	28.72
Unexplained fatigue	1.99	1.94	2.22	0.0096	-2.59	<0.0001*	41.60	<0.0001*	11.41
Jaw pain	0.07	0.00	0.24	0.4565	0.74	0.0979	1.66	0.1986	1.29
Palpitation	1.42	1.29	1.36	<0.0001*	-7.13	<0.0001*	33.56	<0.0001*	5.44
Abdominal pain	6.90	7.44	4.17	0.1374	-1.49	<0.0001*	69.44	<0.0001*	21.63
Coughing	1.21	0.32	0.80	<0.0001*	6.00	<0.0001*	13.07	<0.0001*	9.93
Dizziness	2.28	2.27	2.56	<0.0001*	-14.92	<0.0001*	34.98	0.2310	-1.20
Nausea	4.05	2.91	4.63	<0.0001*	6.97	<0.0001*	51.71	<0.0001*	22.74
Hyperhidrosis	0.78	0.65	0.60	0.1663	-1.38	<0.0001*	12.80	0.0020*	3.10
Epigastric pain	1.99	1.94	1.24	<0.0001*	-19.83	<0.0001*	18.14	<0.0001*	-12.21
Shoulder pain	1.49	1.62	1.20	0.0053	2.79	<0.0001*	7.00	<0.0001*	5.03

‘∗’ or ‘·’ associated with numbers in the table indicating that these values are either significant ‘∗’ or marginally significant ‘·’. Significance codes (Bonferroni-adjusted p-value 0.05/14 = 0.00375): 0; “*” = 0.00375; “·” = 0.05; Abbreviations used: CI - Confidence Interval; P>|z| - two-tail probability value

SupplementaryMaterials1.docx

Assessment of sex and racial biases in electronic health records of emergency department patients with acute coronary syndrome

Status:

Version 1

Abstract

Background

Objective

Methods

Results

Conclusions

Figures

Introduction

Methods

Results

Discussion

Declarations

References

Tables

Supplementary Files

Status:

Version 1