Predictors of missing Electronic Medical Record (EMR) discharge summaries: A retrospective study on data from a large Canadian cohort

Abstract

Background: Healthcare systems worldwide have adopted the electronic medical record (EMR). EMRs are an efficient method of interprofessional communication, and can improve data availability for secondary research purposes. The discharge summary (DS) is a crucial document for both interprofessional communication, and coding of data for research purposes. We aimed to assess the completeness of our EMRs by assessing the presence of a DS in the EMR. Additionally, we evaluated the presence of predictors for a missing DS.

Method: A retrospective chart review was conducted on 3,011 inpatient charts from 3 hospitals in Calgary, Alberta Canada. 893 charts were found to be missing an electronic DS. A 10% sample was drawn to assess for presence of a paper DS. A list of variables was compiled to assess for association between patient and hospital characteristics, patient comorbidities, and the absence of an electronic DS. A Chi-square test, Fisher’s test and logistic regression were conducted to assess for associations.

Results: The univariate analyses showed that age, being a surgical patient, a Charlson Comorbidity Index (CCI) of </1, as well as patients with myocardial infarctions, congestive heart failure, cerebrovascular disease, dementia, chronic pulmonary disease, diabetes, and renal disease were associated with a missing DS. The multivariate logistic regression showed that those that were middle aged, surgical patients, or with fewer comorbidities were more likely to have a missing DS. Within the 10% sample, approximately 50% of all patients were from a surgical department, all of which were missing both electronic and paper discharge summaries.

Conclusion: Our study is the first to describe predictors associated with missing electronic discharge summaries. There is a modern day propensity for adoption of the EMR across healthcare systems worldwide. The EMR, especially the DS, is used for the improvement of interprofessional communication, patient outcomes, and data quality. Therefore, the implications of an incomplete EMR are widespread. Our findings will caution future researchers using EMR data about the potential for incomplete data, particularly for patients who are surgical, middle aged, and have fewer comorbidities. Additionally, our study highlights the need for further investigation into the lack of discharge summaries in surgical units. 

Introduction

Due to recent health care modernisation, health care providers are becoming increasingly dependent on electronic medical records (EMR). Furthermore, the EMR has become a widely accepted source of data for epidemiological, health services, drug utilization, regulatory and safety surveillance research [1]. An EMR accumulates relevant information from a patient’s visit to a health care facility, including course of observation, treatment and discharge plan. Clinical documentation in the EMR can therefore exist in a variety of formats, ranging from free-text documentation to imaging files and numerical bedside monitor trends. With the use of machine learning techniques such as natural language processing and data mining, this complex EMR data can be analyzed to detect patterns in patient health status and healthcare delivery, which can in turn support clinical decision making [2].

Inpatient EMR documentation is also used to generate coded data in several countries [3]. To inform health policies and improve the delivery of health services, health care providers use inpatient coded data. The inpatient database relies heavily on electronic inpatient documents (specifically the discharge summaries). When the discharge summary (DS) is unavailable electronically, trained coders use the paper DS, which often presents challenges in discerning illegible handwriting for the presence of codable diagnoses [4]. As a result, many healthcare facilities are transitioning to a fully electronic-based documentation system [5].

The DS captures extensive information regarding a patient’s hospital visit, such as: care processes, courses of treatment, diagnoses lists, healthcare providers, and admission and discharge dates. Not only is this document essential in ensuring high quality databases, it is also the primary means of communication and collaboration between healthcare providers [6]. The quality and availability of the DS have been shown to have implications on risk of re-admission [7] and patient safety [8]. Therefore, the DS is arguably one of the most crucial documents in the EMR. Consequently, this study aimed to assess the completeness of the EMR through the presence of the DS, as well as to analyze the possible predictors for the presence of the DS.

Methods

Study Setting

The Calgary Health Zone is one of the largest fully integrated, publicly funded, single-payer healthcare systems in Canada. It provides all medical and surgical care to residents of Calgary and surrounding communities in Southern Alberta. Contained in the region are 13 academic and community hospitals. Calgary, Alberta has a population of 1.5 million. In 2016, 70.2% of Calgarians were between the ages of 15 and 64. The average life expectancy in Calgary is 82.9 years. There are 4 large hospitals in Calgary, 3 of which were used in this study. All 3 hospitals have inpatient, intensive care and short-stay beds. The Foothills Medical Centre is an urban teaching hospital with a Level 1 trauma centre and 766 beds. It provides care to over 2 million people from southern Alberta. The Peter Lougheed Hospital is a 600-bed urban teaching hospital. It serves the northeastern part of the city, which has a high influx of immigrants and lower socioeconomic status Calgarians. Lastly, the Rockyview General Hospital is a 650-bed urban community hospital which provides medical, surgical, and psychiatric services to Calgary and Southern Alberta. The fourth hospital, South Health Campus is an urban Calgary hospital that had recently opened upon commencement of the chart review, and was therefore not included in the study. All Calgary hospitals are administered by Alberta Health Services, the single health authority in the province of Alberta.

Study Cohort

In this retrospective study, six registered nurses performed a chart review of 3,045 randomly selected inpatient charts (one chart for one patient) at three acute care hospitals in Calgary. In From January 2015 to January 2016, there were 111,148 inpatient admissions to Calgary hospitals. The nurses had varying clinical experience, both in years of practice and area of expertise. The intent of the chart review was to identify 50 of a derived set of Charlson and Elixhauser health conditions. The chart review took place between August 2016 and June 2017. Patients met inclusion criteria if they were adults (> = 18 years), had an Alberta personal health care number, and had an inpatient visit for any service outside of obstetrics between January 1^st, 2015 and June 30^th, 2015. A final number of 3,011 patient charts were used for this study. From the initial 3,045, 30 were in the field of obstetrics and thus were excluded. An additional 4 patient charts were missing information from the chart review and were excluded (Figure 1).

Datasets Used

The provincial electronic health record system for Alberta Health Services began in 1997, and has since been growing in complexity, as healthcare services continue to expand in electronic availability. Since 2006, the system used by Calgary healthcare providers for documentation in the inpatient electronic medical record is Sunrise Clinical Manager^TM (SCM)- EMR. However, only data collected since 2011 has been validated for research use [9]. SCM-EMR contains the electronic medical record, as well as laboratory and diagnostic imaging reporting. It includes free text (e.g., DS), structured data (laboratory data), and numerical bedside monitor trend data.

In this study, SCM-EMR was used to draw information on the 3,011 patient charts used in the chart review. The nurses were given access to both SCM-EMR and the paper copies of the chart to complete the chart review. For every patient visit, the chart reviewer firstly reviewed the patient’s electronic chart, and then reviewed the paper chart if additional information was needed for data abstraction. For the purpose of a larger study with the same research group at the University of Calgary, the data produced by SCM-EMR on these 3,011 patients was also analyzed. During the analysis, it was found that 893 of the 3,011 inpatient charts were missing a DS.

Extended Chart Review

A random 10% sample (85 charts) was drawn from the 893 missing DS charts to assess for the presence of a DS in paper format, which would not have been detected in the SCM-EMR data. Additionally, the 85 charts were assessed for information on the unit and department from which the patient was discharged, as well as the attending physician’s specialty.

Study Variables

For those with a missing electronic DS, a list of possible associated variables was collected. These variables were compiled using expertise from physicians and nurses. Age was categorized according to the parameters used by the Centre for Disease Control and Prevention. Length of stay parameters were drawn from the Discharge Abstract Database within the Canadian Institute for Health Informatics (CIHI). According to CIHI, the average length of stay for Albertan patients in 2018 was 7.8 days [10]. Therefore, the four categories used to classify length of stay were deemed appropriate by study authors. The day of the week was used to assess whether the presence of a DS was influenced by the patient’s discharge falling on a weekday or weekend. Due to decreased hospital staff during weekends, efficiency and documentation can be affected. Study authors included the day of the week variable as a measure to observe whether decreased efficiency on weekends would result in the absence of a DS. The DS is crucial for interprofessional communication, particularly when a patient is being transferred from one facility to another. Therefore, disposition was included as a variable to assess whether an interfacility transfer affected the presence of a DS in the EMR. Finally, the 17 Charlson comorbidities are a common set of secondary diseases that can accompany the primary disease of a patient [11]. The Charlson Comorbidity Index (CCI) is used to weigh the comorbidities, and to perform risk adjustment in outcomes research from administration databases (such as inpatient mortality). Both the CCI and the individual Charlson comorbidities were included in the analysis for associated variables.

Statistical Analysis

Study variables were reported as numbers and percentages for categorical variables. Chi-Square test or Fisher’s exact test, when appropriate, were performed to assess the differences of associated variables between missing DS and non-missing DS charts. Logistic regression was used to identify the variables that were independently associated with missing DS. Logistic regression results were reported using odds ratios (ORs) and 95% confidence intervals (95% CIs). A p-value of <0.05 was used for statistical significance. Analyses were performed using Stat 14.0 (Stat Corp, College Station, TX, USA).

Results

By comparing the 893 missing DS patients with the remaining 2,118 patients with a DS, the univariate analyses showed that age, being a surgical patient, a Charlson Comorbidity Index (CCI) of </1, as well as patients with myocardial infarctions, congestive heart failure, cerebrovascular disease, dementia, chronic pulmonary disease, diabetes with complications, and renal disease were associated with missing DS (Table 1 and Table 2).

Table 1. Frequencies of patient and hospital characteristics assessed for association with missing discharge summaries in the EMR

Table 2. Frequencies of comorbidities assessed for association with missing discharge summaries in the EMR

The multivariate logistic regression suggested that patient age, surgical patient, and CCI score were all found to be independently associated with missing discharge summaries (Table 3).

Table 3. Variables associated with missing discharge summaries in the EMR by multivariate logistic regression*

Specifically, surgical patients had 4.8 times the odds of having a missing DS than non-surgical patients (OR: 4.87, 95% CI: 4.09–5.80). Compared with patients with a CCI of 0, patients with a higher CCI score (> = 2) were less likely to be missing a DS (OR: 0.67, 95% CI: 0.54–0.83). Additionally, patients between the ages of 45–64 years were close to doubling the odds of having a missing DS than those <45 years (OR: 1.44, 95% CI: 1.13–1.83). Those aged >64 had non-statistically significant increased odds of having a missing DS. The sample size was not sufficiently large to conduct a multivariate logistic regression with each comorbidity.

Within the 10% sample of missing electronic DS charts, 51 of the 85 charts (60%) were also missing paper discharge summaries. Forty of the 51 charts missing an electronic and paper DS contained “Discharge Instructions”, an instructional carbon-copy paper document for the patient most often given post-surgery which outlines the precautionary measures to avoid complications to the surgical site. All 40 of the patients with discharge instructions were discharged from a surgical unit. The remaining 11 patients with no electronic or paper DS or discharge instructions were discharged from non-surgical units (Table 4).

Table 4. Variables associated with missing electronic or paper discharge summaries from the extended chart review

In an effort to assess whether residents and medical students producing the DS contributed to the missing summaries, DS author and level of training was included as a variable of analysis. However, for those with a paper DS, the signature and level of training were often illegible. For the remaining cohort with no DS, author was not identified since no DS was authored. Therefore, this variable was not used in the analysis.

Discussion

The DS is a crucial document within the patient chart, and its absence can have detriments on the level of completion of the EHR. Consequently, patient outcomes, interprofessional collaboration, and administrative data quality are all impacted by an incomplete EMR. Our objective was to assess the completeness of EMR data through analysis of the presence of a DS, and to find variables associated with missing discharge summaries. Alberta Health Services’ current EMR system has been in use for over a decade. It is one of the most validated electronic systems in Canada, and has great utility for interprofessional collaboration and secondary research purposes [9]. Despite this, and the evidence supporting a fully electronic EMR system, our findings demonstrated that EMR incompleteness was associated with patient baseline and setting characteristics. Interestingly, physicians were relying on paper charting or simply not creating discharge summaries when patients are middle-aged, surgical and have fewer comorbidities. The findings of our study will caution researchers when using EMR data pertaining to these 3 patient characteristics.

Upon further assessment of the 85-chart sample, 70% were from outpatient surgical procedures, specifically urological and orthopedic procedures. The one urology clinic and two orthopedic clinics in Calgary were contacted for further information on DS practices. Across all clinics, staff reported limited need for a DS post-procedure, as the surgeon performing the procedure is often the surgeon that will see the patient in follow-up at the clinic. Additionally, given the simple nature of many of these procedures, there is no perceived need for a copy to be made to the family physician. A DS paper template is provided for surgeons at one orthopedic surgery site, albeit not used. Discharge instructions are thus the only document created, with the sole purpose of informing the patient with post-procedure care. However, both orthopedic and urological clinics reported creating a DS in the context of a complex case, wherein a nurse practitioner creates the DS, not the surgeon performing the procedure.

Our finding of missing EMR discharge summaries in surgical departments aligns with existing literature, though our study is the first to report on complete absence of the electronic DS [12]. Most studies report on the illegibility or scarcity of information within the surgical DS, while we found the document to be missing all together [13]. The lack of standardization in the creation of a DS by the surgeon performing the procedure calls for further investigation into departmental- or unit-culture norms. Possibly, there have been no voiced concerns within the healthcare system to incentivize surgeons to create a DS. However, this is dubious, given the extensive national and international literature on both general practitioner and patient dissatisfaction with post-procedure information and outcomes [12,13,14]. There is a dearth in literature on surgeon perspectives toward the necessity of a DS. When broadened from surgical to physician perspectives from other specialties, one study found physicians often do not feel sufficiently informed on the patient’s health history to create a DS. Additionally, avoidance of redundancy of patient information within the EMR is a primary deterrent to creation of a DS [15]. More investigation is needed on the surgeon’s perceived necessity, or lack thereof, for a DS, as a means to understand why surgical cases are often missing a DS within the EMR and paper record.

The reported incentive in surgical departments to create a DS for complex cases can explain the association found in this study’s larger cohort between lower CCI scores and a missing DS. A lower CCI score indicates fewer or less severe comorbidities, which can simplify the procedure. A simple procedure would thus not qualify as “complex” and would not be considered for a DS. Outside of the surgical realm, one study focusing on general medicine found that patients being discharged with a lower CCI score had nearly double the time to dictation from point of discharge (measured in days), compared to those with a higher CCI score, despite low CCI patients having significantly shorter lengths of stay. This apparent lack of urgency to create a DS could be reflected throughout healthcare provider practices for patients with fewer or less severe comorbidities, leading to the absence of a DS in this cohort of patients. Interestingly, the length of the DS (words used) in patients with lower CCI scores did not differ from that of patients with higher CCI scores, which could represent the intrinsic need for communication between inpatient and outpatient physicians [16], regardless of the complexity of the case. Additionally, the associated risks for readmission increase for every 3 days gone without a DS once a patient is discharged [17]. Therefore, for patients where no DS is ever created, the posed health risks could be detrimental.

With regards to the statistically significant finding between those aged 45–64 and the absence of a DS, there is a lack of literature to support this discovery. However, it could be related to the decreased complexity of these patients, and consequent non-urgency for creation of a DS. Compared to those aged 65 and greater, disease prevalence for many of the Charlson comorbidities, as well as the most commonly reported diseases (e.g. hypertension, type 2 diabetes, coronary artery disease), is far less frequent. Therefore, the perceived urgency in reporting on a patient undergoing a simple procedure with few comorbidities could be less compared to the urgency for reporting on a complex patient. This could result in an absence of a DS.

Limitations

Since some of the documents for charts within Calgary medical facilities are still in paper format, this study cannot fully report on the quality of the EMR; rather, it reports on the completeness of the patient medical record. Nonetheless, regardless of paper or electronic documentation, there was still a significant number of records missing a DS entirely.

Within the 893 charts, there is a possibility that charts were captured in the missing DS cohort, when there was actually a paper DS present. Nonetheless, the lack of an electronic DS within the electronic medical record is still relevant. Given the predominantly electronic form of communication between hospitals and general practitioners in Alberta, the inconsistency in availability of documentation in one single location can delay processes for practitioners searching for important health information. Additionally, the delay in receiving a paper document versus quickly accessing the electronic document from a general practitioners office can hinder health delivery services.

Implications of findings

In addition to having negative consequences on patient health outcomes, the absence of a DS can also affect the use of EMR data for secondary purposes [18]. In the process of transforming patient data into a data repository, a coder’s data abstraction and the resulting data quality can be hindered with an absent DS. Within coding practices, the DS is treated as a gold-standard document for coders to abstract, verify, and validate their coding. When the DS is missing, coders must use additional physician documentation (e.g. the History and Physical) to abstract their necessary codes. However, without the DS, if there is inconsistent information across different documents, coders cannot verify their codes against their gold-standard and are forced to omit codes. Therefore, the rigor with which the they can confidently code is consequently. This can lead to missing data within the database the codes are entered in, which decreases the quality of data. This data quality is crucial given its use for national statistics on population diseases and morbidity. Advances in medicine and research will be hindered if the missing DS phenomenon and subsequent poor data quality continues.

Conclusion

In conclusion, our chart review on a large Canadian cohort found a large percentage of incomplete EMR data, as indicated by a missing DS. The absence of a DS within the EMR can present several issues for healthcare processes, including hindered communication between hospitals and general practitioners, heightened risk of readmissions, and poor usability of coded health data. Our study is the first to describe and identify predictors of missing electronic DS. In an era of integrating and relying on the EMR for patient care and secondary data purposes, efforts should be focused on improving DS documentation among surgical wards, middle age patients, or those with fewer comorbidities. The findings of our study will alert researchers to carefully conduct analyses using EMR data, taking into consideration the completeness of the EMR, and its influencing factors.

Abbreviations

EMR: Electronic Medical Record

DS: Discharge Summary

SCM: Sunrise Clinical Manager

CIHI: Canadian Institute for Health Information

CCI: Charlson Comorbidity Index

AIDS/HIV: Acquired Immunodeficiency Syndrome/Human Immunodeficiency Virus

OR: Odds Ratio

CI: Confidence Interval

Declarations

Ethics approval and consent to participate

The Conjoint Health Research Ethics Board (CHREB) approved the ethics application (REB15–0790). Retrospective chart review and administrative data extraction do not require patient consent. A signed data disclosure agreement was signed with Alberta Health Services. Approval from the Health Records Department was obtained to access patient charts.

Consent for publication

Not applicable.

Availability of data and materials

The data that support the findings of this study are available from Alberta Health Services but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of Alberta Health Services.

Competing interests

The authors declare that they have no competing interests.

Funding

This study was funded through CIHR.

Authors’ contributions

The idea for this study and manuscript came from HQ. YX and BB analyzed and interpreted the data. CE, AAS, YX and NW formatted the manuscript. NW performed the sample data abstraction and was a major contributor in writing the manuscript. All authors read and approved the final manuscript.

Acknowledgements

We would like to acknowledge the excellent work from the chart reviewers: Olga Grosu, Chris King, Natalie Wiebe, Ellena Kim, Danielle Fox, and Nicholas Van Kampen. Additionally, we thank the Centre for Health Informatics and CIHI for their ongoing support for this project.

References

1. Cowie MR, Blomster JI, Curtis LH, Duclaux S, Ford I, Fritz F, et al. Electronic health records to facilitate clinical research. Clinical Research in Cardiology 2017; 106(1):1–9. doi: 10.1007/s00392–016–1025–6
2. Dagliati A, Tibollo V, Sacchi L, et al. Big Data as a driver for clinical decision support systems: a learning health systems perspective. Frontiers in Digital Humanities. 2018; 5(8):1–7. doi.org/10.3389/fdigh.2018.00008
3. Centers for Disease Control and Prevention. Use of coded clinical data [World Health Organization-Family of International Classifications Information Sheet]. September 2012 [cited 2019 September 11]. Available from: https://www.cdc.gov/nchs/data/icd/uses_coded_clinicalinfosheet.pdf
4. Doktorchik C, Lu M, Quan H, Ringham C, Eastwood C. A qualitative evaluation of clinically coded data quality from health information manager perspectives. Health Information Management Journal. In-press.
5. Maddox T, Matheny M. Natural language processing and the promise of big data. Circulatory Cardiovascular Quality Outcomes 2015; 8:463–465.
6. Cresswell A, Hart M, Suchanek O, Young T, Leaver L, Hibbs S. Mind the gap: Improving discharge communication between secondary and primary care. BMJ Quality Improvement Reports 2015; doi: 10.1136/bmjquality.u207936.w3197
7. Al-Damluji MS, Dzara K, Hodshon B, Punnanithinont N, Krumholz HM, Chaudhry SI, et al. Association of discharge summary quality with readmission risk for patients hospitalized with heart failure exacerbation. Circulatory Cardiovascular Quality Outcomes 2015; 8(1):109–111. doi:10.1161/CIRCOUTCOMES.114.001476.
8. Schwarz CM, Hoffmann M, Schwarz P, Kamolz LP, Brunner G, Sendlhofer G. A systematic literature review and narrative synthesis on the risks of medical discharge letters for patients’ safety. BMC Health Services Research 2019;19:158. https://doi.org/10.1186/s12913–019–3989–1.
9. Lee S, Xu Y, D’Souza A, Quan H. Unlocking the Potential of Electronic Health Records for Health Research. International Journal of Population Data Science. In-press.
10. Canadian Institute for Health Information. Inpatient Hospitalizations: Volumes, Length of Stay, and Standardized Rates [Quick Stats]. Ottawa Institute, 2019 [cited 2019 September 11]. Available from https://apps.cihi.ca/mstrapp/asp/Main.aspx?Server = apmstrextprd_i&project = Quick%20Stats&uid = pce_pub_en&pwd = &evt = 2048001&visualizationMode = 0&documentID = C6F8B4144B03958E3AE3CAB5DD440EA7&_ga = 2.242198914.1422157057.1568238191–1677101123.1554157741
11. Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD–9-CM administrative databases. Journal of Clinical Epidemiology 1992;45(6):613–619.
12. Spatz H, Engel J, Hölzel D, et al. The surgical discharge summary: a lack of substantial clinical information may affect the postop treatment of rectal cancer patients. Langenbeck’s Archives of Surgery. 2001;386(5):350–356. https://doi.org/10.1007/s004230100240
13. Evans J, Armstrong A. From Zero to Hero, the rise of the Trauma and Orthopaedic discharge summary. BMJ Quality Improvement Reports 2013; doi:10.1136/bmjquality.u201983.w1029
14. Ladds E, Betteridge F, Yamamoto S, et al. Improving the quality of discharge summaries for elective surgical procedures at North Bristol NHS Trust. BMJ Open Quality 2015; doi: 10.1136/bmjquality.u203452.w1552
15. Soong C, Kurabi B, Exconde K, Tajammal F, Bell CM. Design of an orthopaedic-specific discharge summary. BMC Health Services Research. 2016;16(1):545–552. doi:10.1186/s12913–016–1783-x
16. Sakaguchi FH, Lenert LA. Improving continuity of care via the discharge summary. AMIA Annual Symposium Proceedings 2015 November; 1111–1120.
17. Hoyer EH, Odonkor CA, Bhatia SN, Leung C, Deutschendorf A, Brotman DJ. Association between days to complete inpatient discharge summaries with all-payer hospital readmissions in Maryland. Journal of Hospital Medicine 2011;11(6):393–400.
18. So L, Beck CA, Brien S, et al. Chart documentation quality and its relationship to the validity of administrative data discharge records. Health Informatics Journal. 2010;16(2):101–13.

Tables

Table 1. Frequencies of patient and hospital characteristics assessed for association with missing discharge summaries in the EMR

Table 2. Frequencies of comorbidities assessed for association with missing discharge summaries in the EMR