We screened in total 1,309 publications of which twenty-seven studies meet our inclusion criteria for this economic review.[5, 13–38] The process of our literature search as well as the reasons for excluding a number of studies is provided within the PRISMA flow-diagram in Fig. 1. Furthermore, an overview of the characteristics of included studies is listed in Table 2.
Category Number of studies (% of total, rounded)
Country
UnitedStates 24 (89%) [5,13-18,20,22,23,25,26,27-38]
Canada 3 (11%) [19,21,24]
Year published
2019 6 (22%) [13-18]
2018 6 (22%) [5,19-23]
2017 5 (19%) [24-28]
2016 2 (7%) [29,30]
2015 3 (11%) [31,33,34]
2014 5 (19%) [32,35-38]
Study design
Cluster randomized trial 4 (15%) [5,19,26,33]
Cross-sectional 1 (4%) [28]
Retroprospective 9 (33%) [15,17,18,20,21,27,32,36,38]
Quasi-experimental 5 (19%) [14,16,22,24,35]
Comparative 1 (4%) [31]
Observational 1 (4%) [23]
Pre-post-intervention 6 (22%) [13,25,29,30,34,37]
Setting
Inpatient 14 (51%) [14,16,17,20,22-25,27,31,32,34,36,38]
Outpatient 8 (30%) [5,13,15,19,26,30,33,34]
Inpatient & Outpatient 4 (15%) [21,28,29,37]
Emergency department 1 (4%) [18]
Type of economic evaluation
Basic cost calculation 23 (85%) [13-25,27-31,32,34,36-38]
Modelapproach 4 (15%) [5,26,33,35]
Table 2:
Characteristics of included studies (n = 27)
Generally, twenty-two studies (81%) [5, 13–16, 18, 20–25, 28–31, 32, 34–38] out of the included twenty-seven studies report cost savings after implementing an EHR based CDS intervention. Three studies (11%) [17, 26, 33] report a rise in cost expenditure, one study (4%) [19] did not detect significant differences in cost outcomes, and one study (4%) [27] compares the economic outcome of two similar and subsequently explored EHR based CDS interventions. Furthermore, in the majority of included studies the main cost outcome measures were laboratory test cost.[15–17, 20, 21, 25, 28, 29, 31, 32, 38]
Exploitation Of Different Front-end Cds Intervention Categories
According to the taxonomy by Wright et al.[12] we identified twelve (44%) studies [5, 13, 15, 20, 22, 23, 26, 31, 32, 36–38] which explored EHR based CDS interventions based on point-of-care alerts or reminders (category 3). In addition, three interventions (11%) [17, 27, 34] were order facilitators (category 2), two studies (7%) [19, 30] investigated medication dosing support (category 1), while relevant information display as well as expert systems (category 4 and 5) were each reported only once from an economic perspective (4%).[18, 24] In the remaining five studies [14, 16, 28, 33, 35] interventions from two different categories were explored in combination. Finally, we found three studies [21, 25, 29] in which the option to place a certain order or test, e.g. a laboratory test, was removed from the EHR CPOE system or the clinician’s laboratory ordering preference list. That was also partly explored along with the so called “hard-stop” alert, which requires a clicking response from the physician before being able to move forward.[14] These restrictive frond-end CDS intervention types were not yet mentioned in the pre-defined categories by Wright et al.[12]. Thus, for this study we extend their taxonomy by a new category
7. Restriction of choice[39]
The removal of an order option ultimately resulted in less laboratory tests ordered, and therefore in a reduction of healthcare expenditure in all five studies it was implemented [14, 21, 25, 28, 29].
Economic Impact For Prevalent Application Areas
In Table 3, we summarized our findings and created an overview of application areas and cost outcome measures in relation to the applied CDS intervention types. Due to the heterogeneity of included studies with regard to different types of cost outcomes reported and different intervention duration it was not possible to conduct a subgroup analysis considering the economic impact of each CDS-front end category. A detailed evidence synthesis of all included twenty-seven studies as well as a brief description of their intervention types, their application area and the resulting economic impact is provided in an additional file [see Additional file 3].
Table 3
Application areas and cost outcome measures in relation to CDS intervention categories 1.-7.
| Study | Size¹ | Application area | CDS intervention period (in month) | Cost outcome (per year, in US$, if not other stated) ² ³ |
| 1. Medication (dosing) support |
| Tamblyn [19] | Medium | Reduce out-of-pocket costs for patients with uncomplicated hypertension | 60 | No difference⁴ |
| Stenner [30] | Large | ePrescribing tool for therapeutic interchange prescribing | 18 | - $812,956 |
| 2. Order facilitator |
| Bolles [17] | Small | Inappropriate test ordering for specialized HIV laboratory testing | 6 | + $14,000 to + $92,000 |
| Schnaus [27] | Large | The order “complete blood count without differential” unintentionally changed to “complete blood count with differential” | 23 days | + $87,275⁵ |
| Shaha [34] | Small | CDS order sets for managing new-onset stroke patients | 6 | - $460,000 to - $1,130,000 |
| 3. Point of care alerts or reminders |
| Chen D [13] | Large | Reduce unnecessary imaging studies in patients with low back pain | 12 | -$1,872,000 |
| Chin [15] | Large | Decrease routine testing for 25(OH) vitamin D levels | 12 | -$300,000 |
| Gong [5] | Medium | Inappropriate antibiotic prescribing for acute respiratory infection | 18 | -$500,000 per 30 years and 100,000 individuals ⁸ |
| Bejjanki [20] | Large | Reduce 17 frequently used duplicate laboratory tests | 17 | - $51,206 |
| Chen JR [22] | Small | Directing the physician to order penicillin allergy testing for patients receiving aztreonam | 9 | -$673.73 per patient |
| Heekin [23] | Large | Adherence to 18 different Choosing Wisely (CW) alerts | 36 | -$944 per patient |
| Sharifi [26] | Small | Clinical childhood obesity intervention | 12 | + $175mill. in 10 years ⁸ |
| Procop (a) [31] | Medium | Unnecessary duplicate laboratory testing | 12 | - $94,225 (Hard-Stop) - $45,681 (Smart-Alert) |
| Procop (b) [32] | Large | Reduce unnecessary, same day duplicate orders | 24 | - $91,793 |
| Goodnough [36] | Large | Reduce overutilization in blood transfusion procedure | 36 | - $1,620,000 |
| Razavi [37] | Small | Reduce unnecessary waste in transfusion practice and blood use of cardiothoracic surgeons | 12 | - $62,715 |
| Bridges [38] | Small | Reduce unnecessary acute hepatitis profile laboratory tests | 3 | - $13,580 |
| 4. Relevant information display |
| Fertel [18] | Small | Reduce the amount of frequent or high emergency department utilizers | 24 | - $3,306 average monthly per patient care plan |
| 5. Expert systems |
| Nault [24] | Large | Antimicrobial stewardship that facilitates the post-prescription review process | 36 | - CAD $116,666 |
| 6. Workflow support |
| none | - | - | - | - |
| 7. Restriction of choice |
| MacMillan [21] | Large | Reduce unnecessary frequent red blood cell folate tests | 43 | - CAD $78,180 |
| Sadowski [25] | Medium | Reduce admission order sets, which allowed multiple routine tests to be ordered repetitively | 2 | - $152,496 |
| Konger [29] | Large | Define order frequency rules and reduce duplicate tests | 24 | - $157,782 |
| Studies with combined multiple CDS intervention categories |
| 3. Point of care alerts or reminders & 7. Restriction of choice |
| Marcelin [14] | Large | Reduce inappropriate gastrointestinal pathogen panel testing | 15 | -$51,600 |
| Felcher [28] | Medium | Reduce unnecessary Vitamin D testing | 6 | - $2,800,000 |
| 2. Order facilitator & 3. Point of care alerts |
| Goetz [16] | Large | Decrease serum folate laboratory testing | 12 | -$26,719 |
| 2. Order facilitator & 6. Workflow support |
| Michaelidis [33] | Medium | Reduce inappropriate antibiotic prescribing for acute bronchitis | 6 | +$34 ⁷ ⁸ |
| 1. Medication (dosing) support & 3. Point of care alerts or reminders |
| Forrester [35] | Medium | CPOE CDS vs. paper-based prescribing in reducing medication errors and adverse drug events (ADE) | 10 | - $18 mill. per 10.000 Monte Carlo simulations ⁸ |
| ¹ Size is defined as the following: |
| Number of patients or encounters involved |
| 0-999 small size |
| 1,000–10,000 medium size |
| > 10,000 large size |
| If patient count was not reported, we applied this range of criteria to the amount of triggered alerts in total |
| ² Total cost outcome in the EHR based CDS intervention group compared to the control group or pre-implementation phase. |
| ³ All cost outcomes were scaled and thereupon calculated to overall cost outcome per year. Values are rounded to full integer numbers. Because of the predominantly short CDS intervention period time range, a discount factor is not used for calculation. The original reported cost data is mentioned in an additional file [see Additional file 3]. |
| ⁴ No statistically significant differences between control and intervention group regarding out-of-pocket costs per patient: CAD$252.6 (control) and CAD$261.5 (intervention) / patient year. Similar values for currently treated patients as well as mean annual costs of antihypertensive treatment (CAD$370.90 control and CAD$385.70 intervention) |
| ⁵ Comparative study: Expenditure increase resulted from the unintentional change within the EHR based CDS system |
| ⁷ 5-year societal cost per five cases of acute bronchitis |
| ⁸ Cost estimation based on model |
Application areas for cost-savings
Thereupon, we identified four main application areas based on their investigated prevalence that resulted in cost-savings after EHR based CDS implementation. Firstly, two studies report on essentially reducing unnecessary Vitamin D routine testing that led to a decrease of laboratory test cost of
$300,000[
15] and
$1,4mill.[
28] per year.
Secondly, two studies addressed the economic outcomes of the reduction of waste in transfusion practice and red blood cell usage.[36, 37] Acquisition product cost of red cell units were decreased with the help of EHR based CDS and resulted in cost savings of in total $4,821,000 within three years[36] and about $62,715 within one year[37] after implementation, respectively.
Thirdly, two cost-effectiveness-analyses modeled the cost outcome of reducing antibiotic prescriptions for acute respiratory infection as well as for acute bronchitis.[5, 33] Gong et al.[5] include a full accounting of costs into their Markov model and explore that the implemented CDS intervention, called “suggested alternatives”, yielded more quality adjusted life years (QALYs) at a lower cost of $500,000 per 100.000 individuals over thirty years of implementation. Michaelidis et al.[33] on the other hand report a small increase in costs compared to a printed decision support system, i.e. posters. However, the outcome of the latter mainly results from a cost difference between the direct costs of poster printing and the computer programming cost.
Lastly, five studies[20, 29, 31, 32, 38] report on the potential for cost savings through reducing duplicate orders or laboratory tests by using hard-stops[32] or applying order frequency rules[20] to prevent ordering the same test within a certain timeframe. Reducing laboratory duplicate tests resulting in savings of $3,395 in three months for a small patient size cohort[38] and up to $315,565 within twenty-four month for a large patient size cohort.[29]
Application areas resulting in cost increase
Furthermore, we also identified risk areas, which possibly lead to a further increase in healthcare expenditure. One study found that after implementing a CPOE system with default settings, specialized HIV laboratory test cost increased by
$14,000-
$96,000 within six months.[
17] Another study reports that an unplanned change of a pre-selected default order for ‘complete blood count’ to ‘complete blood count with differential’ lead to an average cost increase of
$293.11 per day.[
27] Finally, the implementation of order sets as decision facilitators possibly entail negative economic effects. One study found that only after the uncoupling of Vitamin B12 and serum folate joint orders within predefined order sets, laboratory test cost decreased by about
$26,719 per year.[
16] Similarly, another study removed the option to order daily routine tests from automated admission order sets and found savings of
$26,416 after two months.[
25]
Cost-effectiveness-analyses Models
Table 4 encompasses an overview of studies which conducted a cost-effectiveness-analysis (CEA) of EHR based CDS interventions considering various cost data as well as economic outcome measures, such as the incremental cost-effectiveness ratio (ICER), which depicts the incremental change in costs divided by the incremental change in health outcome or effect.
Table 4
Overview of cost data and cost outcome of model-based studies (n = 4)
| Study | Model time horizon (years) | Choice of model | Implementation and maintenance cost | Total budget impact | ICER |
| Gong et al. [5] | 30 | Markov model | $1.91 base case for 100,000 individuals [preexisting EHR] | CDS intervention $17.32 mill. Control $17.82 mill. | $99.8 per QALY in base case scenario Not directly reported |
| Sharifi et al. [26] | 10 | Monte Carlo micro-simulation | $23,542 per PCP group [preexisting EHR] | CDS intervention +$239 mill. | $237 per BMI unit reduction |
| Michaelidis et al. [33] | 5 | Decision analytic tree | $18 base case - medical record programming [preexisting EHR] | CDS intervention $2,802* Control (usual care) $2,768* | $51.51 per antibiotic prescription safely avoided Not directly reported |
| Forrester et al. [35] | 5 | Decision analytic tree | $1,773,000 five years CPOE system costs | CDS CPOE system $25 mill. Control (paper system) $43mill. | $110 per ADE averted† |
| * Cumulative 5-year societal cost per five cases of acute bronchitis |
| † Documented only for the explored modelling scenario no. 2: The Everett Clinic achieved no reduction in paper chart pulls throughout the 5-year time horizon, to explore the effect of inefficiency from running a paper and electronic system in parallel |
|
|
|
|
Cost-effectiveness-analyses aim to reveal the trade-offs in resource-allocation decisions.[40] In this context, it is essential to investigate when and to what extend upfront and maintenance cost for an EHR based CDS system will be amortized by its benefits, which again can be measured either in health outcomes, such as quality adjusted life years (QALYs) saved or in reduction of unnecessary healthcare utilization.
Generally, two studies report an increase in healthcare expenditure from a societal perspective, [26, 33] while the other two report cost savings from a societal perspective as well as the medical group’s perspective.[5, 35] Notably, the measurement of effectiveness was single study-based estimates in all four studies.
Regarding the consideration of upfront implementation cost, Gong et al.[5] include only base case consolidated cost data of $1.91 for a cohort of 100.000 individuals based on expert opinions. Sharifi et al.[26] include intervention start-up cost for EHR modification of $2.7mill. as well as other direct cost, such as professional care provider training. Michaelidis et al.[33] report implementation and maintenance cost data, which is physician education per hour and medical record and CDS programming per patient of $18 in the base case. Lastly, Forrester et al.[35] report CPOE CDS system cost as hardware, software and maintenance cost starting from $373,000 in year one to $92,000 after five years, as well as personnel, $555,000 in year one, and indirect cost as 3% of the total cost. Interestingly, the latter also include the HITECH Meaningful Use incentives in their model in order to simulate the financial incentives by the Centers for Medicare & Medicaid Services in the US.
Lack Of Considering All Cost Components
Despite revealing major potentials for cost-savings, we could not asses the quality of included studies, because of the lack of cost information provided, or predominantly the lack of considering all relevant cost components. According to the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement, most of the reported recommendations were not satisfied.[41] All twenty-three non-model studies (85%) only calculate the economic outcome based on financial data reported before and after intervention implementation, which for instance, ultimately results from the computation of price per healthcare resource utilization times the quantity of used healthcare resources or services. Thus, even though it was not intended in those studies, it is necessary to mention that only four of them adhered to sound economic evaluations as recommended by CHEERS.[5, 35]
The challenge of heterogeneity for the CEA is also aggravated by considering different cost outcomes considered. Two studies do not directly report an incremental cost effectiveness ratio (ICER) for a predefined threshold, nor include comparative metrics.[33, 35] Other standardized metrics, such as the return on investment or net present value, were also not examined in the included studies. Only one study reported the net monetary benefit (NMB) of the intervention in relation to a predefined threshold.[5, 42]
Additional Studies Worth Mentioning
Notably, five more studies [43–47] meet most of our inclusion criteria, but were excluded due to various, although little, deviations. Three studies [43–45] report cost-savings after a bundle of information technology was implemented simultaneously, but the economic benefit could not solely be attributed to the EHR based CDS intervention. The fourth publication is a NHS health technology assessment (HTA) report.[46] In this HTA, a RCT was conducted in 79 general practices in the UK in which a multicomponent intervention was installed using electronic health records in order to reduce antibiotic prescribing for respiratory infections. The authors perform a basic cost-analysis on whether the cost of healthcare utilization, that is the number of provider consultations, will increase during the time of the trial under the CDS intervention arm, and if patients more often re-consult the physician when not given a prescription. However, the authors explored no difference in cost outcome between the intervention and control period.
The last study worth mentioning compared retrospectively generated alerts by an advanced machine learning CDS system with alerts triggered through the home-grown EHR based CDS system.[47] The authors calculated the healthcare costs of potentially prevented adverse drug events and medication errors, and found that by using the advanced machine learning CDS system 68,2% of alerts were only fired by that new system resulting in cost savings of $60.67 per alert.[47] After extrapolating these results to an local patient population of 747,985 over five years they estimated savings of $1,294,457.[47]