What are the Effects of Computerized Decision Support Systems on Practitioner Performance and Patient Outcomes

Background Computerized decision support systems (CDSS) are software programs that support the decision making of practitioners and other staff. Other reviews have analyzed the relationship between CDSS, practitioner performance, and patient outcomes. These reviews reported positive practitioner performance in over half the articles analyzed, but very little information was found for patient outcomes. The purpose of this review was to analyze the relationship between CDSS, practitioner performance, and patient medical outcomes. PubMed, CINAHL, and Cochrane databases were queried.Methods 27 articles were chosen based on year published (last ten years), high quality source, and discussion of the relationship between the use of CDSS as an intervention and links to practitioner performance or patient outcomes. Reviewers used an Excel spreadsheet to collect information on the relationship between CDSS and practitioner performance or patient outcomes. Reviewers also collected observations of participants, intervention, comparison with control group, and outcomes (PICO) along with those showing implicit bias. Articles were analyzed by multiple reviewers following the Kruse Protocol for systematic reviews. Data were organized into multiple tables for analysis and reporting.Results Fourteen articles (52%) discussed positive practitioner performance, three articles (11%) found no difference in practitioner performance, ten articles (37%) did not discuss practitioner performance. Zero articles reported negative practitioner performance. Fifteen articles (56%) discussed positive patient medical outcomes, two articles (7%) found no statistically signi�cant difference in medical outcomes between intervention and control groups, and ten articles (37%) did not discuss medical outcomes. Zero articles found negative patient medical outcomes.Conclusions Results of this review are commensurate with previous reviews with similar objectives, but unlike these reviews we found signi�cant positive reporting of a positive effect on patient medical outcomes. Our �ndings support adoption of decision support systems.


Background Rationale
Computerized decision support systems (CDSS) are software programs that support the decision making of practitioners and other staff.There are two major classes of CDSS the rst being knowledge based CDSS and non-knowledge-based CDSS (1).Knowledge-based CDSS were the earliest classes of CDSS using a data repository to draw conclusions.Knowledge based systems use traditional computing methods giving programmed results.Non-knowledge based CDSS systems are growing rapidly and use AI assistance to make clinical decisions.AI supported CDSS uses patient data to analyze the relationships between symptoms, treatments, and patient outcomes to make clinical decisions.This patient data is usually derived from electronic health records: digital forms of patient records that include patient information such as personal contact information, patient's medical history, allergies, test results, and treatment plan (2).Data-mining: a process usually assisted by AI is often used by CDSS to identify new data patterns from large data sets (like patient EHR's), while AI is used for data mining, these conclusions can be used by both non-knowledge based CDSS and knowledge-based CDSS (3).CDSS systems are integrated into technologies such as computerized physician order entry (CPOE)(4) tools and electronic medical record (EMR) databases, they can use a wide variety of data to make clinical decisions such as drug data, patient data, treatment data, and more to provide the best recommendations for treatment.CDSS utility varies widely drawing conclusions about many different ailments, disorders, and syndromes, prospects for this technology may employ patient preferences or nancial capabilities.In order to determine the effects of CDSS is having on practitioner performance and patient outcomes, we conducted a systematic review compiling data from studies to draw a renewed conclusion.Our research used a wide variety of peer reviewed journals gathered using PUBMED and CINAHL to ensure the quality of our study.
In prior studies, CDSS has proven to improve practitioner performance but the effects on patient outcomes were inconsistent and require further study.This review was conducted in 1998, evaluating studies since 1992 (5).It found a bene t to physician performance in 66% of studies analyzed (n = 65), but only 14 of those analyzed discussed outcomes, so no conclusions were made.It was repeated in 2005 with a larger sample (6).It found a positive impact on physician performance in 64% of studies analyzed (n = 100), but again, effects on patient outcomes were insu cient to make generalizations.In 2010, a research protocol was registered to repeat the review, but no publication followed.In 2011, the review was repeated with a similar size of articles analyzed (n = 91) (7).It identi ed a positive effect of CDSS on practitioner performance for 57% of articles analyzed, however, consistent with previous reviews, no conclusions could be made concerning patient outcomes.
Since the last publication on this topic in 2011, CDSS has seen signi cant industry growth becoming more accessible, cost effective, reliable, and possessing greater computational power (8).In addition to these hardware improvements, the inclusion of software such as arti cial intelligence (AI) programs are growing rapidly in CDSS use but as of yet these improvements have not been systemically reviewed to see the overarching impacts they are having on patient outcomes and practitioner performance.

Objective
The purpose of this study is to repeat the study from 1998 and 2005 to analyze the effects computerized decision support systems (CDSS) have on practitioner performance and patient outcomes.CDSS employment is rapidly growing especially with increased access to CDSS AI supported software, however the effects are understudied.Our goal is to review the effectiveness of CDSS technologies, their employment, and their overall utility.Our research also intends to identify inhibitors to adoption and causes of poor outcomes.

Eligibility Criteria
The methods followed a measurement tool for the assessment of multiple systemic reviews (AMSTAR) (9).The format of the review uses the Preferred Reporting Items for a Systemic Review and metaanalyses (PRISMA) (10).Conceptualization of the overall review follows the Kruse Protocol for writing systematic reviews in health-related program (11).Articles were eligible for inclusion if they were published in a peer-reviewed journal in the English language within the last 5 years, whose full text were available, and they addressed the elements of the objective statement: measures of effectiveness of CDSS on practitioner performance or patient outcomes.Barriers were not listed in the search terms, but reviewers watched for barriers listed in the articles analyzed.A 5-year window was justi ed because we wanted the research to be up to data and prior research on the same subject was made in the last several years.We limited the search to peer-reviewed journals to ensure some element of quality to the papers we were analyzing.

Information Sources
Three common research databases were queried: PUBMED (the web-based components of Medline, life science journal, and online books) and Cumulative Index of Nursing and Allied Health Literature (CINAHL), and Cochrane (reviews, control trials, methodologies, and health technology assessments).
Searches were conducted from February 25 th to February 28 th 2019.These databases were chosen at the recommendation of the National Institutes of Health which recommends at least three databases: PubMed, Embase, and Cochrane (12).Our university does not have access to Embase, so CINAHL was chosen due to its wide availability to other researchers.This practice also follows the Kruse Protocol (11).

Search and Study Selection
Searches in each database were identical: "Clinical decision support systems" AND ("patient reported outcomes" OR "practitioner performance").Articles were eligible for inclusion if they were published in the last 10 years and discussed both clinical decision support systems and either practitioner performance or patient reported outcomes.

Results
The 41 results from the search string in three databases were placed into an Excel spreadsheet and shared among reviewers for selection and analysis.Filters were applied in each database to capture only the last 10 years (Feb 28, 2009 -Feb 28, 2019).Reviewers independently removed duplicates and screened abstracts.A statistic of agreement, kappa, was calculated.The kappa score produced was .98 showing almost complete agreement on all reviewed articles (13,14).The remaining 30 results were read in full for relevance.Observations for the 27 articles that remained were placed in an Excel spreadsheet for independent data analysis.Observations were summarized into a nity matrices for further analysis.Reviewers collected standard PICO observations plus indications of either practitioner performance or patient medical outcomes.Bias was also noted.Following the Kruse Protocol, observations were distilled into themes for further analysis.A summary table of all observations is listed in Table 1.Articles are listed in reverse chronological order.

Table 1: Summary of analysis
Eleven themes were identi ed for practitioner performance, two of which were not discussed and no difference.These themes are listed in Table 2 and are listed in order of occurrence rst for positive effect followed by no difference and not discussed.
Eight themes were identi ed for patient medical outcomes, two of which were not discussed and no difference.These themes are listed in Table 3 by order of greatest occurrence for positive effect followed by no difference and not discussed.

Bias
Appendix A provides a table of PICO (Patient/participants, intervention, comparison, outcome) and bias.Outcomes are reported in Table 1.Bias was similar across articles reviewed: Most research took place in one facility, organization, or state which is a form of selection bias.A sample taken from a limited geographic area is inherently limited in its ability to generalize results to the general population unless steps have been taken to ensure the sample is representative of the population.

Discussion
Our review methodology enabled a meticulous evaluation of the e ciency and effectiveness of CDSS for practitioners and patients.A summary of the ndings from the review are listed in Table 1.Of the 27 articles analyzed that reported e ciency or effectiveness, 14 and 15 reported positive performance or outcomes, respectively.Ten articles did not report either practitioner performance or patient medical outcomes.
Commensurate with previous reviews on this topic, a majority of articles analyzed reported improvement in practitioner performance (6,7), but contrary to these previous reviews, this review found articles that reported patient outcomes, and a majority of these were positive outcomes.Although ten articles did not discuss practitioner performance, only three articles reported no difference in productivity.
Practitioners using CDSS experienced more accurate prescribing, improved screening, improved productivity, increased awareness of needs of patients, improved follow-up with patients due to enhanced communication channels enabled by the application, improved accuracy, improved documentation, improved benchmarking, improved care plans, and overall improved buy-in for CDSS.Patients experienced improved symptoms, improved e cacy, improved disease management, improved safety, improved mortality, improved screening, and improved feedback.Although ten articles did not discuss patient medical outcomes, only two reported no statistical difference in outcomes between control and intervention groups.

Limitations
The limited group of articles for analysis was a limitation.Only 27 articles met the selection criteria.A larger group for analysis would strengthen the external validity of the results because we could be better assured that our group is representative of the population.The effects of selection bias were reduced through the use of multiple reviewers to screen and analyze articles (9).Only two reviewers screened abstracts and analyzed articles for themes.One additional reviewer might have increased the number of observations.Publication bias was reduced through the inclusion of grey literature that included more than just peer-reviewed material.We considered only articles published in the English language.It is possible that additional observations could have been gained by expanding the search to other languages.This review is also limited by the techniques used in the trials analyzed, and the statistics and effect sizes could not be combined due to the wide range used by the articles.We also did not analyze or compare the heuristics and algorithms used by CDSS systems within the studies.To compensate for a limitation from a similar review in 2005, we expanded our analysis beyond randomized control trials to pre-post and other designs (6).This gure portrays the PRISMA ow diagram including the records identi ed through databases, the number screened out, full-text articles assessed for eligibility, and nal group chosen for analysis.

Table 3 :
of themes identified for patient medical outcomes