Factors that impact the adoption of clinical decision support systems (CDSS) in healthcare settings for evidence-based decision making

Background: Clinical decision support systems (CDSS) are designed to promote evidence-based patient care and shared-decision making in healthcare settings. Despite these benets, adoption and long-term use of the systems remain limited. There is a need to identify different factors that inuence CDSS adoption in healthcare settings. Objective: The Unied Theory of Acceptance and Use of Technology (UTAUT) model was applied to determine different factors that inuence the adoption of CDSS in healthcare settings. Methods. A cross-sectional online survey examining clinicians’ perceptions about CDSS implementation in hospital and primary care settings in Australia was undertaken from June - October 2019. Multivariate logistic regression was used to examine the association of UTAUT moderators (age, gender and experience) and care settings (hospital and primary care) with perceived benets, barriers and facilitators to CDSS adoption. Results: Access to information required for evidence-based decision making, and improvements in quality and safety of patient care were the most common perceived benets of CDSS. Mapping of different barriers and facilitators to the UTAUT model indicated that ease of use (effort expectancy), perceived benet (performance expectancy) and a facilitating environment greatly inuence the adoption of CDSS. Respondents indicated that systems providing a better t between relevance, content and timeliness greatly facilitates uptake. Adoption of CDSS also depends on the ability of an organisation to create a facilitating environment that can help address the lack of users’ trust in these systems. The type of healthcare setting was found to be a signicant predictor of lack of condence in the content within CDSS, threat to professional autonomy, and time constraints as barriers to CDSS implementation. Therefore, setting dynamics as well as user-specic requirements need to be considered to improve the acceptability and use of CDSS. Conclusion: Our study has explored different factors that may help address implementation challenges for CDSS. By combining internal factors of users’ inclination and perceptions about the system’s perceived benets, coupled with external factors of system design requirements, training and support, and stakeholders’ consultations, our ndings highlight the need for a holistic implementation framework to enable effective CDSS adoption.


Introduction
Antibiotic resistance has been recognised as one of ve major global health threats. It is estimated that by 2050 drug-resistant infections will lead to 10 million deaths worldwide [1]. Inappropriate antibiotic prescribing has been identi ed as a major driver to increasing antibiotic resistance [2]. Standardising clinical practices, improving quality and safety of care and reducing inappropriate prescribing have become the top priorities for antimicrobial stewardship programs. Digital health technology tools are quickly becoming an essential part of the healthcare system due to the increasing need to deliver valuebased care [3,4].
Clinical decision support systems are systems that provide clinicians with information for point of care decision making. By connecting evidence-based information on appropriate prescribing (a knowledgebase) with patient information, these systems lter and present the right information at right time to clinicians to assist their decision making and prescribing behaviour [5,6]. Bene ts of CDSS for antibiotic stewardship include optimising the prescribing process by allowing feedback and audit as well as increasing compliance with antibiotic prescribing guidelines [7,8].
Despite increasing evidence regarding CDSS bene ts, the adoption and uptake of CDSS by clinicians (the end-users) remains very limited [9]. Most of the research that has been carried out at the intersection of CDSS and antibiotic management tend to focus on the systems' technical design. The research has overlooked the fact that success of the digital system is equally dependent on end-users inclination to use the system [10]. Previous studies show that CDSS adoption is far less than anticipated with 96% of alerts or recommendations ignored [11,12]. Implementation of these systems can be very challenging due to the complex interaction of system, organisational and human factors. Given this, a socio-technical assessment of the implementation process may identify barriers in the adoption and use of CDSS. Our study aim was to identify different factors at the individual, organisational and system level that in uence the adoption and use of CDSS. Evaluation of the perceptions and insights of clinicians regarding enablers and barriers to CDSS implementation will enhance an understanding of non-technical factors that may in uence the long-term successful implementation of these systems.

Methodology Theoretical Framework
We have used the Uni ed Theory of Acceptance and Use of Technology (UTAUT) model to understand the interplay between different organisational, individual and technical factors that may in uence the successful adoption and use of CDSS [13]. Four basic constructs of this model, namely effort and performance expectancy, social or environmental in uence and enabling conditions, helped us to determine speci c factors related to users' intentions to adopt a CDSS. The UTAUT integrates a range of theoretical models to provide a comprehensive framework for investigating user intention and acceptance of a technology [14]. Fig. 1 provides the theoretical framework utilised in this study.

Questionnaire instrument
The survey questions were designed following an extensive literature review of similar studies [15][16][17][18][19][20]. The questionnaire comprised of 15 questions covering four sections: demographics; availability of electronic prescribing systems and the perceived impact; and barrier and facilitators for use of CDSS. The purpose of the survey was to identify factors that in uence the adoption and acceptance of CDSS in healthcare settings. Questions about the clinicians' demographic characteristics and availability of electronic prescribing systems in respective practices were included to evaluate potential differences in system adoption that might be related to these characteristics. Questions regarding the perceived bene t, barriers and facilitators of CDSS could be answered using ve point Likert-type scales, where 1 represented "Strongly Disagree"; 2= "Disagree"; 3= "Neither Agree Nor Disagree"; 4="Agree" and 5= "Strongly Agree". Most of the survey questions were closed; however, an open-ended question and comments section were also included in the survey to allow respondents to provide additional comments. These responses were then analysed using Braun and Clarke's (2006) thematic analysis process. All the similar codes were grouped into themes which were reviewed iteratively for the validity of each theme.
The survey was conducted online through SurveyMonkey TM (www. surveymonkey.com, San Mateo, California, US). The online survey was initially pilot tested with known clinical contacts (n=10) to evaluate its acceptability by a clinical workforce before it was distributed widely. After reviewing the results in the pilot phase modi cations were made by the research team to the survey's text.

Measures
As it was not mandatory for participants to provide a response to all questions, the number of responses for each question was calculated separately. The information on demographic characteristics and availability of electronic prescribing systems was used to evaluate the impact of these factors on the clinicians' perception regarding CDSS adoption and use. Similarly, moderating factors (age, gender, and experiences) proposed in UTAUT model also helped to investigate how these factors moderate the association between model constructs and the intention to use CDSS.
Dichotomous dependent variables were generated from the Likert 5-point scale for each question by collapsing the responses into binary responses of "Yes" and "No/Unsure" to represent participants' perceptions regarding bene ts, barriers, and facilitators for CDSS implementation. The responses of "Agree" and "Strongly Agree" were combined to formulate a binary response of "Yes", whereas "Disagree" and "Strongly Disagree" responses were coded "No/Unsure". The neutral response of "Neither Agree nor Disagree" was also included in the "No/Unsure" category due to low responses and could not be included as a separate category. The rationale of combining neutral with negative responses was that the focus of analysis was identifying participants who have positively responded to the survey questions.

Statistical Analysis
Bivariate logistic regression analyses were carried out for an exploratory evaluation of the association between binary responses and respondents' characteristics of age, gender, experience, care settings, type of practice and availability of electronic prescribing systems. Associations with a signi cance level <0.2 were included in the multivariate logistic regression model to examine the effect of respondents' characteristics on binary responses. Using backward elimination, variables with a signi cance level of 0.05 were retained in the nal model and included age, experience, and type of care setting. The results are provided as an Odds Ratio (OR) with 95% con dence interval (CI).
All the statistical analysis on the data was performed using Stata 15 (StataCorp LP, College Station, TX).

Qualitative Analysis
The survey incorporated an open-ended question to obtain information on any speci c concern(s) participants had about CDSS. The open-ended responses were analysed for contextual content using NVivo 11 (QSR International Pty Ltd, Doncaster, Victoria, Australia). The preliminary codes were generated through open coding of the qualitative data [21]. Using recursive comparison, these codes were then re ned and merged into conceptual themes.

Ethical Approval
Ethical approval was obtained from the University of Adelaide Human Research Ethics Committee (approval number: H-2019-094). The participants could refuse to provide response to any question. Participation in the survey was voluntary and the data collected was non-identi able. To off-set the expected low survey response rates the respondents were given the opportunity to participate in a draw either to win an iPad or equivalent donation will be made in their name to the Hospital Research Foundation in recognition of their participation.

Response rate
A total of 180 clinicians participated in the survey with 74 from primary care (general practitioners) (response rate 21.1%) and 106 from hospital care (physicians and surgeons) settings (response rate 29.4%). Our response rate is consistent with many other international and Australian studies conducted with a clinical workforce. The literature reports a 30-70% response rate; however, lower than 20% has also been reported [22][23][24][25]. A national longitudinal survey of medical professionals in Australia reported an overall response rate of 19.36%, with 17.6% of GPs replying and 22.3% of specialists [26]. Missing values for questions ranged from 8.9% to 16.4%.

Characteristics of study participants
Participant demographic characteristics are described in Table 1. The majority (66%) were males and aged between 35 -54 years (47%). Most respondents (42%) had 11-20 years of experience working in the healthcare sector. More respondents (56%) resided in the eastern states of Australia.  (6) Hospital-based clinic 12 (7) Mixed 25 (14) Total 63 (35) State and territory, Australia (n=139) Central (SA/NT) 21 (15) Western (WA) 17 (12) Perceived bene t of CDSS Overall, respondents had access to a variety of electronic systems/modules, with 76% having patient information management (PIM) systems in their practices, 69% medication management (EMM) systems, and just over half (52%) had any form of clinical decision support system (CDSS) available for their use. Of all the respondents who had access to CDSS in their practices, there was a higher proportion accessing them in primary care (62%) than in hospital settings (38%).
In terms of perceived bene ts, approximately half agreed (48%) that CDSS can help in reducing the overall cost of antibiotic therapy leading to more cost-effective treatment (Fig. 2). This contrasted with 79% of respondents who agreed that implementation of CDSS can increase the accessibility to information for optimal decision-making (Fig. 2). However, as shown in Table 2, this latter association was strongly associated with healthcare setting. Clinicians in primary care were 76% less likely than those in hospital settings to believe that accessibility to protocols and guidelines can be improved There was variability in responses overall, with several associations having wide con dence intervals. Of all respondents, 69% indicated lack of technical knowledge and training are an important barrier for CDSS adoption. On the other hand, 63% of respondents believed that end users' lack of trust and con dence in the content of the system limits the usability of CDSS. Other frequently identi ed barriers that may make it di cult for end-users to interact with CDSS effectively concern the dynamics of the clinical environment (Fig. 3).  Table  3). We found that 71% of respondents in primary care had clinical experience of >11 years compared to respondents in hospital settings (54%). Therefore, the association of healthcare settings with threat to professional autonomy as a barrier may be related to a higher proportion of experienced respondents in the primary care group. Overall, experienced clinicians were also more likely to believe that lack of con dence in the content of the CDSS would inhibit its use (>20 years OR 1. Perceived Facilitators Figure 4 highlights strong agreement (75%) among respondents that when systems are easy to use this facilitates CDSS adoption whereas 64% believed that organisational support is required for successful implementation. Along with organisational support, 61% also agreed that effective training and technical support ensures clinicians receive adequate technological support and skills to use CDSS effectively.

As shown in
Stakeholders' consultation to ensure systems are more adaptable to speci c users-requirements was also cited as an important facilitator by 60% of respondents.
Healthcare setting was identi ed to be signi cantly associated with perception of certain factors as enablers to CDSS adoption. Compared to hospital settings, respondents working in primary care settings were 39% more likely to believe that ease of use will facilitate CDSS adoption (OR

Qualitative Analysis
Analysis of free-text comments provided three major themes related to factors that in uence CDSS implementation: i. Lack of exibility Respondents expressed concerns regarding CDSS in exibility to change and cited it as a barrier to CDSS adoption. They reported that system usefulness is signi cantly limited if it lacks the ability to re ect the complex clinical context by taking into consideration setting and patient-speci c factors: "Systems I have experienced are comically bad in design mainly because they are in exible in its ability to change "(Hospital) Clinicians require exibility and adaptability in systems instead of "constant rule-making" to tailor recommendations to speci c context. "There is never a 'one size ts all'. So there must always be room to make exceptions" (Primary care) ii. Information overload "My major frustration with it [CDSS] in terms of antibiotic therapy is the presence of excessive allergy alerts, which do nothing to protect patients and simply lead to alert fatigue." (Hospital) Information relevance and precision emerged as important factors that in uence CDSS adoption. Respondents indicated that excessive information with low speci city and relevancy leads to alert fatigue and override, thereby reducing the overall use of CDSS. Furthermore, it was highlighted that time and workload pressures make it di cult for clinicians to distinguish important information from irrelevant data.
iii. Information accuracy "I, as a user, need to know on what basis any recommendation is provided, what is the source of this knowledge and how often it is updated" (Primary care) The accuracy of the content was also identi ed as an important theme for clinicians to trust using CDSS.
Respondents expressed doubts concerning the currency and reliability of the content which determines their overall trust in the system.
"…without knowing how often guidelines are updated in the system, we cannot rely on system alerts" (Hospital) The uncertainty felt by clinicians about the quality and accuracy of evidence negatively impacts their perception of CDSS.

Discussion
This study highlights clinicians' perception regarding the implementation barriers and facilitators for CDSS. We speci cally focused on internal and external factors in uencing users' intention to adopt CDSS by incorporating the framework of UTAUT. The internal related factors were speci c to personal expectations and perceptions, whereas external factors represented organisational, technical or patient related factors. Our ndings indicate a signi cant interplay between constructs of the UTAUT model namely perceived bene t (performance expectancy), perceived ease of use (effort expectancy) and organisational support (facilitating conditions) for CDSS adoption. Along with investigating relevance of different constructs, the association of moderators (age, gender, experience) and healthcare settings within these constructs was also explored.

Barriers and Facilitators External factors
Our ndings highlight that the facilitating environment has a signi cant impact on the adoption and usage of CDSS. One of the most signi cant barriers identi ed was lack of organisational capacity to provide appropriate technical support and training to end users. This can lead to limited understanding of the system's features as well as di culty to interact with the system effectively [27]. This can also be related to individual user's computer literacy as clinicians who routinely use computers are reported to have better adoption of systems such as CDSS [28]. Nonetheless, lack of support and training can limit the users' con dence in the system and ability to resolve any technical issues that may arise, thus discouraging CDSS adoption [29,30]. Interestingly, where the lack of organisational support greatly restricts the effective implementation of CDSS, the availability of appropriate infrastructure and supportive organisational environment was cited most frequently among our respondents as a facilitator.
Different studies that have used organisational theories to explain the technology acceptance behaviour also identify organisational structure and culture as enabling factors for promoting adoption of any new technology [31]. Our results further indicated that young clinicians (aged 18-34 years) are more likely to require organisational support for adoption of CDSS. This might be related to overall clinical hierarchy and lack of familiarity with the clinical environment. The clinical hierarchy and seniors' preferences greatly in uence the perception and practices of young clinicians [32]. Therefore, it is necessary to establish an organisational environment that fosters acceptability, support and trust for CDSS adoption.
One of the basic constructs of UTAUT related to overall system quality is the ease of use of the system. This signi es users' effort expectancy (effort required to use the system). In our survey, ease of use was identi ed as a key factor that facilitates adoption and adherence of clinicians with the CDSS. This is consistent with the measures of information system (IS) success model proposed by DeLone and Mclean (1992) that relates user satisfaction and adoption to ease of use [33,34]. We found that primary care clinicians perceive ease of use to be the most important feature for adoption of CDSS. These results concur with previous studies on adoption of technology in primary care, and is believed to be related to limited consultation time, workload, computer self-e cacy and patients' expectations in primary care [27,35]. Due to time constraints and workload pressure, general practitioners may avoid using CDSS due to risk of encountering irrelevant and overwhelming amount of information, as well as compromising on direct communication with patients [36,37].
Effort expectancy and perceived bene t of the system is related to users' trust that the system is a better t to their speci c requirements. Therefore, more than half of the respondents identi ed stakeholder consultation in the implementation of CDSS as a facilitator. This is related to identifying end-users' concerns, expectations and work ow requirements. Our results also showed that clinicians with longer working experience tended to rate stakeholders' involvement as more important for facilitating CDSS implementation. This is in agreement with the UTAUT theory that identi ed years of experience as a mediating factors for users' inclination to adopt technology [38]. Similarly, our results also indicated that clinicians with longer clinical experience (> 11 years) are more likely to see CDSS as a threat to their clinical autonomy. Therefore, inclusion of experienced clinicians in the CDSS development and implementation process would likely foster increased acceptance, trust and compliance with the CDSS.

Internal factors
In our study, internal factors were frequently reported as barriers to CDSS adoption. The most frequently reported barrier related to CDSS performance expectancy was lack of con dence in the content of the system. Not only is it a result of limited understanding of the system, but also a by-product of other factors concerning lack of trust in personnel involved in development of the system and lack of agreement with the guidelines or content of the system [39,40]. Many surveys have indicated that clinicians believe standard guidelines used in these systems usually do not take into consideration context-speci c requirements or patient-speci c risks, thus reducing their applicability for complex clinical decisions [41,42]. The apprehension of compromising their own clinical judgement and reasoning increases the reluctance of clinicians to adopt CDSS. Our results suggest that these perceptions may vary depending upon individual characteristics and type of healthcare setting. Experienced clinicians are less likely to use decision support tools due to fear of compromising established work practices and reducing autonomous control over processes and the content of clinical decisions [43]. On the other hand, younger clinicians tend to have better computer literacy and skills and so are more con dent in using systems such as CDSS [9,44]. The likelihood of perceiving limited professional autonomy as a barrier to CDSS use was also identi ed in primary care. However, this might be due to the higher proportion of experienced clinicians (11 or more years) amongst our respondents that worked in primary care.
Nonetheless, professional autonomy and freedom of clinical judgement have been identi ed in the literature as key factors to in uence any kind of change process, speci cally in primary care. Marjoribanks et al. (2003) indicated that concerns regarding clinical and nancial freedom, and time constraints greatly contribute to hesitation towards a change process [45]. This again reiterates the need for exibility in CDSS implementation to accommodate setting-speci c circumstances and improve organisational support and infrastructure to improve CDSS adoption.
Clinicians' time and work ow constraints are signi cant factors that in uence the adoption and use of CDSS in clinical settings. Our ndings indicated that clinicians' time constraints and risk of work ow disruptions greatly contribute to end users' resistance to CDSS adoption. This is consistent with previous studies suggesting that failure to provide a t between relevance, format and timeliness of recommendations negatively impact the uptake and utilisation of CDSS [11,46]. Moreover, our ndings also highlight that in primary care there is a greater likelihood of perceiving time and work ow constraints as barriers to CDSS adoption. Despite the fact that work ow disruptions and time sensitivities are relevant to CDSS implementation across healthcare settings, factors such as assessing overwhelming amount of data within short visits and high workloads might contribute to limited CDSS adoption in primary care settings [47]. Further research is required to better understand the dynamics of time and work ow constraints for CDSS adoption in different care settings.
The risk and liability of malpractice was identi ed as an important barrier to uptake of CDSS. The goal of implementing CDSS is to ensure improved safety and quality of care and minimise the risk of adverse events. However, in the unlikely event of any harm to the patient, the legal and medical liability remains unclear [48]. Increased access to clinical information through CDSS, makes clinicians liable to use this information for decision-making. Studies have indicated that clinicians are usually concerned about excessive dependence on an external device or service, even when they believe that the knowledge base of systems may be rooted in experts' professional opinion rather than objective medical evidence [49].
This further contributes to uncertainty regarding the ethical and legal repercussions of overriding or ignoring CDSS alerts given that it can be categorised as professional negligence. Similarly, excessive and irrelevant alerts instead of adding any value to quality of care, increase the reluctance of end users to adopt systems to avoid any such liability as a consequence of alert fatigue [50][51][52].
Our study has several limitations. Because of low participation rates, the results may not be representative of knowledge and perceptions of Australian clinicians. Although the response rate was low, it is not dissimilar to other surveys conducted with healthcare practitioners; however, it may be that participants may have self-selected as a consequence of polarised views on the topic, introducing selection bias. Another limitation is the design of this survey, and perceptions of respondents of factors related to CDSS which may not correlate with actual practice. We allowed open responses to some questions which may have mitigated this to some extent. Future research is required to better understand underlying factors that shape users' perception about CDSS.

Conclusions
The organisational environment along with users' perceptions signi cantly in uence CDSS adoption in healthcare settings. Users' reluctance stems from a lack of exibility and adaptability of systems to users' requirements. The effective diffusion of CDSS in healthcare settings demands effective re-orientation of organisations to establish a facilitating and supportive environment for the uptake of CDSS. This includes providing support and training for users as well as involving end-users of varying levels of seniority in the development and implementation of the systems. Systems need to better re ect user requirements and setting dynamics to reduce end-user resistance to system uptake. These ndings may help policy makers to develop a framework for sustainable and successful implementation of CDSS.

Availability of data and materials
The survey data is not publicly available due to restricted data consent provided by participants concerning sharing of information. A limited dataset is available from the corresponding author on request.
Due to technical limitations, Tables are available in supplementary section. Figure 1 Facilitators to CDSS adoption Reported barriers to CDSS adoption.

Figure 3
Overall perceived bene ts of CDSS Theoretical framework (UTAUT model) of the study Supplementary Files