This report presents the results of a sociotechnical analysis of electronic CDS to aid the management of children with MHT and ICI. Our findings reflect the relatively early stage of this development and implementation effort and also help guide the next steps in this process. The major interview themes covered the anticipated primary and secondary uses of the tool, important stakeholders in implementation, suggestions for improving the tool’s content, along with perspectives on how to integrate the CDS into clinical workflow. Finally, participants highlighted key measures that could evaluate the CDS tool’s impact after implementation.
The focus group interviews highlighted broad support for using the proposed CDS intervention to guide level-of-care decisions, helping to standardize care, build consensus decisions, and expand risk knowledge across specialties. Although less frequently discussed, there was also support for using the tool to inform family counseling. Participants also identified potential unintended consequences to be avoided in future implementation efforts, such as potential misuse of the tool to avoid appropriate consults. Outside of these primary uses, there was mixed and measured support for other possible uses, such as guiding the need for hospital transfer. While recent reports have suggested that routine transfer to a tertiary hospital may not be needed for all children with MHT and intracranial injuries,21 participants explained that changing such practices will require broader acceptance of the underlying evidence. Additionally, changing transfer practices will require addressing social concerns and non-cranial injuries and involving health system administrators that control referral pathways. These areas require further investigation in future work.
Although other researchers have evaluated the implementation of health information technology in an ED setting, those applications have generally focused on tools used by a single clinical specialty.15, 22, 23 By comparison, our study focused on an interdisciplinary care pathway with diverse clinical stakeholders. Reflecting these diverse opinions, our results emphasized the importance of a participatory process involving end-users in both the design and planned implementation of electronic CDS,24 consistent with expert opinion.25 While participants from all specialties generally viewed the CDS favorably, some expressed uncertainty regarding how the tool’s use may differ between neurosurgeons and non-neurosurgeons. The opinion expressed by some ED physicians has previously been summarized as, “really useful but not for me.”26 This finding highlighted the need to explicitly incorporate interdisciplinary education and training during implementation efforts. Additionally, while nurses were not included in the focus groups, participants noted the importance of involving nurses in implementation efforts and considering nursing capabilities when developing level-of-care recommendations. Feedback from the focus groups also impacted the CDS content. For example, concerns about including cost estimates in the CDS tool led us to remove this feature. Finally, focus group feedback helped identify and address potential ambiguities, such as the need to explicitly define the GCS score used based on the patient’s first assessment in the final treating hospital.
Likely the greatest implementation barrier noted in the focus groups was identifying effective approaches of incorporating the electronics CDS into clinical workflow. Universally, participants felt that integrating the CDS tool within the EHR environment was key, compared to a phone app or stand-alone website. In an optimal setting, all elements of the CDS would be auto-populated based on data from the EHR.27, 28 Indeed, increased data entry requirements are associated with clinician dissatisfaction and abandonment of CDS.29 Similarly, the EHR offers the potential for targeting users at the point-of-care, an essential element of CDS success.17 Relying on clinicians to trigger CDS is unlikely to succeed,17 but inappropriate or excessive alerts promoting alert fatigue were a major concern among participants.30, 31 Optimizing the timing of CDS presentation may reduce alert fatigue and increase use among clinicians.32 Currently, the lack of structured data elements corresponding to the CDS inputs (e.g. imaging findings) is a major barrier to either auto-populating the tool components or optimizing its integration within clinical workflow. The absence of such solutions for automating use was a frequent finding among participants.
One potential solution for this problem includes “smart forms” for integrating structured data elements into clinician notes,33 but this approach requires clinicians to initiate the form and has been associated with low use among physicians.34 Alternatively, natural language processing may help improve trigger accuracy,35, 36 but dictated reports are often delayed in the acute care setting. Most encouraging, deep machine learning algorithms have demonstrated strong performance in detecting acute intracranial hemorrhage on CT.37, 38 While still requiring physician review, such tools offer a promising avenue for both provider targeting and automating CDS data capture.
While the process of thematic analysis was inductive in nature, the structure of the focus group interviews was intended to address key sociotechnical elements.14, 19 We found that the themes and sub-themes identified corresponded to almost all key tenets of sociotechnical theory, suggesting that this model captured most considerations relevant to implementation planning. Given that interface design was not considered in this analysis, the human computer interface dimension was expectedly lacking. In addition, we did not identify any significant themes or sub-themes corresponding to external rules or regulations. Likely, this absence reflected the early development stage of the CDS, and further evaluation later in the implementation process may yield additional findings. In addition, we found that one sub-theme – “challenges to using the tool in clinical practice” – could not be effectively mapped to a sociotechnical dimension. We believe this difficulty also reflected the early stage of development, where potential problems, such as ambiguous input variables, were still being identified and remediated.
There are limitations regarding our study. First, while we included a multidisciplinary and multicenter group of physicians, most participants came from academic hospitals, which may limit the generalizability of the results on broader implementation. However, based on the participant feedback, academic clinicians are likely to be the primary targets of future CDS implementation, supporting their higher focus group representation. Additionally, we did not include nurse practitioners or nurses in the focus groups, who may be potential end-users or stakeholders in future implementation efforts. Furthermore, given the early stage of the implementation planning, participants reviewed wireframes of the early CDS prototype, rather a fully developed prototype. An interactive prototype that integrates the focus group feedback will undergo dedicated user testing in future work. Finally, because we depended on volunteer participation, our participants may have reflected a biased sample. For example, more than 80% of our participants were younger than 50 years, and previous efforts have shown older provider age is associated with lower rates of electronic CDS use.39, 40 Therefore, future larger-scale implementation efforts will need to evaluate potential age-related disparities in beliefs in and use of electronic CDS among providers.