Implementation Evaluation of a Multidisciplinary Blunt Chest Injury Care Bundle (ChIP): Fidelity of Delivery

Background: Ineffective or delayed treatment of patients with blunt chest wall injury results in high rates of morbidity and mortality. A blunt chest injury care bundle protocol (ChIP) was developed and implemented to improve evidence-based care for these patients at two regional hospitals in Australia. ChIP is an early notication system to notify specialist clinician ‘responders’ to prescribe and commence treatment for patients with blunt chest injury in the emergency department (ED). A multi-pronged implementation strategy developed using the Behaviour Change Wheel (BCW), including seven intervention functions and 15 behaviour change techniques, guided implementation. Fidelity to the implementation strategy was high, with 97.5% fully or partially implemented. Implementation delity is the extent to which an intervention has been implemented as intended; it affects the internal and external validity of implementation. This study evaluates the delity of intervention delivery (delity, dose and reach) at two hospitals. Methods: Pre-post implementation evaluation study. The characteristics of patients, rate of ChIP activations and components of ChIP received by eligible patients were compared pre (1 July 2015 to 21 November 2017) and post (22 November 2017 to 30 June 2019) intervention. Sample medians were compared using the non-parametric median test, with the 95% condence of the difference estimated using the Hodges-Lehmann estimate. Differences in proportions for categorical data were compared with two-sample z-test. Logistic regression was used to adjust for group differences. Results: Overall, 97.1% of eligible patients received ChIP over the 19-month post-implementation period. Compared to the pre-implementation group the post-implementation group, were more likely to receive evidence-based treatments including high ow nasal cannula (OR=6.8 (4.8,9.6)), incentive spirometry in ED (OR=7.5 (3.2,17.6)), regular analgesia (OR=2.4 (1.5,3.8)), regional analgesia (OR=2.8 5.3)), Patient controlled analgesia (OR=1.8 and multiple specialist team reviews e.g. ICU liaison This study provides an example of successful implementation of a complex multidisciplinary intervention in the acute emergency setting using behaviour change frameworks. Findings from this evaluation can inform the implementation of multidisciplinary interventions in the emergency/acute care context with high delity.

departments and developed based on the Theoretical Domains Framework [23]. Alongside a consultation process that included an APEASE assessment (affordability, practicability, effectiveness, acceptability, side-effects, and equity), a multi-method implementation strategy containing seven intervention functions and 15 behaviour change techniques (BCT) [24] was developed (Additional le 1) [22]. Resources for the delivery of the implementation, including a staff video https://youtu.be/woc4cJGjjQo and storyboard for the video development, have been provided (Additional le 2). A logic map depicts intervention functions, BCTs and modes of delivery used across the two sites (Fig. 2); partially shaded areas represent BCTs partially implemented (21.4%), lled boxes represent BCTs and modes that were fully implemented (76.2%) (Fig. 2) [25].
The delity of the intervention (ChIP), also referred to as ' delity of delivery' or 'treatment delity' [26], has not yet been evaluated. The Medical Research Council (MRC) guideline for process evaluations of complex interventions recommends that ' delity', 'dose' and 'reach' are evaluated as appropriate to the speci c study and intervention [27]. Fidelity will assess if ChIP was delivered as intended. Reach describes whether the intended audience came into contact with an intervention, so this is how many patients received a ChIP activation and if it was appropriate [27]. Dose is the quantity of the intervention implemented; this will assess the components of ChIP that were delivered [27,28]. The purpose of this study was to evaluate the delity, dose and reach of an intervention (ChIP) to discern if ChIP was activated and delivered to patients as intended.

Study Design
This study was a pre-post implementation evaluation of the delity, dose and reach of the ChIP care bundle (intervention). The outcomes were to discern if ChIP was delivered as intended ( delity of ChIP) and whether the intended patient group received the care bundle activation (reach) [27]; and adherence to the intervention components (dose).
This study was part of a larger study testing the e cacy of the care bundle (ChIP) (Figure 3). Research conducted as part of this study adhered to the National Statement on Ethical Conduct in Human Research by the Australian National Health and Medical Research Council [29], and was approved by the NSW Population & Health Services Research Ethics Committee (HREC/17/CIPHS/56). The Standards for Reporting Implementation Studies (STaRI) guidelines were used to guide the reporting of this evaluation [30] (Additional le 3).

Setting
The blunt chest injury care bundle (ChIP) was implemented at two hospital sites in regional NSW, Australia. The two sites were within the same local health district with a 500-bed regional trauma centre (Site A) and a 200-bed rural/regional hospital (site B), representing diverse sites with differing resources.
Site A is a regional trauma centre seeing approximately 70,000 presentations annually, and Site B is a smaller district hospital with approximately 40,000 presentations to its ED annually [31]. Both sites have intensive care units (ICU), EDs, pain specialist teams, and physiotherapists on site.

Patient identi cation
Patients were identi ed via two sources: 1) medical records of patients admitted to hospital with blunt chest injury identi ed using International Statistical Classi cation of Disease version 10 (ICD-10) codes and the Australian Re ned Diagnosis Related Groups version 6 (AR-DRG v6) (Additional le 4) [32]; and 2) patients who had a ChIP call registered on the electronic medical record (eMR) system 'FirstNet' [33] ( Figure 4) .
As part of the activation of ChIP, staff from the ED activated an icon on patients records to help other staff identify patients and to help staff remember to use ChIP. These icons were able to be tracked using the eMR and were used to generate a list of all patients who had an icon logged.
The patient medical records were initially screened for eligibility against the following inclusion criteria: Any mechanism of chest trauma resulting in documented radiological or clinical blunt chest wall injury 18 years or over Presents via the ED

No intubation in ED or prehospital
The records of patients identi ed through eMR who did not meet the above criteria but had a ChIP call were excluded from the primary analysis but had baseline data collected, and sub-analysis reported; as this was important for reach ( Figure 4).
Medical records of patients meeting the above criteria underwent a second screening process, to assess if the patient met ChIP criteria and were eligible for a ChIP call. The ChIP eligibility criteria included patients who had recorded no improvement to chest pain or unable to deep breathe cough after analgesia and an injury that occurred within one week of ED presentation.

Sample
Four groups within the overall sample were analysed according to the outcome measure of interest ( Figure 4). Group 1: The pre-group included admitted patients who presented in the pre-implementation period and met ChIP criteria. Group 2: The ChIP group were patients who met study eligibility criteria and had a ChIP activation, including both admitted and non-admitted patients. Group 3: The ChIP missed group were patients who presented post-implementation, met ChIP eligibility criteria but did not get a ChIP activation. Group 4: The post-group included patients who presented after implementation and met ChIP eligibility criteria, including admitted patients who had activation of CHIP (admitted patients from group 2) or no activation (group 3).

Data collection
The study was conducted over four years. The pre-implementation data period was between 1 July 2015 to 21 November 2017, and post-implementation between 22 November 2017 to 30 June 2019.
Data were extracted from inpatient medical records and entered into a secure electronic database REDcap (Research Electronic Data Capture) [34]. Each data point de ned within a data dictionary and the database was constructed with automatic outlier detection to alert data collectors of outliers. Regular quality checks were performed and ten per cent of records analysed for inter-rater agreement. Records were chosen at random using a random number generator. Inter-rater agreement was checked across 12 pre-agreed items totalling 60 data points. Inter-rater agreement rates were 97.8% for Site A (n=10) and 96.8% for Site B (n=20), both considered acceptable [35,36].
Patient characteristics collected included age and gender. Clinical information included injury(s), mechanism of injury, injury date and time, Injury Severity Score (ISS) the Charlson Comorbidity Index (CCI). The ISS was used as an internationally recognised scoring system for the combined effects of trauma. The score ranges from 1 to 75, with ISS 15 or greater considered severe injuries. The CCI is a scoring system for mortality based on pre-existing comorbidities. ISS and CCI were considered confounding factors.
Data were also collected to identify adherence to the care bundle components (dose) in the areas of analgesia delivery, respiratory support and complication prevention. Data collected included whether vital signs, respiratory assessments, incentive spirometry and high ow nasal cannula use were documented. The dates and times of health service reviews were collected including physiotherapy, surgical, pain team or ICU. The admission team(s) were also collected.

Data analysis
Statistical analysis was performed using SPSS v25 (SPSS Statistics for Windows, Version 25.0. Armonk, NY: IBM Corp) and EpiTools (https://epitools.ausvet.com.au/). Baseline characteristics (age, sex, etc.) were analysed using descriptive statistics. There were two main analyses: 1) between-group analyses of ChIP call vs ChIP missed, and pre-ChIP implementation vs post-ChIP implementation and 2) within-group analyses involving the post group who received ChIP.
Medians and associated interquartile range (IQR) are reported for outcome data such as length of stay, time-based variables and variables based on an ordinal scale. Differences between sample medians were compared using the non-parametric median test, with the 95% con dence of the difference estimated using the Hodges-Lehmann estimate. Generalised Linear Models were also used for adjusted analyses of pre and post ChIP data: logistic regression with logit link was used for binary outcomes and for scale outcomes such as time to speci c treatments the Gamma with log link model was used [37]. Correlation (Spearman's rho) was used to explore the relationship between continuous variables and time to ChIP activation (post group only) and differences in proportions for categorical data were compared using a two-sample z test available in Epitools (https://epitools.ausvet.com.au/ztesttwo ). The z test has the advantage of providing a 95% con dence interval around the difference in proportions. All other tests were performed using SPSS version 25. P-values were considered statistically signi cant at p < 0.05.

Outcome 1: Reach of ChIP intervention
The reach of ChIP activations was calculated as the proportion of eligible patients who got a ChIP call from all patients eligible for ChIP post-implementation (post-group). Hospital sites were compared for reach. ChIP and ChIP missed groups were described and compared on demographics, mechanisms, injuries, medical history and presentation in or out of hours to identify potential reasons for missed calls. Out of hours presentations were de ned as before 8am and after 4pm on weekdays, and anytime on weekends and public holidays. Patients who got a ChIP call but were ineligible were described in subanalysis; also indicating reach as an unintended patient group.

Outcome 2: Fidelity to ChIP intervention
Fidelity was assessed by analysis of the ChIP group. The ChIP group were analysed for patient discharges, discharges over time, and any issues with activation, including time to activation.

Outcome 3: Adherence with ChIP components (dose)
Adherence with ChIP components was evaluated in two ways. Firstly, within the post-group, comparisons were made between ChIP and ChIP missed groups. Secondly, the pre-group was compared to the postgroup to identify how the components were used before and after implementation, identifying changes in practice. ChIP components explored included time to analgesia, pain team and physiotherapist review, use of high ow nasal cannulae (HFNC), patient-controlled analgesia (PCA) or other modes of analgesia.

Results
A total 795 patients were included in the nal data analysis (Fig. 4). There were 284 in the preimplementation group and 453 in the post-implementation group. In the post-implementation period, 533 patients received a ChIP call in the ED. However, 33 patients did not meet study inclusion criteria; leaving 500 who had ChIP activated and met study eligibility criteria (ChIP). There were 13 patients identi ed who were eligible for ChIP but did not get a call (ChIP missed).

Outcome 1: Reach of ChIP intervention
Overall, the reach of ChIP in eligible patients was 97.1%, with 96.1% at Site A and 98.8% at Site B. Eligible patients that did not receive an activation (ChIP missed group, n = 13) were not different in age, ISS, CCI, comorbidities and in-hours presentation, compared to patients who were eligible and received an activation (ChIP group n = 440) (Table 1). However, participants in the ChIP missed group were more likely to be female, have a sternal injury, have less than three rib fractures, or have had a vehicle-related injury compared to the ChIP activated group ( Table 1). None of the ChIP missed group had three or more ribs fractures or a ail chest (Table 1). Reach was consistent over the implementation period as follows: 6 (46.2%) in the rst 6-months post-implementation, 2 (15.4%) in the next six months and 5 (38.5%) in the nal seven months. The ChIP and ChIP missed groups were similar in relation to receiving initial vital sign assessment, respiratory assessment or initial analgesia in the ED (Table 2). In regards to analgesia, the ChIP group were more likely to be seen by the pain service and have a PCA (Table 2). In regards to respiratory support and complication prevention, the ChIP group were more likely to have had a physiotherapy review, HFNC, incentive spirometry, and education for deep breathing compared to the ChIP missed group (  Comparison of pre and post groups: adherence Age, sex, CCI, or mechanism of injury did not differ between the pre-implementation (n = 282) and postimplementation groups (n = 453) ( Table 3). There were more trauma calls in the pre-group compared to the post-group; however, the ISS was higher, and the rib fractures more severe in the post-group (Table 3). Abbreviations: CCI = Charlson comorbidity Index, ISS = injury severity score The post-group were more likely to have regular analgesia charted (day one of admission), regional analgesia, and PCA in compared to the pre-group (Table 4). The post-group had greater odds of receiving HFNC, incentive spirometry and education regarding their injury ( Table 4). The post-group received more reviews by surgery, ICU liaison, ICU, chest physiotherapists and pain team ( Table 4). The post-group also had faster times to initial analgesia, regular analgesia, pain review, physiotherapist review, and ICU review ( Table 5). The rates of ChIP component delivery over the study period are presented in Fig. 5.

Discussion
This study evaluated the implementation of a blunt chest injury care bundle (ChIP) by assessing the delity of delivery ( delity, dose, reach). The implementation strategy based on the BCW resulted in a high and sustained reach (97.1%) and dose of the ChIP care bundle over the 19-month post-implementation evaluation period. Discharges after a ChIP activation decreased over the post-implementation period; this suggests an improvement in the identi cation of patients that were eligible for ChIP activation by staff.
The smaller, rural hospital (Site B) had similar reach and slightly earlier activation times compared to the bigger metropolitan hospital (Site A). Rural implementation is known to be challenging due to fewer resources and staff availability [38]; however, this study has demonstrated it can be successful with strategic theory-based planning [39].
Implementation was considered to be successful in that changes to care delivery were demonstrated across the ChIP components of analgesia administration, respiratory support and complication prevention and in the multidisciplinary response for the duration of the post-implementation period. This result was evident in both the comparisons for the ChIP and ChIP missed groups and also in the preimplementation and post-implementation comparisons.
To assess the need for a ChIP activation, ED staff were required to do a respiratory assessment 30 minutes after analgesia. There were fewer patients in the post-group who got a respiratory assessment post-analgesia compared to the pre-group. This may be due to delayed documentation as data was only collected for the 90 minutes post analgesia; healthcare workers may not always record their assessment or intervention as they have other competing priorities [40].

Implementation strategy scalability
The ChIP intervention and implementation strategy are adaptable and were tailored for implementation at the sites for this study. In a recent systematic review of the blunt chest injury pathways, it was highlighted that pathways need to be highly adaptable to the patient and context [41]. The implementation plan included multi-modal implementation strategies, including educational sessions, a support video, clinical champions, audit and feedback, environmental changes and advertisements, which are common, accessible strategies that can be adapted and used at other sites. Other emergency based implementation studies include multiple strategies; with the most common strategies reported in a systematic review of emergency behaviour change being reminders, educational meetings, educational materials and clinical practice guidelines [42]. The ChIP implementation included a combination of all of these strategies.
The implementation strategies were associated with high delity for care bundle implementation [43]. Minimal costs were incurred for implementation. The funding for clinical champions was not used while still maintaining high delity. Clinical champions were hired for the role from the existing nursing workforce; however, of the eight initial positions, only two clinical champions continued in the role due to secondments and other extenuating circumstances. The two staff that continued as clinical champions were in positions where they could provide support with one being a clinical educator and the other the ICU liaison role and were able to ful l the role su ciently. Demonstrating that sites without funding may be able to implement without the extra resources.

Strengths and Limitations
A limitation is that there may have been some records missed in the screening. The eMR screening identi ed patients who were not picked up by the ICD-10 and DRG codes screening. The initial intent for the eMR screening was to determine the time that the ChIP call was activated; however, the screening also identi ed some patients that were not in the medical record coding screen. The ICD-10 codes were retrospectively checked for the records that were missed. Some examples of the ICD-10 codes given to the missed patients were "chest pain, unspeci ed" and "injury, unspeci ed". These codes were not included in the ICD-10-AM request as they were considered medical-related or vague.
Another limitation is that the ChIP missed group is relatively small, a strength of the intervention reach, but this did lead to limiting power of statistical analysis in comparison to ChIP.
There were fewer patients in the pre-group; this may have been due to poor documentation of chest injury symptoms in the pre-group medical records.
A limitation is that results were based on the effectiveness of documentation of the healthcare professionals, for example, if staff provided education but did not document that they did. Using documentation for implementation evaluation can sometimes lead to low implementation delity [40]; however, implementation delity was high in this study, suggesting it has been an adequate method of evaluation in this case. It was important for this study to evaluate implementation in the real-world without researcher interference. Further, other methods have their problems. For example, direct observation can lead to changes in behaviour due to being observed and may not be feasible due to high cost and ethical considerations around observing patients in direct care [44]. Self-report is also problematic, with a risk of self-report bias overestimating delity [44,45].
This evaluation did not evaluate if components were delivered appropriately or missed, for example, if a patient should have had a regional block. However, the design of ChIP is that it relies on clinician judgement to deliver the most appropriate treatment at the time in relation to the clinical context.
This study provides a unique view of implementation in the emergency context. There are limited studies reporting on the implementation evaluation in an emergency context, with most focusing on facilitators and barriers [46]. A lack of delity to the intervention may be associated with poorer outcomes with a systematic review of care bundles reporting that delity needed to be as high as 95% for improved patient outcomes [47]. The impact of ChIP on patient outcomes is also important and will be presented separately [48]. Implementation of complex interventions in the emergency context have not had successful results in some cases [49][50][51], perhaps needing greater use of behaviour change theories to improve implementation design [52]. This implementation evaluation can inform future spread and scale of ChIP, including for research or clinical implications [53]. Further, it can improve the validity of the ChIP patient outcome studies [54].

Page 23/33
Following a robust theoretical-based implementation plan is associated with high implementation delivery. Implementation evaluation of complex health interventions in the ED context requiring a multidisciplinary response may require multi-modal implementation strategies to be successful. The results from this study can inform future implementation efforts in the acute care environment, such as the ED.  Overall study design evaluating the implementation and e cacy of ChIP, this study indicated in orange

List Of Abbreviations
Page 32/33 Patient identi cation, inclusion and groups (blue) for analysis.