Study Design
This study was a pre-post implementation evaluation of the fidelity, dose and reach of the ChIP care bundle (intervention). The outcomes were to discern if ChIP was delivered as intended (fidelity of ChIP) and whether the intended patient group received the care bundle activation (reach) [27]; and adherence to the intervention components (dose).
This study was part of a larger study testing the efficacy of the care bundle (ChIP) (Figure 3). Research conducted as part of this study adhered to the National Statement on Ethical Conduct in Human Research by the Australian National Health and Medical Research Council [29], and was approved by the NSW Population & Health Services Research Ethics Committee (HREC/17/CIPHS/56). The Standards for Reporting Implementation Studies (STaRI) guidelines were used to guide the reporting of this evaluation [30] (Additional file 3).
Setting
The blunt chest injury care bundle (ChIP) was implemented at two hospital sites in regional NSW, Australia. The two sites were within the same local health district with a 500-bed regional trauma centre (Site A) and a 200-bed rural/regional hospital (site B), representing diverse sites with differing resources. Site A is a regional trauma centre seeing approximately 70,000 presentations annually, and Site B is a smaller district hospital with approximately 40,000 presentations to its ED annually [31]. Both sites have intensive care units (ICU), EDs, pain specialist teams, and physiotherapists on site.
Patient identification
Patients were identified via two sources: 1) medical records of patients admitted to hospital with blunt chest injury identified using International Statistical Classification of Disease version 10 (ICD-10) codes and the Australian Refined Diagnosis Related Groups version 6 (AR-DRG v6) (Additional file 4) [32]; and 2) patients who had a ChIP call registered on the electronic medical record (eMR) system ‘FirstNet’ [33] (Figure 4) .
As part of the activation of ChIP, staff from the ED activated an icon on patients records to help other staff identify patients and to help staff remember to use ChIP. These icons were able to be tracked using the eMR and were used to generate a list of all patients who had an icon logged.
The patient medical records were initially screened for eligibility against the following inclusion criteria:
- Any mechanism of chest trauma resulting in documented radiological or clinical blunt chest wall injury
- 18 years or over
- Presents via the ED
- No intubation in ED or prehospital
The records of patients identified through eMR who did not meet the above criteria but had a ChIP call were excluded from the primary analysis but had baseline data collected, and sub-analysis reported; as this was important for reach (Figure 4).
Medical records of patients meeting the above criteria underwent a second screening process, to assess if the patient met ChIP criteria and were eligible for a ChIP call. The ChIP eligibility criteria included patients who had recorded no improvement to chest pain or unable to deep breathe cough after analgesia and an injury that occurred within one week of ED presentation.
Sample
Four groups within the overall sample were analysed according to the outcome measure of interest (Figure 4). Group 1: The pre-group included admitted patients who presented in the pre-implementation period and met ChIP criteria. Group 2: The ChIP group were patients who met study eligibility criteria and had a ChIP activation, including both admitted and non-admitted patients. Group 3: The ChIP missed group were patients who presented post-implementation, met ChIP eligibility criteria but did not get a ChIP activation. Group 4: The post-group included patients who presented after implementation and met ChIP eligibility criteria, including admitted patients who had activation of CHIP (admitted patients from group 2) or no activation (group 3).
Data collection
The study was conducted over four years. The pre-implementation data period was between 1 July 2015 to 21 November 2017, and post-implementation between 22 November 2017 to 30 June 2019.
Data were extracted from inpatient medical records and entered into a secure electronic database REDcap (Research Electronic Data Capture) [34]. Each data point defined within a data dictionary and the database was constructed with automatic outlier detection to alert data collectors of outliers. Regular quality checks were performed and ten per cent of records analysed for inter-rater agreement. Records were chosen at random using a random number generator. Inter-rater agreement was checked across 12 pre-agreed items totalling 60 data points. Inter-rater agreement rates were 97.8% for Site A (n=10) and 96.8% for Site B (n=20), both considered acceptable [35, 36].
Patient characteristics collected included age and gender. Clinical information included injury(s), mechanism of injury, injury date and time, Injury Severity Score (ISS) the Charlson Comorbidity Index (CCI). The ISS was used as an internationally recognised scoring system for the combined effects of trauma. The score ranges from 1 to 75, with ISS 15 or greater considered severe injuries. The CCI is a scoring system for mortality based on pre-existing comorbidities. ISS and CCI were considered confounding factors.
Data were also collected to identify adherence to the care bundle components (dose) in the areas of analgesia delivery, respiratory support and complication prevention. Data collected included whether vital signs, respiratory assessments, incentive spirometry and high flow nasal cannula use were documented. The dates and times of health service reviews were collected including physiotherapy, surgical, pain team or ICU. The admission team(s) were also collected.
Data analysis
Statistical analysis was performed using SPSS v25 (SPSS Statistics for Windows, Version 25.0. Armonk, NY: IBM Corp) and EpiTools (https://epitools.ausvet.com.au/). Baseline characteristics (age, sex, etc.) were analysed using descriptive statistics. There were two main analyses: 1) between-group analyses of ChIP call vs ChIP missed, and pre-ChIP implementation vs post-ChIP implementation and 2) within-group analyses involving the post group who received ChIP.
Medians and associated interquartile range (IQR) are reported for outcome data such as length of stay, time-based variables and variables based on an ordinal scale. Differences between sample medians were compared using the non-parametric median test, with the 95% confidence of the difference estimated using the Hodges-Lehmann estimate. Generalised Linear Models were also used for adjusted analyses of pre and post ChIP data: logistic regression with logit link was used for binary outcomes and for scale outcomes such as time to specific treatments the Gamma with log link model was used [37]. Correlation (Spearman’s rho) was used to explore the relationship between continuous variables and time to ChIP activation (post group only) and differences in proportions for categorical data were compared using a two-sample z test available in Epitools (https://epitools.ausvet.com.au/ztesttwo ). The z test has the advantage of providing a 95% confidence interval around the difference in proportions. All other tests were performed using SPSS version 25. P-values were considered statistically significant at p < 0.05.
Outcome 1: Reach of ChIP intervention
The reach of ChIP activations was calculated as the proportion of eligible patients who got a ChIP call from all patients eligible for ChIP post-implementation (post-group). Hospital sites were compared for reach. ChIP and ChIP missed groups were described and compared on demographics, mechanisms, injuries, medical history and presentation in or out of hours to identify potential reasons for missed calls. Out of hours presentations were defined as before 8am and after 4pm on weekdays, and anytime on weekends and public holidays. Patients who got a ChIP call but were ineligible were described in sub-analysis; also indicating reach as an unintended patient group.
Outcome 2: Fidelity to ChIP intervention
Fidelity was assessed by analysis of the ChIP group. The ChIP group were analysed for patient discharges, discharges over time, and any issues with activation, including time to activation.
Outcome 3: Adherence with ChIP components (dose)
Adherence with ChIP components was evaluated in two ways. Firstly, within the post-group, comparisons were made between ChIP and ChIP missed groups. Secondly, the pre-group was compared to the post-group to identify how the components were used before and after implementation, identifying changes in practice. ChIP components explored included time to analgesia, pain team and physiotherapist review, use of high flow nasal cannulae (HFNC), patient-controlled analgesia (PCA) or other modes of analgesia.