Study Design
This study was a pre-test/post-test implementation evaluation of the reach, fidelity, and dose of the ChIP care bundle (intervention). This study was part of a larger study testing the efficacy of ChIP on patient outcomes (Fig 3). Research conducted as part of this study adhered to the National Statement on Ethical Conduct in Human Research by the Australian National Health and Medical Research Council [32], and was approved by the NSW Population & Health Services Research Ethics Committee (HREC/17/CIPHS/56). The Standards for Reporting Implementation Studies (STaRI) guidelines were used to guide the reporting of this evaluation [33] (S3 Appendix).
Fig 3. Overall study design evaluating the implementation and efficacy of ChIP, this study indicated in orange
Setting
ChIP was implemented hospital-wide at two hospitals in regional NSW, Australia; with activation occurring in the emergency department and treatment continuing on through the hospital. The two sites were within the same local health district with a 500-bed regional trauma centre treating approximately 70,000 emergency presentations annually (Site A) and a 200-bed rural/regional hospital treating approximately 40,000 emergency presentations annually (site B), representing diverse sites with differing resources [34]. Both sites have ICU, emergency departments, pain specialist teams, and physiotherapists on site. Pain specialist teams were covered by anaesthetics out of hours. There was no physiotherapy coverage outside of hours; patients were seen at the next possible time.
Patient identification
Patients were identified via two sources: firstly, medical records of all patients admitted to hospital with blunt chest injury identified using International Statistical Classification of Disease version-10 Australian modification (ICD-10-AM) codes and the Australian Refined Diagnosis Related Groups version 6 (AR-DRG v6) (S4 Appendix) [35]; or, secondly, patients who had a ChIP call registered on the electronic medical record (eMR) system ‘FirstNet’ [36]. To be eligible for a ChIP call, patients had to have: i) an injury that occurred within one week of emergency department presentation; ii) recorded no improvement to chest pain or were unable to deep breathe or cough 30 minutes after analgesia administration.
To assess the need for ChIP activation, emergency department staff were required to do a respiratory assessment 30 minutes after analgesia. As part of the activation of ChIP, staff from the emergency department activated an icon on patients’ electronic medical record (eMR) to enable other staff to identify the patient and to prompt staff to remember to use ChIP. These icons were able to be tracked using the eMR and were used to generate a list of all patients who had a ChIP icon logged.
The patient eMRs were initially screened for eligibility and were included if they had any chest trauma resulting in documented radiological or clinical blunt chest wall injury, were 18 years or older, presented via the emergency department, and were not intubated in the emergency department or prehospital. The records of patients identified through eMR who did not meet the case ascertainment criteria but had a ChIP activation were excluded from the primary analysis but had baseline data collected, and sub-analysis reported; as this was important to evaluate the reach of the intervention.
Medical records of patients meeting the case ascertainment criteria underwent a second screening process, to assess if the patient met ChIP criteria and were eligible for a ChIP call.
Data collection
The study was conducted over four years. The pre-test-implementation data period was between 1 July 2015 to 21 November 2017, and post-implementation between 22 November 2017 to 30 June 2019.
Data were extracted from in-patient medical records and entered into a secure electronic database REDcap (Research Electronic Data Capture) [37]. Each data point was defined within a data dictionary, and the database was constructed with automatic outlier detection to alert data collectors of outliers. Regular quality checks were performed and 10% of records analysed for inter- rater agreement. Records were chosen at random using a random number generator. Inter-rater agreement was checked across 12 pre-agreed items totalling 60 data points.
Patient characteristics collected included age and gender. Clinical information included injury(s), mechanism of injury, injury date and time, injury severity score, Charlson comorbidity index, and whether patients received a trauma call activation. The injury severity score was used as an internationally recognised scoring system for the combined effects of trauma. The score ranges from 1 to 75, with an injury severity score 15 or greater considered severe injuries. The Charlson comorbidity index is a scoring system for mortality based on 17 pre-existing comorbidities. The injury severity score and the Charlson comorbidity index were considered as confounding factors.
Data were collected to identify which patients had a ChIP activation and whether patients met ChIP criteria. Data were also collected to identify adherence to the care bundle components in the areas of analgesia delivery, respiratory support and complication prevention. Data collected were whether vital signs, respiratory assessments, incentive spirometry and high flow nasal cannulae use were documented, and the times these were commenced. Initial, regular and as required charted analgesia within 24 hours were collected including the times commenced. Data on regional analgesia, patient-controlled analgesia, and/or continuous analgesia were collected for the hospital stay, including the times commenced. The dates and times of health service reviews were collected including physiotherapy, surgical, pain team or ICU admission. The admitting in-patient team(s) was also collected. Data was collected for documented patient education on chest injury.
Outcome Measures
The outcomes were to discern whether the intended patient group received the care bundle activation (reach); if ChIP was delivered as intended (fidelity of ChIP) and the adherence to the intervention components (dose [30]) as shown below.
- Reach of ChIP intervention: the proportion of eligible patients who received a ChIP call from all patients eligible for ChIP post-implementation (22 November 2017 to 30 June 2019).
- Fidelity – ChIP Intervention delivery as intended: analysis of all patients who received a ChIP activation in the post-implementation period (22 November 2017 to 30 June 2019).
- Dose – adherence with ChIP components: Pre-and post-test comparison for care-bundle components and between groups analysis of ChIP and ChIP-missed patient
Data analysis
Statistical analysis was performed using SPSS v25 (SPSS Statistics for Windows, Version 25.0. Armonk, NY: IBM Corp) and EpiTools (https://epitools.ausvet.com.au/). Baseline characteristics (age, sex, etc.) were analysed using descriptive statistics.
Medians and associated interquartile range (IQR) are reported for outcome data such as length of stay, time-based variables and variables based on an ordinal scale. Differences between sample medians were compared using the non-parametric median test, with the 95% confidence intervals of the difference estimated using the Hodges-Lehmann estimate. Generalised Linear Models were used for adjusted analyses of pre- and post-test ChIP data: logistic regression with logit link was used for binary outcomes and for continuous outcomes such as time to specific treatments the Gamma with log link model was used [38]. Correlation (Spearman’s rho [rs]) was used to explore the relationship between continuous variables and time to ChIP activation (post group only) and differences in proportions for categorical data were compared using a two-sample z test available in Epitools (https://epitools.ausvet.com.au/ztesttwo ). The z test has the advantage of providing a 95% confidence interval around the difference in proportions. All other tests were performed using SPSS version 25. P-values were considered statistically significant at p<0.05. Four groups within the overall sample were analysed according to the outcome measure of interest are discussed below.
Outcome 1: Reach of ChIP intervention
The reach of ChIP activations was calculated as the proportion of eligible patients who received a ChIP activation from all patients eligible for ChIP post-implementation (1 July 2015 to 21 November 2017). The two groups used in this analysis were the ChIP and ChIP-missed groups. The ChIP group were patients who met study eligibility criteria and had a ChIP activation, including both admitted and non-admitted patients (22 November 2017 to 30 June 2019). The ChIP-missed group were patients who presented post-implementation (22 November 2017 to 30 June 2019), met ChIP eligibility criteria but did not get a ChIP activation. Hospital sites were compared for reach. ChIP and ChIP-missed groups were described and compared by demographics, mechanism of injury, injuries, medical history and presentation in or out of hours to identify potential reasons for missed calls.
Out-of-hours presentations arrived before 8am and after 4pm on weekdays, or anytime on weekends and public holidays. Patients who received a ChIP call but were ineligible were described in a sub-analysis; also indicating reach as an unintended patient group.
Outcome 2: Fidelity - ChIP intervention delivered as intended
Fidelity was assessed by analysis of the ChIP group from the post-implementation group (22 November 2017 to 30 June 2019), including admitted and non-admitted patients. The ChIP activation group were analysed for time to ChIP activation: by site, transfers; in hours (8am – 4pm Monday to Friday) versus out of hours, comorbidities, age, injury severity, mode of hospital arrival. In addition, analyses were undertaken examining the length of emergency stay, and patient discharges from the emergency department.
Outcome 3: Dose - Adherence with ChIP components
There were two groups included in the analysis of dose. The pre-group included admitted patients who presented in the pre-implementation period (1 July 2015 to 21 November 2017) and met ChIP criteria. The post-group included patients who presented after implementation commencement (22 November 2017 – 30 June 2019) and met ChIP eligibility criteria, including admitted patients who had activation of ChIP or no ChIP activation.
Adherence with ChIP components was evaluated in two ways to identify practice change. Firstly, the pre-implementation group was compared to the post-implementation group to identify how adherence to the ChIP components before and after implementation. ChIP components explored were: time to analgesia, pain team and physiotherapist review, use of high flow nasal cannulae, patient-controlled analgesia or other modes of analgesia. Secondly, within the post-group, comparisons were made between ChIP and ChIP-missed groups as a sub-analysis.