Primary outcome: Feasibility of the study
A summary of the results for the feasibility of the study is reported in Table 2.
Table 2: results for the feasibility of the study.
Outcome
|
Criteria for feasibility
|
Feasibility proved
|
Implementation of the new diagnostic PE protocol at the participating hospitals
|
Successful implementation in at least two of the three participating sites.
|
Yes, intervention implemented in two centers.
|
Identification of the population
|
Capturing ≥80% of all CTPAs ordered
|
Yes, 90% captured.
|
Access to all the required electronic data
|
Successful access to all the data
|
Yes.
|
Timely manual data extraction
|
Completed 5 days before the end of the following month ≥80% of the times.
|
Yes, 100% of the times.
|
Implementation of individual emergency physician audit and feedback
|
Feedback on the previous month ready to be emailed before the end of the following month ≥80% of the times.
|
Yes, 100% of the times.
|
Number of hours of research assistant time to extract the required data and synthesize the physician feedback reports.
|
≤two days/week.
|
Yes, 14 hours/week.
|
PE: pulmonary embolism; CTPA: computed tomography pulmonary angiography.
1. Implementation at participating hospitals
The intervention was implemented at the Hamilton sites, but not at the Montfort hospital in Ottawa. The PE diagnostic order set was initiated on October 28, 2019. The topic of PE diagnosis was discussed at the HHS ED physicians rounds and three educational podcasts were recorded and remain freely available for streaming and download.18–20 All emergency physicians received an email explaining the rationale for the intervention and its objective, accompanied by a Frequently Asked Question section and educational material (email text available in Appendix B). As a reminder to the ED physician group, we attached laminated stickers with a logo and the invitation to “rule out PE without CT” to each computer in their offices (Figure 2). A team of three nurse educators engaged in individual meetings with the ED nurses, explaining the aim of the intervention and the new workflow. A multidisciplinary team including managers, radiologists, emergency physicians, educators, radiation technologists, and nurses met monthly to review progress and problems arising with the new order set. A number of logistical changes were made including automatic population of the CTPA request form with the D-dimer result and estimated glomerular filtration rate for the radiology technicians and streamlining of process.
We collaborated with regular project meetings with the Montfort Hospital ED. In May 2019, our colleagues let us know they were not able to participate in the study due to lack of data access and resources.
2. Identification of the population
We found that the two selected CEDIS presenting complaints (chest pain and shortness of breath) only captured 70% of all CTPAs ordered by emergency physicians. Therefore, inclusion criteria were expanded to 13 presenting ED presenting complaints (Table 3), allowing us to capture 90% of the ED ordered CTPAs. The remaining 10% were dispersed among 40 presenting complaints, each one accounting for 0.1–0.9% of all CTPAs (Table S1).
Table 3: distribution of CTPAs by presenting complaint in 2018
CEDIS description
|
CTPAs (n)
|
CTPAs (%)
|
Chest Pain (cardiac features)
|
261
|
25%
|
Chest Pain (non-cardiac features)
|
98
|
9%
|
Shortness of Breath
|
366
|
35%
|
Palpitations/irregular heart beat
|
43
|
4%
|
Syncope/Pre-Syncope
|
30
|
3%
|
Hemoptysis
|
16
|
2%
|
Cardiac arrest (non-traumatic)
|
7
|
1%
|
Respiratory arrest
|
1
|
0%
|
General Weakness
|
43
|
4%
|
Back Pain
|
24
|
2%
|
Cough/Congestion
|
18
|
2%
|
Abdominal Pain Unspecified
|
25
|
2%
|
Hyperventilation*
|
0
|
0%
|
Other
|
105
|
10%
|
Total
|
1037
|
|
* Hyperventilation was added even if it is seldom used in our emergency departments and no CTPAs was ordered. The rationale is that it can be a typical presentation of PE.
CEDIS: Canadian Emergency Department Information System
CTPA: Computed Tomography Pulmonary Angiography
Obtaining electronic data
We requested data from three sources: the hospital decision support services, an internal hospital research database and the eHealth Information Technology Services (eHITS) office. The first two sources were unable to provide timely data (at the end of each month). The eHITS department was able to provide us with the required data in the required turnaround time. After working together to define the database queries and to validate the data, we received the first finalized dataset in January 2019. The system is now automated with monthly updates.
4. Manual data extraction
We found the variable “ordering doctor” for the CTPAs, in the electronic medical record (EMR) was not accurate due to some scans being ordered by an admitting service but the order was logged under the emergency physician’s name. It was crucial for us to have accurate data for the ordering physician or the physician audit and feedback would lose credibility. Therefore, we manually checked the ordering physician data. Despite this increase in the workload, the manual extraction was always completed before the end of the following month.
5. Implementation of individual physician feedback
Audit and feedback intervention are effective for improving healthcare professionals' compliance with desired practice,21 and individual feedback has a potential additional role as compared to group feedback alone.22 The path toward the implementation of physician feedback proved to be challenging and many adaptations of the original plan were required. Initially, we aimed to provide individual feedback to physicians on a monthly base. When reviewed, we realized that the number of CTPAs ordered per month per physician was too low (range 0–6 CTPAs per month per physician). The proportion of inappropriate CTPAs would have been subject to enormous variability for very small variations in the actual number of non-appropriate CTPAs ordered. We decided to reduce the frequency of feedback from monthly to quarterly and the feedback was issued at the end of the first three months post implementation. An example of the physician feedback is reported in Figure 3.
6.Estimate of research assistant time
Based on the first three months, we calculated that a research assistant is required for twelve hours per week on the project. In addition, a physician should spend two hours per month to resolve queries. Preparing and checking individual feedback requires an additional six hours per month, on average. Therefore, the total amount of time required to complete these tasks was approximately 14 hours per week (less than two days); thereby, meeting the feasibility criterium.
Secondary outcomes: preliminary estimates of effect
The patients’ characteristics and outcome distribution are reported in Table 4. In total, 81,103 patients accessed the HHS EDs for one of the selected presenting complaints between January 1, 2018 and February 28, 2020 (70,932 in the before-intervention period and 10,171 after the intervention). Of these, 5,993 patients were tested for PE and 2,267 patients underwent CTPA or VQ scanning. 285 patients (0.4% of the study population) were diagnosed with acute PE.
Table 4: Characteristics of the study population (all patients accessing the ED with a relevant presenting complaint), effect outcome, and balancing measure.
|
Denominator
(for proportions)
|
Before
N([%)*
|
After
N (%)*
|
Percentage difference for outcome (95% CI)
|
Total # of patients (n)
|
NA
|
70,932
|
10,171
|
NA
|
# of patients per month [median (Q1, Q3)]
|
NA
|
3073
(3206, 3324)
|
3038
(3437, 3696)
|
NA
|
Age, years [median (Q1, Q3)]
|
NA
|
58 (40, 73)
|
58 (40, 72)
|
NA
|
Female, n (%)
|
All patients
|
38,120 (54%)
|
5,519 (54%)
|
NA
|
Outcomes
|
|
|
|
|
Tested for PE
|
All patients
|
5,094 (7.2)
|
899 (8.8)
|
1.6 (-0.3; 3.4)
|
Test appropriate
|
Patients tested for PE
|
3,782 (74.2)
|
740 (82.3)
|
8.1 (5.0; 11.2)
|
Imaging use
|
All patients
|
1,903 (2.7)
|
364 (3.6)
|
0.9 (-1.1; 3.4)
|
Imaging positive yield
|
Patients with imaging
|
247 (13.0)
|
38 (10.4)
|
-2.6 (-13.1; 8.0)
|
Imaging not appropriate
|
Patients with imaging
|
432 (22.7)
|
22 (9.1)
|
-13.6 (-26.3; -0.9)
|
Imaging missed
|
Patients with positive D-dimer
|
665 (35.0)
|
98 (26.0)
|
-9.0 (-18.4; 0.4)
|
PE prevalence
|
All patients
|
247 (0.4)
|
38 (0.4)
|
0.0 (-2.0; 2.1)
|
Central PE
|
All patients
|
221 (0.3)
|
34 (0.3)
|
0.0 (-2.0; 2.1)
|
Distal PE
|
All patients
|
26 (0.04)
|
4 (0.04)
|
0.0 (-2.1; 2.1
|
D-Dimer use
|
All patients
|
4,643 (6.6)
|
862 (8.5)
|
1.9 (-0.1; 3.9)
|
*Unless otherwise specified
CI: confidence interval, Q1: first quartile, Q2: third quartile, PE: pulmonary embolism
7.2% and 8.8% of the study population were tested for PE before and after the intervention, respectively. There was an 8.1% (95% CI 5.0; 11.2) increase in the adherence to the proposed protocol. The imaging positive yield showed a trend toward reduction (-2.6%, 95%CI -13.1; 8.0). The time trends for PE testing is reported in Figure 4 and the time trends for the remaining secondary outcomes are reported in the appendix (Figures S1-S6).
Secondary outcomes: balancing measures
In our study population, a D-dimer was ordered in 6.6% and 8.5% of the patients before and after the intervention, respectively.