Design
This single-site study has a prospective pre-post design using the RE-AIM framework19 to evaluate the implementation and sustainability of the enhanced PR program. The study consisted of two phases: 1) the implementation phase (first six months immediately after implementation) and 2) the sustainability phase (18 months after implementation). Both quantitative measures and open-ended survey-questions were used.
Participants
The program implementation was assessed on individual (patients) and organization levels (PR program/HCPs). The inclusion criteria were:
-
Patients enrolled in the enhanced PR program, with a diagnosis of COPD confirmed by post-bronchodilator FEV1/FVC ratio of less than 0.71.
-
HCPs involved in the PR program were invited to participate. This included individuals licensed in nursing, respiratory therapy, physiotherapy, occupation therapy, social work and nutrition.
Exclusion criteria included: patients with a diagnosis other than COPD, with cognitive impairments who were unable to accurately complete questionnaires, and who could not understand either English or French.
Study site and recruitment
The study took place at the PR program of the Montreal Chest Institute (MCI) of the McGill University Health Centre, in Montreal, Canada. The existing outpatient PR program included both weekly group education sessions, as well as supervised exercise 3 times per week, for 6 weeks. No changes were made to the timing or duration of the program. The main changes made in the enhanced PR program related to the education material used as well as the format for delivery of education sessions. The previous educational sessions and materials had been developed by PR staff and contained a mixture of didactic and self-management focused education styles, the sources were not standardized (e.g. they do not all refer to the LWCOPD program or care guidelines), and their quality and content were dependent on the professional expertise. The enhanced PR program contains 12 essential education topics including slides and patient handouts which were designed by a team of content experts and they encouraged patient participation and principles of self-management education. HCPs were asked to refer to the reference guide for self-management delivery and communication with individual and group patient sessions. All the new material contains standardized references to the LWWCOPD (log in to www.livingwellwithcopd.com, section “Rehabilitation” for details of education sessions and materials).
For the implementation phase, patients were recruited from the first three cycles of PR (6 weeks each) after implementation of the enhanced PR program (August 2017 until February 2018). Patients who were enrolled in the program 18 months after implementation (two cycles – March 2019 until June 2019) were included in the sustainability phase of the study. All HCPs working in the PR program during the implementation and sustainability phases were invited to participate and included when they provided informed consent.
The enhanced PR program
The enhanced PR program is based on current evidence and international guideline recommendations for PR with self-management education based on the LWWCOPD program. The LWWCOPD is effective in promoting behaviour change, improving quality of life, and reducing hospital admissions in individuals with COPD16–18. More information on the enhanced PR program is shown in the Additional file 1.
Implementation strategy
Before implementation, all HCPs involved in the PR program at the MCI received an initial one-day group training with time set aside for questions and discussion regarding the implementation plan. During the training, the program content, goals and motivational communication techniques were presented and discussed. All HCPs involved in the program were shown how to access the resources of the program (e.g. reference guides, assessment forms as well as facilitator notes and slides for group education sessions). Feedback from HCP’s involved in the program was incorporated into the final implementation plan. Members of the study team were available throughout the initial phase to answer questions and assist PR staff as required. One of the main adaptations done by the HCPs was to schedule the 12 essential topics of the CPRP during the available 10 group education sessions (e.g. some group education sessions would need to include 2 topics).
Measures
The RE-AIM framework19 guided the outcomes for both objective 1 (implementation phase) and objective 2 (sustainability phase). The domains of the RE-AIM Framework are depicted in Table 1. Objective 3 pertained to both phases of the study.
Table 1
Domains of the RE-AIM framework.
Domain
|
Definition
|
Assessment level
|
Operationalization within study
|
Reach
|
The absolute number, proportion, and representativeness of individuals who are willing to participate in a given initiative
|
Individual level
|
Number of patients referred to program, number of patients accepting to participate in the program and characteristics
|
Effectiveness
|
The impact of an intervention on important outcomes, including potential negative effects, quality of life, and economic outcomes.
|
Individual level
|
Clinical outcomes of patients pre and post program
|
Adoption
|
The absolute number, proportion, and representativeness of settings and intervention agents who are willing to initiate a program.
|
Organization level
|
The proportion of HCPs following the enhanced PR program out of the total HCPs involved in PR at the study site and their characteristics
|
Implementation
|
At the setting level, implementation refers to the intervention agents’ fidelity to the various elements of an intervention’s protocol. This includes consistency of delivery as intended and the time and cost of the intervention.
|
Organization level
|
Fidelity on content and delivery of the group education sessions by the HCPs involved
|
Maintenance
|
At the individual level, maintenance has been defined as the long-term effects of a program on outcomes after 6 or more months after the most recent intervention contact.
|
Individual level
|
Clinical outcomes of patients at 3-month follow-up
|
|
The extent to which a program or policy becomes institutionalized or part of the routine organizational practices and policies.
|
Organization level
|
Outcomes on domains Reach, Effectiveness, Adoption and Implementation as described above within new PR cycles 18 months after initial implementation of the program.
|
available from www.re-aim.org |
Implementation phase
Reach
Number of individuals referred for PR, number of patients with COPD participating in the enhanced PR program within the study period, reasons for not participating and characteristics of the included individuals.
Effectiveness
Clinical outcomes pre and post program (functional exercise capacity, health status, patient report of difficulty or ease in daily activities, knowledge and self-efficacy) (Table 2).
Table 2
Measures and instruments used to assess effectiveness.
Outcome measure
|
Measurement instrument
|
Clinical important difference
|
How data was collected
|
Functional exercise capacity
|
6MWT
|
≥ 33 m ↑
|
Part of program, obtained by HCP’s
|
Health status
|
CAT
|
≥ 2 points ↓
|
Patient report of difficulty or ease in daily activities
|
FPI-SF
|
≥ 0.5 points ↑
|
Additional questionnaires, obtained by researcher
|
Knowledge
|
LINQ
|
≥ 1 point ↓
|
BCKQ
|
≥ 3% ↑
|
Self-efficacy
|
SEAMS, SEWS and MSEES*
|
≥ 10% ↑
|
↑an increase of clinical important difference is a positive outcome. |
↓ a decrease of clinical important difference is a positive outcome. |
*score from the MSEES is split into 4 subscores following the example of Selzler et al. 2019. |
Abbreviations: 6MWT = Six Minute Walk Test, CAT = COPD Assessment Test, HCP = Health care professionals, FPI-SF = Functional Performance Inventory Short Form, LINQ = Lung Information Needs Questionnaire, BCKQ = Bristol COPD Knowledge Questionnaire, SEAMS = Self-Efficacy for Appropriate Medication use Scale, SEWS = Self-Efficacy for Walking Scale, MSEES = Multidimensional Self-Efficacy for Exercise Scale |
Adoption
The proportion of HCPs that follow the enhanced PR program out of the total HCPs involved in PR at the study site. Characteristics of the HCPs of the site (e.g. experience in PR and professional qualifications). Adoption also includes the willingness of HCPs to adopt the new program which is covered in the survey for HCPs.
Implementation
Fidelity of the group education sessions using a checklist developed by investigators (Additional file 2) that evaluated the content (4 or 5 items) and delivery (3 items) of the education sessions. The components of this checklist were scored on a 7-point Likert scale ranging from (1) Not addressed to (7) All aspects covered clearly and in depth. One of the researchers attended a sample of the group education sessions to score.
Maintenance
Clinical outcomes (knowledge, self-efficacy and the patient report of ease or difficulty in daily activities) at 3-month follow-up.
Sustainability phase
As the implementation phase was a learning process, some outcomes were changed or added for the sustainability phase. The outcomes for the Reach and Effectiveness domains were the same as the implementation phase. Changes in staff were tracked for the domain Adoption.
For the implementation domain, the same fidelity checklist was used to assess the group education sessions. In addition, the exercise sessions were assessed for fidelity. The duration, number of sessions, home exercise, types of exercise, monitoring and the use of strategies to determine exercise intensity were compared to the programs’ recommendations described in the Additional file 1. These data were extracted from the patients’ medical records. To understand how the HCPs were dealing with the enhanced PR program 18 months after implementation, one of the investigators observed and communicated with the all dedicated HCPs in their weekly meetings during the study period. At the group education sessions when one of the investigators was present, she observed and communicated with patients as well. This type of observation is called observer-as-participant by Gold (1958)20. Notes and ideas were put in a digital logbook.
As all outcomes above were assessed 18 months after implementation, further outcomes for the domain “Maintenance” would be redundant and therefore were not included in this phase.
Satisfaction with, facilitators of and barriers to implementing and sustaining the enhanced PR program
For objective 3, patients and HCPs were asked to fill out surveys in both the implementation and the sustainability phases (Additional file 3). Patients were asked about their satisfaction with the program components and potential barriers for participation. HCPs were asked about their satisfaction with the content and introduction of the program and potential barriers for implementation and maintenance of the program.
To further assess the barriers and facilitators of the implementation and maintenance as perceived by the HCPs in the sustainability phase, HCPs were asked to complete an adjusted version of the Determinants of Implementation Behavior Questionnaire (DIBQ) which is based on the Theoretical Domains Framework (TDF)21,22.
Analysis
Outcomes on reach, adoption and implementation were analyzed descriptively. T-tests for patient characteristics were performed to identify statistical differences in participants that completed and participants that did not complete the program in the implementation phase. This was not done for the sustainability phase as there was only one participant that did not complete the program. From the fidelity scores of the group education sessions, a percentage was calculated for each group education session, with ‘good’ fidelity ≥ 5.
Changes in patient outcomes were calculated. Because of low sample size no statistical tests were performed. Therefore, the number and percentage of participants that improved more or equal to the minimal clinical important difference (MCID) were calculated. This data was included in the additional files. The MCIDs used were based on the literature as depicted in Table 2. No MCID was found in the literature for the Functional Performance Inventory Short Form (FPI-SF) and the self-efficacy questionnaires. Based on clinical expertise we used an increase of 0.5 points as minimal change for the FPI-SF and an increase of 10% as minimal change for the self-efficacy questionnaires. This method was also used for the patient outcomes at the 3-month follow-up.
To identify barriers and facilitators, open-ended questions from the surveys were summarized into a comprehensive list.
Domain-scores were calculated for the DIBQ. Following the example of Kunstler et al.23, the domain-scores were categorized in disagree (values 1 to 3.5), neither agree or disagree (values 3.5 to 4.5) and agree (values 4.5 to 6). Frequencies were calculated for all categories. SPSS (IBM Corp. Released 2015. IBM SPSS Statistics for Windows, Version 23.0. Armonk, NY: IBM Corp) was used for the descriptive analysis.