Aims and Objectives
The primary aim is to identify and describe the context in which implementation occurs and the factors that influence implementation [22, 23]. Based on the literature and our pilot assessment of SurgeCon, we have included a multi-faceted implementation strategy to help support the successful uptake of SurgeCon. Thus, our secondary implementation aim is to describe the use of these strategies in different ED contexts and identify any changes that can be made to improve the uptake and sustained use of SurgeCon.
Study Setting
EDs with 24/7 on-site physician support in the Eastern Health (EH) region of NL were selected in this clinical trial: (1) Health Sciences Centre (Urban/Tertiary), (2) St. Clare’s Mercy Hospital (Urban/Tertiary), (3) Dr. G.B. Cross Memorial Hospital (Rural/Secondary), and (4) Burin Peninsula Health Care Centre (Rural/Secondary)[9]. Participants of this study consist of patients who visit any of the four selected EDs during the study period, providers/staff, and health system managers.
Implementation Team Composition
The planning and implementation team for this study is multi-disciplinary, including qualitative and quantitative researchers, clinical trial experts, implementation scientists, healthcare economists, health informatics researchers, healthcare professionals, health system managers, frontline healthcare staff, and patients, all of whom play significant roles as patient research partners, facilitators, site coordinators, champions, and members of various working groups and committees (Implementation, Innovative Clinical Trial, Patient Engagement, Steering, Executive) in implementing and evaluating the intervention program (see Table 1).
Table 1
Implementation Personnel List
|
Implementation Personnel
|
Explanation
|
1
|
SurgeCon facilitator
|
The facilitator is a member of the Implementation Working Group who was involved in the development of SurgeCon in Carbonear. The facilitator will be designated to be one of the main points of contact for local teams during the implementation process.
|
2
|
Site coordinator
|
Site coordinators are nurses who will be performing on-site research-related tasks and assisting with the interview recruitment.
|
3
|
Frontline SurgeCon Champions
|
Once the site assessment is complete, ED management will be asked to select a member of the local ED frontline team who will receive additional training either in-person or remotely. That individual will then be an ongoing point of contact for ED staff at their site who have any questions related to SurgeCon. They will also liaise with the research team at regular intervals to discuss any practical or technical issues with using SurgeCon. Also, observation will be utilized by champions to collect information during the readiness assessment period to ongoing implementation evaluation
|
4
|
Implementation working group
|
The SurgeCon Implementation Working Group is responsible for overseeing and guiding the implementation of the intervention at each of the selected sites.
|
5
|
Innovative Clinical Trial (iCT) working group
|
The iCT Working Group supports the team with their expertise in methodology and ensures the validity and precision of the study.
|
6
|
Patient Engagement Working Group
|
The Patient Engagement Working Group will oversee and guide patient engagement and patient-oriented research in all sites.
|
7
|
Executive Committee
|
The Executive Committee (all Steering Committee members, plus the patient advisor, payer representative, key strategic area leaders, and a policymaker), will oversee the whole project and have the authority to determine priorities and supervise the general course of operations.
|
8
|
Steering Committee
|
The Steering Committee (nominated PI, clinician PI, SurgeCon facilitator, research manager) will manage daily operations to ensure the project adheres to the Rewarding Success agreement and that the highest standards of scientific rigour are maintained.
|
9
|
Patient Research Partners
|
Patient Research Partners are patients who are also members of the research team. Patient research partners provide their perspective and help guide decisions to ensure the research produces outcomes and knowledge that can be used to help address the needs and priorities of the local populations they represent.
|
10
|
ED Team
|
The ED team includes all physicians, nurses, allied health, and other personnel who work at the four selected EDs. They will participate through interviews and report questions, concerns, and issues to site coordinators and champions.
|
We involve stakeholder groups (e.g., patients, decision makers, researchers, clinicians, etc.) in all stages of SurgeCon’s implementation. Collaboration is not limited to the providers and managers, as the role of patients is crucial. Through our implementation framework, we contribute to the literature by guiding researchers on how to engage patients in a systematic way to redefine and reassess intervention programs to make them more culturally and contextually relevant. We also used four essential pillars to shape our patient engagement strategy [24]: (1) Patient Initiation (allowing patients to participate in the research process), (2) Building Reciprocal Relationships (all individuals being treated as equal partners), (3) Co-learning (researchers and patients learning from each other), and (4) Re-assessment & Feedback (routinely consulting with patients and making improvements accordingly). Our framework gives patients a variety of opportunities to engage in the implementation and evaluation process, ranging from limited commitment (e.g., surveys, interviews, etc.) to full, ongoing participation through team membership.
Data Collection
Implementation outcomes will be collected through a mixed-methods approach, including 1) semi-structured, in-depth interviews, 2) observation by researchers and champions, 3) survey instruments, 4) wait times data, and 5) data from SurgeCon’s dashboard system and evaluated by patients, providers/staff, health system managers.
Semi-structured, in-depth interviews (conducted via phone, digitally recorded, and lasting 45 to 60 minutes) and observation will be utilized to collect information during the exploration period to ongoing implementation evaluation (e.g., assessing ED’s physical layout, barriers, and enablers to SurgeCon adoption and sustained use, and examining ED contextual constructs and factors).
In the first stages of qualitative data collection, purposeful sampling will be applied to maximize the chance of obtaining rich data for central research topics from “Information-rich cases” which will subsequently be followed by theoretical sampling [25]. However, theoretical sampling will indicate that the sample is “not selected from the population based on certain variables prior to the study; rather the initial sample is determined to examine the phenomena where it is found to exist” [26].
Quantitative data for this study includes primary and secondary data. Primary data will be collected using a survey instrument via telephone interview. Secondary data will be provided to our team by the Newfoundland and Labrador Centre for Health Information (NLCHI). Additional details on quantitative data collection and analysis are available via other works published by our team [27].
Individuals selected to be site coordinators will be responsible for assisting with the scheduling and coordination of interviews with ED staff. Staff members who indicate that they do not want to participate in the study will not be contacted. The site coordinator will facilitate the pre-interview process for staff members who have expressed interest in completing the interview (e.g., consent form, selecting a private location for the interview to be conducted) or will provide contact information of staff members to the research team. The research team member will then complete a web-conference/telephone interview with the participant in a private setting. Recruitment will continue until data saturation has been achieved.
Data Analysis
Qualitative data from interviews and observations will be analysed based on Straussian Grounded Theory (GT) (1990) [28]. During the open coding process, the first stage of data analysis is to break down collected data into concepts, their dimensions, thoughts, and ideas. This provides an opportunity for the researcher to find similarities and differences to categorize similar occurrences and behaviours into the same group. Data generated during the open coding process that resemble one another are subdivided into different codes. This subdividing of data assists in the development of a comprehensive explanation of the phenomenon, which is the purpose of axial coding, the second stage of the open coding process. The extraction of a core category from this initial two-stage process is the task of the investigator during selective coding. All these stages in the coding process will be conducted by a qualitative researcher. Codes and categories will then be reviewed by members of the Implementation Working Group to reach a consensus. At the end of the study, the credibility of results will be enhanced by member checking, data triangulation, and peer debriefing.
COVID-19 Impact on Research Operations
Due to the pandemic, staff training and research team meetings will be delivered or carried out through web-conferencing/a virtual platform. Additionally, some of the data collection was originally intended to be carried out by the research team; however, we changed the plan by involving hospital employees. For instance, hospital employees will utilize observation to collect information during the ongoing implementation evaluation. Also, the coordination of research operations at each of the sites will be completed by EH staff since the intervention sites are restricted to Eastern Health personnel. Eastern Health employees responsible for research related tasks will be jointly supervised by the research team and Eastern Health managers. Additionally, the SurgeCon platform and research instruments will be adjusted to capture and report COVID-19 data.
Implementation and Evaluation
ED Implementation and Evaluation Framework
SurgeCon’s implementation plan include an iterative improvement process that is divided into four stages: 1.) Exploration; 2.) Adoption; 3.) Active Implementation; 4.) Sustainment [29]. For each stage, we applied the RE-AIM framework and CFIR domains to identify the criteria that applied to our context. Table 2 provides details on the evaluation and timelines according to the implementation stages. All measures, including how each domain is mapped in the RE-AIM framework are provided. We have expanded RE-AIM’s implementation outcomes to include outcomes recommended by CFIR [30].
Table 2
Implementation frameworks used to evaluate implementation outcomes
Dimensions/ Variables Description
|
Implementation Stage (Time Period)
|
|
Exploration Months 1–10 at all hospitals
|
Adoption
2 Months at each hospital (months 11-
12, 17–18,
23–24, 29-
30)
|
Implementation Evaluation/iCT Months 13–31
|
Sustainment Months 17–48
|
Reach
(RE-AIM Framework)
Who is intended to benefit and who actually participates or is exposed to the intervention?
|
Exclusions: The percentage of eligible ED sites that are excluded pre- randomization.
|
√
|
√
|
√
|
|
Participation rate: The number of EDs that participate divided by all EDs that meet the eligibility criteria.
|
|
√
|
√
|
|
Characteristics of the participant sites and non- participants sites: Assessment of the following variables: the average number of patients, the average number of staff, staff mix, staff characteristics (age, sex, years of practice) and patient characteristics (age, sex, CTAS score). This will also include assessment of potential moderating factors such as organization readiness for change.
|
√
|
√
|
√
|
√
|
Understand Barriers and Enablers to Reach
|
√
|
√
|
√
|
√
|
Effectiveness
(RE-AIM Framework)
What is the most important benefit you are trying to achieve and what is the likelihood of negative outcomes?
|
Health System Level
• Length of Stay
• Time Until Physician Initial Assessment
• Left Without Being Seen
|
|
|
√
|
|
Patient-reported Level
• Satisfaction
• Patient reported experiences of ED service
|
|
|
√
|
|
Understand Barriers and Enablers to Effectiveness
|
|
|
√
|
|
Adoption
(RE-AIM Framework)
Where is the program or policy applied? Who applied it?
|
The proportion of Health Care Providers who engage in SurgeCon activities among those who agreed to participate in the study (Acceptability, appropriateness)
|
√
|
√
|
√
|
|
Understand Barriers and Enablers to Adoption
|
√
|
√
|
√
|
|
Implementation
(RE-AIM Framework)
How consistently is the program or policy delivered? How will it be adapted? How much will it cost? Why will the results come about?
|
Fidelity of staff training
|
|
|
√
|
|
Fidelity of intervention delivery
|
|
|
√
|
|
Adaptations
|
|
|
√
|
|
Implementation Cost
|
|
|
|
√
|
Understanding Implementation
• Acceptability
• Appropriateness
• Feasibility
• Barriers and Enablers
|
|
|
|
√
|
Maintenance
(RE-AIM Framework)
When will the initiative become operational; how long will it be sustained (setting level); and how long are the results sustained (individual level)?
|
Institutionalization: long- term adoption of SurgeCon
|
|
|
|
√
|
Cost of maintaining the intervention
|
|
|
|
√
|
Sustainability - Barriers and Enablers
|
|
|
√
|
√
|
Scalability
(CFIR Framework)
|
|
|
|
|
Intervention Characteristics (e.g. stakeholders’ perception, complexity of intervention)
|
|
√
|
√
|
|
Outer Setting (health system policy, patients’ needs)
|
√
|
√
|
√
|
√
|
Inner Setting (resources, leadership)
|
√
|
√
|
√
|
√
|
Characteristics of
Individuals Involved (knowledge, attitude)
|
|
√
|
√
|
√
|
Process of Implementation (planning, training)
|
|
√
|
√
|
|
Stage 1: Exploration (Site assessment)
The working group will complete five activities during the exploration phase including: delivering a virtual presentation to inform ED staff (paramedics, nurses, and physicians) and management of upcoming operational changes; observing the ED’s physical layout to make improvements necessary for the implementation; recording the clinical and a demographic characteristics of EDs and patients, collect information related to the ED’s organizational and workflow structures; conducting patient telephone interviews to capture their live experiences and feedbacks; and conducting semi-structured interviews to clarify barriers and key performance issues. Since there is a high degree of variability that exists between EDs, the information collected during the site assessment allows for the customization of SurgeCon’s underlying protocols and determines whether certain components of the intervention are appropriate or applicable for implementation. Information collected during the site assessment will allow the SurgeCon Implementation Working Group to tailor components of the intervention to address site specific needs (see Table 3).
Table 3
Description of activities during the exploration phase
Activity
|
Objective
|
Target groups (ED, Staff, Patients, patient advisors)
|
Measurement tool and method
|
Outcome of interest
|
1. Deliver a presentation
|
Prepare staff for upcoming changes
|
ED staff
|
Virtual presentation
|
Improved understanding of SurgeCon and obtaining the highest level of support possible.
|
2. Observing the ED’s physical layout
|
To gain insight into the best next steps to improve layout and gain the greatest efficiency
|
ED
|
Qualitative Observational Method/site assessment checklist
|
Improved area of operations
|
3- Recording the clinical and demographic characteristics of hospitals and patients
|
To assess ED’s organizational and workflow structures
|
ED
|
Site assessment checklist
|
Understanding ED characteristics
|
4-Conducting patient-reported experiences and Patient Satisfaction telephone interviews
|
To consult with patients, get feedback from them, study their lived-experiences
|
ED patients
|
Patient-reported experiences and patient Satisfaction survey
|
Develop strategies to overcome the challenges of poor patient satisfaction
|
5-Conducting interview about barriers and enablers to Reach
|
To study the organizational climate of the ED before the intervention and consult with ED staff to gauge interest in implementing a new ED management system.
|
ED staff
|
Semi-structured interview
|
Resolve potential contextual barriers to successful implementation
|
Stage2: Adoption (SurgeCon Installation)
During the adoption stage, the working group will complete eight activities, which will be initiated by the SurgeCon initialization across the four study sites in a stepwise manner to help ED staff (paramedics, nurses and physicians) manage their actions to actively reduce patient surges and wait times and increase patients' access to emergency medical care. To test SurgeCon’s digital whiteboard application, we will validate information extracted from the platform’s data repository and analyze end user feedback to help inform iterative software updates. SurgeCon staff training will occur during this stage, while additional training and support will be made available to all sites throughout the study period. The installation of hardware and software assets for the intervention’s e-Health component as well as training will be carried out a month in advance of the adoption phase start date. We also plan to post department level data in a prominent area of the ED monthly (depending on the discretion of the sites) and also circulate individual provider key performance indicator (KPI) data anonymously. This will give physicians and also individual nurse practitioners the opportunity to know their monthly physician’s initial assessment (PIA) times compared to the ED average and targets set by ED management. Providing this data may increase physician motivation to use components of SurgeCon flow education since it will demonstrate the intervention’s capacity to reduce their door-to-doctor time – a key metric for health standards of care. At this stage, the SurgeCon facilitator who will assist local ED teams with the development of clear roles, goals, and communication strategies. The Facilitator will review SurgeCon process improvement activities, usage of the whiteboard application, and adherence to the protocol. Local champions who are leading internal change initiatives along with members of the Implementation Working Group will help prepare EDs for the Active Implementation stage of the study. Improving the overall appearance of physical spaces in the ED (e.g., waiting room, fast-track zone, examination rooms, treatment space, etc.) to improve patient satisfaction is another goal in this stage. In consultation with our local patient partners, we will renovate, redecorate, and declutter ED spaces (see Table 4).
Table 4: Description of activities during the adoption phase
Activity
|
Objective
|
Target (ED, Staff, Patients, patient advisors)
|
Measurement tool and method
|
Outcome of interest
|
1- Installing SurgeCon’s digital whiteboard application
|
To advise when to use volume- based staffing (shifting staff between areas of the hospital based on workload), appropriate and timely involvement of hospital management, and overcapacity protocols, which may otherwise be overlooked by distracted frontline ED staff.
|
ED
|
Protocol-driven software platform
|
Online Illustration of ED organization and wait time
|
2- Data collection
|
To enhance the platform and update ED staff
|
ED
|
SurgeCon platform and ED secondary data
|
Enhancing ED functioning
|
3-Training frontline SurgeCon Champions
|
To provide ongoing learning related to using the new system and the successful uptake and adoption of SurgeCon
|
ED staff
|
Participant observation
|
Solving practical or technical issues with using SurgeCon
|
4- Staff Training
|
To encourage ED staff to become active participants in the improvement process
|
ED staff
|
Interactive Simulation Course[11], SurgeCon eHealth Platform Training[12], and Patient Centeredness Training[13]
|
Facilitating the implementation of quality improvement
|
5- Site Assessment
|
To determine whether the site is capable of implementing the platform as intended given the level of resources and staff commitment
|
ED staff
|
Champion observation and site coordinator reports
|
Assisting ED staff in development of SurgeCon
|
6- Establishing a Patient-Centric ED Environment
|
To improve the overall appearance of physical spaces in the ED
|
ED
|
Patient advisors’ observation and Site assessment checklist
|
To improve patient satisfaction
|
7-Conducting patient-reported experiences and Patient Satisfaction telephone interviews
|
To consult with patients, get feedback from them, study their lived-experiences
|
ED patients
|
Patient-reported experiences and patient Satisfaction survey
|
Develop strategies to overcome the challenges of poor patient satisfaction
|
8-Conducting interviews about barriers and enablers
|
To study the organizational climate of the ED before the intervention and consult with ED staff to gauge interest in implementing a new ED management system.
|
ED staff
|
Semi-structured interviews
|
Resolving potential contextual barriers to successful implementation
|
Stage 3: Active Implementation (SurgeCon monitoring and evaluation)
During the implementation phase, five activities will be completed to evaluate the intervention and to determine whether implementation efforts made are sufficient to overcome barriers to a successful implementation of the intervention and produce improved outcomes. This includes fidelity of intervention (i.e., the degree to which the intervention was implemented as intended) and fidelity of training. Furthermore, we will examine contextual constructs and factors influence the intervention. For instance, mediators (factors that increase the intervention’s effectiveness in terms of producing positive outcomes), moderators (factors that influence the degree of change in targeted outcomes), and underlying mechanisms that lead to sustained organizational changes at all levels of EH’s organization [31, 32]. The ultimate goal is to culturally embed SurgeCon in each ED if it can produce consistently positive results. Adaptations will also be made to the intervention by the research team during the study period. This process will identify intervention components and implementation methods that produce the best response in staff and management in terms of intervention acceptability, commitment to change, and improved performance. Local ED teams and champions will work closely with the research team to ensure all parts of the intervention are applied as intended and evaluated appropriately. We will outline the parameters of operational processes developed during the adoption stage of the implementation plan (see Table 5).
Table 5
Description of activities during the active implementation phase
Activity
|
Objective
|
Target (ED, Staff, Patients, patient advisors)
|
Measurement tool and method
|
Outcome of interest
|
1-Measuring fidelity of training
|
To determine if the staff are trained to a well-defined performance criteria
|
ED staff
|
Training evaluation survey
|
Trained/Competent staff
|
2- Measuring fidelity of intervention delivery
|
To determine if the sites have implemented SurgeCon as intended
|
ED
|
Software platform and champion participant observation
|
Successful implementation
|
3- Examining contextual constructs and factors
|
To identify and describe the context in which the intervention occurs and the factors that influence implementation.
|
ED staff
|
Champion observation, Site coordinator report
|
Understanding contextual constructs and factors to develop a plan for overcoming the challenges and ensuring successful implementation
|
4-Conducting patient-reported experiences and Patient Satisfaction telephone interviews
|
To consult with patients, get feedback from them, study their lived-experiences
|
ED patients
|
Patient-reported experiences and patient Satisfaction survey
|
Develop strategies to overcome the challenges of poor patient satisfaction
|
5-Conducting interview about barriers and enablers
|
To study organizational climate of ED before, during and after the intervention and consult with ED staff to gauge interest in newly implemented ED management system.
|
ED staff
|
Semi-structured interview
|
Resolving potential contextual barriers to successful implementation
|
Stage 4: Sustainment (SurgeCon maintenance and sustainability)
During the sustainment phase, the extent to which SurgeCon has become institutionalized or part of routine ED practices and policies will be examined through five activities. We will measure (1) Institutionalisation: the long-term adoption of SurgeCon including if all elements of the intervention (e.g., the action-based protocol) and its implementation strategies (e.g., champions) were retained and changed from the pilot study in Carbonear, (2) Feasibility: the extent to which the intervention can be carried out in other ED departments besides Carbonear, and (3) Cost of maintaining the intervention (e.g., system updates, staff training, etc.) and maintaining the restructured ED and its patient-centred environment. Also, the perceived barriers and enablers of Regional Health Authorities (RHA) managers and ED staff to the set-up and use of the SurgeCon intervention will be assessed. This will include an assessment of the different procedural components (e.g., site assessment) of SurgeCon, the day-to-day use of the different elements of SurgeCon (e.g., fast track zone, interaction with the technology and software), and the education, training, champion, and feedback strategies to determine if they were perceived to be enablers to implementation. We will use the CFIR guidelines to underpin the interview guides. We will create analytic matrices to conduct a cross-case analysis for identifying patterns of barriers and enablers and develop a collaborative approach to maintain and refine the platform over time to scale up. Additionally, to improve SurgeCon’s sustainability, all participating EDs will meet quarterly by teleconference, sharing ED team success stories, and identifying barriers to implementation, as well as approaches used to mitigate negative outcomes. They will prepare a one-page summary for the SurgeCon Executive Committee which will include the following: how ED teams plan to maintain SurgeCon’s use and operation, how to improve ED staff capacity and skills required to properly use the SurgeCon platform, and how SurgeCon can be better adapted to address general and site-specific needs. Furthermore, at this stage, it is important to identify individuals in leadership roles at both the system and hospital levels who can help drive SurgeCon’s process improvement initiatives beyond this study (see Table 6).
Table 6
Description of activities during sustainability phase
Activity
|
Objective
|
Target (ED, Staff, Patients, patient advisors)
|
Measurement tool and method
|
Outcome of interest
|
1-Assessing scalability of SurgeCon
|
To identify the extent to which the intervention can be maintained and carried out in other ED departments
|
ED
|
RE-AIM scoring and CIFR instrument
|
Scalable SurgeCon Platform
|
2-Measuring the cost of implementing the intervention
|
To measure the cost of maintaining the intervention (e.g. system updates, staff training, etc.) and maintaining the restructured ED and its patient-centred environment
|
ED
|
Economic analysis
|
Cost of implementation
|
3- Assessing feasibility of maintaining SurgeCon
|
To identify the issues raised and their relevance to maintaining the intervention
|
ED
|
RE-AIM scoring instrument, champion observation, and site coordinator report
|
Making changes or resolving challenges as a result of intervention
|
4-Conducting patient-reported experiences and Patient Satisfaction telephone interviews
|
To consult with patients, get feedback from them, study their lived-experiences
|
ED patients
|
Patient-reported experiences and patient Satisfaction survey
|
Develop strategies to overcome the challenges of poor patient satisfaction
|
5-Conducting interviews about barriers and enablers to institutionalization
|
To study organizational climate of ED after the intervention implementation
|
ED staff
|
Semi-structured interview
|
Resolving potential contextual barriers to successful maintenance
|
[9] The Carbonear ED was excluded since it was the pilot site of the intervention and the Janeway Children’s hospital ED was excluded since it only provides care to children and adolescents and did not have excessive wait times.
[11] Interactive Simulation Course. This module focuses on an ED improvement flow sustainability strategy. The aim will be to provide information on the rationale underpinning SurgeCon and how it works in the practice setting using case-based scenarios. It will be interactive allowing for questions and answers and ensuring the learning outcomes are achieved.
[12] This module will help ED staff: (1) become familiar with the digital whiteboard application, (2) learn how the system collects and reports information, (3) learn how to interpret system generated notices and warnings, (4) assign protocol tasks, and (5) learn effective response strategies to system notices and warnings.
[13] This module includes an educational session which reinforces the following topics: (1) providing quality ED care to all patients regardless of urgency, (2) treating all patients with respect, and (3) demonstrating that the patient’s visit to an ED must always be considered necessary as they may have no other option.