Ethical approval to proceed with this study was granted by the local health service ethics board (HREC/18/WH/120). This study utilized a retrospective mixed method process evaluation to address the study aim. Process evaluations aim to build workforce capacity in the use of evidence informed interventions, and explore not only ‘what works’ but ‘why’ (17). This approach frequently employs mixed methods, and while it can be time and resource intensive, there is increasing recognition of the significant role of process evaluation in understanding the reality of using complex interventions in practice (18). In particular, process evaluations can provide valuable information about the lived experience of implementation, including the barriers to and facilitators of effective translation of evidence into daily practice (19).
To ensure the study drew on established implementation theory, the Promoting Action on Research Implementation in Health Services (PARiHS) framework was used as an organizing theoretical structure. The PARiHS framework is designed to guide the implementation of evidence-based practice and is widely used in implementation across a range of disciplines (20). The PARiHS consists of four domains – the characteristics of the setting or context (Context), the way in which evidence is facilitated (Faciliation), the individuals and team engaged in implementation (Recipients) and the quality and type of evidence (Innovation) – all of which are believed to influence the success of implementation. In this framework, successful implementation is proposed to result from the interaction between facilitation, innovation, recipient characteristics and context (21).
Specialized vestibular physiotherapy was implemented in the ED of a tertiary metropolitan Australian health service. The service operates three acute public hospitals, and this study was undertaken at a single hospital site. This hospital provides a range of support, including acute and community-based services to aged, adult, paediatric and maternity populations. The local community has a population of approximately 800,000 people, is culturally and linguistically diverse, and is located in one of the fastest growing regions in Australia (22). Approximately 250 patients present to the hospital’s ED daily (11).
A specialized vestibular physiotherapy service was implemented in the ED over the second half (8 weeks) of the 16-week recruitment period for the feasibility study. It was made available to eligible patients during clinician business hours (8am to 4 pm), Monday to Friday due to available resources. Each weekday morning, a study investigator attended the ED and reviewed the triage record of admissions since 4 pm the preceding day to identify patients with specific symptoms (dizziness, vertigo, imbalance, loss of balance, or fall). If any identified patients remained in the ED, the investigator consulted with the medical officer to determine whether the patient was suitable for assessment and intervention by this service. The investigator also periodically reviewed the triage notes throughout the day to identify further qualifying patients who presented during business hours.
Description of the Intervention
The specialized vestibular physiotherapy service targeted adults aged ≥ 18 presenting to the ED with symptoms of dizziness, vertigo or imbalance. Exclusion criteria included; 1) Primary diagnosis by medical officer at initial assessment (or thereafter) with a clear non-vestibular cause; 2) Patients unable to understand instructions and/or willingly participate in vestibular assessment with physiotherapist (due to language, cognition, symptoms or any other reason); and 3) Patients presenting to ED outside of specialized vestibular physiotherapy service hours (16).
The SVP intervention involved assessment and treatment as per clinical practice guideline recommendations (4, 5) Once patient symptoms improved enough to allow safe mobilization, they were discharged from the ED with education about their condition and a referral for specialist physiotherapy follow-up as an outpatient. A range of outcomes from the specialized vestibular physiotherapy service were evaluated as part of the feasibility study (16), and these are provided in Additional file 1.
Eligibility and recruitment for Implementation Study
Eligible participants for this study included physiotherapy, medical and nursing clinicians from the ED, and inpatient acute care and community rehabilitation clinicians who were impacted by the trial implementation of the specialized vestibular physiotherapy service, and therefore considered stakeholders. Clinicians were identified as key stakeholders and invited to participate if they were directly involved in the clinical management of patients diagnosed with peripheral vestibular dysfunction (either in the acute or sub-acute setting).
All eligible clinicians were invited to participate by email, and asked to return a written consent form. Participants were able to indicate on the consent form what modes of data collection they wished to participate in (i.e. ORCA only, interview/focus group only, ORCA and interview/focus group). As per the feasibility study to which this evaluation was linked, the sample size was informed by purposive selection, which identified participants with specific experience of the implementation process under investigation. The study sample included 11 staff members (4 care coordinators, 3 doctors, 5 physiotherapists, and 1 nurse), as not everyone invited chose to participate.
Outcomes of Implementation
The outcomes of interest in this process evaluation were organizational readiness for change, and clinician perceptions of the implementation process (including aspects such as acceptability, appropriateness, feasibility and contextual factors). Data related to patient perceptions of the implementation process were also collected, and will be reported elsewhere. Four outcomes measures were used to gather quantitative data– the Organizational Readiness for Change Assessment (ORCA) (23), Acceptability of Intervention Measure (AIM), Intervention Appropriateness Measure (IAM) and Feasibility of Intervention Measure (FIM) (24).
The ORCA is a 77-item checklist designed to operationalize the constructs of the PARiHS framework (23). The psychometric properties of the ORCA have been established, with its reliability and validity both found to be acceptable across most sub-scales (apart from those related to evidence). In this study, the ORCA was used retrospectively to enable clinicians to identify needs and conditions they perceived to be important to the process of specialized vestibular physiotherapy implementation.
The AIM/IAM/FIM is a combined suite of implementation outcome measures, developed to monitor and evaluate the success of implementation initiatives. The measurement properties of these brief scales have received preliminary confirmation, with reasonable structural validity, known groups validity, test-retest reliability and sensitivity to change indicated (24). Higher scores are interpreted as indicating greater acceptability, appropriateness and feasibility, and scale scores are calculated with mean responses.
Qualitative data was collected via semi-structured interviews and focus groups. A bespoke set of interview prompts were developed, as no suitable existing prompts or schedules were available. The schedules were developed from previously published reflective questions for facilitators using the PARiHS framework (20), and began with prompts which specifically focused on each assessment and treatment component in the clinical practice guidelines. The schedule then become more general in approach, enabling clinicians to reflect and comment on the implementation process as a whole. All focus groups and interviews were digitally recorded, and transcribed verbatim for analysis. The interview prompts have been provided as Additional file 2.
Data was analyzed using a convergent parallel model of mixed methods. This model entails the concurrent collection of quantitative and qualitative data, and values both forms equally when they are combined to produce the overall interpretation (25). All data was transformed to align with the PARiHS Framework, to maintain alignment with implementation theory and also enable more direct contrast and comparison.
Ordinal quantitative responses from the outcome measures were aligned to the domains of the PARiHS Framework. The process of alignment was undertaken by reviewing definitions and descriptions of the four domains (Context, Facilitation, Recipients and Innovation), and assigning each item on the ORCA and AIM/IAM/FIM to its related domain. This process was initially undertaken by one researcher (DH) before being independently reviewed and confirmed by a second researcher (KI). The alignment between the items and the domains are shown in Additional file 3, and not all aspects of the PARiHS Framework were addressed by the quantitative question. This data was analyzed using SPSS Version 25.0, and reported using descriptively to illustrate the proportion of responses.
Qualitative transcriptions were subjected to content analysis, in alignment with the PARiHS framework. A codebook was developed to ensure consistency in analysis, providing definitions for all concepts related to the PARiHS Framework and examples of its presence in the dataset. The codebook has been provided as Additional file 4. Data were systematically coded and categorized to identify trends and patterns, their frequency and relationships. The analysis was completed independently by two researchers (KI, DH), and compared for consistency. A third researcher (ML or AL) also reviewed all coding, and the few instances of disagreement identified were resolved by consensus. The reporting of this study meets the requirements of the Standards for Reporting Implementation Studies (StaRI) statement (26), which has been included as Additional file 5.