The overall approach will involve 1) evaluating external facilitation to expand IDEAS implementation across multiple occupational therapy clinical sites, and 2) measuring the effect of IDEAS on stigma beliefs of practicing occupational therapists.
Convenience and chain sampling will be used to recruit up to ten implementing occupational therapists – internal facilitators – to lead the IDEAS intervention within their respective clinics. Participants will intentionally be recruited from a diverse range of sites with respect to geographic location and clinical setting type (e.g. pediatric, skilled nursing, inpatient rehabilitation). In addition to the ten implementing occupational therapists who will be internal facilitators, participants of this study will include the staff occupational therapists within the clinic who participate in the IDEAS training, and clinical managers who participate in interviews to provide information about the implementation climate.
Partnering sites will include a variety of urban, suburban, and rural occupational therapy clinical settings that serve a wide range of patients across the lifespan. Confirmed sites include outpatient pediatric clinics, adult inpatient rehabilitation hospitals, and extended care facilities. Sites will vary from smaller occupational therapist-led clinics to occupational therapy departments within large hospital networks. IDEAS is implemented as a group training for staff occupational therapy practitioners via a combination of in-person and virtual interactions. The initial training of the internal facilitator occurs via zoom. The IDEAS implementation can occur during an in-person or Microsoft Teams staff meeting. If staff meeting is in-person, expert panel speakers join the meeting via zoom or teams on a screen in the meeting room. If virtual, the panel speakers join the staff on the virtual platform.
Using a hybrid type 3 design, a formative evaluation of IDEAS implementation will be conducted using external facilitation as the implementation strategy.15,16 Evaluative objectives are to 1) collect and analyze quantitative data on provider (occupational therapists IDEAS participants) pre/post stigma beliefs using the AAQ-S; and 2) collect qualitative pre/post data from IDEAS participants, implementors, and clinical managers regarding their experiences of implementation within their site. The latter will be analyzed using CFIR analysis guidelines (described below).
Implementation Strategy – External Facilitation
The external facilitator (SW) will support occupational therapists/internal facilitators from diverse practice settings throughout the US in implementing a single IDEAS intervention within their local clinical sites via a one-time external facilitation virtual meeting. During this meeting the external facilitator will provide and review with each implementing occupational therapist/internal facilitator an e-resource manual (see Appendices 1 and 2), three IDEAS performance video links (sites select which performance is the best fit with respect to their priorities), and a list of contact information for panelist speakers for each video. Following the initial external facilitation meetings, the external facilitator will provide support via phone, text, and email exchanges as needed. Dose of external facilitation will be tracked via an external facilitation tracking log in which the external facilitator notes type of communication (email, phone, zoom), frequency, date, and duration of each encounter.
Our primary measures of implementation success are self-efficacy of implementors, experiences of external facilitation, and feasibility and acceptability of IDEAS.17-19
The research team (excluding the external facilitator) will conduct 3 pre/post stakeholder interviews at each implementation site; one with the implementing occupational therapist/internal facilitator, one with a clinical manager, and one with an occupational therapy practitioner who intends to participate in the IDEAS training. These interviews are created using the CFIR interview generator, and include the constructs listed in Table 1.20
Following the initial external facilitation meeting between the external and internal facilitators, the internal facilitator will complete an online survey containing the items of the Acceptability of Intervention Measure (AIM), the Intervention Appropriateness Measure (IAM), and the Feasibility of Intervention Measure (FIM) regarding their experience of external facilitation. Following IDEAS implementation, the internal facilitator and the occupational therapist who participate in both the IDEAS training and stakeholder interview will complete the AIM and IAM questions regarding the intervention itself. The survey questions are listed in Table 2 and scored on a 5-point scale from (1) “completely disagree” to (4) “completely agree.”
Occupational therapists who attend the IDEAS training will complete pre/post AAQ-S surveys electronically; the pre-survey also contains demographic questions such as age, ethnicity, race, and whether the person identifies as a member of the minoritized population the IDEAS intervention is focused on (yes, no), as well as a brief description of the study and a space for providing written informed consent.
Evaluating Implementation. Members of the research team will use the CFIR codebook template, which contains CFIR definitions and coding guidelines, to code each stakeholder interview from 10 implementation sites. The team will highlight data corresponding with each of the constructs listed in Table 1, while remaining open to additional emerging codes as needed to highlight pertinent data related to implementation. The team will work together on coding until consistent agreement is reached on how to code the data. The team then will code remaining transcripts in pairs, returning to prior transcripts and recoding as needed if/when the codebook is revised.
Following qualitative coding, the team will independently review the data for a single site and meet to assign a quantitative rating to each code based on qualitative data from that site, rating from -2 to +2 based on whether each construct has a positive or negative impact on implementation success. For example, if the packaging of the intervention is described as problematically interfering with successful implementation (e.g., the film does not work), that construct of ‘quality packaging’ will receive a -2 rating. If the packaging is problematic (the film quality is poor but still works) the construct of ‘quality packaging’ will receive a -1. If the packaging does not influence implementation, it will receive a 0, and if it has a positive or highly positive impact on implementation, it will receive a +1 or +2, respectively. The team will enter their ratings in a blinded vote and will discuss any discrepancies, re-voting until consensus is reached for each construct.
Implementation success will be rated as high or low, based on average scores from the AIM, IAM, and FIM items as well as consensus ratings on the CFIR constructs of self-efficacy and experiences of external facilitation. Configurational analysis will then be conducted to determine difference-making combinations of CFIR-related conditions uniquely distinguishing higher- from lower-performing sites. Configurational analysis will draw upon Boolean algebra and set theory to identify a “minimal theory” or a crucial set of difference-making combinations that remove redundancies and that uniquely distinguish one group of cases from another (i.e., those with versus those without implementation success).21-27 Major strengths of this approach include its ability to model equifinality, when multiple paths lead to the same outcome; its capacity to identify complex relationships, when several conditions work together jointly as a whole; and its versatility with small studies.28-32 Configurational analysis in this project will be conducted using Coincidence Analysis (cna) and is supported by using the R package “cna" as well as the software applications R and R Studio.
IDEAS Effectiveness on Provider Stigma and Correlations with Implementation Success. IDEAS effectiveness will be measured regarding its impact on provider stigma using repeated measures ANOVA to compare average provider AAQ-S pre-post change scores within and between sites. A priori power analysis will be performed to determine the required sample size needed to obtain adequate statistical power for the IDEAS intervention. Using the following parameters based on repeated measure ANOVA within- and between- factors of two measurement time points, it assumes effect size of 0.80, α at 0.05, β at 0.80 and the correlation between repeated measures of 0.5. The required total sample size is determined to be 80. The aim, therefore, is to have 8 occupational therapist complete IDEAS, including the pre/post AAQ-S, at 10 sites. Effectiveness outcomes will be included in configurational analyses described above to explore whether/how implementation success and associated factors are related to effectiveness. For example, it is hypothesized that observations of high implementor self-efficacy combined with organizational cultures that embrace change may correlate with greater intervention effectiveness at that site, demonstrated via larger AAQ-S changes scores.