Study protocol is published elsewhere. [21] A brief description follows.
Study design
The design was a 2-arm, 3-year pragmatic hybrid type III [22–23] mixed method randomized controlled trial conducted at 18 waiver sites in the state of Michigan. [24–26] A hybrid design was chosen as it examines the effects of implementation strategies and intervention effectiveness for beneficiary outcomes. [23] The Knowledge-to-Action [27] model underpinned examination of outcomes, and the Consolidated Framework for Implementation Research [28–29] guided implementation. Data from clinicians were collected 9 months after training. Beneficiary data for the same period of time were extracted from the electronic health records (EHR), which included the last assessment prior to the clinician training (10/1/2019) and all assessments following implementation (10/2/2019 to 6/30/2020).
Study setting and participants
The settings were 18 Medicaid home and community-based waiver sites in Michigan that used the EHR system. Two Michigan sites using a different EHR were excluded. The waiver supports low-income at or below 300% of the Federal Poverty Level, nursing home eligible, disabled and older adults in the community to avoid institutionalization. Sites care for 400 to 2,500 beneficiaries and have 10 to 125 clinicians. Sites were contracted and clinicians were recruited via email at each site. Beneficiaries were recruited during usual care by clinicians, using a pocket aid to examine beneficiary needs in regard to the intervention. Beneficiaries could opt out and continue to receive care provided by the site, but their data were not extracted from the EHR or analyzed.
Randomization and blinding
To assure similarity of two trial arms, sites were paired in blocks using quality assessment scores (2015-17) and number of beneficiaries. A coin was flipped to determine arm assignment for each pair. Clinicians and beneficiaries were blinded to arm assignment.
Power analysis
Given 18 sites (9 in each arm) available in Michigan, in the comparison of site-level outcome of adoption and sustainability, the detectable effect size with power of 0.80 in two-sided tests at .05 level of significance was Cohen’s d=1.41. Given the sample size of clinicians of 539, the average cluster size was approximately 45, and with an assumed intra-class correlation coefficient (ICC) of 0.01, the design effect factor was 1.45, and the detectable effect size was Cohen’s d=0.29. Given the sample size of 7,030 beneficiaries, the average cluster size was approximately 390, and with an assumed intra-class correlation coefficient of 0.01, the design effect factor was 4.9, and the detectable effect size was Cohen’s d=0.15 for power of 0.80 in two-sided tests at .05 level of significance. These effect sizes are below d=0.33-0.5, the thresholds commonly used for clinical significance, therefore the study was powered to detect any meaningful differences between arms on clinician and beneficiary outcomes. [30, 31]
Usual care
Usual 1915(c) home and community-based services waiver care includes annual assessments, case management, and supports coordination by RNs and social workers (SWs) via home visits and phone calls. [32] Nineteen services are provided as needed and include adult day care, chore services (e.g., cleaning, laundry), community health worker, community transportation (e.g., to doctor’s appointment), counseling, environmental modifications, and a fiscal intermediary. In addition, goods and services, home delivered meals, nursing services (e.g., medication management), personal emergency response system, private duty nursing/respiratory care, specialized medical equipment and supplies, training, personal care, medication management, lawn care, snow removal, cleaning, grocery shopping, and laundry are provided as needed.
Evidence-based Intervention
The 16-week intervention was implemented by OTs who conduct 6 home visits and provide assistive devices, RNs who conduct 4 home visits, and a handyman who provides home alterations such as installing devices and home modifications. [6–9] The team consults with the individuals to identify daily activity goals (e.g., taking a shower and walking to the bathroom) and evaluate barriers to achieving the goals to attain outcomes. OTs assist individuals to carry out ADLs and IADLs that are challenging, such as meal preparation, bathing, and dressing. RNs target pain, mood, fall prevention, incontinence management, and medication management. The intervention was adapted to include SWs to address social and emotional needs. [10]
Implementation strategies
Our bundle included 9 implementation strategies. First, informal relationships were built with sites which included monthly virtual meetings; and formal relationships were by memorandum of understanding, delineating the role and duties of IFs in each site. IFs were identified by each site’s management team and approved by the Principal Investigator (PI). The EF was selected by the PI and was an OT who was an intervener at one of the sites in the pilot project. Second, readiness to implement and leadership were examined. Third, clinician attitude and self-efficacy were examined over time. Fourth, clinicians were trained in the intervention and IFs and the EF were trained in facilitation. IF training included an online 9-module program which included an overview and the implementation plan and topics on problem solving, feedback, reflection, counseling, motivational interviewing, and remediation and a 60-minute synchronous session with the PI. EF training included completion of IF modules plus a 60-minute synchronous session with the PI which included a review of the IF role, challenges an IF may face, and how and when to facilitate with an IF. Fifth, a coalition of IFs met monthly to build capacity by sharing best practice use of implementation strategies. Sixth, IFs prompted interdisciplinary coordination among RNs, SWs, and OTs to promote teamwork, brainstorming, and problem solving to support beneficiary goal attainment. Seventh, IFs facilitated clinician implementation at all sites and the EF facilitated IFs at 9 sites (Arm 2). Eighth, an audit of implementation strategy and intervention fidelity occurred with feedback to IFs who worked with clinicians. Finally, the EF worked with IFs in Arm 2, providing feedback weekly. See the published protocol paper for detailed information and tools. [21]
Measures
Site, clinician, and beneficiary level data were collected using the Stages of Implementation (SIC) [33, 34] (sites); Organizational Readiness for Change (TCU-ORC), [35]; Evidence-Based Practice Attitude Scale (EBPAS), [36], General Self-efficacy (GSE), [37] (clinicians); and Minimum Data Set-Home Care (MDS-HC) [38] (beneficiaries) tools as described in detail in the protocol paper. [21] We also measured clinician and beneficiary characteristics and training completion.
Outcomes
Primary outcome was adoption and sustainability of the intervention measured via the SIC scores. Scoring of the SIC tool (range 0-100) is described in detail in the protocol paper. [21] Secondary outcomes were clinician attitudes toward evidence-based practice and self-efficacy and beneficiary ADLs (sum of 11 ADL items), IADLs (sum of 8 IADL items), pain (sum of 4 self-reported pain items), depression (sum of 3 self-reported mood items), and number of falls, ED visits, and hospitalizations. Intervention fidelity was not collected due to difficulty extracting data from the EHR.
Study procedures
Internal Review Board approvals were obtained, contracts (site, state, and EHR company) were executed, and IFs (each site) and the EF were selected and trained. Data (quality assessment scores, number of beneficiaries) were obtained from the state and sites were randomized. Clinicians were recruited, consented, and baseline data (characteristics, EBPAS, GSE, TCU-ORC) were obtained. Clinicians were trained and the intervention was provided to beneficiaries. We planned to collect the SIC (phone surveys) data monthly for 12 months, however, due to COVID-19, data were not collected in April through August 2020, with the exception of 3 surveys from some of the sites. Clinician EBPAS and GSE were collected (survey) at baseline and 9 months. Beneficiary data prior to and after the intervention were obtained.
Data analysis
Stages of implementation scores were compared between trial arms using t-tests, and effect sizes (Cohen’s d) were estimated as differences between means expressed in standard deviation units. The cut-offs for the interpretation of Cohen’s d are 0.2 (small), 0.5 (medium), 0.8 (large) [39]. Characteristics of clinicians and beneficiaries were summarized by trial arm at baseline. Because of the turnover of clinicians at each site, constrained longitudinal model, with measures at baseline and at 9 months and a constraint of equality of means at baseline due to randomization, was used for the analysis of clinician data. With this analytical technique, data from all clinicians who completed baseline only, 9 months only, or both surveys were used. Random effects were used to account for nesting of clinicians within sites. For all beneficiaries, baseline data were available, and characteristics of beneficiaries without post-intervention data were compared by trial arm to evaluate potential bias due to missing values. Post-intervention data were analyzed in relation to trial arm with the adjustment for baseline version of the outcome, age, sex, and any baseline factors that differed in terms of missing values.