Several sustainability definitions, frameworks, and assessments have emerged recently (11-13, 17-21). Yet, application barriers exist due to the diversity of programs, organizations, and staff (20, 21). In Figure 3, we depict the integration of sustainability tools with our implementation process, determinants, and outcomes frameworks. The effort to identify dynamic, adaptable implementation strategies early in the implementation process using the DSF helped prepare for success despite typical and atypical healthcare setting changes (e.g. staffing changes, transition to tele-health) (9). Although formal sustainability assessments were not completed until year four in this project, repetition of evaluative tools is recommended (e.g. RE-AIM) (22) and using sustainability assessments, (e.g. CSAT & the NHS Sustainability Model) to promote a program’s sustained effectiveness.
Use of existing sustainability frameworks aids in identifying and addressing sustainability barriers. Changing barriers over time could result in “program drift,” or deviation from the program’s original aims (9). The DSF promotes the maintenance of high fidelity interventions through identification of adaptable implementation strategies (9). In this study, a facilitator worked closely with trained, clinical champions at each site who monitored and led program adaptations. The champion meetings, continued adaptations of clinical tools, and electronic resources for PAPT users were sustained through healthcare staffing changes (including champion transitions), as well as less common changes such as moving facilities and a pandemic. At one clinic, the champion had autonomy to create a tracking system to address a process sustainability barrier. At the other site, the champion trained an additional therapist and educated the clinical-team due to staff sustainability barriers. Use of evaluative sustainment frameworks and assessments enabled the appraisal of ongoing implementation strategies and further tailor to site-specific needs. After administering the CSAT & the NHS Sustainability Model, the facilitator could assist the sites to identify adaptations to meet their site-specific sustainability barriers.
This program evaluation uniquely focuses on sustainability of a clinical program targeting an outpatient physical rehabilitation setting. In two recent reviews on program sustainability, none occurred in similar departments (20, 23). However, in one quasi-experimental study, behavior change strategies were implemented in a PT clinic with good uptake during the intervention, but poor sustainment at the three, six, or twelve-month follow up (24). Our sustained implementation strategies using the EMR, centralizing education, and programmatic support of a facilitator and champions have helped this program to be successful for more than four years.
Following initial implementation, demand from stakeholders, including referrers, patients, therapists, and managers, led to the spread of PAPT through our regional system of care. PAPT program growth was associated with minor population changes in the PAPT program (Table 2), and increased PT referrals from the initial referrers, which may help to address the PT referral and utilization gap noted in the health services literature (2). Adoption at new sites was accompanied by site-specific processes and implementation strategies that have been essential for the success in unique contexts.
Unintentional spread also occurred through adoption of the proactive, consultative care model by speech and occupational therapy for people with PD, as well as for other patient diagnoses, like Huntington’s disease. Data from these clinical areas were excluded from this analysis, but are well supported in the literature and bolstered PAPT sustainability (25-28). Organization leaders were empowered to lead these new programs by adapting the implementation strategies associated with the success of PAPT.
Two limitations include the lack of generalizability of retrospective quality improvement data and lack of well-studied sustainability measures. The use of retrospective measurement in a single health system limits generalizability to other organizations. However, documenting strategies to spread from academic to suburban clinics in the same health system may be impactful in similar contexts. The retrospective methods also present challenges such as inability to control potential confounding factors, including new, leadership positions within the organization. The second limitation is that the existing clinical sustainability measures have limited reliability and validity data (12, 13). Implementation practitioners should consider using new sustainability tools as they are validated; however the varied contexts of sustainability between communities and clinical settings may require measurement adaptations (21). Despite their limitations, our clinical and research team felt that the chosen assessment tools provided valuable information for program sustainment.
Demonstrating the application of sustainability frameworks has implications for implementation practice and research. Figure 3 summarizes the process for integrating sustainability principles during implementation. Key features are identifying and addressing sustainment early, repeating implementation evaluations over time, and completing sustainability assessments. More research can improve our understanding of the complexities of sustainability. Systematic implementation research studies should measure the benefits of assessing and addressing sustainability at different implementation stages.