A theory-informed, rapid cycle approach to identifying and adapting sustainability strategies: Optimizing depression treatment in primary care clinics seeking to sustain collaborative/integrated care (The Transform DepCare Study)”

doi:10.21203/rs.3.rs-1605401/v1

Download PDF

Methodology

A theory-informed, rapid cycle approach to identifying and adapting sustainability strategies: Optimizing depression treatment in primary care clinics seeking to sustain collaborative/integrated care (The Transform DepCare Study)”

https://doi.org/10.21203/rs.3.rs-1605401/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 25 Jan, 2023

Read the published version in Implementation Science Communications →

You are reading this latest preprint version

Background: Few if any examples exist of how best to select and adapt sustainability strategies. We used a collaborative/integrated care use case to describe a novel, theory-informed, stakeholder engaged process for operationalizing sustainability strategies.

Methods: Informed by the Behavior Change Wheel, we used mixed methods to identify key sustainability behaviors and determinants of sustainability before conducting a “sustainability mapping” phase to identify acceptable and equitable intervention functions and behavioral change techniques. Drawing on user centered design principles, we enlisted 22 national and local stakeholders to operationalize and adapt (e.g., content, functionality, workflow) a multi-level sustainability strategy to maximally target behavioral determinants of sustainability.

Results: We identified ongoing provider treatment optimization and patient initiation as key sustainability behaviors in settings receiving external facilitation and fiscal implementation strategies. The most acceptable, equitable and feasible intervention functions included environmental restructuring, education, training, modeling, persuasion, and enablement. We determined that a waiting room delivered shared decision making and psychoeducation patient tool (DepCare) the results of which are delivered to providers as well as ongoing problem-solving/implementation team meetings with care managers would be the most acceptable and equitable multi-level strategy in diverse settings seeking to sustain CC programs. Key adaptations in response to contextual factors included expanding the DepCare tool to incorporate anxiety/suicide screening, triage support, multi-modal delivery, and patient activation (vs. shared decision making); pairing preference reports with decisional support as well as yearly onboarding/motivational provider and staff educational videos; shifting problem solving meetings from billing support to triage/reducing inappropriate referrals.

Conclusion: We provide a roadmap for designing theory-informed, sustainability strategies and employing user centered design principles to adapt strategies to changing mental health landscapes.

This study provides a novel roadmap for leveraging behavior and implementation science as well as user centered design principles to design and rapidly adapt theory-informed sustainability strategies.
In addition to external support, sustaining collaborative care may require ongoing patient activation, provider decisional support/motivation, and local implementation support.
User centered design principles can be used to operationalize and refine sustainability strategies.

Nearly 100 randomized trials demonstrate that collaborative care (CC), a team-based approach to managing depression in primary care, improves access to mental healthcare, improves depression, quality of life and productivity, and is cost-effective compared to usual care by primary care providers (PCPs) alone. ^1–3 CC is also one of the few evidence-based interventions shown to reduce health disparities,⁴ with minorities experiencing better improvements in depressive symptoms, physical and/or mental functioning and unmet treatment needs⁵ as well as receipt of preferred depression treatment;⁶ and reduced perceived racial discrimination.⁷

CC expanded exponentially across the U.S., facilitated by Patient Centered Medical Home initiatives, Accountable Care Organizations, and new reimbursement approaches as part of primary care redesign⁸ and through the Centers for Medicare and Medicaid Services.⁹ Several recent initiatives suggest that implementation strategies such as providing flexible funding for staff, training, monitoring, quality improvement and external facilitation improve CC implementation.^10–13 Unutzer et al (2020) recently found that external support is essential to CC implementation and real-world effectiveness.¹⁴ However, few studies focus on how best to identify strategies for CC sustainability, which has been defined as “the continued use of program components and activities for the continued achievement of desirable program and population outcomes.”¹⁵

In 2015, building upon years of experience implementing CC for depression, New York State (NYS) Department of Health (DOH) and NYS Office of Mental Health (OMH) successfully implemented CC in diverse, low socioeconomic status settings using a strategy consisting of technical assistance, fee-for-quality reimbursement, external facilitation, and ongoing training and monitoring; the program now involves more than 300 primary care clinics across the state.¹⁶ In a mixed-methods study of 32 clinics more than 4–6 years into the sustainability phase, we demonstrated that having a full-time depression care manager dedicated to CC and early success (i.e., average improvements in depressive symptoms of 50% in the initial 2 years of implementation) determined whether a clinic would sustain CC or opt out.¹⁷ We found that even with external technical assistance and financial resources, clinics seeking to sustain CC often encountered low patient, staff (e.g., depression care managers) and PCP engagement as well as limited resources (e.g., # of depression care managers), perhaps due to implementation drift (e.g., decay in fidelity to depression screening) and turnover of program champions over time.¹⁷

The NYS findings suggest that strategies designed for early implementation may not be sufficient for sustainability, particularly in diverse settings with the most to gain from the model. There is recent focus on sustainability as a dynamic concept, recognizing that the implementation of evidence-based interventions and strategies will likely evolve over time, due to complex and changing real-world healthcare settings and systems.^18,19 The Dynamic Sustainability Framework (DSF) argues for the continuous refinement and improvement of interventions and implementations during the sustainability phase, through learning and evaluation, problem solving and ongoing adaptations to enhance fit to patients, PCPs, healthcare settings and broader contexts.¹⁹ Although experts have called for the scientific assessment and operationalization of sustainability, few real-world examples of how to best operationalize this process exist.

The NYS DOH/OMH initiative for implementing CC for depression provides a unique, rare opportunity to identify how best to develop and test strategies for sustainability in real world settings. This paper provides a roadmap for designing adaptable, theory-informed strategies for sustainability, drawing from the Behavioral Change Wheel (BCW),²⁰ Expert Recommendations for Implementation Change (ERIC) project,²¹ and user centered design principles^22–24 to design acceptable and equitable strategies in diverse settings. Dynamic and adaptable sustainability frameworks are particularly needed in the post-Coronavirus Disease 2019 (COVID-19) era of rapidly evolving primary care and behavioral health delivery systems, which will increasingly be delivered remotely. To our knowledge, this is one of the first studies to describe the design of a multi-level, theory-informed, adaptable, sustainability strategy for settings seeking to sustain integrated/collaborative care programs.

Overview

Our development of implementation strategies to enhance CC sustainability consisted of the following phases: (1) a mixed methods “sustainability mapping” phase informed by the BCW Framework and ERIC strategies; and (2) rapid cycle adaption phase drawing from user centered design principles,^22–24 involving multidisciplinary stakeholders to operationalize and refine multi-level sustainability strategy and maximize fit and effectiveness (Fig. 1).

Theoretical frameworks used to develop CC sustainability strategies

For Phase 1, we used the BCW framework (Additional File 1) based on our previous findings that CC sustainability hinges on sustained PCP and patient behavior change.²⁵ The BCW Framework can be applied to both individual behavior (e.g., patient-level smoking cessation) and multi-level (system, provider, patient) behavior in clinical settings.²⁰ The multi-step BCW framework first requires identifying and specifying a primary behavior (Additional File 1). The underpinning of this framework is that changing behavior requires increasing capability, opportunity, and/or motivation for a behavior (COM-B Model) by removing barriers and/or augmenting facilitators to that target behavior. The BCW framework involves: 1) understanding the behavior; 2) identifying intervention options; and 3) identifying related content and implementation options, using acceptability, practicality, effectiveness, side effects, and equity considerations to guide selection of implementation strategy component.²⁰ The BCW has been used extensively to develop implementation strategies.^26,27 For Phase 2, we aimed to maximize the fit between CC, our sustainability strategies and context using rapid cycle adaptation of the strategy itself. To operationalize our multi-component sustainability strategy, we leveraged user centered design principles, an approach to product development that grounds the process in data collected from end users at the individual and settings level; this process draws from a clear identification of the end users and their needs, prototyping/rapid iteration, simplification of existing procedures and exploiting natural constraints.^22,24,28 Here, we describe iterations made by the advisory board, intervention development team, and creative team and will describe patient level usability testing, cognitive interviews, and heuristics evaluation separately (Fig. 2).^29,30

Setting

We developed the multi-component sustainability strategy in a large academic primary care clinic in the Ambulatory Care Network (ACN) at New York Presbyterian Hospital in upper Manhattan serving low socioeconomic and racially/ethnically diverse communities of Washington Heights, Inwood, Harlem, and the Southwest Bronx. Implementation is currently occurring in separate demographically similar ACN clinics in the sustainability phase of the NYS OMH CC Initiative (i.e., receiving external technical assistance/facilitation/audit and feedback as well as fee-for-quality reimbursement for Medicaid patients) as well as integrated care settings not receiving external implementation/sustainability strategies.

Phase 1: Understanding determinants of CC sustainability, and identifying sustainability strategies

Understanding determinants of CC sustainability.

Overall, our work was informed by three previously published mixed-methods studies on sustainability of CC, including (1) 30 semi-structured interviews with PCPs, care managers, psychiatrists and administrators at clinics that sustained or opted out of CC across in NYS¹⁷; (2) three focus groups and four one-on-one interviews with racial/ethnic minority patients (n = 12) referred to CC programs in the sustainability phase (both enrollees and no-shows)³¹; and (3) 10 interviews with CC experts from diverse settings across the U.S. focused on determinants of patient engagement in mature (i.e., at least one year after implementation initiation) CC programs.³² First, we defined the problem in behavioral terms and selected and specified target behaviors at every level (patient, provider, and system). Drawing from our previously published qualitative interviews with key stakeholders on barriers to CC sustainability (above),^17,31,32 two authors (NM, MAPA) classified themes by level (patient-, provider- or system-level) and COM-B construct. For example, we deemed the ‘lack of PCP time and competing demands’ theme to be a provider-level “opportunity” barrier. There were no major disagreements. We resolved any differences by consensus discussion, with researchers referring to original manuscripts to reassess context of the codes.

Identifying intervention functions, behavior change techniques, and sustainability strategies.

After coding the COM-B categories, our research team (NM, MAPA) followed the BCW to map the above COM-B barriers to corresponding intervention functions (i.e., nine broad categories by which an intervention can change behavior, e.g., education, training, persuasion), policy categories (i.e., seven policies representing types of decisions made by authorities that help to support and enact interventions), and behavior change techniques (i.e., a standardized language for describing the active ingredients in behavior change interventions via which intervention functions and policy categories are delivered (Additional File 2). As several interventions, policies and behavior change techniques may map to each COM-B construct, the BCW framework recommends applying the APEASE criteria to narrow-down intervention components to those that are affordable (within an acceptable budget for all), practical (can be delivered as designed), efficacious (long-term effectiveness and cost-effective related to designed objectives in real world context), acceptable (judged appropriate by relevant stakeholders), safe (no unwanted side effects), and equitable (does not increase disparities).²⁰ We adapted this process for sustainability (e.g., does not worsen long-term disparities, would result in long-term effectiveness). For example, enablement at the patient-level (which includes social/peer support) was an intervention function that would be acceptable, effective and promote equity, but not practical to organize or affordable in the long-run. Only those intervention functions, policy categories, behavior change techniques and modes of delivery that met all APEASE criteria long-term and were agreed upon by both coders were selected. Differences were resolved through consensus discussion.

To guide our mapping process, we used our previously published systematic review of interventions shown to improve depression treatment initiation in primary care settings: we found that patient-preference matching, and a combination of patient activation/motivation were particularly efficacious.³³ Given calls for shared language and conceptual clarity around implementation strategies, we then mapped behavior change techniques to implementation strategies from the ERIC project. ^21,34 We also specified the dose, mode of delivery and targeted implementation outcome of each strategy component.³⁵

Phase 2: Rapid cycle adaptation of the sustainability strategy

After initial implementation mapping described above, we conducted sequential group and one-on-one meetings with multidisciplinary teams of stakeholders (Fig. 2) from January, 2018 to May, 2021: (1) a creative team led by our creative director of developers and experts in user experience who used results from Phase 1 (above) to map system, patient and provider experiences and operationalize/develop patient and PCP facing materials; (2) an intervention development team of racial/ethnic minority patient stakeholders, care managers, and PCP stakeholders who ensured strategies fit to rapidly shifting outer and inner contexts in diverse, low socioeconomic settings; (3) an advisory board of experts in behavior change and patient activation/experience to refine the strategy and finally (4) depression care managers, PCPs and racial/ethnic minority patients for user testing patient-facing interventions (the protocol for which will be described separately) to maximize the usability, safety, feasibility, and sustainability of our multi-component strategy. For intervention team stakeholders, we used snowballing techniques to identify CC champions in the ACN clinics. We identified advisory board members from literature reviews on behavioral, user experience and shared decision-making experts. Stakeholders were invited by email to serve in an advisory role for our study. Stakeholders attended meetings as a group with one-on-one follow-ups as needed to clarify recommendations. Patients and providers who underwent user-testing were consented and the protocol was approved by the Columbia University Irving Medical Center Institutional Review Board.

The core team of staff and implementation science researchers recorded and transcribed meeting notes, identified themes and arrived at consensus to guide strategy prototype refinements when technically and logistically feasible. We aimed to leverage user-centered design principles that crosscut ERIC implementation strategies (i.e., co-design solutions, understand systems and context, consider user needs and experiences, develop and test solutions rapidly)^{22–24, 28} to operationalize the multi-component sustainability strategy. We rapidly reviewed meeting notes for usability themes (content, usability, usefulness, understand-ability/functionality, visibility, workflow, navigation, content)^36–38 related to maximally targeting multi-level barriers to sustainability (i.e., motivation, opportunity [i.e., outer, inner setting] and/or capability). We also tracked contextual factors and events related to the CC program itself based on input from key stakeholders during meetings. The creative team met after each cycle to refine educational materials before presented prototypes to the advisory board and intervention team members for feedback, consensus, and adaptation. We conducted three iterative adaptation cycles. At the end of the last cycle, we reviewed and refined the mapping process to ensure the mode of delivery/behavior change techniques fit the contextual considerations of the telemedicine/post-COVID-19 era (e.g., we expanded the mode of delivery via iPad in waiting rooms to include home delivery via personal phone, computer, or tablet). An external expert trained in the BCW framework with no prior knowledge of our intervention then coded our final multi-level sustainability strategy components (e.g., shared decision-making tool, PCP marketing videos) for behavior change techniques to ensure that behavioral components determined in Phase 1 were well represented in our final strategy.

In our final meetings, we administered the brief feasibility and appropriateness of intervention measures³⁹ informed by Proctor’s implementation outcomes to ensure the feasibility and appropriateness/fit of our multi-component sustainability strategy. We used descriptive analyses of our quantitative implementation outcomes to report percentages for categorical variables and means with 95% confidence intervals for continuous variables.

Understanding Sustainability Behaviors

Informed by our previous qualitative methods with stakeholders across NYS as well as U.S. experts,^17,31,32 we identified sustained engagement in CC by care managers/psychiatrists (system-level), depression treatment optimization/referrals (provider-level) and treatment initiation/persistence (patient-level) as essential behaviors for collaborative/integrated care sustainability in racially/ethnically/socioeconomically diverse settings (Additional File 3). We combined system and provider level constructs given marked overlap in interviews and themes. Given how essential patient enrollment in programs was to reimbursement and thus program sustainability, continued patient enrollment was identified as a “primary sustainability behavior”.

The determinants of sustainability behaviors are presented in Table 1. Key capability constructs at the patient-level were lack of awareness of depression treatments while providers/systems faced lack of CC knowledge/training and error-prone referral processes (e.g., inappropriate referral of patients with serious mental illness were ineligible for CC). From an opportunity perspective, competing initiatives, limited resources/complex psychosocial needs, complex workflows (e.g., for warm handoffs) as well as PCP and care manager time/schedules/workloads were key provider/system level barriers, while stigma as well as accessibility, convenience, and quality of mental care were key patient level barriers. Key provider/system level motivation barriers related to ongoing engagement by PCPs, lack of PCP-care manager teamwork/communication and infeasible warm handoffs in the sustainability phase. Stakeholders felt that NYS’ external implementation strategy of technical assistance and audit/feedback ensured high care manager/psychiatrist fidelity to the program itself but noted high care manager and psychiatrist turnover in the sustainability phase. Other motivation related themes included patient fear of treatment side effects and concerns around treatment efficacy (e.g., due to prior treatment failure). Experts also noted unaddressed patient-level concerns and the need for tailoring (e.g., for Spanish speaking participants).

Table 1

Capability, Opportunity, and Motivation Constructs of Behavior (COM-B) at the Provider/System and Patient Levels
COM-B Construct	Provider/System Level Themes	Patient Level Themes
Psychological Capability	PCP Training/ Knowledge -Lack of knowledge about CC Error prone referral processes	Patient education about treatment options Patient’s lack of awareness of their own mental health
Social Opportunity	External Environment -Competing primary care initiatives -Restrictive enrollment requirements -Inadequate resources and complex psychosocial needs	Stigma about mental health
Physical Opportunity	Funding -Complex funding streams -Insufficient funding Information Technology / Infrastructure -Antiquated data management/ information technology infrastructure PCP Time / Resources / Personnel -Time constraints on PCPs -Competing PCP demands -Inflexible PCP schedule -High PCP workload limits depression diagnosis and treatment Complex Workflows -Infeasible warm handoffs in sustainability phase (also motivation barrier) -Complicated screening, referral, and triaging (also capability, motivation barrier) DCM Time / Resources /Personnel -Competing DCM roles, insufficient DCMs/personnel -Inadequate space -Inflexible mental health workers’ schedule	Treatment Accessibility & convenience Providers’ expertise & quality of mental care
Reflective Motivation	Provider engagement -Lack of primary care physician pro-activeness -Poor continuity of care -Poor psychiatrist engagement -Lack of DCM engagement -Inadequate teamwork/communication	Patients’ beliefs about treatment being ineffective Unaddressed patient needs and preferences Patient engagement/self-efficacy -Depression treatment stigma -Patient non-adherence -Limited language/literacy/cultural beliefs of patients -Infeasible warm handoffs (also opportunity barrier)
	Workflow logistics -Complicated screening, referral, and triaging (also opportunity, capability barriers)
	Beliefs about Consequences -Inadequate PCP buy-in -PCP Concern for validity of measures
Automatic Motivation		Patients’ fear of treatment
Abbreviations: CC Collaborative Care; DCM Depression Care Manager; PCP PCPs

Identifying sustainability options/strategies

Tables 2 and 3 describe the operationalized multi-component sustainability strategy with corresponding behavior change techniques, targeted COM-B constructs, intervention functions, ERIC strategies, final mode of delivery (see adaptation Phase 2 section below) and implementation outcomes targeted⁴⁰ (e.g., CC fidelity, acceptability, sustainability) at the patient and provider/system levels. We determined that NYS OMH’s ongoing implementation and scale up strategies (i.e., technical assistance, audit and feedback, reimbursement) targeted CC uptake, quality/fidelity, acceptability, feasibility, and costs, but that sustainability would require strategies that promoted ongoing multi-level behavior change. Key intervention functions for promoting ongoing behavior change that met all APEASE criteria included environmental restructuring, education, training, modeling, persuasion, education and enablement.

Table 2

Operationalizing a multi-component sustainability strategy for promoting collaborative care enrollment at the patient-level: behavioral change techniques, intervention functions, mode of delivery as well as targeted behavioral constructs and implementation outcomes.
Actor	Behavioral Change Techniques	Intervention function	COMB Construct					ERIC strategies	Strategy	Mode of delivery, Dose, and Implementation Outcome targeted
Actor	Behavioral Change Techniques		Psych Capability	Social Opportunity	Physical Opportunity	Reflective Motivation	Automatic Motivation	ERIC strategies	Strategy
Patient	Adding objects to environment: add objects to the environment to facilitate performance of the behavior	Environmental Restructuring Enablement	X	X	X		X	Change physical structure and equipment	eSDM tool for patient activation, psychoeducation, and referral with flexible modality (takes place of warm handoff) Provide therapy by phone/online/video/self-help Treatment options Spanish language, low literacy tool for Spanish speaking and low literacy groups	iPad, phone, computer, waiting room/ home, Mailer Once Acceptability/Usability/Fidelity
	Prompts / cues: introduce or define environmental or social stimulus with the purpose of prompting or cueing the behavior. The prompt or cue would normally occur at the time or place of performance	Environmental Restructuring/ Education	X	X	X	X		Change physical structure and equipment	Reminder calls for appointments/Reminder of what they wrote/time tool for prior to PCP visit	Phone/Epic/Brochure Once Fidelity, sustainability
		Environmental Restructuring/ Education	X	X	X	X		Change physical structure and equipment	eSDM prior to PCP visit to facilitate conversation	Tool Once Fidelity
	Demonstration of behavior: provide an observable sample of the performance of the behavior, directly in person or indirectly e.g., via film, pictures, for the person to aspire to or imitate	Persuasion, Training, Modeling	X	X	X	X	X	Model and simulate change	Video with minority patient modeling engagement, going to treatment, content hub with demonstration of engaging in treatment, convenience	Tool (Content hub, how to talk to provider/therapist) Once Acceptability
	Information about social and environmental consequences: provide info (written, verbal, visual) about social and environmental consequences of performing the behavior	Persuasion, Education	X			X	X	Develop educational materials / Distribute educational materials	Spanish and English video with Information about depression, treatment options & CC, motivational messaging	Tool (patient video) Once Acceptability
	Credible source: present verbal or visual communication from a credible source in favor of or against the behavior	Persuasion, Environmental Restructuring		X	X	X	X	Model and simulate change	Care manager/Patient video about treatment	Tool (care manager video) Once Acceptability
	Verbal persuasion about capability: Tell the person that they can successfully perform the wanted behavior, arguing against self-doubts, and asserting that they can and will success	Persuasion, Enablement		X		X			Care manager/Patient video about ability to do treatment and improve	Tool (Patient/Care manager Video) Once Acceptability
	Information about health consequences: provide info (written, verbal, visual) about health consequences of performing the behavior	Education, Persuasion	X			X	X	Develop educational materials / Distribute educational materials	Care manager/Patient video about ability to do treatment; information about health consequences (pain)	Tool (Content Hub)/Brochure Once Acceptability
	Self-monitoring of behavior: establish a method for the person to monitor and record their behavior as part of a behavior change strategy (brochure)	Education, Training, Enablement	X		X	X			Brochure with monitoring of appointment attendance, adherence to meds	Brochure Once Fidelity
Abbreviations: CC Collaborative Care; eSDM electronic shared decision making; PCP PCPs

Table 3

Operationalizing a multi-component sustainability strategy for promoting collaborative care enrollment at the provider/system-level: behavioral change techniques, intervention functions, mode of delivery as well as targeted behavioral constructs and implementation outcomes.
PCP/System	Behavioral Change Techniques	Intervention function	COMB Construct					ERIC strategies	Strategy	Final Mode of delivery, Dose Implementation Outcome targeted
	Instruction on how to perform behavior: advise or agree on how to perform the behavior (includes skills training)	Training	X		X		X	Conduct ongoing training / Develop educational materials / Distribute educational materials	Educational Session about ordering/referring	Onboarding PCP video Once Acceptability, fidelity
	Demonstration of behavior: provide an observable sample of the performance of the behavior, directly in person or indirectly e.g., via film, pictures, for the person to aspire to or imitate	Training, Modeling	X	X			X	Model and simulate change	Shows how to refer patients for mental health (demonstrates example of ideal note)	Onboarding PCP video Once Acceptability, Fidelity
	Information about social and environmental consequences: provide info (written, verbal, visual) about social and environmental consequences of performing the behavior	Education, Persuasion	X			X	X	Develop educational materials / Distribute educational materials	Describes social/environmental effects of referral (physician burnout, resources, wait times due to inappropriate referral)	Onboarding PCP video Once Acceptability, Fidelity
	Information about health consequences: provide information (e.g., written, verbal, visual) about health consequences of performing the behavior	Education, Persuasion	X			X	X	Develop educational materials / Distribute educational materials	Provides information on health consequences among referred patients	Onboarding PCP video Once Acceptability, Fidelity
	Prompts / cues: introduce or define environmental or social stimulus with the purpose of prompting or cueing the behavior. The prompt or cue would normally occur at the time or place of performance	Environmental Restructuring, education	X	X	X	X	X	Remind clinicians	Summary report/decisional support to cue referral (delivered to providers/care managers) and Job aid on mental health management	EPIC/Email to PCP Once Acceptability, Fidelity, Feasibility, Sustainability
	Adding objects to environment: change or advise to change the physical environment to facilitate performance of the wanted behavior or create barriers to the unwanted behavior (other than prompts/cue, rewards, and punishments)	Environmental Restructuring, enablement	X	X	X	X		Change physical structure and equipment	Smart phrase/ eSDM tool (providers/care manager/clinic)	Tool/EPIC to PCP Every encounter Fidelity, Feasibility, Sustainability
	Action planning: prompt detailed planning of performance of the behavior (must include at least one of context, frequency, duration, and intensity). Context may be environmental (physical or social) or internal (physical, emotional, or cognitive)	Enablement	X	X	X		X	Facilitation/implementation meetings/ongoing training/ Conduct educational meetings	Ongoing Facilitation with care managers around key determinants of successful CC sustainability (e.g., how to identify and engage PCP champion)	Onboarding PCP video/ Zoom ID team meetings/ newsletter Once/Quarterly Fidelity, Sustainability
	Feedback on outcome(s) of the behavior: monitor and provide feedback on the outcome of performance of the behavior	Education, Persuasion, Training	X		X	X	X	Audit and provide feedback	Feedback on screening, appropriate referral, attendance rates frequency, intensity, duration (clinic/care managers/Providers)	Zoom ID team meetings/newsletter and via Care manager messages in EPIC Quarterly Acceptability, Feasibility, Fidelity, Sustainability
	Restructuring physical environment: change or advise to change the physical environment to facilitate performance of the wanted behavior or create barriers to the unwanted behavior	Environmental restructuring, Enablement	X	X	X		X	Change physical structure and equipment	Triage patients based on eSDM tool input Screening support and education Billing support Access to culturally tailored, high-quality CC	Zoom ID team meeting Phone Fidelity, Feasibility, acceptability, Sustainability
	Problem solving, analyze, or prompt the person to analyze, factors influencing the behavior and generate or select strategies that include overcoming barriers and/or increasing facilitators	Enablement	X	X	X		X	Facilitation/implementation meetings/ongoing training/ Conduct educational meetings	Review CC and DepCare implementation, brainstorm how to improve every 2–3 months	Zoom ID team meetings/newsletter Quarterly Fidelity, Sustainability
	Feedback on behavior: monitor and provide informative or evaluative feedback on performance of the behavior (e.g., form, frequency, duration, intensity)	Education, Persuasion, Training	X		X	X	X	Audit and provide feedback	Feedback on screening, appropriate referral, attendance rates frequency, intensity, duration	Zoom ID team meetings/newsletter Quarterly Acceptability, fidelity, feasibility, Sustainability
	Credible source: present verbal or visual communication from a credible source in favor of or against the behavior	Persuasion, Environmental Restructuring		X	X	X	X	Inform local opinion leaders	Engage clinic directors/opinion leaders to provide feedback	Zoom ID team meetings/newsletter Quarterly Acceptability, fidelity, feasibility
Abbreviations: CC Collaborative Care; EPIC Epic Systems Corporation electronic health record; eSDM electronic shared decision making; PCP PCPs; ID Intervention Development

Preliminarily, we determined that the most feasible multi-component strategy (with corresponding behavioral change techniques and ERIC strategies) for promoting sustainable behavior change would center around a patient-level video-assisted electronic shared decision making (DepCare) tool to provide culturally targeted psychoeducation, motivational messaging from care managers and patients with lived experiences, and treatment preference matching/automated shared decision making all delivered in the waiting room for every patient with elevated depressive symptoms prior to primary care visits to ensure equitable access to technology (prompts/cues, information about consequences, credible source, verbal persuasion, and restructuring the environment; develop/distribute educational materials, Change physical structure and equipment). A web application that could be delivered to patients who screened positive by medical assistants was considered the most feasible, acceptable, and sustainable mode of delivery (Table 2). We were not able to achieve consensus for APEASE criteria for several behavior change techniques. For example, it was not deemed feasible to provide feedback on behavior (whether patients initiated/optimized treatment) and feedback on outcomes of behavior (i.e., distress levels) particularly as it related to practicality, unintended side effects (e.g., shame) and equity (might disproportionately shame patients facing more structural barriers to treatment).

At the provider/system-level, an efficacious, preliminary sustainable strategy would involve yearly mental health/CC general medicine grand rounds as well as automatically delivered DepCare tool summary reports to PCPs/care managers on treatment preferences/barriers at the time of a visit to support patient-provider communication, triage, and referrals (prompts/cues and information about consequences; ongoing training, remind clinicians, audit and provide feedback). The strategy would also involve local (as opposed to external NYS delivered) technical assistance/implementation meetings with PCPs and care managers to discuss contextual factors key to CC sustainability (e.g., quality improvement, billing education/support, long wait times) (problem solving/action planning; implementation meetings, facilitation). Several provider/system level behavioral change techniques, such as self-monitoring of referral behaviors, did not meet any of the APEASE criteria. Initial concerns by coders about acceptability of action planning and problem solving were resolved by imbedding these into quarterly system-level implementation team meetings with mental health staff (vs. at the PCP only level) as well.

Rapid Cycle Adaptations to Fit context

Stakeholder characteristics are described in Additional File 4. These included our creative team (n = 4), intervention development team (n = 7) and advisory board (n = 11) who attended team member meetings to revise the multi-level sustainability strategy. In Additional File 5, we describe outer/inner contextual factors impact depression screening, treatment, collaborative care programs and adaptations to the multi-level strategy by the creative team, intervention development team and advisory board (i.e., stakeholders) categorized by usability/workflow themes and COM-B constructs targeted.

Contextual Factors.

Key outer and inner contextual factors that emerged during team meetings included competing initiatives (e.g., iPad delivered social determinants of health screening in the waiting rooms), transition to EPIC Systems Corporation electronic health record, care manager turnover, suicide screening mandates, and expanded CC reimbursement for anxiety. During COVID19, CC transitioned to remote delivery and eventually a hybrid delivery model, which improved referrals and show rates but increased inappropriate referrals: “[Primary Care] Providers using CC as walk-in mental health clinic…Not their purpose and not everyone is guaranteed treatment…Providers too comfortable sending all patients to CC, not asking about depression, treatment in assessments resulting in inappropriate referrals.” In 2021, depression screening became imbedded in the pre-visit “e-check in” processes as well as an Accountable Care Organization metric of interest.

Patient-level strategy adaptation.

The initial DepCare prototype consisted of depression screening (with the validated Patient Health Questionnaire (PHQ)-9), a depression “score report” (describing the meaning of their score), a treatment preference checklist (e.g., in-person vs. video, therapy vs. medication), a barriers to treatment checklist (e.g., time/convenience, stigma), a care manager motivational video, a patient video, assessment of patient’s interest in seeing a care manager and starting a medication, and suggestions for communicating with PCPs. Details of the iterative user-centered design process involving cognitive interviews and user testing of the DepCare prototype with patients and physicians will be described separately.

Content/Usefulness: Based on initial review of the DepCare prototype, stakeholders recommended further personalizing the experience based on patients’ history of depression and prior lived experiences with depression treatment (e.g., recommendations for augmenting treatment and speaking to one’s therapist, directing patients with mild symptoms to self-care) to maximally address capability, motivation, and opportunity constructs. Relatedly, one patient stakeholder in the intervention development team also suggested describing atypical symptoms, such as anger (opportunity). In iterative adaptations, we added a “content hub” that patients could navigate on their own at the end of the tool (e.g., with additional information about depression and treatment, connection to free resources/hotlines, and internet-based cognitive behavioral therapy) to personalize psychoeducation (capability) and expand treatment options (opportunity/outer setting). Over time, focus transitioned from decisional support to activating patients to meet with care managers to address high CC no-show rates (motivation) and finally goal setting, personalization, and treatment optimization (capability, motivation, opportunity) to address inappropriate referrals. One stakeholder remarked: "Messaging should include language about -- things can be different. You are taking a step to do something today. Even though this is scary, this could be the start of getting your life back. We're happy to be partners in your care”. Finally, in response to contextual factors related to NYS OMH program reimbursement for anxiety, stakeholders suggested we also include anxiety and suicide screening in the tool to further align with and support clinic initiatives (opportunity/inner and outer setting).

Functionality/Workflow

Telemedicine/COVID19 related strategy adaptations included deploying the tool not just in the waiting room but via patient portals and personal devices prior to primary care visits (opportunity/inner setting). Stakeholders also recommended flexible delivery based on a clinics’ screening rates (e.g., options to bypass PHQ questions in those patients already screened by the healthcare system) (opportunity/inner setting). We also included voicers to address literacy (capability). Paper versions/PDFs of the entire tool were created to address digital literacy (capability, opportunity).

Provider/System level strategy adaptation.

The preliminary strategy included a PCP preference report, PCP education and ongoing quality improvement/problem solving staff meetings.

Content/Usefulness

To address shifts in contextual barriers (e.g., from addressing high no show rates to optimizing triage/reducing inappropriate referrals), the PCP educational strategy focused on treatment optimization and maintenance (i.e., augmenting antidepressants by providers, referring more severe patients inappropriate for CC to psychology/psychiatry/long-term treatment options) (opportunity/inner setting). We expanded the ongoing quality improvement/problem-solving meetings across the institution so that behavioral health clinicians could learn from each other (both integrated and collaborative care settings) and address barriers specific to our healthcare system not addressed with external technical assistance. Care managers initially reported sustainability barriers related to billing codes, which improved over time while issues related to triage and identifying physician champions became more salient (opportunity/inner and outer setting, capability). To support system-wide interest in meeting depression screening metrics for the Accountable Care Organization, we also added educational and marketing videos (for medical assistants and staff) around valid, equity-informed depression screening in the telemedicine era (capability).

Functionality/Workflow

We adapted provider education modalities from yearly mental health grand rounds on CC (capability) to multidisciplinary Zoom meetings and more persuasive marketing and educational onboarding videos around both mental health treatment and the DepCare tool, which could be widely disseminated and automatically delivered yearly to ensure sustainability (opportunity, motivation, capability). Stakeholders noted that busy providers and patients may opt to focus on physical symptoms and deprioritize mental health during a given visit, recommending options for bypassing providers with the tool (opportunity/inner setting, motivation, opportunity). This was eventually deemed infeasible to busy care managers who agreed to instead receive preference reports to better prepare for visits and avoid inappropriate referrals. Instead, we opted for both care managers and PCPs to receive preference reports. Stakeholders also recommended preference reports include pictures, pictograms, and colors to expedite clinical decision-making and appropriate referrals (capability). We further created an EPIC Systems Corporation electronic health record imbedded “smart phrase” (or template with checklists for treatment/referral options) for additional treatment decisional support after the system transitioned to EPIC.

Re-mapping behavior change techniques

An external BCW expert reviewed all final adapted materials. The combination of the brochure and DepCare tool appropriately represented all the initially mapped behavior change techniques, except for restructuring of the physical environment, which we reconsidered a system level strategy. The external BCW expert mapped all behavioral change techniques to the provider marketing video, except for demonstration of the behavior and add objects to the environment. We refined the provider video to better demonstrate how a provider would refer to treatment and use the “smart phrase” and strengthened descriptions of the preference report and EPIC Systems Corporation [i.e., electronic health record] smart phrase as additions to the environment. The expert appropriately identified most behavioral change techniques related to the quarterly implementation team meetings, except for feedback on the behavior, action planning and credible source. During meetings we differentiated behaviors (e.g., screening/referrals) from outcomes of behaviors (e.g., clinic-level depression symptom burden) and engaged care managers/clinic administrators to lead meetings and send newsletters as credible sources. If proven effective in our ongoing trial, links to all materials will be made widely available.

Final sustainability strategy

The final multi-component sustainability strategy is presented in Table 4. The patient-level centers around the DepCare tool (IR CU19184), which includes enhanced depression and anxiety screening, diagnosis recognition support, patient activation, personalized psychoeducation, videos promoting patient engagement in treatment, and personalized medication selection support. The provider-level strategy includes an educational and motivational video on CC and optimal management of depression and comorbid anxiety, invitations to quality improvement/implementation team meetings and automatically generated DepCare tool decisional support on individual patient treatment preferences delivered to both the provider and care managers. The clinic level strategy includes quality improvement support and education around valid depression screening as well as local technical support/problem solving for mental health treatment optimization. In the last advisory board meeting (n = 4) prior to launch, the mean appropriateness of the intervention based on the validated scale was 4.56 and the mean feasibility was 4.36.

Table 4

Final Multi-Level Multi-component DepCare Sustainability Strategy
	Experimental Arm	Enhanced Usual Care Arm
Clinic	(1) Support and education around valid depression and anxiety screening (2) Local Technical assistance for collaborative care program	(1) Support and education around valid depression and anxiety screening (2) Local Technical assistance for collaborative care program
Provider	(3) One-time presentation or video with education and motivational messaging around collaborative care, functionality of the DepCare patient tool and optimal management of depression and comorbid anxiety (4) 2–4 quality improvement/implementation team meeting per year on optimizing mental health treatment in primary care and DepCare strategy (i.e., multi-level, multi-component intervention) implementation (5) Automatically generated decisional support on individual patient treatment preferences (i.e., for every patient who receives the DepCare patient tool)	(3) Usual Care (social workers are notified of suicidal patients)
Patient	(6) Patient tool comprised of enhanced depression and anxiety screening (includes option for voice-over questions, point-and-click responses), and for those who screen positive for depressive symptoms (with or without comorbid anxiety), diagnosis recognition support, psychoeducation, videos promoting patient engagement in treatment, and personalized medication selection support.	(4) Usual Care (patients are intermittently screened by depression/anxiety screening based on clinic resources or provider indications)

Using CC sustainability as a use case, we provide a roadmap for designing sustainability strategies, which involves identifying key “sustainability behaviors” and their barriers, mapping barriers to behavioral theory-informed sustainability strategies, and finally operationalizing and adapting these strategies using rapid cycle, multi-stakeholder processes informed by user-centered design principles. We found that a multi-level, multi-component strategy centered around an electronic automated shared decision making, triage and referral tool (DepCare), provider education/activation/decision support and ongoing problem-solving meetings would be the most acceptable and equitable strategy for supporting multi-level behaviors around optimizing depression treatment in settings sustaining CC programs in diverse, low socioeconomic status settings. We go on to demonstrate how user centered design principles can be used to operationalize strategies and guide adaptations related to content, usability and workflow to address contextual factors and maximally target behavioral constructs. The results of our ongoing cluster randomized control trial will confirm whether our multi-level sustainability strategy improves treatment optimization rates in settings seeking to sustain CC/integrated care programs (both those with and without external implementation strategy support).

Extensive research on CC implementation concludes that external support is essential for real-world effectiveness.¹⁴ Centralized (policy-level) strategies for CC implementation (i.e., technical assistance, reimbursement, audit and feedback) often focus on care manager behavior and CC quality/fidelity/implementation processes.¹⁶ Our study is based on the premise that sustainability requires ongoing behavioral change, here continued healthcare systems/staff screening, provider referral/optimization and patient treatment initiation/persistence.¹⁶ In fact, patients in the sustainability phase of CC may be those who are particularly treatment resistant. Researchers have long shown that providers often fail to recognize depressive symptoms, have low mental health treatment competency, are ill-equipped to discuss depression, and fail to optimize treatment in depressed patients,^25,41 contributing to suboptimal patient engagement rates.^42–45 Strategies not attuned to multi-level, sustained behavior change (not just of care managers but staff, PCPs and patients) may lose ground in the sustainability phase. Our theory-informed, multi-level approach to targeting behaviors key to sustainability suggests that automated, technology supported psychoeducation, activation, CC program marketing, treatment preference matching (that specifically address contextual factors as opposed to widely available patient facing tools) may be key to ongoing patient activation; in turn ongoing provider-level decisional support/reminders/education to facilitate communication may be key to supporting provider and patient mental health treatment engagement alike in settings seeking to sustain CC programs.

A recent systematic review of sustainability strategies found few studies that employed a conceptual framework to guide in strategy development and identifying key barriers (i.e., limited funding/resources) and facilitators (i.e., need for adaptation/alignment and funding).⁴⁶ We posit that as policy-driven reimbursement for evidence-based interventions improve (as seen in the case of CC), sustainability efforts will need to incorporate frameworks and models centered not just on contextual factors/adaptation but also on sustained behavior change of end users (as it relates to both the intervention and strategy). The BCW framework has been extensively used to design behavioral interventions and develop or map implementation barriers to strategies including ERIC strategies.^34,47,48 For example, Mangurian et al (2017) used the BCW framework to develop a cardiometabolic screening for people with mental health illness and identified key behavior change techniques related to additional resources, social support problem solving and action planning.⁴⁷ To our knowledge, we are amongst the first to apply the BCW framework to develop sustainability strategies by selecting behavior change techniques and modes of delivery with the greatest potential for sustainable, equitable behavioral change. Implementation experts should choose frameworks that fit sustainability needs however (e.g., we chose the BCW framework after identifying behavior as a key determinant of sustainability in settings with external and fiscal support). Experts recommend using sustainability frameworks like DSF which considers how interventions, contexts and the broader ecological system (here NYS OMH’s external support) change over time and the dynamism of healthcare systems as demonstrated in our study.⁴⁹ A recent mixed methods study also found that several elements of the Consolidated Framework for Implementation Research (CFIR) (e.g., ongoing coalitions, networks and partnerships, infrastructure and capacity to support sustainability, community need for programs, ongoing evaluation of performance and outcomes) are integral to sustainability of evidence-based mental health and behavioral interventions.⁵⁰ The results of our ongoing trial will elucidate the impact of our multi-level strategy on key CFIR constructs.

Finally, it is often unclear how investigators operationalize behavior change techniques to arrive at strategies that fit local context. Experts have remarked on the need to consider user centered design for improving the fit between evidence-based practices and implementation contexts and that the two fields can be complementary.²² There are few use cases for integrating the two disciplines, however. We posit that user centered design may also be one mechanism for facilitating ongoing adaptations, refinements and improvements integral to sustainability.¹⁹ We demonstrate here how strategy adaptations are needed to maximally target determinants of behavior change and address outer (e.g., COVID19, rapidly worsening mental health rates, CC program evolving to reimburse to reimburse for CC) and inner (interest in depression screening as a modality of interest, waiting room restrictions) contextual factors. Future research will describe the effect of our rapid cycle, user centered design-informed testing of our DepCare tool on patient usability, acceptability, and barriers to treatment optimization.

There were several limitations to our study. Our use of the BCW Framework to focus on behavioral determinants of sustainability may have failed to elucidate key contextual barriers to sustainability, such as reimbursement and resources. ^46,50 Relatedly, ERIC strategies and the BCW framework have yet to be tested or validated for sustainability. Since study completion, newer frameworks for tracking adaptations of interventions and strategies have also emerged, which may have allowed us to better operationalize our process. For example, the Framework for Reporting Adaptations and Modifications-Expanded for implementation strategies (FRAME-IS) would have allowed us to systematically track processes and reasons for adaptation.⁵¹ While our adaptation process innovatively utilized user centered design procedures and multiple stakeholders to adapt (not just track adaptations), this was an intensive process and may not be replicable. In addition, adaptation of the sustainability strategy occurred in experimental settings with a subset of individuals (vs. real time) which did not allow us to evaluate the effects of strategy adaptation on key constructs of sustainability (other than usability and perceived feasibility).

Our study provides a rigorous, multi-step process for applying behavioral science, implementation science and user center design to select and adapt sustainability strategies to fit dynamic local contexts and interventions. Our strategy is currently being implemented in settings with collaborative/integrated care, during which we will track further adaptations to both the strategy and evidence-based intervention and assess effects of our multiple level “sustainability” strategy on sustained behavior change. This study has marked implications to developing, adapting, and testing sustainability strategies in broad range of populations and settings.

ACN - Ambulatory Care Network

BCW - Behavioral Change Wheel

CC - Collaborative Care

COM-B – Capability, Opportunity, Motivation – Behavior Model

CFIR - Consolidated Framework for Implementation Research

COVID-19 - Coronavirus Disease 2019

DSF - Dynamic Sustainability Framework

ERIC - Expert Recommendations for Implementation Change

NYS DOH - New York State Department of Health

NYS OMH - New York State Office of Mental Health

PHQ - Patient Health Questionnaire

Ethics approval and consent to participate. This study does involve human subjects.

Consent for publication. N/A

Availability of Data and Material. The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Competing Interest: The authors declare that they have no competing interests

Funding: This work was supported by funds from the Agency for Health and Research Quality (R01HS025198). AHRQ had no role in the design and conduct of the study, including the collection, management, analysis, interpretation of the data, preparation, review or approval of the manuscript and decision to submit the manuscript for publication.

Author Contributions: NM secured funding, analyzed data, drafted manuscript and supervised the manuscript. APA analyzed data and drafted sections of the manuscript. LB oversaw intervention development and adaptation. JMB oversaw data collection, interpreted data, intervention development and adaptation. KDB analyzed data and provided critical revision of the manuscript. SF collected data. MM contributed to intervention development and adaptation. MS collected and analyzed data. DS collected data. SP provided critical revision of manuscript. SY oversaw collection of data and critical revision of the manuscript. AD supervised data collection, critical revision, and interpretation of data. All authors read and approved the final manuscript.

Financial Disclosures: None

Acknowledgements: None

Unutzer JHH, Schoenbau M, Druss B. The collaborative care model: An approach for integrated physicial and mental health care in meidcad health homes. Health Home Information Resource Center Brief May 2013. http://www.medicaid.gov/State-Resource-Center/Medicaid-State-Technical-Assistance/Health-Homes-Technical-Assistance/Downloads/HH-IRC-Collaborative-5-13.pdf Accessed April 2015.
Thota AB, Sipe TA, Byard GJ, et al. Collaborative care to improve the management of depressive disorders: a community guide systematic review and meta-analysis. Am J Prev Med. 2012;42(5):525–38.
Gilbody S, Whitty P, Grimshaw J, Thomas R. Educational and organizational interventions to improve the management of depression in primary care: a systematic review. JAMA. 2003;289(23):3145–51.
Hu J, Wu T, Damodaran S, Tabb KM, Bauer A, Huang H. The Effectiveness of Collaborative Care on Depression Outcomes for Racial/Ethnic Minority Populations in Primary Care: A Systematic Review. Psychosomatics. 2020;61(6):632–44.
Wells K, Sherbourne C, Schoenbaum M, et al. Five-year impact of quality improvement for depression: results of a group-level randomized controlled trial. Arch Gen Psychiatry. 2004;61(4):378–86.
Arean PA, Ayalon L, Hunkeler E, et al. Improving depression care for older, minority patients in primary care. Med Care. 2005;43(4):381–90.
Wells KB, Tang L, Miranda J, Benjamin B, Duan N, Sherbourne CD. The effects of quality improvement for depression in primary care at nine years: results from a randomized, controlled group-level trial. Health Serv Res. 2008;43(6):1952–74.
Goodrich DE, Kilbourne AM, Nord KM, Bauer MS. Mental Health Collaborative Care and Its Role in Primary Care Settings. Curr psychiatry Rep. 2013;15(8):383–3.
Press MJ, Howe R, Schoenbaum M, et al. Medicare Payment for Behavioral Health Integration. N Engl J Med. 2016;376(5):405–7.
Bao Y, Druss BG, Jung H-Y, Chan Y-F, Unützer J. Unpacking Collaborative Care for Depression: Examining Two Essential Tasks for Implementation. Psychiatric Serv. 2015;67(4):418–24.
Tai-Seale M, Kunik ME, Shepherd A, Kirchner J, Gottumukkala A. A case study of early experience with implementation of collaborative care in the veteran’s health administration. Popul Health Manag. 2010;13.
Whitebird RR, Solberg LI, Jaeckels NA, et al. Effective Implementation of collaborative care for depression: what is needed? Am J Manag Care. 2014;20(9):699–707.
Solberg LI, Crain AL, Jaeckels N, et al. The DIAMOND initiative: implementing collaborative care for depression in 75 primary care clinics. Implement Science: IS. 2013;8:135–5.
Unutzer J, Carlo AC, Arao R, et al. Variation In The Effectiveness Of Collaborative Care For Depression: Does It Matter Where You Get Your Care? Health Aff. (Millwood). 2020;39(11):1943–50.
Scheirer MA, Dearing JW. An Agenda for Research on the Sustainability of Public Health Programs. Am J Public Health. 2011;101(11):2059–67.
Deborah J, Bowen A, Heald E, LePoire, et al. Population-based implementation of behavioral health detection and treatment into primary care & early data from New York State, 07 January 2021, PREPRINT (Version 1) available at Research Square [https://doi.org/10.21203/rs.3.rs-140050/v1].
Moise N, Shah RN, Essock S, et al. Sustainability of collaborative care management for depression in primary care settings with academic affiliations across New York State. Implement Sci. 2018;13(1):128.
Shelton RC, Cooper BR, Stirman SW. The Sustainability of Evidence-Based Interventions and Practices in Public Health and Health Care. Annu Rev Public Health. 2018;39:55–76.
Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013;8(1):117.
Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011;6:42.
Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10(1):21.
Dopp AR, Parisi KE, Munson SA, Lyon AR. Aligning implementation and user-centered design strategies to enhance the impact of health services: results from a concept mapping study. Implement Sci Commun. 2020;1:17.
Lyon AR, Bruns EJ. User-Centered Redesign of Evidence-Based Psychosocial Interventions to Enhance Implementation-Hospitable Soil or Better Seeds? JAMA Psychiatry. 2019;76(1):3–4.
Lyon AR, Koerner K. User-Centered Design for Psychosocial Intervention Development and Implementation. Clin Psychol (New York). 2016;23(2):180–200.
Moise N, Davidson KW, Chaplin W, Shea S, Kronish I. Depression and clinical inertia in patients with uncontrolled hypertension. JAMA Intern Med. 2014;174(5):818–9.
Moise N, Phillips E, Carter E, et al. Design and study protocol for a cluster randomized trial of a multi-faceted implementation strategy to increase the uptake of the USPSTF hypertension screening recommendations: the EMBRACE study. Implement Sci. 2020;15(1):63.
Mangurian C, Niu GC, Schillinger D, Newcomer JW, Dilley J, Handley MA. Utilization of the Behavior Change Wheel framework to develop a model to improve cardiometabolic screening for people with severe mental illness. Implement Sci. 2017;12(1):134.
Dopp AR, Parisi KE, Munson SA, Lyon AR. A glossary of user-centered design strategies for implementation experts. Transl Behav Med. 2019;9(6):1057–64.
Proceedings from the 12th Annual Conference on the Science of Dissemination and Implementation. Implementation Science. 2020;15(1):25.
Monane RGL, Ye S, Odhiambo Z, Blanco L, Carter E, Bakken S, Moise N. User-centered design and usability of an electronic tool to improve collaborative care implementation. Academy Health Dissemination and Implementation Conference; 2018; Washington, DC.
Dong M, Salamanca LF, Medina V, et al. Patient-level barriers and facilitators to sustaining collaborative care programs for underserved minorities: A qualitative study. Gen Hosp Psychiatry. 2020;67:169–70.
Carter E, Monane R, Peccoralo L, et al. Missed opportunities to engage patients in collaborative care challenge program sustainability: A qualitative study. Gen Hosp Psychiatry. 2020;67:158–9.
Moise N, Falzon L, Obi M, et al. Interventions to Increase Depression Treatment Initiation in Primary Care Patients: a Systematic Review. J Gen Intern Med. 2018;33(11):1978–89.
Oliwa JN, Nzinga J, Masini E, et al. Improving case detection of tuberculosis in hospitalised Kenyan children—employing the behaviour change wheel to aid intervention design and implementation. Implement Sci. 2020;15(1):102.
Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013;8(1):139.
Richardson S, Mishuris R, O'Connell A, et al. "Think aloud" and "Near live" usability testing of two complex clinical decision support tools. Int J Med Inform. 2017;106:1–8.
Press A, McCullagh L, Khan S, Schachter A, Pardo S, McGinn T. Usability Testing of a Complex Clinical Decision Support Tool in the Emergency Department: Lessons Learned. JMIR Hum Factors. 2015;2(2):e14.
Dauber-Decker K, Basile M, King DA, et al. Developing a decision aid to facilitate informed decision making about invasive mechanical ventilation and lung transplantation among adults with cystic fibrosis: lessons learned from usability testing (Preprint). JMIR Human Factors. 2020;8.
Weiner BJ, Lewis CC, Stanick C, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017;12(1):108.
Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76.
Overbeck G, Davidsen AS, Kousgaard MB. Enablers and barriers to implementing collaborative care for anxiety and depression: a systematic qualitative review. Implement Sci. 2016;11(1):165.
Williams MD, Jaeckels N, Rummans TA, Somers K, Nesse RE, Gorman RS. Creating value in depression management. Qual Prim Care. 2010;18(5):327–33.
Savard M. Bridging the Communication Gap Between Physicians and Their Patients With Physical Symptoms of Depression. Prim Care Companion J Clin Psychiatry. 2004;6(suppl 1):17–24.
Fortney J, Enderle M, McDougall S, et al. Implementation outcomes of evidence-based quality improvement for depression in VA community-based outpatient clinics. Implement Sci. 2012;7:30.
Tai–Seale M, Kunik ME, Shepherd A, Kirchner J, Gottumukkala A. A Case Study of Early Experience with Implementation of Collaborative Care in the Veterans Health Administration. Popul Health Manage. 2010;13(6):331–7.
Hailemariam M, Bustos T, Montgomery B, Barajas R, Evans LB, Drahota A. Evidence-based intervention sustainability strategies: a systematic review. Implement Sci. 2019;14(1):57.
Mangurian C, Niu GC, Schillinger D, Newcomer JW, Dilley J, Handley MA. Utilization of the Behavior Change Wheel framework to develop a model to improve cardiometabolic screening for people with severe mental illness. Implement science: IS. 2017;12(1):134.
Truelove S, Vanderloo LM, Tucker P, Di Sebastiano KM, Faulkner G. The use of the behaviour change wheel in the development of ParticipACTION’s physical activity app. Prev Med Rep. 2020;20:101224.
Walugembe DR, Sibbald S, Le Ber MJ, Kothari A. Sustainability of public health interventions: where are the gaps? Health Res Policy Syst. 2019;17(1):8.
Palinkas LA, Spear SE, Mendon SJ, et al. Conceptualizing and measuring sustainability of prevention programs, policies, and practices. Transl Behav Med. 2020;10(1):136–45.
Miller CJ, Barnett ML, Baumann AA, Gutner CA, Wiltsey-Stirman S. The FRAME-IS: a framework for documenting modifications to implementation strategies in healthcare. Implement Sci. 2021;16(1):36.

Download PDF

Journal Publication

published 25 Jan, 2023

Read the published version in Implementation Science Communications →

Reviewers agreed at journal
31 May, 2022
Reviewers invited by journal
31 May, 2022
Reviewer #1 agreed at journal
30 May, 2022
Submission checks completed at journal
22 May, 2022
Editor invited by journal
22 May, 2022
Editor assigned by journal
05 May, 2022
First submitted to journal
28 Apr, 2022

You are reading this latest preprint version

A theory-informed, rapid cycle approach to identifying and adapting sustainability strategies: Optimizing depression treatment in primary care clinics seeking to sustain collaborative/integrated care (The Transform DepCare Study)”

Status:

Journal Publication

Version 1

Abstract

Figures

Contributions to the Literature

Background

Methods

Overview

Theoretical frameworks used to develop CC sustainability strategies

Setting

Phase 1: Understanding determinants of CC sustainability, and identifying sustainability strategies

Phase 2: Rapid cycle adaptation of the sustainability strategy

Results

Understanding Sustainability Behaviors

Identifying sustainability options/strategies

Rapid Cycle Adaptations to Fit context

Re-mapping behavior change techniques

Final sustainability strategy

Discussion

Conclusion

Abbreviations

Declarations

Consent for publication. N/A

References

Supplementary Files

Status:

Journal Publication

Version 1