Contextual Factors and Mechanisms that Inuence Sustainability: A Realist Evaluation of Two Provincially Scaled Evidence-Based Initiatives

Background: In 2012, Alberta Health Services created Strategic Clinical Networks TM (SCNs) to develop and implement evidence-informed, clinician-led and team-delivered health system improvement in Alberta, Canada. SCNs have had several provincial successes in improving health outcomes. Little research has been done on the sustainability of these evidence-based implementation efforts. Methods: We conducted a qualitative realist evaluation using a case study approach to identify and explain the contextual factors and mechanisms perceived to inuence the sustainability of two provincial SCN evidence-based interventions, a delirium intervention for Critical Care and an Appropriate Use of Antipsychotics (AUA) intervention for Senior’s Health. The context (C) + mechanism (M) = outcome (O) congurations (CMOcs) heuristic guided our research. Results: We conducted thirty realist interviews in two cases and found four important mechanisms facilitating sustainability: the use of a collaborative approach, audit & feedback, the informal leadership role, and patient stories. Informal leaders were often hands-on and inuential to front-line staff. Learning collaboratives broke down professional and organizational silos and encouraged collective sharing and learning, motivating participants to continue with the intervention. Continual audit & feedback interventions motivated participants to want to perform and improve on a long-term basis, increasing the likelihood of sustainability of the two scaled, multi-component interventions. Patient stories demonstrated the interventions’ impact on patient outcomes, motivating staff to want to continue doing the intervention, and increasing the likelihood of its sustainability. Conclusions: There are important contextual factors and mechanisms within sustainability processes that may apply to systems change implementers and decision makers. Our research revealed the causal relationship between implementation and sustainability and how outcomes from implementation shape sustainability contexts. Future work is needed to evaluate the effectiveness of informal leadership, learning collaboratives, audit-feedback, and patient stories as strategies for sustainability, to generate better guidance on planning sustainable improvements with long term impact. components, feedback components, the nature of behavior change required and target, goals and action plan. Taking study ndings into account, we concur with these best practice recommendations. Our results further emphasize the presence of variance in contextual factors (e.g., resource allocation), intervention design (e.g., mode of delivery of feedback, frequency of feedback,), recipient characteristics (e.g., profession, role, years of experience) and behavior change characteristics (e.g. readiness for change, practice change) that inuence the effect of A&F on sustainability. Future research is needed to examine the process of delivery, effectiveness, and impact of A&F on the sustainability of multi-component, scaled interventions, even in a single provincial system undertaking coordinated, provincial implementation and scale.

outcomes from implementation shape sustainability contexts. Future work is needed to evaluate the effectiveness of informal leadership, learning collaboratives, audit-feedback, and patient stories as strategies for sustainability, to generate better guidance on planning sustainable improvements with long term impact.

Contributions To The Literature
Understanding contextual factors and mechanisms perceived to in uence the sustainability of scaled research-based, healthcare improvement initiatives Demonstrating the ripple-effect between implementation factors and contexts for sustainability The potential of four interventions to facilitate sustainability: learning collaboratives, audit and feedback, patient stories and informal leaders The need to test the use and effectiveness of these interventions beyond implementation, for sustainability Background It is well known that sustainability planning and processes are required well in advance of the implementation of evidence-based interventions for healthcare improvement (1). However, little research has evaluated what in uences the sustainability of such interventions and what strategies are most effective to enhance sustainment (2). Sustainability research is both fundamental to the eld of implementation science and critical to the long term viability of a publicly funded healthcare system (3). Informed by a recent synthesis, our conceptualization of sustainability is comprised of a program, clinical intervention, and implementation strategies, including individual behavior change (e.g., clinician, patient) that continue to be delivered and are maintained after a de ned period of time; during which the program and individual behavior change may evolve or adapt while continuing to produce bene ts for individuals/systems (4).
The past decade has marked a period of health system transformation in Alberta. The province created Canada's rst province-wide, fully integrated health system in 2008. One key objective of this integrated system is to embed evidence into healthcare practice to continuously improve health outcomes and health service delivery, ensuring high quality care and value for every Albertan. To support these objectives Alberta Health Services created Strategic Clinical Networks™ (SCNs) in 2012. SCNs comprise of multi-stakeholder teams (e.g., patients, leaders and managers, clinicians, and researchers) that work collaboratively to identify care gaps and implement evidence-based interventions that improve health outcomes and health service delivery (6,7). Clinical healthcare networks, like SCNs, are intended to break down professional, organizational, and geographical boundaries by bringing multi-stakeholder groups together to codesign evidence-based interventions aimed to improve health care delivery and outcomes (8). SCNs are embedded in Alberta Health Services (AHS), Canada's rst province-wide health care system servicing 4.3M people (9). Currently, there are 16 SCNs across Alberta, each with a speci c scope and mandate, focused on various areas of health (i.e. cancer) or, areas of care (i.e. emergency care) or, provincial programs (i.e. senior's health) or, speci c populations (i.e. maternal, newborn, child and youth health) or across multiple disease areas (i.e. diabetes, obesity, nutrition) (6).
Previous research on SCNs have focused on implementation (10,11), cost analysis (12,13), or speci c interventions (14,15). However, while these evidence-based interventions themselves have been evaluated, no studies to date have explicitly examined sustainability.
As SCNs mature and continue to embed evidence into practice through province wide implementation efforts, learning to spread and scale these interventions and to ensure sustainability is critical (16,17). Failure to sustain effective evidence-based interventions poses signi cant risks to individuals, healthcare systems, funding systems, and communities (18). Recognizing and explaining key factors that have hindered and facilitated SCN intervention sustainability will contribute to systematic and comprehensive sustainability planning. This realist evaluation case study examines two multi-component interventions that have been spread and scaled across Alberta, providing an opportunity to better understand contextual factors and mechanisms that in uence sustainability at scale.

Realist evaluation
We conducted a realist evaluation (19) using an explanatory case study research design (20) to study factors that enabled or hindered the sustainability of two provincially scaled and spread multi-component interventions or "cases". Realist evaluation unpacks and explains the possible causes and contextual factors of change by examining "what works for whom, under what circumstances, and why?", rather than merely assessing "does it work?" (19). We followed the realist heuristic context (C) + mechanism (M) = outcome (O) con guration, whereby an intervention works or not (O), (CMOcs) because of the action of some underlying mechanism (M), which only comes into operation in particular contexts (C) (19, 21). We followed the realist cycle of theory hypothesis generation, theory hypothesis observation and speci cation (19) according to realist terms previously detailed (22). We followed the Realist and Meta-narrative Evidence Synthesis: Evolving Standards (RAMESES) II reporting standards and SQUIRE 2.0 checklist (23, 24) (additional le 1 & 2). Initial program theory development Following the realist evaluation cycle, we rst developed an initial program theory (IPT) to hypothesize how, why, for whom and under what contexts we expected these interventions to be sustained. The rst step in our IPT development was to review key implementation science (n=15), sustainability (n=11) and SCN documents (n=19), including the identi cation of relevant theoretical links between implementation and sustainability. The National Health Services Sustainability Model (25), Dynamic Sustainability Framework (17) and Normalization Process Theory (26) were used to identify key contextual factors and mechanisms that in uenced the likelihood of sustainability. The Diffusion of Innovations (27) theory was applied to help understand key characteristics that in uence successful adoption. The Theoretical Domains Framework (28,29) provided a validated way to link elements that in uenced implementation, to a broad range of behavioral theories. Similarly, the Consolidated Framework for Implementation Research (30) and the Consolidated Framework for Sustainability (31) were used to make sense of diverse factors that in uence implementation and potentially sustainability including intervention, contextual, individual and implementation process characteristics.
Second, we conducted key stakeholder meetings with three senior leaders from different SCNs, to explore their perspectives and experiences on sustaining such large scale, multi-component interventions in their organization. We used meeting notes to supplement information gathered from key documents. Information from our key stakeholder meetings and key documents informed the initial 64 CMOcs. Our team iteratively re ned and thematically organized these CMOcs, yielding a nal set of ten CMOcs. The IPT and ten CMOcs are provided in additional le 4. We subsequently tested and re ned these 10 CMOcs through realist interviews with multi-disciplinary healthcare providers (HCPs) involved in the two purposefully selected cases.

Ethics approval
Ethics approval for this study was granted by the University of Alberta Health Research Ethics Board (Pro0096202). Institutional approval was provided by Alberta Health Services Northern Alberta Clinical Trials and Research Centre.

Case selection
We purposefully selected two scaled, evidence-based multi-component interventions based on a) their maturity, b) scale of implementation (province wide), c) demonstration of improved outcomes and impact and, d) context variation (community and acute healthcare). We de ned a 'case' as an intervention that was evidence-based, had been formally implemented by the SCNs either within Alberta Health Services and /or with partner organizations. Case A is the Intensive Care Unit (ICU) Delirium intervention implemented from 2016-19 across all 22 ICUs in Alberta. Case B is the Appropriate Use of Antipsychotics (AUA) implemented in two different sectors, long-term care (LTC, 170 sites) and designated supportive living (DSL, 140 sites). The AUA intervention was rst piloted in 2013-14 in 11 early adopter sites and was spread provincially during 2014-15 to 170 LTC sites (both public and private); DSL implementation occurred from 2016-18 in 140 spaces both public and private settings (see additional le 3 for case descriptions).

Recruitment and data collection
We purposefully selected interview participants involved with the implementation of each intervention across different levels of the healthcare system (i.e., front line staff, middle management, and senior management) and geographically across the province. We contacted potential study participants through an open letter of invitation circulated to staff by Alberta Health Services leaders.
Interested participants were invited to voluntarily contact the research assistant at their convenience for more information.
We conducted qualitative realist interviews using a semi-structured interview guide to test and further re ne our initial program theory and explore new emerging CMOcs. Interviews explored participants' perceptions of each intervention, implementation, and sustainability processes, as well as the contextual factors and mechanisms that enabled or hindered sustainability. All interviews were conducted by telephone by the research assistant (AC), audio recorded and transcribed.

Data analysis
Following a case study analysis approach (20), we analyzed case-speci c CMOcs, followed by cross-case comparison of Case A and Case B CMOcs. It became clear during cross-case comparison analysis that similar patterns emerged across cases. Categorizing and connecting strategies outlined by Maxwell (32) were used to categorize CMO patterns, with our IPT as an extraction guide. We also inductively coded new CMOcs that emerged across cases. We then connected CMO patterns across cases using NVIVO 11 software to code CMOc patterns. The aim of our analysis was to identify causal patterns of contextual factors and mechanisms between cases which reportedly affected the outcome of sustainability. In this paper, we report the most frequent CMOc patterns that emerged across both cases.

Participant demographics
We conducted thirty realist interviews (case A, n=17 and case B, n=13) from July 2019 -October 2019. Participant demographics, by case, are presented in Table 1.

CMO con gurations
From our initial ten CMOcs, three were evident across both cases and subsequently re ned through cross-case comparison of the realist interviews: (1) The in uence of a collaborative approach on the sustainability of a scaled, multi-component intervention; (2) The degree of importance of continuous monitoring, audit and feedback on the sustainability of a scaled, multi-component intervention, and (3) The in uence of different layers of leadership on the sustainability of a scaled, multi-component intervention. A fourth, novel CMOc emerged across both cases that we had not hypothesized in our IPT: (4) The in uence and impact of patient and family stories on the sustainability of a scaled, multi-component intervention. These four CMOcs are presented in Table 2.
The in uence of a collaborative approach on the sustainability of a scaled, multi-component intervention Participants from both cases explained how the interventions were implemented through a collaborative approach, using several strategies including learning collaboratives (LCs), tailored to each case (see additional le 3 for LC case description).
LCs encouraged cyclical reinforcement of the intervention, continuous learning, and the desire to continue and sustain the work. They were perceived to break down organizational and professional silos by facilitating conversations among groups across the province, who may not otherwise interact. Most participants felt that collaborative provincial sharing and learning were key mechanisms to sustaining both interventions.
Participants reported time constraints, nancial and geographic barriers as major hindrances to bringing people together, provincially. Front-line staff involved in case B expressed concerns regarding their ability to attend every collaborative in person. In some instances, key providers were absent, typically due to staff shift coverage and the inability to secure time off, reduced budgets to nance staff attendance, or travelling distances. To overcome these barriers, LCs in case B were offered virtually. However, most front-line staff felt part of the value of the LC was bringing people together face-to-face. In contrast, directors and managers felt offering the LCs virtually would be bene cial, especially considering anticipated future budget restraints, such as reduced staff travel funding. These participants considered virtual learning as a way to evolve, adapt and provide exible learning in current scally restrained healthcare climates. It is unclear what, if any, impact differences there are in provincial "face-to-face" versus virtual LCs. Quotes to support this CMOc are presented in Table 3.
The degree of importance of continuous monitoring, audit, and feedback on sustainability of a scaled, multi-component intervention Monitoring, evaluation, and feedback of intervention data, such as provincial and local performance metrics, health outcomes and patient experiences were viewed as a vital component to sustainability of both interventions. Feedback was delivered to participants in each intervention, however different types of feedback were viewed as more important, depending on the intervention and stakeholders involved. Different stakeholders had different preferences for the type of feedback that was meaningful to them. In case A, participants reported that quantitative provincial and local performance metrics "drove" the continuation of the intervention. For front-line staff, this data allowed them to understand how they were performing locally, and provincially in relation to other similar sites across Alberta in terms of reducing delirium rates in the ICU. The feedback of this type of data kept the intervention on people's radar and motivated staff to continue with the intervention long-term.
In case B, provincial and local performance metrics were viewed and interpreted differently because the purpose of the intervention was to reduce the inappropriate use of antipsychotics, rather than totally reducing all antipsychotics use. In some instances, leaving a resident on an antipsychotic was most appropriate. Participants valued more re ned data that reported on inappropriate antipsychotic use and use of alternative therapies (e.g., behavior therapy), rather than reports detailing total antipsychotic use. As such, while the provincial and local performance metrics did hold some value in monitoring the intervention, it was especially important to consider contextual elements affecting these metrics. Informal feedback, through the sharing of success stories between sites, and receiving positive feedback from families and other staff, was viewed as more valuable data in this intervention. Importantly, all participants felt that the data being fed back had to resonate and be meaningful to its recipients and it was important for the data to "make-sense" to those reviewing it. Sense-making of data was viewed as a critical aspect of implementation that could enable sustainment.
The modality to provide monitoring and feedback was also perceived as an important factor for sustainability. Multiple communication channels such as emails, scorecards, quality boards, and staff meetings were used. Different channels were effective for different stakeholders. For example, emails were not an effective way to share data with front-line staff, because the emails were often overlooked, however, email was often the most important way to share data for managers or executive directors. For case A, monthly scorecards were provided to each ICU and metrics for all ICUs were shared to enable provincial comparisons. After the implementation period of the delirium intervention, quarterly performance metrics continued. Sharing data in a way that made sense to different stakeholder groups kept the intervention at the forefront of practice and provided a better understanding of the intervention's long-term impact. Quotes to support this CMOc are presented in Table 4.
The in uence of different layers of leadership on the sustainability of a scaled, multi-component intervention Participants perceived that strong and in uential leadership presence was important to the sustainability of both interventions. Participants identi ed different leaders to be front-line staff, unit managers, SCN Practice Leads and SCN executive directors. Front-line staff who were considered leaders were viewed by others to be "making the gains" and improving antipsychotic use or incidences of delirium in everyday practice. SCN Practice Leads and executive directors were viewed by others as "overseeing" the interventions, by monitoring the data, and by providing sites with learning and support.
Front-line staff participants valued learning about the importance of the intervention from informal leaders who were embedded in everyday practice and whom they related to, rather than learning from those in management positions alone. Engaged leaders were those visible to front-line staff. These leaders were "hands-on" and used their in uence to positively communicate the impact and successes of the intervention and came from multiple disciplines (nursing, medicine, physiotherapy, and pharmacy). The presence of such leaders created an enabling, positive work environment with a unit culture conducive to sustaining any gains made from the intervention. Quotes to support this CMOc are presented in Table 5.
The in uence and impact of patient and family stories on the sustainability of a scaled, multi-component intervention For both cases, sharing patient and family stories was one of the most important mechanisms for sustainability of the interventions. In both cases, these stories were formally shared as part of learning collaborative sessions. Some patient stories were shared in-person by family members and some were shared in video format (digital stories). In the AUA intervention, stories were shared by family members of residents from sites across the province, whereas the delirium intervention used a combination of stories from patients and families across the province, as well as videos that were already publicly available, such as those from delirium.org. Patient and family stories were used to illustrate the impact of the intervention to multiple stakeholder groups, which facilitated an understanding of why the intervention was important and how the intervention bene ted patients and improved daily work. Participants responded that these stories had a positive impact and gave them motivation to continue the work. Participants explained how watching videos of patient and family stories conveying positive patient outcomes changed their perspectives on how and why the intervention was important and gave them motivation to continue with the work. Quotes to support this CMOc are presented in Table 6.

Discussion
Our research ndings explain important contextual factors and mechanisms that had a perceived effect on the sustainability of two provincially scaled, multi-component interventions. The discussion that follows outlines four key mechanisms that were perceived by our participants to facilitate intervention sustainability. These mechanisms include implementation strategies of collaboration, audit and feedback, informal leaders, and patient stories.
Learning collaboratives as a mechanism for sustainability Collaborative research approaches are becoming increasingly used by healthcare systems, research funders and government organizations as part of health services research and implementation science (33). A collaborative research approach provides the opportunity for patients, healthcare providers and other key stakeholders to be active participants in the design process rather than the traditional approach of being a passive recipients of design work (i.e. intervention) (34). Participants from both cases discussed LCs as the key mechanism for a collaborative approach that facilitated intervention sustainability. In accordance with the Dynamic Sustainability Framework (17) our ndings suggest that active partnership among all relevant stakeholders is essential to sustaining interventions within care settings. As in the Consolidated Framework for Sustainability (31), our research highlights the importance of relationships, collaboration, and networks for sustainability.
A LC is an organized, multifaceted approach that includes teams from multiple healthcare sites coming together to learn, apply and share improvement methods, ideas and data on performance for a given healthcare topic (35,36). In our evaluation, LCs occurred inperson for case A with virtual components introduced in case B. While there is clear evidence on the effectiveness of in-person LCs to enhance learning, less is known about the effectiveness of virtual LCs (37). Similar to other research, our ndings suggest that creating a culture of continuous learning, promoting accountability, and creating an inter-organizational support network from which sites can learn from others' successes and challenges are some of the main bene ts of LCs (38). Despite the bene ts of LCs identi ed in our study, and others, questions remain about the effectiveness of LCs for behavior change, the use of skills gained in the LCs, the impact of LC for sustained improvement, the effectiveness of LCs as a strategy for sustainability and the and cost-analyses of LCs over time (36,38,39).
A systematic review by Wells et al., (36) found that LCs characteristics, such as the number, length, and delivery mode (i.e. virtual vs inperson) varied across studies. This highlights the existing variability in the design and delivery of LCs; there is a paucity of evidence on how best to design and implement a learning collaborative. Similar to Hoekstra et al., (33) we argue the need for research to examine how and why collaborative research approaches and interventions (such as LCs) work, including the key principles, strategies, outcomes, impacts and contextual conditions these approaches function under. This knowledge may allow for more tailored and e cient stakeholder engagement in future.
Continuous monitoring, audit, and feedback for sustained change Monitoring, audit, and feedback (A&F) of interventions are important strategies to facilitate buy-in, maintain compliance and ensure the continuation of improved outcomes (40). Our ndings pertaining to how A&F supports ongoing staff engagement, by hearing, and seeing data in a group atmosphere are well aligned with the literature (40)(41)(42).
The use of data to monitor local implementation is not just a means of promoting accountability, but also a mechanism to solve problems that impair performance. In the absence of regular, careful monitoring, implementation may be more liable to fail or revert to previous practices (40). From our ndings, it is evident that careful and regular monitoring needs to happen from early implementation of an intervention to support sustainability. Implementation teams and operational leaders need to plan a monitoring, A&F system that makes sense and is meaningful to all of those involved and can demonstrate impact.
Previous research has been done to synthesize the effectiveness of A&F for implementation research. One Cochrane systematic review on 140 studies found that A&F can lead to important improvements in professional practice. However, the effectiveness of A&F as an intervention to change provider behavior depends on both the content of and how the feedback is provided (41). The Dynamic Sustainability Framework (17) suggests that ongoing feedback on interventions should use practical, important measures of progress and relevance. The framework recommends the use of measures that are feasible, relevant to desired outcomes of patients and align with the ' t' between intervention and context. There is a lack of guidance on what dose of feedback and which modalities are most effective to support the sustainability of scaled interventions over time. A&F is most effective when provided more than once (41), however it is unclear from the literature and our study, how often the intervention is required for sustainable impact. Another study that examined the use of theory in A&F studies found that there was an overall lack of use and consistency of explicit theory to guide A&F interventions (42). As a result of these issues, the most important active ingredients and mechanisms that enable successful A&F intervention for healthcare improvement remain unclear (43).
In an effort to bridge this knowledge gap, Ivers et al., (43) provided potential best practice guidance recommendations for A&F interventions in relation to audit components, feedback components, the nature of behavior change required and target, goals and action plan. Taking study ndings into account, we concur with these best practice recommendations. Our results further emphasize the presence of variance in contextual factors (e.g., resource allocation), intervention design (e.g., mode of delivery of feedback, frequency of feedback,), recipient characteristics (e.g., profession, role, years of experience) and behavior change characteristics (e.g. readiness for change, practice change) that in uence the effect of A&F on sustainability. Future research is needed to examine the process of delivery, effectiveness, and impact of A&F on the sustainability of multi-component, scaled interventions, even in a single provincial system undertaking coordinated, provincial implementation and scale.
The in uence of informal leadership for sustainability Previous implementation research studies have established the in uence of formal (e.g., administrators) and informal leaders (e.g., champions) and their activities (e.g., facilitation, support) on sustainability (1,44,45). Informal leaders, sometimes referred to as champions, opinion leaders, change agents, or knowledge brokers, are considered front-line practitioners, driving the implementation of a wide range of change interventions in healthcare settings (46)(47)(48).
A focus on informal leaders is essential because this is where the quality of care ultimately affects patient outcomes (49). In alignment with our study, a Cochrane review determined that the effectiveness of informal leaders as an intervention for the implementation of evidence-based interventions appears comparable, or sometimes even superior, to other interventions (50). As in our study, Ennis et al., (51) found that informal leaders contribute to creating a positive work environment. Informal leaders in uence workplace culture and have signi cant impacts on team e cacy and performance by seeking out opportunities to promote, improve and negotiate best care practices (51).
Our ndings suggest that front-line informal leaders are valued and play an important role in the implementation and sustainability of multi-component, scaled interventions. In our study, front-line informal leaders were active participants in the intervention and were encouraging and motivating for others. This aligns with existing evidence that informal leaders are effective because they socially in uence other professionals, and that this in uence is a function of the respect of their peers (48,50).
Furthermore, it was recognized that senior leaders (i.e. executive directors, unit managers) may not necessarily be the best people to promote continuation of interventions due to their lack of understanding of the daily work of front-line staff. Informal leaders were viewed as more in uential based on their credibility amongst colleagues. This same phenomenon has been found in similar work (52).
Engaging in uential individuals across organizations can help to secure the credibility of interventions and strategies to develop "informal leaders" have shown to be effective in implementing changes at the clinical level (52). Hence, implementation strategies should recognize and seek to engage with and develop individuals who have not traditionally been perceived as leaders. In the later stages of implementation, senior leadership should plan for strategies to help informal leaders emerge, ensuring they have the capacity and capabilities to lead in sustaining efforts. Like the Consolidated Framework for Sustainability Constructs in Healthcare (31) our research highlights the importance of the people involved (e.g., champions) for sustainability.

Impact of sharing patient and family stories
In our initial program theory, we did not hypothesize patient stories as an important mechanism for the sustainability of an intervention.
Patient stories have previously shown merit, with reported improvements in care practices, positive staff engagement, a way for staff to "remember why we're here", and combat burnout (53,54). In this study, patient stories provided a way to connect with patients, to understand their experiences, and to remind staff why the intervention was important, facilitating sustainability.
Stories have a degree of emotional power that can spark attention, resonance and change (55)(56)(57)(58). Like our ndings, other studies have found that sharing patient success stories enables HCPs to feel energized after watching them, as these stories are "impactful, heartwarming, and understandable" (54). Foster et al., (59) found that listening to patient stories not only had profound emotional effects on HCPs, but motivated practice change as they developed newly formed intentions to improve patient outcomes. Similarly, Haigh and Hardy (60) found that patient stories shown to HCPs led to re ection, empathy and discussions surrounding practice change aimed at service improvement. These studies mirror our ndings in that sharing patient stories can in uence better service and patient outcomes through staff motivation and re ection of current practice. Despite the clear impact our study, and others, have shown of patient stories on staff motivation, it is less clear how these stories are being used, to what end they are collected, and how often they need to be shared to sustain initial levels of motivation (54).

Research and practice implications
Our ndings found four key strategies (use of collaborative approach, A&F, informal leadership, and patient stories) perceived by participants to positively in uence intervention sustainability. However, our research also highlighted knowledge gaps that require further research. There is a lack of rigorous evaluations on the use and effectiveness of LCs as a strategy to aid behavior change to reduce the knowledge to action gap. More research needs to be done to look at the design, components, delivery, and impact of LCs as a strategy to help with implementation and more critically, sustainability of an intervention. For A&F further research needs to be done to evaluate different approaches to the design, delivery, and dose of this intervention. We also recommend research that can unpack and try to explain theory used in A&F design and effect modi ers of A&F. Lessons from such research can help researchers and decision-makers plan, design and execute improvement interventions in a way that can be done before implementation and that can lead to sustainable outcomes and impact. Our research recommends that senior leadership needs to plan for strategies to help informal leaders to emerge and to ensure that they have the capacity and capabilities to lead intervention implementation and sustainability efforts. Patient stories have been identi ed as powerful strategy to translate knowledge, however evaluations are needed in relation to the use and impact of patient stories for sustainability.
Like previous research on sustainability (22) our ndings illustrate the important relationship and "ripple-effect" between implementation and sustainability; where there is a causal relationship between implementation processes and outcomes, and sustainability. We found that implementation factors and decisions made for implementation were critical to facilitating or hindering contexts for sustainability. Sense making of monitoring and outcomes data was also a common mechanism at early implementation that enabled or hindered the likelihood of sustainment. Our work also aligns with and extends existing theoretical approaches for sustainability. For example, the Consolidated Framework for Sustainability presents 40 determinants that in uence the sustainability of healthcare interventions, such as leadership and champions, monitoring progress over time stakeholder participation and involvement (31). Our research offers potential strategies (i.e. learning collaboratives, A&F, and patient stories) to increase the likelihood of intervention sustainability and impact. Understanding how to sustain scaled interventions, through which strategies is a novel area in sustainability research. We recommend future research that tests the effectiveness and validity of these strategies for sustainability across other scaled interventions.
In this current evaluation of two provincial wide, scaled, multi-component interventions, many of the important factors and mechanisms that had a perceived effect on sustainability were contextual factors in existence prior to implementation (e.g., leadership) or elements related to implementation (e.g., interventions designed for implementation). Future research is needed to examine how these factors have an important role to play in sustainability, not just implementation.

Resource allocation is challenging in health systems, thus it is important for implementers to understand what they 'need to do' vs 'what
is nice to do' in order to create and maintain interventions that have sustainable impact. Our research has shown that a collaborative approach that includes A&F, informal leaders and shared patient stories has a perceived positive in uence on sustainability; yet it remains unknown which of these strategies are a 'need to do' versus a 'nice to do' for long-term sustainability and impact. There is also a clear tension between implementation and sustainability, it is unclear for operational leaders how much effort to put into sustainability planning prior to implementation when it is unknown if an intervention will be successful or not. Nonetheless, our research emphasizes a clear relationship between implementation and sustainability; we anticipate that if SCNs can understand key components of sustainability earlier, their implementation and sustainability planning could become increasingly deliberate and e cient.

Limitations
The contextual factors and mechanisms identi ed in this evaluation are based on the perceptions of our participants from two scaled interventions; additional research is needed to test the in uence of these factors on sustainability, in situ, and among other scaled interventions. It was beyond the scope of this study to examine the sustainment of the interventions in terms of impact on clinical outcomes. To mitigate this limitation, we purposely sought out several data sources (SCN leaders, documents, including theory and existing evidence to inform the link between implementation and sustainability, participant interviews) to inform our work across all stages of the research. Our sampling of individuals within each intervention attempted to access those who could best re ect on intervention implementation and sustainability. During our Case B interviews, we learned emergently that health care aides may be a key informant role that we had not yet accessed. We subsequently attempted but were unsuccessful at recruiting individuals to participate in study interviews, and this may have negatively impacted our ability to fully characterize unique aspects of that intervention in our study.

Conclusion
Our ndings provide important lessons and considerations for other scaled interventions and healthcare systems looking to adopt and sustain scaled, multi-component evidence-based interventions. We identi ed four key strategies (i.e., learning collaboratives, audit and feedback, informal leaders, and patient stories) that enabled the likelihood of sustainability. Future research that tests these strategies for sustainability can help to provide evidence-based recommendations to healthcare innovators, leaders, researchers, and decisionmakers on how to optimize impact of interventions by thinking of sustainability from the outset. We believe that until that is done, we will continue to see potential resources wasted on what becomes failed interventions.

Declarations
Ethics approval and consent to participate Ethics approval for this study was granted by the University of Alberta Health Research Ethics Board (Pro0096202). Institutional approval was provided by Alberta Health Services Northern Alberta Clinical Trials and Research Centre. Written informed consent was required and obtained from all participants in this study.

Consent for publication
Informed consent was obtained from participants, for the publication of quotes in this manuscript.

Availability of data and material
The qualitative data supporting this study is not available as participants did not consent to having their data publicly available. As a result, we are not authorized to share the dataset.
The authors declare that they have no competing interests.

Funding
This work was funded by a small grant ($23,587.30) provided by the Strategic Clinical Networks™ and AHS awarded to R Flynn and S Scott.

Authors' contributions
RF & KM conceptualized this study and secured study funding from Alberta Health Services. RF led this study and coordinated the study team. AC coordinated recruitment and data collection. RF, SDS and KM provided methodological guidance. TW was the principal knowledge user for this study. AC led analysis with methodological assistance from KM & RF. All authors contributed to manuscript drafts and reviewed the nal manuscript. Provincial (more than one zone) 1 6 Not applicable -1 Workplace Location Urban 14 -Regional 3 - Table 3. Evidence to support CMOc1: The influence of a collaborative approach on the sustainability of a scaled, multi-component intervention Case A-002: "So, every aspect of this intervention was collaborative and when I say that, the creation of it [the intervention] came from input and collaboration of operations, from units, to patients and families and to SCN staff. So, it was never done in silo of just a [name of SCN]. It was always done with an approach that there was representation from across the province." Case A-009: "We decided to use the innovative learning collaborative methodologies, which involved bringing together all 21 provincial teams, to be five learning sessions. And at these learning sessions, teams came together. We shared best practices. We shared guest presenters speaking about implementation. Speaking about clinical best practice for [name of intervention] and [name of work environment]. And teams had an opportunity to come together and network. They could work on...specific clinical best practices. There were four management metrics. And then they could choose two-unit specific metrics for which they chose best practices and clinical recommendations from the framework. And worked on implementing those through action plans of the learning collaboratives." Case B-005: "I think too another big piece was not having the intervention be just the responsibility of one person. So I think having, having the team actively engaged and involved and the team including families as well in that process. As we just talked earlier about the collaborative approach that you know, our medical director pitched in with the physicians. We had our program managers helping, coaching, mentoring the front line. You know our front-line nurses coaching and mentoring health care aides. So I think that was really key in that it wasn't reliant on just one person to roll out the intervention that really, required a team effort and for everyone to be bought in. So I think that helped as well." Case B-010: "I really think it was the collaborative being an innovative collaborative. Having those three learning workshops. And the touch points in the middle, as opposed to having those one in done educations. Because you go to an education day, you get all hyped up, "oh my God! This is great information! We're so excited!" And then you go back to your site and you are excited, but not all the other staff went to that education. And they have no idea what you are talking about. And then it is hard to implement something. Whereas when we do our collaboratives, we take a whole team. They come together and they make a plan on how to make change." When an intervention is implemented at scale through a collaborative approach using a provincial learning collaborative that brings working groups, committees, and operational leaders across the province together

Supplementary Files
This is a list of supplementary les associated with this preprint. Click to download. Table 6. Evidence to support CMOc4: The influence and impact of patient and family stories on the sustainability of a scaled, multi-component intervention Case A-002: "So, we have five learning collaboratives. We always strive to have a patient and family story presented where we had a previous patient share their story with the audience of pictures and feedback and talking about what it felt like to be a patient. And our feedback that we received on that part of it was always very, very positive and that it was a patient story that really helped people to continue to push forward to make change and to continue with the work in terms of you know, I'll say just continuing with our motivation to try. Because [name of clinical issue] is not a new practice in critical care and people often have said that they're just you know, [name of clinical issue] fatigued. That they're sick of hearing about it. They're sick of doing the same kind of work and trying to make changes with it never happening. But one thing that we've heard loud and clear and continuing to hear is the patient story, really...I'll say helped to overcome that fatigue." Case A-009: "So I think...and that's been one of the most powerful things [patient and family stories]. A lot of people at the beginning said like this work is...not that they said it was dumb. But they said you know, "this is pointless. You're never going to impact delirium. You're not going to stop it. It's still going to happen." But once they saw the patient perspective...it really changed their motivation and why they wanted to do this work." Case A-013: "Like when we first started doing delirium...we used a lot of the videos online...from the ICU delirum.org where there's young people and the effects of their delirium on them and how it changed their long-term ability to manage was impactful actually for the staff" Case B-011: "We got videos of teams talking about when a resident woke up. So you know, and we posted all of those stories on the toolkit so that people could use them and we talked about it as a strategy of using good news stories to encourage people and motivate them. So when health care aides say things like it's actually easier to take care of people who can help then it was trying to take care of somebody who was so sedated that they couldn't help themselves at all.