The Sustainment of Population Health Programs Requires a Systems Approach

Background: Achieving sustained implementation of health promotion programs (HPPs) at the population level (i.e. population health) is needed to reduce prevalence of chronic disease. However, population health is highly inuenced by social, economic and political environment changes at the local, state and national level. The aim of this research was to examine barriers to the sustainment population-level HPPs and identify key enabling factors that overcome such barriers. Methods: We conducted a qualitative study through interviews with population health experts working within Australian government and non-government agencies with experience in sustaining public health programs at the local, state or national level (n=13). We used an inductive thematic approach to analyse data. Results: Four key barriers to the sustainment of HPPs were identied: 1) short term political and funding cycles; 2) competing public priorities within health and the broader public agenda; 3) silo thinking within health service delivery; and 4) the population t of the program to population needs. To overcoming these barriers various approaches centred on the importance of long-term planning and resourcing, exible program design and management, leadership and partnerships, evidence generation, and system support structures. Conclusion: This study provides key insights towards overcoming challenges to the sustainment population health programs amidst complex system-wide changes.


Background
Reducing the population burden of chronic non-communicable diseases (NCDs) requires prolonged investment in prevention (World Health Organization 2013) as changes in health behaviour or outcomes may only be evident after sustained delivery of health promotion programs (HPPs). Sustainment in public health has been de ned as 'the sustained use or delivery of an intervention in practice following external implementation support' (Shelton, Chambers et al. 2020, Shoesmith, Hall et al. 2021). This is understood to have occurred after full program implementation is achieved and start-up funding is withdrawn (Stirman, Kimberly et al. 2012). Sustainment also largely depends on the nature of the intervention and outcomes (Stirman,  Sustained implementation of individual HPPs is a challenge. Effective programs may evolve or be terminated, as funding, political or organisational priorities, or public support change over time (Brownson, Allen et al. 2015, Hailemariam, Bustos et al. 2019. Termination may be an appropriate policy or resourcing decision, particularly if the program is less effective delivered at scale, sometimes described as 'voltage drop' (Chambers, Glasgow et al. 2013, Shelton, Cooper et al. 2018). However, more often HPPs terminate as a result of funding ending, or a lack of support from policymakers, agency leaders or community (Brownson, Allen et al. 2015). If HPPs terminate prematurely, they may fail to elicit individual health bene ts (Glasgow and Chambers 2012), or positive return on investment (Walugembe, Sibbald et al. 2019).
Several frameworks and tools have been devised to identify or measure factors that facilitate or hinder sustainment of public health programs (Chambers, Glasgow et  Less is known about how programs are sustained at the population-level (

Study design, participants and recruitment
We undertook a qualitative study that was relevant to public health policy decision makers and practitioners in Australia. We used purposive sampling techniques to recruit a broad range of health promotion experts from diverse organisations and who worked at different levels within health promotion (i.e. local, state or national level). Using initial networks, we identi ed prominent health promotion practitioners or decision makers (stakeholders) within Australia, supplemented by other recommendations by initial participants. Eligible participants were selected based on having experience in health promotion at a population level and/or working in a management position for more than one term of government (i.e. +three years), and whose role was to oversee the delivery of HPPs. Prospective participants were sent an information statement via email to consider participation and called to con rm their interest and schedule the interview. Invitations to participate in the study continued until we reached a saturation of ideas.

Data collection
One-on-one virtual face-to-face interviews were conducted with consenting participants between June and August 2020 using Zoom software. All interviews were conducted by one experienced qualitative researcher (MC). Participants provided verbal or signed consent before the interview was conducted.
Interview were audio recorded and took 52 minutes on average to complete.
We developed a semi-structured interview guide to examine the factors in uencing HPP sustainment at a population level. We invited study participants to re ect on their experiences while managing, implementing or sustaining public health intervention(s) throughout their career. Participants were asked to consider population-health HPPs that were/ were not sustained as changes in government or government priorities or social, political or technological changes occurred. The interviewer took eld notes during the interviews to inform the analysis and used follow-up questions to prompt the participant.
We developed interview questions using the Program Sustainability Assessment Tool (PSAT). The PSAT is a quantitative instrument developed to assess the capacity of public health programs to be sustained. The PSAT is comprised of eight domains/sub-scales of factors associated with the sustainability of public health programs (Luke, Calhoun et al. 2014). Interview questions were designed to explore these domains; with a focus on how/whether disruption in broad scale implementation affected program sustainment. Our interview questions assessed the role of partnerships; social-political support from decision makers and stakeholders; funding for ongoing programs; strategic planning for sustaining implementation; organisational processes, learning and resilience; program adaptations to ensure programs were sustained; the role of program evaluation and research; communication strategies; as well as general re ections (Supplementary le 1). Co-authors (NN and KL) pilot tested and re ned all interview questions prior to the start of the study

Data Analysis
We transcribed interview recordings verbatim and checked for accuracy. We applied thematic analysis methods using a grounded theory analysis approach to the data (Clarke, Braun et al. 2015). MC reviewed transcripts and recordings and made initial observations. We used an inductive analysis line-by-line coding with descriptive words or phrases to describe data segments and then developed common constructs to derive higher-level codes (Saldaña 2015). MC reviewed initial coding and emerging themes with co-authors (NN and KL). Key themes were then discussed with all co-authors as a process to establish theoretical notes. We re ected on the themes that emerged from our experience managing HPPs or in uencing health promotion practice at some level or capacity. Participants were invited to check qualitative themes and provide feedback. We used NVivo software (QSR International Pty Ltd. Version 12, 2018) to code and explore qualitative data.

Participant characteristics
We identi ed 14 eligible stakeholders. At the recruitment stage, one eligible stakeholder did not respond when contacted. All stakeholders who did respond, agreed to participate in an interview. In total, we conducted 13 interviews with stakeholders who were government (n = 7) and non-government (n = 6) health promotion experts. They held senior leadership (n = 6) or mid-management (n = 7) roles within their organisation. Many participants had experience in both government and NGOs, and all had been in public health for at least 10 years. Four participants had a position with a research focus or adjunct academic positions. Most participants were female (n = 10), and from the state of New South Wales (n = 8).

Key barriers to sustaining population HPPs
We identi ed four key barriers to the sustainment of population-level HPPs: 1. Short-term political and funding cycles; 2. Competing public priorities; 3. Silo thinking within the health system delivery; and 4. Population t. We summarize these ndings in Table 2.

Barrier -Short-term political and funding cycles
Political and economic environments in which HPPs were implemented was highlighted as a fundamental population health challenge. Participants explained that programs designed to change health behaviours and reduce NCDs generally require a long-term program effort. However, this was not achieved in practice. HPPs were most often planned for short-term implementation, supported by shortterm funding (usually three years maximum), and measured against short-term outcomes. Thus, programs had limited capacity to improve health in a cost-effective way. Short-term programs and funding cycles generated sta ng insecurity. In turn, a short-term workforce resulted in corporate memory loss. 'Short-termism' was broadly discussed at the national and state level, in government and NGOs, where funding streams, grants and commitments were time-bound. One health promotion manager described that the usual reasons to justify this approach was to minimise potential risks to nances or reputation. On the ip side, they also stated that health promotion tended to be too risk averse. "Diet and obesity interventions are incredibly complex and they're multi caused and there are different levels. So the interventions that are required are likely to be interventions that require many decades of intervention." #11 State NGO manager "the short-term funding cycles can actually breed, in some respects, bad practice. So, what you're actually developing and what you're looking to implement is in uenced by the fact that you only have three years. You might go for low-hanging fruit, you might go for solutions that you might not have gone for if you knew you had a longer period of time. You don't learn from your mistakes. You don't get to evolve and massage and improve your initiative" #13 National NGO senior manager The political environment was a key contributor to program sustainment. Participants spoke of the challenges in sustaining population-level programs when government priorities changed. State government priorities dictate the direction of health services and public health. Public HPPs either adapt to t with government priorities or risk being terminated. Termination was not necessarily viewed as a negative: half of the participants noted the need to end programs that were unproductive, or 'refresh them' to 'sustain impact appropriately.' NGO organisations could support programs being continued despite new political priorities. However, priority changes often stretched public health funds and resources. Political interest in a health issue or program was not always regarded favourably. Some participants were concerned that partisan program popularity might reduce the likelihood of retaining support for a HPP should a change in government leadership occur.
A general response was to persevere through uctuating political or social climate cycles and accept short incremental 'wins'. Potentially longer-term funding, such as that provided by Australia's National Partnership Agreement on Preventive Health (NPAPH), was mentioned by most participants as a major driver of HPP implementation -and may have been perceived as a stable environment for HPPs. Given 'scarce health resources', competing for funding and resources with other priorities in health (or government ministries more broadly) was perceived as constant challenge. Within health, HPPs competed with other public health and health service priorities with immediate or short-term visible impacts on health; having longer-term outcomes often beyond the tenure of senior health service executives or political leaders.
Participants recognized that senior leadership strongly in uenced the extent to which health promotion was supported or sustained. Some participants spoke about the health promotion background or experience of current or former executives. Relationships between HPP directors and senior organisational leadership was regarded as critical to promote speci c HPP pro les and health prevention more broadly. Maintaining relationships and defending programs was considered both a necessary and resource-intensive activity.
"I think one of the things that can interfere is of course how much your executive are on board with health promotion and that's certainly seen us with stronger or maybe sometimes the strategies are different and I think for health promotion you really have to be constantly looking up ways to promote yourself." #04 Local health manager Competing priorities existed outside the health sector, as population health strategies to address health determinants often required actions outside the jurisdiction of health departments and health ministers.
In uencing these meant engaging politicians with different priorities. One example was the need to work with a government o ce responsible for liquor licensing to reduce the health harm from alcohol use, whose responsibilities also intersect with other government priorities including the night-time economy. Participants in senior government positions noted a greater need to communicate with counterparts in other sectors to support public health programs. Mature thinking about the role of other organisations in health promotion, and caution about the capacities and interests of other agencies was also deemed important.
"we get caught up in our sense of what we're doing -health is the most important thing and that education should be doing this and transport should be doing that et cetera. And I think it's that question about when you're working with other people, health's not their core business... But they've got their organisational priorities and they've got the things that they have to deliver." #06 State government senior manager 1. Barrier -Silo thinking within the health system The sustainment of HPPs across the health system sometimes required 'passing responsibility' to another organisation or another part of an organisation, as a way of program institutionalisation. One state-level manager noted a common mindset-as soon as scal responsibility for an HPP was transferred to the local health services department, state responsibility ended. Programs were sustained by transferring program costs and responsibility to other sectors such that the HPP was 'embedded in a sector'. Such thinking came at the expense of value, in terms of the economies-of-scale and e ciency.
Implications of compartmentalised program actions and activities was re ected in funding needs. Funding was either discussed as 'critical' or 'mildly necessary' to sustain HPPs. This view depended on what level within the health system the person worked. Those in state oversight positions did not perceive funding as a barrier to sustain HPPs. They looked to other mechanisms such as policies or partnerships to achieve program sustainment. Whereas those in local government health services or NGOs viewed funding as essential to sustain HHPs. This strongly in uenced the way these groups related to funders including state government. Innovations were considered risks that local level government and NGOs were cautious about as they could jeopardise potential funds.
"I think for NGOs it has to come down to funding. There has to be a commitment for funding if NGOs are to deliver something." #10 State NGO manager "just don't do anything too ashy that can be aligned with one particular government or one particular minister, which is really hard because they want you to do that work because they want the pro le. …big ashy things don't survive because when there's a change -…even with the new CEO… the new one wants to make their own mark. So that doesn't continue, the new stuff does." #11 State NGO manager Failure to recognize the role and value each part of the larger system required to implement and maintain large-scale programs created tensions between levels of government, and between government and NGOs. State operations teams were regarded by local delivery managers as not understanding contextual issues that in uenced implementing HPPs in local areas. This was said to result in unachievable key performance measures and program outcomes in some areas.
"we started seeing, 'this is the program'; 'these are the practices'. It became much more regimented around: 'this is what the program looks like'; these are the deliverables'; this is the outcome we're looking for and you will be assessed … and whether they pass or fail are based on whether they meet the practices'… I was burning out staff trying to get this model to work." #09 Local health senior manager "If we're the local implementers, it should all sing together, we should be a band. We should be a very good band or orchestra, whatever you want to call us, different components… But down here we're probably all the ones making the -they might be the bigger sound, we're doing all the blending…" #04 Local health senior manager 1. Barrier -Population t Managers were generally responsible for a suite of HPPs, and it was widely accepted by participants that no one HPP could effectively address population-level NCD prevention. Individual HPPs were viewed as a component of a wider population health commitment. Many participants considered sustaining health promotion more critical, and of higher priority, than sustaining a speci c HPP -which placed single HPPs or program elements at risk for defunding.
HPPs designed for a speci c purpose were di cult to adapt to changing population needs or align with new priorities. One participant explained that HPPs were generally developed with a speci cally de ned format and purpose. As such, some participants considered it necessary to retire programs. Other participants were cautious about committing to long-term partnerships where they may be unable to adapt the program to the population needs. Regardless, some HPPs [Cancer Council Skin Cancer Prevention Program and the Heart Foundation's Jump Rope for Heart] were successfully delivered over decades.
'we set them up initially to deliver against certain outcomes and often there is radical changes, particularly in how people consume information, their lifestyle factors, even the people who come into the program' #05 State government manager "Health promotion, it's never one single thing that it's going to make a difference. It's all those things happening at once." #10 State NGO manager Delivery of HPPs at-scale directed by state or national level teams across local health services offered advantages and disadvantages for sustainment. Localised small-scale HPPs were more easily terminated, whereas HPPs delivered at-scale across local levels were thought to withstand political pressures, especially if they had a greater public pro le and a central implementation structure. However, lack of local health service teams' involvement in decisions on state-wide HPPs was considered a potential barrier for HPPs tting local contexts. Managers at the local level asserted that their knowledge of local issues and capacity to adapt at-scale HPPs to their local population context would ensure not only implementation but also ongoing sustainment. From a resource management perspective, two senior managers discussed that while resourcing of at-state state-led programs was more stable, it was critical for staff to spend time across more than one HPP to ensure job variety, and therefore staff retention.
Because otherwise it's like that little thing of, "Well that's just a little thing over here, and yeah that'll be sad if that goes,"... But with some of these agship programs -I mean they can't all be big programs, they all have to start somewhere as well -but these as agship programs … part of the reason why it was able to survive. #06 State government senior manager Key enablers for sustaining population HPPs Several approaches were used to overcome barriers. We highlight these below as 'enablers' and contextualise them in Summary Table 2. However, most participants recognised that to fully address barriers required transformational changes in the system-including funding structures, performance measures, planning and design.

Long term strategic planning and resourcing
Strategic planning was important particularly in the face of changeable political, economic and social climates. Participants spoke pragmatically about how they spread and allocated health promotion staff and other resources to maximise the capacity to sustain multiple HPPs. Longer-term planning meant looking for partners to share responsibilities and invest in the HPP over time. Embedding HPPs into medium and long-term strategic documents was also considered an important strategy to sustain HPPs within organisations. Some participants mentioned using formalised partnerships with local agencies and community organisations to support HPPs. Service agreements were used to embed program delivery into local health services. Key performance indicators for HPPs were used to clearly identify program deliverables. State and local government health managers viewed NGOs as important partners despite political changes and challenges. NGOs viewed themselves as more resilient to government disruption or change; they fostered bipartisan political relationships to remain exible and sustain their HPPs.
"in the NGO sector, where we are perhaps, we are subject to economic uctuations and to some extent, changes of government because of government funding, but generally speaking we can make longerterm commitments because a change of government doesn't affect what we choose to prioritise." #13 National NGO senior manager "having a commitment for whoever it is to deliver it -it doesn't always have to be NGOs -but it needs to be a long-term commitment. And build in that evaluation so that at the end of the four years of funding … they [decision makers] can see what realistic impacts it has had so that they can continue on. Because they might not see a decrease in prevalence within four years. #10 State NGO manager System support and structure Most participants viewed population health from a 'systems perspective'. That is, as part of a complex health system, with many levels and where different organisations contributed to HPP delivery and sustainment. This contrasted with 'silo thinking' as HPPs were discussed in terms of multilevel implementation by multi-level implementers. This perspective was enabled by 'system structures', such as governance structures and processes, and local community setting services. System supports included policies, strategic planning documents and service agreements and quality frameworks, sustainable funding, health promotion institutions and other mechanisms or 'infrastructure scaffolding'. Intangible supports, such as relationships, were also recognised as mechanism sustaining HPPs across levels. These supports enabled state-wide HPPs to withstand the pressures of competing priorities and improved the t of HPPs to local contexts. State level support to deliver an HPP across local municipal networks was thought to make it easier for local level health service providers (e.g., those operating within local health service boundaries) to allocate time and resources.
"what matters more than anything else is the systems that sit behind that intervention. So, to what extent are there robust medium-to long-term implementation plans, funded action plans, robust institutions … institutional and governmental systems -building blocks without which, or in the absence of which, sustainability is really di cult." #13 National NGO senior manager So for us we implement state programs and with the funding that we get for those programs, that's us a sustainability strategy because it's funded…. the health promotion directors will also want it in writing … it's really important for us to use as evidence when we try to protect our budgets. #04 Local Health senior manager

Flexible design and management
Evolving population health needs could be met through program adaptability,. Flexibility was considered necessary in terms of how at-scale HPPs are designed and implemented to t local contexts. From a management perspective, one way of increasing exibility while maintaining delity at-scale was, as one state program manager describes, to use a 'tight-loose-tight' approach (i.e. enabling 'loose' adaptable program activities within the con nes of core 'tight' activities). The local implementers also saw that as essential because local populations and their needs differ. Flexibility provides opportunity for testing different implementation approaches in speci c contexts, without which programs were likely to fail. "There's a point at which they need to be refreshed, or reinvigorated, or revised in some sort of constructive way." #05 State government senior manager The funding goes to the health districts and they're told what they need to deliver, that's the 'tight'. The 'loose' is 'do it as you see t', but obviously with a lot of support and involvement in the o ce. The 'tight' is you now have to report on your performance indicators, and formally report on those" #03 State government manager Leadership During a crisis (such as unexpected funding cuts) organisational and political leadership were highlighted as key factors that ensured a program survived. Where leadership support failed, public support for programs also helped to in uence decisions. Gaining a public pro le and public support was important but viewed as a lower priority than building in uential relationships and communicating program bene ts directly to senior decision makers.
"the one at the top of my list would be high level senior executive or political support. Without that, no matter how good an intervention is, it's just not going to be sustained, particularly if we're talking about government, but I think the same can apply working in not-for-pro t." #01 State NGO area manager Partnerships Non-health partners were deemed necessary to sustain programs when resources were limited. However, for some partner organisations health may not be their 'core business' and they may have a limited understanding of health promotion. Finding key people or champions in delivery organisations was thought to be important at state and local levels. However, relying on individual champions for program sustainment was also thought to be a vulnerability. For health promotion to remain a priority, HPP actions or activities needed to be routinized. To do this, organizations needed to clearly identify how HPPs aligned with end-user goals.
"There's no point us inventing something that no one can pick up and is not scalable and can't be funded. So I've got a target agency to make my intervention routine practice…the answer is you design it with your end user, the end provider organisation in mind." #07 Local Health senior manager "I think the most crucial thing is whatever we are developing in terms of an intervention needs to be part of whatever organisation is delivering it, part of their core business. It's just an adaptation of what they do. If it's too different, then there's no sustainability. #10 State NGO manager

Research and evaluation evidence
Research evidence was necessary to design, implement and adapt a HPP delivered at-scale to achieve best t for local contexts. Some participants had a research role which in uenced their thinking. However, the importance of embedding research into program development and its ongoing adaptation to achieve population level health bene ts was prominently discussed. Building research into practice enabled programs to be continually adapted to meet evolving needs. Evaluation and routine data collection generated tangible evidence of effective HPPs to communicate bene ts to decision makers and sustain funding.
"…The key thing there is you build your health promotion systems, the organisations, the processes, etc, to integrate research into its governance and build it into its recruitment, build it into its training …in this case research is being part of health promotion." #07 Local Health senior manager "if I have a program evaluation that is demonstrating a program to be effective beyond reach …it increases the likelihood for me of either at least protecting it, if not ideally actually increasing or sustaining investment into the long term. That makes a really huge difference compared to a program that isn't evaluated." #11 State NGO area manager  . Therefore, we extend the current literature by capturing real-world experiences of participants directly involved in delivering HPPs at the population level across levels of in uence. We offer important insights into barriers and key enabling factors that created challenges and opportunities to sustaining HPPs.
Decision makers and practitioners identi ed four main factors perceived to in uence the sustainment of HPPs at the population level. Two barriers were associated with changing socio-economic and political environments -political and funding cycles and competing public priorities. Changes in government, short budget cycles and internal political pressures are 'new factors' in uencing sustainment (Bodkin and Hakimi 2020). But this highlights the current literature gap. Barriers to sustainment depend largely on the setting or context for implementation (Shelton, Cooper et al. 2018). To overcome short-termism and competition for nite public health resources requires a public health systems approach.
'Systems thinking' is about how programs work within a complex system and is characterised by feedback learning and adaptation (Meadows 2008 . For health promotion activities to in uence health behaviour at the population level requires a whole-of-system approach of actors and actions working at multiple levels targeting general and sub-population groups. This is the opposite of 'silo thinking' which compartmentalises the actions of organisations and their short and long-term contribution to HPP sustainment. It also changes the perspective on the fourth barrier, population t of programs, as population needs shift, as evident in the case of the current pandemic. Enabling factors for the sustainment HPPs supported systems thinking included: exible agile programs; supportive structures and agencies (internal leadership and committed partners); and long-term planning and resourcing. Evidence from research and evaluation also enables an information feedback cycle for how an HPP should evolve and adapt in response to changed actors, environments or contexts (Newell, Proust et al. 2007). This is where partnerships between researchers and service providers can contribute to ongoing development. Flexible and agile programs are more able to withstand a changing sociopolitical environment and respond to shifting population needs. Adaptation of HPPs is the essence of However, the notion of adaptation is contentious as it elicits a 'dynamic tension' when an HPP deviates from its original [effective] design and implementation approach to meet the needs of a new population

Strengths and limitations
Strengths of our study include being one of few studies that explore how population health programs can be sustained at the population level. Participants represented extensive, diverse experiences and realworld insights as to factors that in uenced sustaining HPPs. This scope of experience may have contributed to them contemplating factors at the systems level, whereas most of the current literature addresses how one organisation operates or achieves implementation success over shorter periods.
Limitations include our sample size was relatively small, with senior stakeholders occupied by infectious disease efforts, and participants were primarily from one state within Australia. This limits the capacity to generalize ndings to other groups and jurisdictions in Australia and elsewhere. However, themes that arose re ected themes discussed with participants from other states and the literature.

Conclusion
The sustainment of HPPs at the population level is fraught with many challenges however are necessary to improve long-term population health outcomes. Factors work independently and in concert within and across numerous levels of a complex, dynamic system. The implications of embedding HPPs into the Australian health system, the need for long-term stable funding in ever-changing socio-political environments and funding cycles, and the need to embrace the role of multiple organisations, holds relevance for other countries.