We undertook a rapid literature review on evidence use in LG, followed by a national online scoping survey of approaches to evidence use in LG (n=8) and facilitated workshops in three UK sites involving 54 participants to explore how evidence is currently used in LG, alongside the opportunities and challenges of using evidence in these three contexts. In addition, in-depth 1:1 interviews (n=14) were conducted in one Local Authority (LA) to explore perceptions of what, why and how evidence is currently used in LG in more detail. The combined data informed the development of a logic model which was tested and refined in two further workshops with stakeholders in Rivertown (n=13) and nationally (n=27).
In this paper we report on the finding from the workshops, as these sought to identify mechanism for improving evidence use, which is the aim of this paper. Findings of the scoping review, the interviews and the logic model will be reported elsewhere. More details on each work package and their findings are also available from the Health Foundation report (Adamson et al., 2020).
Facilitated workshops
Based on the findings from the literature review and scoping survey, four workshops were organised with LA staff, which explored how evidence is currently used in LG, alongside the opportunities and challenges of increasing evidence use in three different areas.
Following an initial workshop in Rivertown, subsequent workshops were held in Belltown and Castletown attended by 54 participants in total (more details on participants in each workshop can be found in Table 1). Topic guides were used to focus discussions on different topics identified by PHPs in each workshop: ‘school readiness’ in Rivertown, ‘health inequalities’ in Belltown, and ‘health in all policies’ in Castletown. Facilitated workshop discussions provided an opportunity to scope local needs in relation to the chosen topic and examine the existing networks that drive decision making and use of evidence. In each workshop, we explored beliefs about the value of research evidence, the value of different models for evidence-informed decision making, the routine application of existing evidence and participants’ views on the potential for co-creation of new evidence that would address their challenges and priorities.
Although each authority chose different public health topics to frame their discussions, workshops were facilitated using similar prompt questions. Written notes by table facilitators and flipcharts from each of the interactive table discussions at the workshop were analysed collectively by the research team. The findings were then presented at a fourth workshop by means of a ‘sense check’, generating group discussion among LA participants (n=13) regarding the appropriateness of the logic model along with practical considerations of its application.
Table 1. Description of participants in each workshop (n=54)
Workshops
|
Participants roles
|
Total
|
Rivertown
|
Senior Specialist - Public Health (Children and Young People), Public Health Intelligence Specialist, and Service Manager - Early Help and Family Support, Performance Analysts, Information Managers
|
14
|
Belltown
|
Representatives from Belltown City Council, Belltown HSC Trust, Health and Social Care Board, South EHSC Trust, Libraries NI, Belltown Health Development Unit, West Belltown Partnership Board, Northern Ireland Housing Executive
|
22
|
Castletown
|
Representatives from Hampshire County Council and Castletown City Council, including Assistant Director, Internal Provision and Front Door, Head of Insight and Engagement, Head of Research and Intelligence, Head of Corporate Customer Service and a newly appointed embedded researcher. Academics in Public Health and Medicine from Castletown University
|
18
|
Rivertown (2nd/ follow-up workshop)
|
Participants included: Senior Public Health Specialists, Insights Manager, Performance Analysts, Service Improvement Leads, Community Safety Specialist, Communities Officer, Policy and Communications lead, Public Health Intelligence Lead, two Directors and a voluntary organisation chief executive.
|
|
Workshop structure
Each workshop started by exploring the chosen topic by LA stakeholders in more details, to get participants thinking about the topic area and to clarify terms, for example, in Rivertown the initial discussions focused on how participants defined school readiness. This discussion was followed by a future visioning exercise, asking participants to imagine that it’s ten years later (2029), with their chosen subject area completely embedded in LG and achieved in their local area (e.g. and all children in Rivertown are ready for school) and asked to note down on post-it notes what three things pleased them most and what had happened to ensure this. This exercise was designed to help participants define outcomes in relation to their topic area. Participants were then asked to collectively cluster individual notes in groups on a flip chart. In this paper, the results from this exercise will be compared across workshops to illustrate how LA staff would like to use and co-produce research in LG.
The future visioning exercise was followed by table discussion on current practices for using evidence (e.g. statistics, performance measures, research, local intelligence) in their chosen subject areas (e.g. how do you know that a child is ready for school?) and to inventory where this evidence was coming from; e.g. what information sources were used to evidence their selected outcomes? This discussion also aimed to get an understanding of who had access to this evidence and who held and used what data.
In the follow-up table discussions, participants looked at what helps and hinders evidence use in decision making and how evidence use could be improved. A member of the research team facilitated discussions using an agreed topic guide at each table, and recorded notes on a flipchart. Finally, in the last set of table discussions, participants were asked about the potential role of research in the LA, particularly who and what was currently missing to achieve this. This session aimed to get a better understanding of what types of evidence may be missing and what the strengths were of inter and intra-organisational relationships to support evidence use.
Findings
Although each workshop focused on a different topic chosen by participants in advance, there were similarities in the themes emerging from each workshop, which point to significant potential triggers for evidence use in LG. The five overarching themes were:
- Aligning national and local policies
- Local system-wide approaches
- Evaluation of local programmes
- Political and cultural barriers
- Collective spaces for reflection
We will discuss each theme separately below with illustrations from the four workshops.
Aligning national and local policies
In all three workshops, participants differentiated evidence at different spatial levels; often starting with national government and the implications of policy development at this level for statutory data requirements at the local level. In each workshop, participants emphasised the need for a change in policy focus and related outcomes, and more integration of national policies across different government departments, to be able to change their own focus in local outcomes and data needed to account for these outcomes.
For example, in the Rivertown workshop participants discussed a definition of school readiness that incorporated two broad categories: academic skills and social skills of children at age 5. In the table discussions, acquisition of social skills (and a broader definition of skills for life) emerged as the more dominant category and preferred way to define school readiness. Participants concluded that social and soft skills were most important in order to create an individual environment in which the young person could thrive. However, when reviewing the current evidence that was being collected within the LA on school readiness, indicators for academic skills were much more prevalent. For instance, Early Years Foundation Stage Profiles at end of reception, the Ages and Stages Questionnaire, pre-school assessments all tended to focus on cognitive and language skills.
Moreover, these data were collected by different organisations, such as schools, health visitors, nurseries and including private providers, which made it difficult to compare and data was often not shared between organisations or only available at population level. Many of these data collection processes were dictated by national government as part of the statutory delivery and monitoring of services and, therefore, LAs reported minimum wriggle room in deciding what data to collect because of these national mandated targets and practical challenges surrounding data sharing. In sum, LAs were collecting data which did not match their preferred definition of school readiness to inform local policies, and were not able change this.
Local system-wide approaches
The need for more integration of policy and data collection was repeated at LG level with participants advocating for closer collaboration between LG departments on data sharing, pooled budgets and integrated services. This need for integration links to our first building block for co-production: a process to facilitate better data integration across LG departments and between LG, NHS and other partners.
Adopting a system wide approach to improving health that included other LG departments and facilities, such as transport, sports and leisure, was seen by participants as more effective in achieving outcomes. Getting the right people around the table from different local departments and aligning research to political language, was deemed as essential. For example; the word ‘intervention’ means very different things to a researcher and a local councillor, with the latter often assuming a negative connotation, such as children being taken into the care of the LA. This alignment points to the second building block of co-production in which research evidence is codified to make it fit with the political context and agenda in which it is to be used. Researchers need to understand how decision-making actually works and learn the politics of the local decision-making processes.
The need for co-production as an integration process, which codifies knowledge for different audiences, was also extended to wider partners, including NHS and VCS organisations. Participants across all three workshops pointed out the need to work closely with external partners outside their LA, such as schools, parents and the wider public (process). Communication was seen as key for this and, therefore, evidence needed to reflect the objectives and needs of these partners by answering the question ‘what is in it for them’ (codification). Moreover, partners needed to be confident and knowledgeable to engage and therefore dialogue was considered important. This introduces the third building block of co-production: developing capacity and capability to engage through training, both for LG staff to understand their partners’ needs and for external partners to engage confidently with LG.
Codification in co-production was deemed particularly relevant at the community level, by giving local communities a greater voice in decision making and resource allocation. More community ownership of services was seen as better way of targeting those that need the services the most by enabling communities to design where and how services would be delivered. Research data should support and reflect this by highlighting areas with the greatest inequalities and service needs. Not surprisingly, health and social inequalities were seen as key focus for local and national policies.
In the Belltown workshop, participants mentioned an ongoing struggle to incorporate community data (described as soft intelligence from these communities) in their decision making, which they were keen to do. They recognised the importance of a community voice in their decision making in order to target areas of highest need. However, the LA culture of evidence in Belltown prioritised quantitative survey data (e.g. from the national Statistics and Research Agency), which they were trained to use, over qualitive data from local communities, which participants did not know how to access or use effectively. In other words, the LA lacked the absorptive capacity (31, 32) to use the knowledge from communities as it was not codified in way that the LA could embed in their cognitive structures. These cognitive structures depend on effective communication and a shared language between the different co-production partners; in this case, Belltown LG staff and representatives from local communities.
To create this shared language, participants in the Belltown workshop suggested a codification based on the concept of social and economic inequalities in all policies. This would facilitate a move away from policies for the ‘average person’ (which is the focus of quantitative survey data) and instead advocated targeted policies for more deprived areas for which qualitative data is more appropriate, as it provides richer insights in specific cases.
Evaluation of local programmes
Participants agreed that a different codification of knowledge required a different way of collecting, collating and using evidence within LG. All three workshop participants expressed a need for ongoing assessment and evaluation of programmes within LG, which they felt was currently lacking. These assessments and evaluations were essential to answer different research questions: What worked well for whom and where? How can we make better use of existing data within LAs, including survey data and population statistics, combined with qualitative data? To support effective and relevant evaluation, participants suggested the need to build in reflection and evaluation from the start of new programmes: thinking through how a programme might work and understanding key outcomes and, therefore, what evidence is needed to assess these outcomes to demonstrate a change in health outcomes and the effect of a programme on health inequalities.
Currently, participants complained that fragmentation in LG and silo-thinking between departments hindered data sharing and joint learning: different professional cultures within LG and, in some cases, a lack of an organisational culture that supports research use was not conducive for collecting, collating and using evidence within LG to facilitate more programme evaluations and assessments.
For example, in the Castletown workshop participants reported a lack of incentives within their LA to evaluate local programmes and services. What was valued by programme leaders and commissioners was the delivery of projects and not their evaluation. Incentives and performance targets for projects were based on delivery of the service (outputs and numbers) and not what difference the service made to the health and wellbeing of service users. Evaluations could even be counterproductive: some Council staff admitted that they did not value evaluations, as they did not want to hear what did not work in their project, which could endanger future commissioning and funding of the service.
Similar to the participants in the Belltown workshop, participants in Castletown identified that their LA lacked the absorptive capacity to use research evidence. In this case not because the evidence was not codified in a fitting way with LG decision making processes but because the cognitive structures were not able to absorb any research evidence. Perverse incentives caused a culture of anti-evaluation with a complete lack of willingness to undertake co-produced research.
Political and cultural barriers
What emerged from the three workshop discussions was a deeply politicised and fragmented system in LG that put different demands and constraints on evidence use, depending on the context and decision making process in which evidence was used. The political system was described as disorderly by participants, but also as opportunistic in nature. Participants highlighted that there were opportunities in this disorderly system to insert research evidence into decision making processes at the right time.
To maximise these opportunities, researchers need to understand the wider social and political systems in which these processes operate and utilise contextually specific knowledge to identify levers of influence. This includes acknowledging that there are multiple sub-cultures of evidence use within the council; each with their own policies and legislation. Understanding the timings of the political process (e.g. four yearly election cycles with peaks and troughs and different windows of policy making) and identifying trusted contacts in LG that could act as the ‘go-to-person’ for different cultures, were seen as essential pre-requisites for researchers.
Previous studies (35), (36) have highlighted the valuable role that boundary spanners and knowledge brokers can play in in translating research evidence by acting as ‘evidence champions’ (9) and ‘credible intermediaries’. In our study, we found that these roles already existed in various departments across LG; however, these roles and the people fulfilling them were underutilised and could be used more effectively in co-production evidence between academia and LG.
Moreover, political understanding and facilitated access to decision makers point to the concept of adaptive governance (27). In this case, co-production of research evidence for LG required relationship building between researchers and LG staff and capacity building for researchers to engage effectively in this process. Skills are required in political sensitivity, negotiating, influencing, persuasion, change management, problem solving, teamwork and leadership (37).
However, and as we have seen from examples in the Belltown and Castletown workshop, adaptive governance is not limited to researchers. What emerged from the workshops were very different cultures of evidence use in academia and in different departments in LG. Participants thought that academic research was geared towards academic outputs: with researchers prioritising papers in peer reviewed journals and large funding application to prestigious research programmes, for which local evaluations were often not a good fit. On the other side, LG professed to a culture of risk aversion not conducive to academic research, as negative evaluation findings could endanger future commissioning and funding of a service, and a prioritisation of front line delivery where time is of the essence and academic research often takes too long.
Collaborative spaces for reflection
Workshop participants kept turning back in the table discussions to a lack of collective shared spaces for reflection. In the daily rush to support frontline delivery of services with a lack of resources, participants complained that no time and space was available to look at their commissioning plans and reflect on desired outcomes and how best to deliver them, informed by research evidence.
According to participants, this problem was exacerbated by silo working within different government departments, each with their own policies and culture. This fragmentation is perpetuated by national government mandates around statutory deliverables and data collection, which structures different outcomes in different silos of LG (that are not always shared across departments). These silos will need to be integrated first before a joint-up research culture can be embedded in LG. Some participants suggested temporary suspension of statutory legislation within LG to enable this; however, this is unlikely to be realistic option, at least in the short term.
Therefore, participants were keen to create spaces for reflection within LG, not just in one department, as this would potentially sustain the existing silos, but by creating spaces between departments for collective reflection. These spaces were envisioned as group of people from different departments coming together on a regular basis to share knowledge and training, to reflect on the evidence and knowledge that they use, and to make connections between people for mobilising knowledge across LG. These spaces could also include wider collaborations with external partners, such as the police to discuss what they do and what evidence they use. Examples of these spaces can already be seen in some local community safety partnerships but this approach could be adopted wider across the council in other topic areas.