How knowledge brokering on implementation science can improve implementation outcomes: a case study of a public mental health project

Background Schools need mental health interventions that are deliverable, scalable and effective. However, rates of successful implementation of evidence-based practices in schools is low. An underpinning factor is lack of understanding, and attention, by commissioners, providers and schools, to evidence about what works to bolster effective and sustained implementation. Knowledge brokering is one way to improve use of implementation science for delivery of interventions in complex settings. To date, evidence for the use of knowleldge brokers in public health settings is scarce, and their use for brokering implementation related knowledge is non-existent. The aim of this study was to examine whether it was feasible and acceptable for knowledge about implementation to be brokered to a steering group responsible for a wide scale, public mental health, school-based intervention, and what impact this might have on their implementation decisions. Methods The primary researcher attended 13 monthly steering group meetings during the initial implementation phase of the public mental health program and shared implementation knowledge with them. With consent, meeting notes were taken by the primary researcher, and a journal kept of 12 meetings; 5 meetings were also audio recorded and steering group minutes analysed. All data were amalgamated and ordered into month-by-month summaries. The analysis attempted to identify when implementation knowledge was shared with the group (coded a ‘key moment’) and what the steering group did with this knowledge (coded a ‘key outcome’). Results Over the 13 meetings, 15 key moments led to 14 key outcomes, 10 of which involved implementation decisions being made based on brokered knowledge about implementation science. Discussion The majority of key moments led to a significant number of evidence-informed implementation decisions being made by the steering group. Some brokered knowledge was not acted upon. Implementation knowledge was not shared with the program funders which may have meant they did not appreciate the importance of implementation. Conclusion Knowledge brokering on implementation science to a group responsible for public mental health is feasible and acceptable and a potentially viable implementation strategy for future public health programs.

implementation decisions being made based on brokered knowledge about implementation science.
Discussion The majority of key moments led to a significant number of evidence-informed implementation decisions being made by the steering group. Some brokered knowledge was not acted upon. Implementation knowledge was not shared with the program funders which may have meant they did not appreciate the importance of implementation.
Conclusion Knowledge brokering on implementation science to a group responsible for public mental health is feasible and acceptable and a potentially viable implementation strategy for future public health programs. 3 Background There is increasing pressure for schools to deliver mental health programs in order to address the significant burden of youth mental health difficulties (1,2). Schools are promising sites for intervention as they circumvent usual barriers to support and have a wide reach (3). Yet, despite the availability of evidence based programs for mental health, there are significant challenges in implementing these in schools (4,5) and failures due to poor implementation are common (6)(7)(8)(9)(10).
EBPs for schools could be optimised if more attention is given to their implementation (6, 11) but often those responsible for implementing school-based programs lack awareness about the importance of considered implementation and the fact that there is evidence available to bolster it.
The failure to secure the potential positive outcomes from EBPs, and the failure to utilize evidence based implementation, falls under what Graham describes as a knowledge-to-action gap (KTA, (12).

Knowledge transfer
This gap can be addressed with good knowledge transfer (KT) between evidence producers and evidence users (13). Knowledge transfer is "an interactive interchange of knowledge between research users and researcher producers" (14)(p729). Regarding school use of EBPs, this definition of KT would suggest that schools should be able to incorporate an EBP, and its associated implementation protocol, in a straightforward way. However, in practice, KT into schools is often complex and challenging. Barriers have included poor engagement of school staff (15), lack of head teacher support (16,17), lack of organisational structure (including administrative support; Langley et al., 2010), low teacher engagement, high staff turnover, poor alignment of the intervention with the school's philosophy, goals, policies and programs (16), and overwhelming demands on staff (18,19).
School EBPs rarely include comprehensive implementation guidance. The result is very often incomplete implementation, limited sustainability and limited spread (7,20,21). The impact of such interventions on intended beneficiaries is thus significantly reduced (21)(22)(23)(24). There is also an economic cost to consider, Romney, Israel, and Zlatevski (2015) demonstrated that within community centres responsible for implementing an evidence-based public health program, the average cost per graduate was over seven times higher for the community centres which had not prepared for 4 implementation.

Knowledge brokers
Given this, some KT researchers have recommended the use of an intermediary known as a knowledge broker (KB) to support use of implementation knowledge (and not just knowledge about interventions themselves) by research users (25,26). Evidence shows that KBs can successfully mediate between researchers and intended users to increase their understanding of each other's language and break down barriers to the use of evidence (27).
A review by Elueze (2015) concluded that knowledge brokering is an effective way to promote evidence-based decision-making, evidence-based practice and / or collaboration between researchers, health practitioners and policymakers. Bornbaum et al.'s (2015) systematic review found that KBs can ensure ongoing interaction between stakeholders, build an understanding of stakeholders' goals and contexts over time, identify challenges, barriers to information flow and particular areas of concern, and speed up how quickly relevant knowledge is identified, evaluated and translated into practice and/or policy.
Despite their potential to address the important KTA gap, evidence for the use of KBs in public health settings such as schools is scarce. Traynor et al.'s (2014) RCT explored the effectiveness of KT strategies of varying intensities for promoting the use of research evidence in decisions related to childhood obesity prevention by public health practitioners in 30 Canadian health departments.
Results indicated that practitioners who worked closely with the KB showed a significant change in their ability to find, evaluate and apply research evidence to practice-based issues (average increase of 2.8 points along a scale of 36 points (95% CI 2.0 to 3.6, P < 0.001)). These participants also showed a significant increase of 49% in evidence-informed decision-making (EIDM) behaviours from baseline.
KBs, schools and sharing implementation knowledge 5 To date, the application of KB initiatives in education is a relatively new phenomenon and a number of knowledge brokering initiatives have begun contributing to the process of linking research based knowledge to decisions in policy and practice (40,41). There has been little examination of whether a KB could support the implementation of an EBP by brokering implementation science knowledge. Use of KBs could represent a promising KT strategy to narrow the KTA gap on implementation. Studies are needed to examine how far KBs can facilitate the successful implementation of EBPs in schools.

Steering groups
Many stakeholders are involved in delivering public health programs, i.e. commissioners, program deliverers, the target population. Knowing which group of stakeholders might be best to target with an implementation KB is currently unknown. Head teachers or certain school staff may be appropriate; however, at a regional or organizational level it may be more effective to work with intermediary teams such as steering groups. Key decision makers in public health programs sometimes form a steering group (SG), defined as "a group of people who are chosen to direct the way something is dealt with" (dictionary.cambridge.org). It is not uncommon for public health interventions to have a SG who have responsibility for implementing an intervention, e.g. the WHO's Health Literacy Program (http://www.healthliteracy.org.uk/index.php/steering-group). If a SG responsible for delivering a public health program could respond well to a KB sharing implementation knowledge, implementation success could be improved leading to more of the public benefitting and a potential reduction in financial wastage.

Study aims
In the UK, a large county had received funding to trial an 8 week mindfulness program in 300+ primary and secondary schools. A SG was established to oversee the implementation and evaluation of the program. However, the program came with only a basic implementation protocol, namely to train teachers in mindfulness and then train young people. The SG's situation thus mirrors that of many project teams and SGs trying to apply EBPs into schools and were therefore considered a good test case to examine the potential impact of an implementation science KB. The present study aimed to determine whether knowledge about implementation could be brokered to a SG responsible for a wide scale, public mental health, school-based intervention, and what impact this might have on the 6 SG's implementation decisions.

Design
The primary researcher (KH) was an initial member of the SG (as an expert in mindfulness) but was later invited to also act as an implementation KB, given his knowledge in this area. The study was therefore semi-ethnographic as KH was an active SG participant and immersed in their culture (42) while at the same time was systematically observing implementation developments within this context, carrying out participant observations and recording interactions with the SG members both in and outside of meetings.

Ethics and recruitment
Ethical approval was obtained from the University of Leeds Faculty of Medicine and Health (School of Psychology) Research Ethics Committee (reference: . Participants consented to KH taking the role of KB and to steering group meetings being recorded. All six members of the SG agreed to take part. The sample, therefore, included a public health consultant, a local GP, two local head teachers, a mindfulness trainer, and an emotional resilience project officer from the funders (See Table 1 in supplementary information for details of SG members' job, role in the group, gender and reason for joining the SG). The SG members had no prior knowledge or experience of implementation science. The project's funder required that all SG actions be evidenced based and this was a prerequisite to obtain future funding. This meant the SG were particularly motivated to receive and use evidence-based knowledge.
Participant identities were anonymised, but their job titles were not. The personal details of participants were stored securely and separately to study data.

Knowledge broker
At the time of appointment to the KB role, KH had extensive knowledge of mindfulness interventions and implementation science having conducted relevant research in schools, hospitals and the community. Their practical experience of working in healthcare settings, working in teams, good research knowledge of schools, and research skills put them in a good position to carry out the KB role effectively, although this was their first experience of knowledge brokering. The KB's focus was 7 on promoting findings from implementation science research to the SG to support them in making evidence-based decisions regarding their implementation strategy for a whole school approach to mindfulness. During each SG meeting, the KB was given a time slot to share knowledge with the group which was relevant to their implementation decisions or to answer questions they had regarding implementation (some asked previously, some asked in the present moment). The SG members' questions on implementation often gave the KB insight into what knowledge to share in the next meeting to ensure they could make the best implementation decisions possible. Sometimes this was knowledge the KB had already, and sometimes the KB needed to identify or review evidence.

Data collection
The principal aim of the study was to explore whether knowledge about effective implementation could be shared with the SG and if and how it impacted their implementation decisions. KH attended 13 monthly SG meetings over 14 months, beginning at the start of mindfulness interventions implementation. At this point, the project was in a test and learn phase, with the potential of further funding to upscale the intervention the funders (a large national charity) decide it was a good investment. There is no standardised taxonomy of KB activities. KB activities very often depend on, and are dictated by, stakeholders' needs which are themselves often unanticipated and in the moment (43). Thus, a pragmatic approach to data collection was taken.
Notes were taken, and a journal kept of all SG meetings (n = 13; each approximately 3 hours long), and 5 of these 3 hour meetings were audio recorded (as soon as ethical approval and individual consent was secured). Also with consent, all other interactions with the SG (telephone calls (n = 19) and e-mails (n = 27)) were logged, noting conversations, issues discussed, if any implementation knowledge was shared and/or reportedly adopted. The researcher also visited some of the target schools and interviewed a sample of school staff to learn more about their perspective on implementation challenges (reported in Hudson et al (submitted)).

Analysis
The journal, meeting notes, communication logs and audio recordings constituted the study data. To expedite analysis in a time constrained project, audio recordings were worked from directly during analysis. Data from all four sources was amalgamated and ordered into month-by-month summaries 8 of KB and SG activity detailing who was present, what was discussed, what implementation information was shared and what happened as a result (based on reports at subsequent meetings; see supplementary material for an example of a meeting summary). These summaries (n = 13) were then subjected to line by line analysis. The analytic intention was to identify (i) when implementation knowledge was shared with the SG (coded as a 'key moment'); when the SG acted on this knowledge (coded as a 'key outcome'). This analysis was a pragmatic way to address the study aim of understanding whether KB could occur, and with what effect, in a SG. A random sample of 20% of the summaries (totaling 8 pages) were subjected to independent coding by one other person and the coding showed agreement on all coding of key moment and key outcomes. Key moments, and key outcomes are explained in detail below.

Findings
15 key moments occurred over the course of the study and led to 14 key outcomes as shown in Figure 1. It can be seen that the majority of key moments (n = 12 of 15) led to affirmative key outcomes (n = 10 of 14) where the SG used the knowledge shared to make evidence-based decisions on how the intervention should be implemented. There were four key outcomes where the SG did not take into account the implementation evidence shared with them at three key moments. Figure 2 shows that key moments occurred regularly up until meeting 9. Key outcomes began to occur in November 2015, three months after the KB had joined the SG, and continued up until November 2016 when the SG did not receive funding to continue. At this point, the KB stopped attending meetings. The SG were shown just one part of the guide -an implementation rubric which included a series of ten steps that made up a full implementation cycle as well as six sustainability factors that were essential to high quality, sustainable implementation (See supporting materials for more information).
They found that two key sets of activities (10 implementation steps and 6 sustainability factors) combined with essential elements of effective leadership, were vital to effective SEL implementation and sustainability. In the present study, any references to SEL programs were replaced with references to MT. The SG were invited to see if they could use these steps to help them implement MT 10 into schools.
The SG were also introduced to the idea of the 'readiness' of schools and the 'readiness of individuals' (Key Moment 10). This referred to how schools' and individuals' motivation and capacity to adopt a mental health program affected its implementation success (Scaccia et al., 2015). The group were then advised to use only evidenced-based forms of MT, and that unlike MT delivered in groups, there was no evidence that one-to-one sessions of MT in schools could improve the wellbeing of young people (Moment 11). They had planned to include one-to-one MT in their offer, but they were advised that evidenced-based programs are likely to lead to better outcomes than non-evidence-based ones What were the key outcomes?
All of the key moments led to key outcomes. Moments 1-6 were responsible for bringing implementation into the group as a core topic of conversation and ultimately ensured the group decided to make 'getting implementation right', a key aim of the mindfulness project (Key Outcome 1). This outcome itself led to Key Outcome 9 where the SG began to talk about incorporating various aspects of implementation science into their phase 3 bid going forward. This, in turn, led to Key Outcome 10, where they applied for funding to ensure there were 'local implementation coordinators' in phase 3, whose primary focus would be on making sure MT was implemented successfully in the schools. Halfway through the KB process, the SG went on to develop their 'implementation guidance for headteachers' which they planned to distribute to schools during phase 3 (Key Outcome 11). It talked about 'making it happen' (Key Moment 2) and encouraged schools to think about various aspects of implementation, create an action plan and agree to a number of implementation related goals in order to decide if they were 'ready' to implement an offer of MT (Key Moment 10). The document was informed by the CASEL framework.
Key outcomes 10 and 11 were also inspired by the CASEL framework (Key moment 9) which, prior to the development of the 'implementation guidance for headteachers', had led to the SG using the CASEL framework as a reference point in meetings (Key Outcome 2). The CASEL framework also influenced how the SG approached schools, e.g. by trying to achieve a high level of leadership engagement (Key Moment 7). Learning about the 'readiness of schools' (Key Moment 10) meant the SG began offering MT only to schools deemed 'ready for MT' (Key Outcome 14). Making their implementation strategy more selective in this way, was also driven by the KB's shared research findings which found school readiness to be a key distinguishing determinant of implementation success between schools, and the CFIR framework which highlights organisational readiness as a key driver of implementation (Key Moment 15). Another key outcome was the removal of the need for schools to sign a memorandum of agreement when offered MT (Key Outcome 7) which meant schools had to implement MT exactly as the SG instructed. This was a result of Key Moment 14 where the SG learnt that allowing schools some adaptations to 'fit' mindfulness into the school may increase buy-in, ownership and enhanced 'fit'.
A number of key moments had no impact on the SG's implementation related decisions. Interestingly, the SG went ahead and included one-to-one MT in its phase 3 bid (Key Outcome 5) despite them being informed that there was no evidence of its effectiveness (Key Moment 11). They also never incorporated any measures to monitor implementation across the schools (Key Moment 12). Although the funders of the mindfulness project, seemed to acknowledge the value of a KB and wanted more KBs in their other mental health projects (Key Outcome 13), the SG perceived that the funders did not fully understand the value of the SG's decision to make getting implementation right a priority. This perception arose after two members of the SG met with the funder's representatives to discuss phase 3 funding. Some SG members felt the funder had not understood the value of getting implementation right in the meeting. The SG believed this might have been the main reason why they did not obtain phase 3 funding (Key Outcome 12). The funders decision to not provide phase 3 funding had little impact on the KB process as the SG were keen to find ways to continue, and were keen to receive ongoing help from the KB. Finally, despite being informed about the dangers to implementation posed by offering multiple interventions at once (Key Outcome 3), the funders insisted that schools be offered, and be encouraged to use, multiple interventions at the same time (Key Outcome 4).

Discussion
This study aimed to explore whether knowledge about implementation could be brokered to a SG responsible for a wide scale, public mental health, school-based intervention, and what impact this might have on the SG's implementation decisions. Over 14 months, the SG made 10 evidenced-based decisions in response to the implementation knowledge that was shared with them. Findings show that, over an 8 month period, KB on implementation occurred 15 times, and secured 10 positive outcomes for program implementation. The approach used to broker knowledge was feasible, acceptable and seemed to work well as a method. The SG was able to retain the implementation science knowledge brokered to them and ensure the project encompassed implementation science practices and principles going forward. They did this by incorporating these ideas into their phase 3 bid. These findings echo Traynor et al.'s (2014) which also found participants who had received 13 knowledge from a KB exhibited a significant change in knowledge and skill as well as greater EIDM.
Findings show that the KT process occurred in two stages. In the first stage, the SG came to understand and prioritise implementation, and in their second stage, the SG learned how to apply implementation science to their project. These processes took The majority of key moments led to a significant number of evidence-informed implementation decisions being made by the SG. They perceived the KB to be a legitimate source of information.
There was also an active research culture amongst the SG with members including a public health consultant and GP who were familiar with scientific research studies and journals. This familiarity meant it was not difficult for key members of the group to understand and relate to the scientific knowledge being shared with them. Researchers and intended users often come from very different cultures, creating a gap or "semantic distance" which can weaken the broker-recipient interactions and lessen the KB's effects on EIDM (35,44). In the SG, there was no evidence of this being the case, but in some SGs and schools, the translation of research may not be so easy. In circumstances such as these, it will become increasingly important that the KB is able to tailor knowledge in a way that the stakeholders find acceptable.
The fact that some knowledge shared by the KB was rejected (e.g. advice not to implement one-toone mindfulness training as it is not an EBP) may have been because the SG was passionate about mindfulness and believed in its ability to help young people become more resilient to mental health problems. This posed a challenge to EIDM, but likely represents how many project teams feel about the interventions they are trying to deliver.
That the SG also failed to act on the KB advice to establish formal evaluations of implementation success seemed related to limited resources. Well-specified and evidence-based measures of 14 implementation are rare in public health programs (45) and generating evaluation measures of implementation progress would require a significant amount of time. Thus, brokered knowledge can sometimes be relegated due to practical project constraints. Furthermore, the SG was unable to persuade the funders to encourage schools to disseminate one program offer at a time, representing another challenge to the use of KB, namely that decision making often includes other stakeholders who do not hold the same implementation knowledge. Dagenais et al.'s (2015) critical analysis of a qualitative knowledge brokering study concluded that interpersonal contact is an essential condition for effective KB interventions and it has been suggested that KB interpersonal contact improves the likelihood of behaviour change (35,46). In their systematic review, Bornbaum et al. (2015) found that maintaining a physical presence among stakeholders, facilitating relationships among them, encouraging teamwork and facilitating interactions was found to be useful to promoting the KT process in health-related settings. Although the KB was immersed in the SG, implementation knowledge was not brokered to funding decisionmakers. This may have contributed to the SG not receiving further funding (Key Outcome 12).
The program funder's main priority was sound investment for clear outcomes. In theory, making evidence-informed implementation decisions should improve the outcomes of a program like MT.
Without implementation, the cost benefit is incalculable, because to invest in a programme without follow-through is, according to implementation science research, likely to be wholly wasteful.
However, it was never clear whether the funders fully understood this point of long term reductions in cost.
Some researchers have pointed to the 'dark side' of KB (47) and suggest it may be negatively affected by some tensions stemming from the inherent complexity of knowledge and brokering and from the 'in-between' position occupied by KB's. KBs take up a position where they are not officially part of the group they are sharing knowledge with but also not completely separate and external to that group. Stakeholders can find it challenging to trust and accept them and KBs can receive hostility and skepticism from other professionals, feeling isolated and incompetent as a result (47),(48). The KB in this study never felt incompetent, but did receive some hostility and scepticism from the 15 funder's representative in meeting 7 when encouraging schools to implement one program at a time.
The funder's representative believed introducing multiple programs was likely to lead to better outcomes.
The KB's position was the most junior in the SG. This meant none of the other SG members had their decision-making powers challenged by them. Had the KB's status been higher, they might have encountered power struggles and more resistance from the SG to the KT process.
The KB also felt isolated from decisions at times around the phase 3 bid, that were mainly made outside the group by the funder representative and the public health consultant. Implementation science was included in the phase 3 bid without much involvement from the KB. It may have been that the funder's representative and the public health consultant did not explain the importance of implementation very well, or that they simply could not convey its importance to the funder as a whole.

Study Evaluation
The findings of this study should be considered within its limitations. The study could have recorded more precise indicators that the SG had understood the knowledge being shared with them. Use of specific evaluative questions (rather than meeting notes for example) might have helped clarify the extent of the SG's understanding. However, we decided not to ask the SG to complete an exit questionnaire or interview because the main researcher was researching their own input which created a potential for bias, and was a further weakness of this study. The SG might have been positively biased in their answers to an exit questionnaire as they had built up a good interpersonal relationship with the primary researcher and they may not have wanted to paint the KB in a negative light. There was also a risk of bias from the KB as the research was not independent. Had an independent researcher conducted the interviews in this study and analysed the results, the findings may have been different. To lower the risk of bias in the analysis process, data was checked by a third party for inter-rater reliability.
KT may have been more effective had the KB brokered knowledge directly to the funder and been independent of the research process.

Future studies
Creating teams of KBs may be of value. Kislov et al. (2016) suggest that creating isolated knowledge brokering roles is not enough to produce a long-term sustainable impact on the KTA gap. The authors suggest shifting from individual KBs to collective KBs where multi-professional teams of KBs are supported at the organisational and policy level in order to impact the KTA gap.
A standard criterion for evaluating the effect of KB on KT processes is also needed. A number of researchers have begun making promising attempts to create standard criteria of evaluating KBs (49-

51)
Feasibility and cost-effectiveness data on KBs as a KT strategy are also needed. Knowing about the potential cost savings a KB can bring may motivate public health organisations to use them (35).

Conclusions
Our findings show that KB on implementation can secure better implementation decisions by a project SG. These findings, therefore, offer a possible KT strategy within public health settings. To promote the use of implementation science in public health initiatives, KBs may need to broker knowledge to commissioners and funders as well as SGs.

Consent for Publication
Not applicable Availability of data and material The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Competing interests
The authors declare that they have no competing interests.