Building Capacity for Implementation – The KT Challenge

doi:10.21203/rs.3.rs-150988/v1

Download PDF

Short report

Building Capacity for Implementation – The KT Challenge

https://doi.org/10.21203/rs.3.rs-150988/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 28 Jul, 2021

Read the published version in Implementation Science Communications →

You are reading this latest preprint version

Background

The KT Challenge program supports health care professionals to effectively implement evidence-based practices. Unlike other knowledge translation (KT) programs, this program is grounded in capacity building, focuses on health care professionals (HCPs), and uses a multi-component intervention. This study presents the evaluation of the KT Challenge program to assess the impact on uptake, KT capacity and practice change.

Methods

The evaluation used a mixed-methods, time-series design involving surveys and review of documents such as teams’ final reports. Online surveys collecting both quantitative and qualitative data were deployed at four time points (after both workshops, six months into implementation, and at the end of the two-year funded projects) to measure KT capacity ( knowledge, skills and confidence), and impact on practice change. Qualitative data was analyzed using a general inductive approach and quantitative data was analysed using non-parametric statistics.

Results

Participants reported statistically significant increases in knowledge and confidence across both workshops, at the 6-month mark of their projects, and at the end of their projects. In addition, at the 6-month check-in, practitioners reported statistically significant improvements in their ability to implement practice changes. In the first cohort of the program, of the teams who were able to complete their projects, half were able to show demonstrable practice changes.

Conclusions

The KT Challenge was successful in improving the capacity of HCPs to implement evidence-based practice changes and has begun to show demonstrable improvements in a number of practice areas. The program is relevant to a variety of HCPs working in diverse practice settings and is relatively inexpensive to implement. Like all practice improvement programs in health care settings, a number of challenges emerged stemming from the high turnover of staff and the limited capacity of some practitioners to take on anything beyond direct patient care. Efforts to address these challenges have been added to subsequent cohorts of the program and ongoing evaluation will examine if they are successful. The KT Challenge program has continued to garner great interest among practitioners, even in the midst of dealing with the COVID-19 pandemic, and shows promise for organizations looking for better ways to mobilize knowledge to improve patient care and empower staff. This study contributes to the implementation science literature by providing a description and evaluation of a new model for embedding KT practice skills in health care settings.

Decision Sciences

Scientific Communication

Capacity building

Evidence-based practice

Knowledge translation

Health care professionals

Implementation science

Disseminates information about a promising and innovative KT capacity-building program/model
Characterizes unique elements of the program including its focus on health care professionals who have responsibility for implementing practice changes, a multi-component support model, low cost, and applicability to a variety of professionals in diverse practice settings.
Provides evidence from early evaluation of improvements in knowledge, confidence, and ability to implement practice change

There is an urgent need to reduce the gap between the creation of evidence and its implementation into practice. However, alarmingly, up to 70% of all organizational improvement efforts fail (1). To address this shortcoming, knowledge translation (KT) initiatives and training programs have proliferated in recent years (2, 3, 4). Key barriers to successful KT include lack of knowledge, skills, time and leaders’ endorsement, and competing priorities of organizations (5, 6, 7). Important enablers include health care professionals’ (HCPs) positive beliefs about the benefits of participation in training programs, combined with expert guidance and organizational support (8). HCPs want to provide the best care possible, but many lack the knowledge and skills to effectively move evidence into practice. A 2012 survey of health care providers, administrators and researchers revealed that nearly 80% of respondents wanted to improve their knowledge and skills related to implementation (7). Moreover, research has demonstrated that high-quality implementation strategies are associated with successful implementation (2), thus supporting the need to offer KT and implementation capacity building, training and support to clinicians. A recent systematic review stressed the importance of increasing the number and reach of training opportunities to address the lack of dissemination and implementation training options (9).

Providing KT training takes dedicated effort from health care organizations. Much of the research conducted on how to close the evidence-to-practice gap has been led by and/or focused on academic researchers with minimal involvement from those who provide or use health services (10). First-generation interventions used co-creation models. For example, a program in the United Kingdom used a partnership approach to KT and reported that implementation projects co-produced by researchers and end users were more likely to successfully adapt new evidence into practice, and the co-production experience encouraged future collaboration between the parties (10). Researchers and health care leaders have called for training that builds capacity amongst those working at the point of care (11, 12, 13), who often have the responsibility to implement, but do not have adequate skills and support to do so effectively. Involving those who use research knowledge in efforts to implement it has multiple benefits, including enhancing the effectiveness of implementation efforts (10). Eames and colleagues, for example, found changes in clinician-reported behaviour, especially use of strategies for implementing a change in practice, following participation in their KT capacity-building intervention for occupational therapists, as well as changes in the culture to one in which clinicians engaged in KT as part of their clinical practice (6). Similarly, a longitudinal evaluation of a program for implementers, called Practicing KT, showed increased use of, knowledge of, and self-efficacy in KT among those who completed the program (5).

Many programs exist to increase KT capacity, including some that offer training in implementation skills, but there is lack of information on their effectiveness. Programs that use active forms of learning to promote the acquisition of KT skills have been shown to be effective in transferring knowledge, but there is scant evidence of sustained practice change resulting from these sessions (2, 5, 14, 15). KT scholars have acknowledged the need for longitudinal evaluation of KT capacity-building initiatives to assess sustainability of outcomes from these programs (5, 11).

Armed with these findings, we developed an implementation support program for HCPs, aimed at building capacity to move research evidence into practice. The program, called the “KT Challenge,” was modeled on a similar research capacity-building initiative for clinicians. That program, in place at our organizations for more than 10 years, has been shown to be effective in supporting practice changes, enhancing evidence-based practice and increasing interest in research engagement, including increased interest in graduate school (16, 17). The KT Challenge program was implemented at two health organizations in British Columbia, Canada, and offered to all HCPs employed at the organizations. We report an evaluation of the KT Challenge program with respect to uptake by HCPs, impact on KT capacity (knowledge, skills, and confidence) of HCPs, and impact on evidence-based practice changes.

Overview of the KT Challenge program

The KT Challenge is an implementation support program that involves training, funding, peer review, and mentorship. The key components of the KT Challenge program are briefly described below.

Letter of Intent (LOI):

In the LOI, teams identify the practice change they want to implement, document the need for this change in their practice context, and summarize evidence of its effectiveness. LOIs also require the signature of a manager to ensure management support and endorsement, and the identification of team members. The LOIs are formatively reviewed and, in keeping with the capacity-building approach of this program, revisions are suggested when required. Some teams are screened out of the program at this point if their proposed LOI does not identify an evidence-based practice change but most teams are given feedback and invited into the next stage.

Workshops:

Teams attend two half-day workshops, focusing on developing an implementation plan, and evaluating the effectiveness of the implementation of the practice change. Topics include implementation theories and frameworks, stakeholder engagement, identifying barriers and facilitators, implementation strategies, and evaluation planning.

Mentorship:

Teams are supported to find a mentor within their clinical area to support implementation of the practice change and assist with navigating facilitators and barriers. Mentors are also invited to participate in the evaluation of the KT Challenge program throughout the project.

Online resources:

Teams have access to an online learning site where a curated set of readings and resources is posted.

Management support:

Teams are required to obtain their manager’s support for conducting the implementation project and dedicating time to it.

Funding:

Successful teams are awarded $5,000 to cover costs related to personnel, materials and supplies, stakeholder engagement, and services.

Program Leads:

The KT Challenge is run by a program lead from each organization. The leads promote the program, coordinate the review processes, monitor team progress, support teams to successfully navigate barriers, and lead the evaluation of the program.

Figure 1 outlines the timeline of activities for each cohort. Funded teams are asked to submit a quarterly report to provide an update on their progress, as well as a final report on the impact of their project on the intended practice changes.

Evaluation Plan

To evaluate the KT Challenge, we used a time-series, mixed-method design to assess the impact of the program on workshop participants and funded team members. The evaluation focused on the program uptake and changes in KT capacity (knowledge, skills, and confidence) among HCPs, and the impact on practice change. Online surveys were used to collect data at four time points: after each of the two workshops, 6 months into implementation, and at the end of their two-year funding. Only team leaders and mentors of funded teams complete the 6-month and end-of-project surveys. The online surveys collect both quantitative and qualitative data on participants’ knowledge, skills, and confidence. Data on the impact of the practice change is gleaned from the final reports submitted by the team leads.

Data Analysis

The qualitative data were analyzed using a general inductive approach, which involved the identification of themes and sub-themes. The quantitative data were analyzed in Excel (Microsoft Corporation), using the Wilcoxon Signed-Ranks Test to compare pre and post scores on self-ratings of knowledge, skills, confidence, and ability. Statistical significance was set at p < 0.05.

Uptake of KT Challenge by practitioners

Four cohorts of participants have taken part in the program, with a fifth cohort recently launched. The first cohort (2016) has completed their projects. To date, 24 teams have been funded, comprising 185 HCPs. Participants have included a wide range of HCPs involving 23 types of practitioners working within a range of practice settings. (See Figure 2).

Interest in the program has remained steady, with 13 teams participating in Cohort 3 (6 were funded). While 9 teams initially joined Cohort 4, the number of teams dropped significantly when the COVID-19 pandemic struck, and only 2 teams submitted funding proposals. However, Cohort 5 was launched in October 2020, and 13 teams applied to participate, including teams proposing COVID-related projects.

KT Capacity

The average survey response rate from the three cohorts was 76% for post-workshop #1 survey, 59% for post-workshop #2 survey, and 100% for both the 6-month check-in survey and the end-of-project survey.

Participants reported statistically significant increases in knowledge and confidence at the 6-month mark of their projects, compared with before the initiative (p < 0.05). In addition, at the 6-month check-in, practitioners reported statistically significant increases in their ability to implement practice changes.

Numerous comments were provided on the surveys and in the final reports attesting to the knowledge, skills, and confidence acquired to effectively support implementation:

“Thinking about addressing barriers and not simply providing MORE information and MORE education has stuck with me and has impacted how I approach other initiatives and projects.”

“Changing how I think about implementing or addressing any practice change. I am constantly thinking about what REAL barriers to change might be.”

Impact on Practice Change

Data on the impact of the program on practice changes is available from the first cohort only, as funded projects in subsequent cohorts are still in progress. Eight teams were funded in the first cohort, but two were unable to complete their projects due to management changes and shifting priorities within their departments. Of the remaining six teams, three showed demonstrable practice changes across their respective practice areas, giving the program a 50% success rate:

In a cardiac inpatient unit, depression screening increased from none to 75%, and follow-up discussions with primary care providers were recorded at 36%.
In a tertiary hospital setting, malnutrition screening increased by 50%.
In a hospital-based physical rehabilitation program, 67% of physical therapists and occupational therapists reported increased uptake of the Canadian Stroke Best Practice Recommendations.

Three teams completed their projects but were not able to demonstrate measurable practice change. Cohort 2 (n = 9 teams) and Cohort 3 (n = 6 teams) are currently completing their projects, with promising results, including one team that has significantly increased the percentage of patients with spinal cord injury receiving functional electrical stimulation treatment. Another team has garnered strong interest in their topic – treating problematic methamphetamine use – and was approached by local healthcare policymakers as well as staff from the United States Office of National Drug Control Policy, wanting information on their implementation plan.

Reported Challenges

Despite the positive impacts on practitioners’ KT capacity and demonstrable practice changes, many teams experienced challenges, common to most KT programs (5,6,15), and including:

Team member turnover
Manager turnover
Gaps in communication with mentor
Projects taking more time than anticipated, and
Lack of support from key stakeholders for the practice change.

Steps were taken in response to these challenges, as outlined below.

The evaluation of the KT Challenge program demonstrates that a multi-faceted implementation support program for HCPs is effective at moving evidence into practice and can be conducted within a modest budget ($5,000 per team). The steady numbers of applicants for the KT Challenge program indicate sustained interest, including in the most recent cohort (October 2020), in the midst of the global pandemic, 13 teams submitting LOIs to join the program.

Among challenges faced by teams, lack of time, changes in personnel, and shifting work priorities were the most commonly cited. The following steps were taken to improve the program for future cohorts:

Program leads provide feedback and suggestions to teams in response to their quarterly reports
Workshops and program documents were revised to specifically highlight the need for the project evaluation to collect data on the uptake of the practice changes
Program leads undertook more promotion of the program within their organizations to strengthen management’s commitment to the funded teams.

However, even the teams who were unable to complete, or whose projects did not lead to measurable practice change, agreed that participation in the program was beneficial. We had hoped to see all projects result in demonstrable practice change arising from their interventions, but given the many barriers to practice improvements, it is unreasonable to expect 100% success. As Durlak and Dupre have noted, “Expecting perfect or near-perfect implementation is unrealistic. Positive results have often been obtained with levels around 60%...no study has documented 100% implementation for all providers.” (18, p. 331). In comparison, 50% of our completed teams reported evidence-based practice changes, demonstrating the effectiveness of the KT Challenge program. Furthermore, the 185 HCPs who participated in the program gained knowledge and confidence in KT practice skills, as well as improved ability to implement practice change.

We revised the KT Challenge program to enhance the relevance of the projects to patients and families, and to improve dissemination of the findings to audiences that include patents and families. Beginning with the 2019-2020 cohort, all KT Challenge teams were required to include a Patient-Family Partner on their implementation team, and all funding proposals are now reviewed by a panel that includes Patient-Family Partners.

Our evaluation shows that the KT Challenge has been very effective at increasing capacity in KT skills and has provided information on areas where the program can be improved. We have carefully reviewed evaluation comments from each cohort in the KT Challenge and have made adjustments to the program based on feedback from participants, offering progressively improved support with each cohort. We hope this support will lead to a higher percentage of teams demonstrating measurable practice changes.

The KT Challenge program is a promising strategy that can be applied across a variety of clinical settings to support the effective uptake of evidence-based practice. This evaluation demonstrates that clinicians will respond to opportunities for KT training, that enhanced capacity for KT skills is achievable with support, and – most importantly – that successful practice change can result from small-scale, mentored and funded KT projects in clinical practice settings. This evaluation study contributes to the implementation science literature by providing a description of a new and effective model for embedding KT practice skills in health care settings.

HCP Health care professional

KT Knowledge Translation

LOI Letter of Intent

OT occupational therapist

PT physiotherapist

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Competing Interests

The authors declare that they have no competing interests.

Funding

The KT Challenge program is supported by Providence Health Care (PHC) Research Institute and the PHC Professional Practice Office, and Vancouver Coastal Health Research Institute and the Robert H. N. Ho Enhancing Patient Care Fund.

Authors’ Contributions

AB and MS contributed to the overall KT Challenge program design and evaluation. AB and AC oversaw the program management and operations at their respective health organization sites. MS developed the workshop content, delivered the presentations, developed the evaluation, and consulted with the teams. KC, AH, JK, AL, MM, AS, and MES formed the program advisory committee to review project proposals, and inform program decisions. All authors contributed to writing and revising the manuscript.

Acknowledgements

We would like to acknowledge the 185 HCPs who have taken part in the KT Challenge, the numerous Patient-Family Partners who have advised teams and reviewed proposals, and the leaders in our organizations who provide funding for the KT Challenge program

National Health Service Sustainability Model. Downloaded 24 Sept 2020 from: https://webarchive.nationalarchives.gov.uk/20160805122935/http://www.nhsiq.nhs.uk/media/2757778/nhs_sustainability_model_-_february_2010_1_.pdf ;2010, ISBN: 978-1-907045-36-3
Davis D, Evans M, Jadad A, Perrier L, Rath D, Ryan D, et al. The case for KT: shortening the journey from evidence to effect. BMJ; 2003; 327:33e5.
Madon T, Hofman KJ, Kupfer L, Glass RI. Public health. Implementation science. Science; 2007; 318:1728e9.
McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, et al. The quality of health care delivered to adults in the US. N Engl J Med; 2003; 348:2635e45
Moore J, Rashid S, Park J, Khan S, and Straus S. Longitudinal Evaluation of a Course to Build Core Competencies in Implementation Practice. Implementation Science. 2018; 13:106.
Eames S, Bennett S, Whitehead M, Fleming J, Low, Shanling O, Mickan S, Caldwell E. A pre‐post evaluation of a knowledge translation capacity‐building intervention. Australian Occupational Therapy Journal; 2018; Vol 65(6).
The Michael Smith Foundation for Health Research. Training and resources to support research use: a BC needs assessment; 2012.(downloaded 31 Aug 2020 from https://www.msfhr.org/sites/default/files/KT_Needs_Assessment_FINAL.pdf
Wenke R, Noble C, Weir KA, et al. What influences allied health clinician participation in research in the public hospital setting: a qualitative theory-informed approach. BMJ Open 2020;10:e036183. doi: 10.1136/bmjopen-2019-036183
Davis R and D’Lima. D. Building capacity in dissemination and implementation science: a systematic review of the academic literature on teaching and training initiatives. Implementation Science; 2020;15:97; https://doi.org/10.1186/s13012-020-01051-6).
Heaton J, Day J, Britten N. Inside the 'black box' of a knowledge translation program in applied health research. Qualitative Health Research; 2015; Vol 25(11).
Park JS, Moore JE, Sayal R, Holmes BJ, Scarrow G, Graham ID, Jeffs L, Timmings C, Rashid S, Johnson AM, Straus SE. Evaluation of the "Foundations in Knowledge Translation" training initiative: preparing end users to practice KT. Implement Sci; 2018; ISSN: 1748-5908, Vol. 13 (1).
Thomson D, Brooks S, Nuspl M, Hartling L. Programme theory development and formative evaluation of a provincial knowledge translation unit. Health Res Policy Syst; 2019; ISSN: 1478-4505, Vol. 17 (1). PMID: 30971263.
Bennett S, Whitehead M, Eames S, Fleming J, Low S, Caldwell E. Building capacity for knowledge translation in occupational therapy: learning through participatory action research. BMC Med Educ; 2016; ISSN: 1472-6920, Vol. 16 (1).
Goodenough B, Fleming R, Young M, Burns K, Jones C, Forbes F. Raising awareness of research evidence among health professionals delivering dementia care: Are knowledge translation workshops useful? Gerontology & Geriatrics Education; 2017; Vol 38(4).
Gaglardi A, Berta W, Kothari A, Boyko J, Urquhart R. Integrated knowledge translation (IKT) in health care: a scoping review. Implement Sci; 2016; 11:38.
1. DOI 10.1186/s13012-016-0399-1.
Black A, Ali S, Baumbusch J, McNamee K, Mackay M. Practice‐Based Nursing Research: Evaluation of clinical and professional impacts from a research training program. Journal of Clinical Nursing; 2019; 1:7. DOI: 10.1111/jocn.14861.
Black A, Balneaves L, Garossino C, Puyat J, Qian H. Promoting Evidence-Based Practice through a Research Training Program for Point-of-Care Clinicians. Journal of Nursing Administration: 2015; 45(1).
Durlak, J, Dupre E. Implementation Matters: A Review of Research on the Inﬂuence of Implementation on Program Outcomes and the Factors Affecting Implementation. Am J Community Psychol; 2008; 41:327–350 DOI 10.1007/s10464-008-9165-0

StaRIchecklistBlackEtAl.docx

Download PDF

Journal Publication

published 28 Jul, 2021

Read the published version in Implementation Science Communications →

Editorial decision: Major revision
24 May, 2021
Review #2 received at journal
17 Feb, 2021
Review #1 received at journal
17 Feb, 2021
Reviewer #2 agreed at journal
30 Jan, 2021
Reviewers invited by journal
26 Jan, 2021
Reviewer #1 agreed at journal
26 Jan, 2021
Editor assigned by journal
18 Jan, 2021
Submission checks completed at journal
18 Jan, 2021
Editor invited by journal
18 Jan, 2021
First submitted to journal
17 Jan, 2021

You are reading this latest preprint version

Building Capacity for Implementation – The KT Challenge

Status:

Journal Publication

Version 1

Abstract

Figures

Contributions To The Literature

Background

Methods

Overview of the KT Challenge program

Letter of Intent (LOI):

Workshops:

Mentorship:

Online resources:

Management support:

Funding:

Program Leads:

Evaluation Plan

Data Analysis

Results

Uptake of KT Challenge by practitioners

KT Capacity

Impact on Practice Change

Reported Challenges

Discussion

Conclusions

Abbreviations

Declarations

Ethics approval and consent to participate

Consent for publication

Availability of data and materials

Competing Interests

Funding

Authors’ Contributions

Acknowledgements

References

Supplementary Files

Status:

Journal Publication

Version 1