Identifying a region-level ‘Knowledge Translation Signature’ in rectal cancer surgery – an observational pilot study

Background Knowledge translation (KT) interventions can facilitate the implementation of evidence-based practice and help close quality gaps. Across Ontario, since approximately the year 2006, numerous KT interventions have been implemented by the provincial cancer agency or by individual researchers. Ontario is divided administratively into 14 Local Health Integrated Networks (LHINs). We piloted a methodology to summarize and score at the LHIN level all KT activities implemented to improve the quality of rectal surgery (i.e., a KT Signature score). Methods We interviewed stakeholders to identify KT interventions used in respective LHINs over years 2006 to 2014. Results were summarized into narrative and visual forms. KT experts reviewed and scored final summaries using a 20-item KT Signature Assessment Tool. Scores for each item ranged from 1 – 5. Thus scores could range from 20-100 for each LHIN. Results There were thirty interviews. KT experts produced KT Signature scores for each LHIN that were bimodally distributed with an average score for 2 LHINs of 78 (range 73-83) and for 12 LHINs of 30.5 (range 22-38). Conclusion Related to region level KT interventions to improve rectal cancer surgery quality, we identified two KT Signature types. Scores in 12 Ontario LHINs were low reflecting minimal efforts. Two LHINs had high scores reflecting implementation of numerous KT interventions in addition to those encouraged by the provincial cancer agency. Our methods and results require further validation. But they should be of interest to stakeholders implementing interventions designed to improve medical care at a population level.

Conclusion Related to region level KT interventions to improve rectal cancer surgery quality, we identified two KT Signature types. Scores in 12 Ontario LHINs were low reflecting minimal efforts. Two LHINs had high scores reflecting implementation of numerous KT interventions in addition to those encouraged by the provincial cancer agency. Our methods and results require further validation. But they should be of interest to stakeholders implementing interventions designed to improve medical care at a population level.

Background
Knowledge translation (KT) interventions in health care are meant to facilitate the implementation of evidence-based practice and help close quality gaps. 1 Examples of KT interventions include guidelines, audit and feedback, and use of opinion leaders.
Stakeholders have suggested that KT intervention effectiveness may be enhanced through the use of 'integrated knowledge translation' (iKT); the use of theory to plan, implement and evaluate any KT strategy; and, sustained iterative approaches that allow KT efforts to be modified as barriers to practice change are recognized. [2][3][4][5] With iKT, the target subjects of an evidence-based intervention (e.g., front-line surgeons) are involved in all aspects of the research initiative including design, implementation and evaluation. 2,3 The Knowledgeto-Action (KTA) Cycle is informed by key behavioural theories (e.g., social theory) that may drive health care worker behaviour and reinforces the importance of an iterative sustained effort to close quality gaps. 4,5 The province of Ontario, Canada (population 14 million) is divided into 14 Local Health Integrated Networks (LHINs). 6 Clinical care for patients diagnosed with rectal cancer can be challenging. Cancer Care Ontario, the governing body responsible for cancer care across the province, has used various KT interventions to improve the quality of care received by patients diagnosed with cancer. These include use of guidelines, communities of practice, diagnostic assessment programs, and multidisciplinary cancer conferences. [7][8][9][10][11][12] There are reports of KT interventions used in other LHINs to improve rectal cancer surgery care in addition to those encouraged by Cancer Care Ontario. 13,14 We piloted a methodology to summarize and score at the LHIN level all KT activities implemented to improve the quality of rectal surgery. This included identifying KT interventions implemented using progressive KT approaches such as integrated KT and use of theory. Our study should allow us to ascribe a 'KT signature' score for rectal cancer surgery to each of the 14 Ontario LHINs. We define 'KT signature' score as a qualitative summary and quantitative scoring of KT activities directed at a specific clinical area in a given geographic region over a specific period of time. We are unaware of previous efforts to summarize and qualify (i.e., score) KT interventions delivered in a specific geographic region.

Methods
We used semi-structured interviews, and, the subsequent evaluation and scoring of interview results using a KT Signature tool to identify a KT Signature score for each of the 14 Ontario LHINs.

Study Setting-Cancer Care Ontario and KT activities in Ontario
Cancer Care Ontario has over the years implemented numerous KT interventions in an effort to improve the surgical care received by patients with rectal cancer. These include use of guidelines, communities of practice, diagnostic assessment programs, and multidisciplinary cancer conferences. [7][8][9][10][11][12] The intention of these latter three interventions, respectively, is to have surgeons work together in a region to develop methods of optimizing care; to facilitate the timely and appropriate testing and treatment of people with cancer; and, to ensure that patients receive coordinated treatment recommendations from a range of specialists. As well, Cancer Care Ontario routinely reports on wait times for cancer surgery and occasionally executes limited audit and feedback to LIHIN administrators (e.g., number of lymph nodes counted in pathology specimens). Of note, these interventions are delivered or encouraged in a top down manner; Cancer Care Ontario administrators have no mechanism to force surgeon engagement or response with any intervention, nor has there been an effort to evaluate the impact on patient care of these interventions. This is somewhat understandable since such an evaluation would be influenced by numerous confounding factors.

Design of Interview Guide
A 25-page interview guide helped identify if an intervention or activity did or did not occur. (Supplemental Index I) The Cochrane Effective Practice and Organization of Care taxonomy outlined an exhaustive list of KT interventions that may have been used including: education materials (e.g., guidelines), education meetings, audit and feedback, practice demonstrations, education outreach or detailing, reminders, and, tailoring interventions. 15 Activities potentially provided though Cancer Care Ontario but not specifically listed in the taxonomy such as communities of practice, diagnostic assessment programs, and multidisciplinary cancer conferences were also included. Positive responses were probed further to understand the processes of intervention implementation. Probes considered the following: was the activity selected by an individual or group; what body did such individuals or groups represent; were interventions selected to address specific quality gaps; were interventions delivered at the individual surgeon, hospital or LHIN level; and, how was intervention success evaluated? There was special interest in identifying surgeon-led iKT targeting regional (e.g., LHIN-level) performance, evidence of sustained iterative approaches (e.g., data exercises that were repeated through time and not simply one-off evaluations), and, any intervention not in the a priori list.

Participants
The Surgical Oncology Program at Cancer Care Ontario assigns a surgical oncology lead and a colorectal cancer surgery lead for each of the 14 Ontario LHINs. These leads were invited to participate under the premise that they were the most likely surgeons to be familiar with rectal cancer surgery KT initiatives in their respective LHINs. In addition, heads of general surgery at high volume hospitals (i.e., performed >10 rectal or rectosigmoid cancer procedures per year) were approached for interviews. Snowball sampling was used to identify other key informants well positioned to provide relevant information. 16

Data Collection and Organisation
In advance of interviews, participants received an introductory package that included the purpose of the study and a summary of the interview guide. The summary listed preidentified KT interventions and relevant processes of intervention selection. A single research coordinator conducted telephone interviews. Following participant consent, the interviews were recorded and transcribed verbatim.

Outcomes and Analysis
The primary outcome for this study was to assign to each of the 14 Ontario LHINs a KT signature score that would reflect KT efforts designed to improve the quality of rectal cancer surgery at the population or region level. The study team could find no validated instruments that would allow summarizing and scoring of KT activities delivered across a large region and over an extended period of time in any clinical area. As well, there was no evidence of related previous qualitative efforts in any geographic region or in any clinical context. Therefore, a KT Signature Assessment Toolwas devised. (See Additional File 1) This tool listed 20 items corresponding to processes of implementation (e.g., the use of LHIN-level data to identify quality gaps) and specific KT interventions (e.g., guidelines). For this initial attempt at assigning KT signatures, a priori it was decided that each of the resulting 20 items would be scored on a Likert scale from 1-5, where 1 and 5 represented the item or process used 'not at all' or 'to a great extent', respectively. The maximum and minimum score for each LHIN was therefore 100 (20 items x 5 = 100) and 20, respectively. As well, it was decided a priori that individual item scores would be added for an overall LHIN score; and, scores from raters would be averaged. Experts in KT reviewed the tool and provided feedback on its design prior to use. A priori it was also decided there would be consideration of grouping LHINs with similar activity patterns and scores.

Assigning a KT Signature to Ontario LHINs
Four experts in knowledge translation agreed to participate in a modified Delphi process to assign KT signatures to individual LHINs using our collected data. 17,18 Experts were first provided with the study objectives and methods, a copy of the KT Signature Assessment Tool, and, the LHIN summaries (narrative and KTA Cycle). Face-to-face meetings were then arranged. Meetings began with a study overview, and then presentation of each LHIN summary. Primary data were also available for direct review. Following each LHIN presentation, raters independently scored activities using the KT Signature Assessment Tool. Scores were entered into a summary table and LHINs were rank-ordered according to mean overall score. Experts discussed average scores and the LHIN summaries to formulate LHIN groupings. Consensus was reached following face-to-face discussion and re-confirmed through post-meeting email.

Results
Interviews were held between January 2014-March 2015.Two to four interviews were completed per LHIN, for a total of 30 interviews. For illustrative purposes, Figure I presents KTA cycle summaries for LIHN-A and LHIN-H, respectively.

Ascribing LHIN-level KT Signature Scores
The KT Signature scores from our four raters for each LHIN are presented in Table I. There are potential weaknesses with this study. First, the method of ascribing a KT signature score to a region has not been properly validated, including the use of focused interviews and scoring withour KT Signature Assessment Tool. However, the KT Signature Assessment Tool did have face validity since it was put together with the input of KT experts, and, was based on the Cochrane Effective Practice and Organization of Care taxonomy of interventions and the KTA cycle. As well, through a Delphi process, four KT experts consistently produced a bimodal distribution of KT Signature Scores -higher scores for 'KT Signature I' versus 'KT Signature II' LHINs, further providing some face validity for our approach. Regardless, this method of summarizing and scoring KT interventions implemented at a region level should be considered a pilot. Stakeholders from other regions are encouragedto replicate, evaluate, and refine our methods. We also plan to use the identified KT signature groupings for future quantitative analyses-this may provide construct validity for our methods. Second, we scored numerous KT interventions and processes delivered over a number of years. This precluded the use of a checklist approach to describe content for each and every intervention. 19 But our KT experts were aware of concepts such as the presumed superiority of iKT and iterative, sustained efforts. We are confident such relevant factors were integrated into KT Signature Tool scores. Regardless, we emphasize the need to further validate our methods and results. Finally, it is possible that in a given LHIN, interviews did not identify all relevant KT interventions. This is unlikely. Following interviews, in an effort to ensure data accuracy and to seek additional information, LHIN results were mailed to respective Chiefs of Surgery at all hospitals in the LHIN, and to other stakeholders such as Cancer Care Ontario Regional Vice Presidents. Very little additional data were forthcoming, and no additional LHIN-level interventions or relevant implementation processes were identified.

Conclusions
Related to improving the quality of rectal cancer surgery, interviews with stakeholders across Ontario, a Delphi process with KT experts, and a novel KT Signature Assessment Tool, were used to ascribe a KT Signature score to each of the 14 Ontario LHINs. The distribution of scores was bi-modal. Two LHINs had interventions that were iterative, informed by data, and initiated by front-line surgeons while the other 12 LHINs ostensibly executed interventions that originated from Cancer Care Ontario in a top-down fashion.
These two groupings will be used for future quantitative analyses. These methods and findings require validation, though they should be of interest to researchers and stakeholders attempting to summarize and score KT