Fidelity and Maintenance of Motivational Interviewing Skills in Diabetes Prevention Program Coaches: A Pilot Study


 Background: Motivational interviewing is an effective counselling style for changing lifestyle behaviours. Few studies have examined brief motivational interviewing training for non-healthcare practitioners to deliver motivational interviewing-informed health programs. The purpose of this study was to pilot a brief motivational interviewing workshop on non-healthcare practitioners to deliver a community-based diabetes prevention program. Methods: This pilot study used convenience sampling to obtain seven participants naïve to motivational interviewing who wanted to become diabetes prevention program coaches. Participants attended a two-day motivational interviewing workshop, were then shadowed by an expert coach delivering the diabetes prevention program, and finally, were shadowed by an expert coach and received feedback. The primary outcome was whether coaches were able to maintain a level of at least client-centered motivational interviewing skills for the six months post-training, as assessed by the Motivational Interviewing Competency Assessment (MICA). Two independent coders used the MICA to assess a random selection of participants’ audio recordings of interactions between with diabetes prevention program clients. One session for each client in coaches’ first six months post-training was coded. Motivational interviewing-competency scores were generated using MICA scores for six months. Results: Coaches were 25B2 years old, 71% female, and 43% had less than a bachelor’s degree. Mean motivational interviewing-competency was at a level of client-centered (total MICA score of 3.3a0.24) over six months. The majority (71%) of all sessions were client-centered for all of the MICA categories. Conclusions: This pilot study offers preliminary evidence that non-healthcare practitioners attending a brief motivational interviewing training were able to deliver a client-centered level of motivational interviewing in a community-based diabetes prevention program up to six months post-training without the use of any booster training sessions. This suggests that the training used within this study may be sufficient to train future non-healthcare practitioner diabetes prevention program coaches in the community.

It is necessary to ensure that the training that health coaches receive is su cient so that program participants are receiving the program as intended. Measuring treatment delity (the degree to which the program is being delivered as intended) allows for con dence that any observed changes from an MI intervention were due to the intended delivery of MI. Assessing delity also prevents researchers from measuring the effects of MI on a behaviour change when MI was not in fact delivered (type III error), and increases the likelihood that researchers will detect an effect of MI when that effect is present. Finally, the measurement of delity allows for researchers to be con dent in implementation of interventions into community settings (17) and can facilitate skill development by providing ongoing feedback about the program implementation. Unfortunately, lack of monitoring of treatment delity is common in the MI literature, speci cally in health behaviour change (18). A meta-analysis by O'Halloran, Blackstock (19) observed a small effect size for studies that examined the effect of MI on increasing physical activity levels in populations with chronic health conditions, however this small effect increased to moderate when studies that did not assess delity were removed from analysis. Similarly, studies that assess delity of MI in populations with T2D tend to be of higher quality than those that do not assess delity (9,11). The assessment of treatment delity allows interventionists to better understand the elements for an intervention's success or failure. Assessing treatment delity ensures that counselors are actually providing MI in MI-based interventions. Frost, Campbell and colleagues (18) called for higher quality research to deal with issues relating to monitoring and reported delity of MI interventions. Without the assessment and reporting of delity of MI intervention, readers cannot be con dent that MI is truly being used as intended in interventions.
Despite this limitation in the literature there are several validated coding manuals for assessing MI delity that have been published to date (20,21). The manuals generally score and count skills associated with MI to ensure that counselors are applying MI in interventions. The delity of training is also necessary to assess to ensure those that are expected to deliver an intervention in an MI-informed approach received appropriate training (17). Two-day MI workshops that combine didactic and experiential methods repeatedly prove effective in improving counselors' MI skills (22,23). Furthermore, the addition of supervision and feedback following two-day MI workshops demonstrates additive bene t compared to a two-day workshop alone (24,25).
Although MI workshops have been shown to be effective, the maintenance of high-delity MI delivery has been questioned. It appears that important MI skills decline between two-and four-months post-workshop (22,23). However, providing newly trained MI counselors with post-workshop enhancements, such as feedback and supervision appears to decline the loss of skills post-training and skills are maintained over a 6-month period post-workshop (26). Speci cally, post-workshop training enhancements that were 5-12 contact hours or more showed greater MI skills at 6-month follow-up compared to enhancements that were less than ve hours (26). Review of previous literature does not give a conclusive overview of which skills decline, nor the timeline of when the skills begin to decline. This study aimed to address these important gaps in the literature by pilot testing a two-day MI workshop with post-workshop training enhancements to examine the skill acquisition and maintenance of newly trained coaches who delivered a diabetes prevention program called [PROGRAM NAME].
[PROGRAM NAME] is a community-based lifestyle program for individuals with prediabetes. The primary aim of [PROGRAM NAME] is to help empower individuals at risk of developing T2D to make long-lasting exercise and dietary changes to decrease their risk for T2D. [PROGRAM NAME] clients attend six sessions with an MI-trained coach over three weeks. Each session, the coach engages in discussion with the client for approximately 40 minutes using an MI approach. Approximately every four weeks new [PROGRAM NAME] clients begin the program, which means that each coach begins sessions with one (or more) new clients each month. Currently, [PROGRAM NAME] is only being offered in one community site. In order to scale-up [PROGRAM NAME], a necessary step is to pilot test the coach training to ensure that it is meeting program objectives.
The objective of the present pilot study was to examine the MI skills of newly trained coaches delivering [PROGRAM NAME] up to six months after their initial three-phase MI-training to determine whether the training was su cient to deliver [PROGRAM NAME] as intended. It is imperative that this training is pilot tested to know if it is su cient to train future [PROGRAM NAME] coaches who are non-healthcare practitioners.

Study design
Due to the nature of [PROGRAM NAME] being a community-based diabetes prevention program, this pilot study used a non-randomized design and all coaches received the same MI training. Since this is a community-based program, the research team wanted to ensure that all program clients were receiving a high-quality program. Based on the extensive literature surrounding the effectiveness of MI on health behaviour changes, speci cally in the eld of T2D prevention and management, the research team decided that all clients would meet with a coach trained in MI.
All [PROGRAM NAME] sessions between clients and coaches were audio-recorded. One session per client was randomly selected to be coded for each coach for six months post training.
Participants and setting [PROGRAM NAME] was delivered in a tness facility in [CITY, PROVINCE, COUNTRY]. All participants were recruited through word of mouth, the university job board, and advertising to undergraduate students about the opportunity through class visits and emails offering practicum and research opportunities.
Eligible participants were naïve to MI and agreed to be coaches. No speci c educational background was required.

MI training for [PROGRAM NAME] coaches
The three phase MI-informed training protocol for new [PROGRAM NAME] coaches included a two-day MI workshop (phase one) in which they learned the skills and processes needed to deliver the [PROGRAM NAME] content using MI. The workshop used a mixture of didactic and experiential learning, covering topics such as the spirit of MI, the four processes of MI, and the four MI micro-skills. Coaches also learned how to listen to and respond to change talk, ambivalence, and sustain talk. Following the workshop, coaches shadowed an expert in delivering [PROGRAM NAME] (phase two), and the MI instructor shadowed them and provided feedback to ensure their understanding and comfort in delivering [PROGRAM NAME] using MI (phase three). Following this three-phase MI training, coaches worked independently with clients.

Measures
Demographics. Participants self-reported their age, gender, ethnicity, highest education level completed, area of degree completed, years of counseling experience, and if they had ever received MI training. (27) is a coding tool developed to evaluate a practitioner's MI competency from a quality assurance perspective, and to help provide feedback to practitioners on how they can improve their MI skills. The MICA codes MI micro-skills, strategies and intentions. The two micro-skills that are counted are the number of re ections and questions given by the coach. The MICA codes the ratio of re ections to questions (R:Q) ≥ 2:1 as MI-competent. In the present study, it was not expected for coaches to reach this 2:1 ratio because the [PROGRAM NAME] coaches are instructed to deliver speci c [PROGRAM NAME] intervention content, which mandates a number of closed-ended questions needed for brief action planning and goal setting. The MICA also codes two MI strategies: 1) strategically responding to change and 2) strategically responding to sustain talk; and ve MI intentions: supporting autonomy and activation, guiding, expressing empathy, partnering, and evoking. Each strategy and intention are scored out of ve, and a score of three is considered 'clientcentered' care and four and higher designates MI-competent and MI-pro cient, respectively (see Table 1 for full scoring rubric). The MICA permits half scores.

MICA. The MICA
In the case of sustain talk or change talk being absent from the coded session, the MICA instructs coders to give a score of 3/5. A total MICA score is generated by summing the average score of the two MI strategies with the average of the ve MI intentions.

MICA Coding and Analysis of [PROGRAM NAME]
Coders listened to 20-minute segments (according to MICA standards) of each selected coach-client session audio recording. As coders listened to the audio recordings, they coded statements as questions, re ections, or no code. After listening to the 20-minute segment, coders counted questions and re ections, and scored the quality of each MI intention and strategy out of ve. Any disagreements between coders (i.e., giving a MICA score that was not within 1 point of what the other coder gave), were resolved through discussion between the two coders.
MICA scores were reported as descriptive statistics, with means, standard deviations, and ranges of scores reported. Analysis of whether the pilot study was successful was determined by the overall MICA scores. Overall, if scores showed that coaches were delivering [PROGRAM NAME] at a level of client-centered MI or better, the pilot study would be determined to be a success.

Participants
Seven participants completed the MI training, coached clients independently, and were included in the current pilot study. The coaches were thee undergraduate-and four graduate-level students. Areas of study included psychology and kinesiology/exercise sciences. The mean age of coaches was 24.71 years old (SD = 2.43), and ve of the seven coaches identi ed as women.
Of the seven coaches, two were considered to be "incompleters" of the study, as one only coached for one-month post-training, and the other only coached for two months post-training. Both of the "incompleters" completed their undergraduate degrees at approximately the same time that they stopped coaching.
The remaining ve coaches completed sessions with clients for the entire 6-month duration of the study. Analyses for the MICA in months one and two include averages for all seven coaches, but also show averages for just the ve coaches who completed all six months (Table 2). coaches had one to three clients. For coaches who had more than one client in a month, their scores for each client were averaged for each strategy and intention to create one score per month per coach. Refer to Table 3 for more information on the number of clients per month for each coach.  Coach 1  1  1  2  2  3  1   Coach 2  1  2  2  3  2  1   Coach 3  2  2  3  1  1  2 Coach 4 1 2 n/a n/a n/a n/a Coach 5 1 1 3 2 2 1 Coach 6 1 n/a n/a n/a n/a n/a Coach 7 1 2 1 2 2 2 Note. n/a for months after a coach stopped taking on clients.

MICA scores
The means and standard deviations of the R:Q, scores for each strategy and intention, and the total MICA score for each of the six months post-training are provided in Table 2. Generally, coaches who completed the six months were operating at a level of client-centered care immediately post-training and for the next six months. No major increases or decreases in any of the strategies or intentions were observed during the six months post-training. Coaches who did not complete the 6-month duration were inconsistent in applying most MI strategies and intentions (MI-inconsistent). Sustain talk was absent from the majority of coded sessions and as per the MICA coding instructions a score of 3/5 was given for 'absent sustain talk'. This explains the lack of variability between coaches' scores for strategically responding to sustain talk.

Discussion
Treatment delity is an often overlooked yet crucial aspect of pilot research. Understanding the degree to which an intervention is delivered as intended allows for accurate interpretation of results, appropriate conclusions to be drawn, and replicability (17). The objective of this pilot study was to monitor the MI skills of newly trained [PROGRAM NAME] coaches for the six months following their three-phase MI-training to determine if the training was su cient.
Immediately post-MI training, the majority of coaches were operating at a level of client-centered MI, and this level was maintained for the next six months. The ve coaches that completed the study and independently trained clients for up to six months were able to deliver the [PROGRAM NAME] intervention in a client-centered MI approach. No decline in any of the MI strategies or intentions were seen, and conversely, no increases were seen. No booster or follow-up training sessions were provided to coaches after their initial two-day workshop. These ndings suggest that the coach training was successful in having coaches maintain MI skills for six months post-training. This leads to promising implications for the sustainability of delivering [PROGRAM NAME] in the community.
Even though coaches were operating at a level one step below that of "MI competence", as de ned by the MICA, providing client-centered care has been emphasized as an important clinician skill. The idea of client-centered practice rst appeared in psychology (28) and has since been adopted by rehabilitation clinicians (29). Speci cally, a client-centered approach is foundational in the eld of occupational therapy 35 and has been shown to improve health outcomes (30,31). A client-centered approach "embraces a philosophy of respect for, and partnership with, people receiving services" (32). Clientcentered care closely aligns with many of the core ideas of MI such as compassion, empathy, and active listening, all of which have been shown to enhance positive outcomes associated with care (33,34). In contrast, Jackson, Butterworth (27) de ne MI competence as operating at a level that is intentionally and purposefully focused on MI intentions and skills, which differs slightly from their de nition of client-centered care: operating at a level that "naturally, intentionally, or unintentionally hits elements of MI". Because of the novelty of the measure, the authors are not aware of any studies that have reported the frequency in which novice coaches reach either of these categories, or what effect either category has on client outcomes. The [PROGRAM NAME] coaches reached and maintained a level of client-centered care that may be effective at helping individuals make health behaviour changes.
Contrary to previous literature demonstrating a decline in MI skills over time (22,23), there was no decline in skills observed among [PROGRAM NAME] coaches. This maintenance of skills may be due to several factors. First, phase two and three of the training (i.e., shadowing and reverse-shadowing) may have improved coaches' understanding of these skills and processes allowing them to maintain skills for longer periods of time, as also demonstrated in previous work (26). Second, the ve coaches that participated in the full duration of this study were all involved in other areas of research within [PROGRAM NAME]. Perhaps these coaches were more invested in their clients' outcomes due to the implications on their own research projects (i.e., adherence to physical activity, evaluation of [PROGRAM NAME]). As well, these coaches self-selected to be [PROGRAM NAME] coaches that would go through MI training.
Individuals who choose to take part in the MI training might be interested in this counseling style or see it as aligning with their own personality (e.g., an individual high in empathy might be keener to learn a client-centered counseling style compared to an individual who has a more directive style of communication). Third, it is possible that by assessing the coaches' audio recordings, the coaches were more cognisant of the skills and processes they were being judged on and were therefore more likely to use those skills learned in the MI-workshop. It might also be the case that 'client-centered' communication might be easier to maintain when compared to 'MI-competent'. It is possible that no drift in skills was observed because none of the It is di cult to conclude if there was anything unique about the two coaches who were at an MI-inconsistent level of coaching, thus the authors can only speculate as to why these coaches were MI-inconsistent and non-completers. Both of these individuals completed their undergraduate degrees during this study and one moved away while the other accepted full-time employment unrelated to this study. Unfortunately, no qualitative data or exit-interviews were conducted to ask these individuals about their perception of their MI skills.
The rigorous measurement of the delity of MI skills across time allows for the conclusion that the [PROGRAM NAME] intervention is, in fact, being delivered using a client-centered level of MI. Fidelity assessment is crucial for advancing the science of MI. Researchers demonstrating the delity of MI being delivered as intended can begin to understand what level of MI, and the speci c MI skills or intentions that are most important for guiding individuals to make health behaviour changes. The assessment of delity advances the evidence of MI as an effective communication style for delivering behaviour change interventions.
The ndings from this study show that the [PROGRAM NAME] coaches delivered an intervention using MI that was considered client-centred after the brief training. This study adds to the literature by demonstrating that this three-phase MI training style is effective at teaching coaches a client-centered style of MI that can be maintained for at least six months. None of the coaches had previous counseling experience and were all university students, and yet they were able to learn MI and operate at an effective level of counseling after a two-day workshop plus shadowing time. Lastly, the MICA tool to assess delity has minimal burden on researcher's compared to other options of using more time-intensive coding tools (i.e., MISC (20), MITI (21)). This is a strength, as a minimally time-consuming delity assessment tool would be sustainable to continue monitoring MI skills of future [PROGRAM NAME] coaches over time.
Despite these strengths, this study is not without limitations. First, the lack of a control group limits the conclusions that can be drawn on which aspects of the training led to the maintenance of skills (the workshop, shadowing, or reverse-shadowing). The sample of coaches is also a limitation. This sample of human kinetics undergraduate and graduate students might not generalize to other individuals who would be working within a diabetes prevention program, such as diabetes educators or dietitians. The small sample size and lack of variability between coaches did not allow for statistical analyses to be conducted. A larger sample size and more variability between coaches' scores would have allowed for the analysis of whether clients with coaches who were operating at a higher level of MI experienced better outcomes (i.e., physical activity adherence, decreased A1C, BMI reduction) than those who had coaches applying MI skills at a level below client-centered care. Another limitation was that [PROGRAM NAME] coaches were providing a very speci c intervention and had topics that they aimed to discuss with clients within each session. Therefore, given the structured nature of the content, it is possible that this is why coaches were unable to provide an MI-competent level of care.
Future research should focus on aspects of the post-workshop enhancements to determine what is driving the maintenance of skills. Further research is needed to examine if and how skills can improve from client-centered to MI-competent.

Conclusion
This study provides evidence that the brief MI training used within the study was su cient in effectively teaching [PROGRAM NAME] coaches to deliver client-centered MI. Based on the results from this study, the training can continue with little-to-no modi cations for new [PROGRAM NAME] coaches. This knowledge is important as client-centered care has been associated with improvements in health outcomes (28, 30,31), and now future work can examine the true impact of [PROGRAM NAME] on client outcomes associated with reducing the risk of developing T2D. Availability of data and materials