Developing a Successful Implementation Plan for a High Frequency, Low Touch Care Model at Specialized Type 1 diabetes Clinics: The Type 1 Diabetes Virtual Self-Management and Education support (T1ME) Trial

Stephanie de Sequeira (  stephanie.desequeira@unityhealth.to ) St Michael's Hospital Justin Presseau Ottawa Health Research Institute: Ottawa Hospital Research Institute Gillian Booth Unity Health Toronto Lorraine Lipscombe Women's College Hospital Isabelle Perkins Ottawa Health Research Institute: Ottawa Hospital Research Institute Bruce A. Perkins Mount Sinai Health System Rayzel Shulman Hospital for Sick Children Research Institute: SickKids Research Institute Gurpreet Lakhanpal Unity Health Toronto Noah Ivers Women's College Hospital


Introduction
Individuals with type 1 diabetes (T1D) are more likely to achieve optimal glycemic control when they have frequent visits with their healthcare team (1). Further, prior clinical trials suggest a potential bene t of frequent, telemedicine interventions as an effective strategy to lower hemoglobin A1c (HbA1c) (2,3).
However, it is not just the frequency of visits that have a signi cant effect on clinical and quality of life outcomes, it is also the type of interaction. There is a growing body of research on the effect of synchronous -or real-time -interactions (in-person, phone calls or video visits) compared to asynchronous interactions (email or text). For example, Verhoeven et al., showed that synchronous telemedicine interactions lowered costs for both patients and the healthcare system by reducing unscheduled visits compared to usual care (4). A model of care which includes frequent, synchronous interactions between individuals with T1D and their healthcare teams is particularly bene cial to patients who are not meeting glycemic targets and need to make changes to their diabetes self-management (5). Unfortunately, this was di cult to deliver in the context of pre-COVID-19 care, which typically involved time-consuming in-person visits during working hours. The necessary move to virtual care during the COVID-19 pandemic has provided a window of opportunity to address this gap in T1D management through virtual models of care. The Type 1 diabetes virtual self-Management Education and support Trial (the T1ME trial) aims to test the effectiveness of a model of "high frequency, low touch (e.g. virtual, remote)" care with real time visits for individuals with T1D who are not meeting glycemic targets (HbA1c > 8%). The T1ME trial is comprised of three components aimed at supporting self-management changes and goal advancement: 1) Virtual care software that enables video (or audio and instant messaging) visits between patients and their healthcare providers; 2) automatic appointment reminders and goal setting prompts; and 3) a centralized virtual library which houses curated and vetted education and self-management resources for individuals living with T1D.
If the T1ME trial is to be successful during the COVID-19 pandemic and beyond, the trial must be implemented using evidence-based processes (6). Implementation of virtual healthcare interventions are complex due to a wide range of factors, such as organizational support (7), willingness of staff to participate in the intervention (8), and current healthcare delivery systems (9). Implementation strategies must be developed to effectively target all these contextual factors. Moreover, most published multifaceted implementation strategies do not provide an explanation for why certain components were chosen (10), making it di cult to assess whether interventions su ciently address known barriers. Therefore, in this study, we sought to clarify the complex provider and system factors that need to be considered when implementing a high frequency virtual care model in T1D diabetes clinics prior to and during the COVID-19 pandemic. Additionally, we sought to comprehensively describe how we developed an implementation plan suited to address identi ed factors.

Study design
This was a theory-informed, qualitative study seeking to understand determinants of engagement with high frequency virtual care in general and the T1ME trial components in particular; and to map those determinants to feasible implementation strategies. Ethics approval was received from Women's College Hospital and the Ottawa Health Science Network Research Ethics Board.

Context
Clinical practice guidelines suggest that people with T1D have visits with their diabetes team every 3 months unless their glycemic control is already optimized (11). More frequent visits with certi ed diabetes educators (CDEs) and other care providers are often needed to help patients to recognize glucose patterns, adjust their insulin doses, and to offer education and technological support on the use of insulin pumps and continuous or ash glucose monitoring. In Ontario, people with T1D may be eligible to receive government funding for insulin pumps and related supplies through Ontario Health's Assistive Devices Program (ADP). Individuals who are registered in this program are required to receive frequent care from a certi ed pump team. Within this model of care, patients may need to wait three to six months for appointments with their diabetes team to trouble-shoot issues with diabetes self-management. Prior to the pandemic, most visits were conducted in person, requiring individuals with T1D to take time off work or school to visit their team members. This model of care may not be well-suited to patients who needed additional support or timely enough to enable them to make real-time changes to their diabetes self-management.
The description of the T1D care model described above, featuring mainly in-person care, was applicable up until the COVID-19 pandemic (12). As of March 2020, diabetes clinics in Ontario, Canada were mandated to adopt a virtual care model rapidly and with minimal preparation due to COVID-19 lockdown measures implemented. As of December 2021, most T1D care continues to be delivered virtually. However, despite virtualization, indicators suggest that care is still provided with longer, infrequent appointments every three to six months. Even though second vaccination rates have surpassed 80%, and booster doses have surpassed 30% in Ontario (13), it is unlikely that T1D care will return to the prepandemic norm, especially with new variants arising. Instead, diabetes clinics will most likely adopt a "new-normal" model of care that will include virtual options when in-person visits are not feasible or needed, as there will be lingering concerns regarding social distancing for some time, and many clinics have invested in virtual care technologies. Virtualization of diabetes care offers an opportunity to consider shorter, more frequent contacts through more feasible virtual modalities.
The T1ME trial seeks to improve this T1D care model and focus on more patient-centered care. If our high frequency, virtual model is to be successful, we must target key work ow processes and behaviours among diabetes clinic staff. Firstly, many traditionally in-person visits will need to be changed to virtual.
This includes understanding and targeting work ow processes relating to the uptake of new telecommunication technology. Secondly, we must understand the behaviour changes needed to accommodate a high frequency care model. Within this model, patients will meet with their CDEs for shorter, but more frequent touch points. This will change the nature of the interaction and affect work ow processes. Additionally, we will also need to evaluate current resource allocation and the potential impact of our high frequency, low touch model on clinic resources. Therefore, in the current study we sought to understand work ow processes, resource allocation, and other factors in implementing a high frequency, low touch care model in diabetes clinics.

Participants and Recruitment
We recruited nurses or dietician CDEs and managers in diabetes education programs at specialized T1D clinics in Southern Ontario. Sites were purposefully selected for variation in factors thought to potentially affect implementation of the intervention including the number of patients, number of patients under age 25, number of health professionals, number of patients with most recent HbA1c above 8%, and rurality.
For each site, a recruitment email was sent to the lead physician or clinic manager, inviting them to participate in a 30-45 min telephone interview. We also sent invitation emails to CDEs and managers identi ed by the investigators' personal networks. We then recruited additional key-informants at each site using snowball sampling. In particular, we sought a team member of the chosen T1D clinic who provided clinical care or support and/or with knowledge regarding the organization of the clinic processes, including technological processes (e.g. electronic medical records).

Data collection
Firstly, an electronic survey was sent to the clinic manager at each clinic to obtain descriptive information about the clinic. We collected information such as the number and type of healthcare professionals, types of communication with patients, wait times, and history with implementation of new programming. SD then conducted semi-structured, one-on-one, 30-45 minute telephone interviews during working hours.
Interviews were recorded, de-identi ed and transcribed. Oral informed consent was obtained before beginning the interviews. Field notes were made after each interview.
Interviews followed a semi-structured guide (developed by NI, JP, SD, GB, LL) that aimed to i) explore current processes and procedures for management of T1D patients under routine and semi-urgent scenarios and ii) examine the determinants of uptake and implementation of our high frequency, low touch model of care using the theoretical domains framework (TDF). The TDF is an integrated theoretical framework synthesized from 128 theoretical constructs from 33 theories judged most relevant to implementation questions. Domains of the TDF include items such as knowledge, goals, optimizing, and belief about capabilities (14) Analysis Research team members with a range of disciplinary backgrounds (endocrinology [GB], psychology [JP], family medicine [NI], public health [SD]) reviewed the electronic survey data and transcriptions in depth to understand the current processes in clinic and, importantly, the changes required for the intervention to be implemented as intended. The transcripts were examined to explore how the changes required might vary across clinics (15).
Transcriptions of the interviews were then coded using the TDF domains by two independent researchers (SD, IP) using a word processor. Coding was mainly deductive, involving assigning utterances to the relevant TDF domains; open coding was used if and when important issues were identi ed that do not seem to t any existing domain. A codebook was maintained and updated regularly to ensure inter-coder reliability.
When all transcripts were coded, we (NI, JP, SD, GB) identi ed the most important determinants (domains) to be addressed in the implementation and training plan by 1) frequency (which domains, for which key targeted behaviour, most commonly arise as issues to be addressed in the transcripts); 2) con ict (presence of disagreement across participants on certain domains representing a potential need for tailored strategies); 3) strongly held and strongly emphasized beliefs amongst participants about the targeted behaviour; and 4) most important determinants to be addressed (determinants which have the highest likelihood of impeding or facilitating implementation) (16). Additionally, we grouped domains into higher-level barriers and enablers. Then, we mapped out how each domain interacted with other domains. This allowed us to generate a list of theoretical domains most likely to in uence the targeted behaviours for successful implementation of the T1ME trial.
Finally, we (NI, JP, SD) used the Behaviour Change Techniques Taxonomy Version 1 (BCTTv1) developed through an international consensus process, to identify actions that would enable the interventions to become more easily adopted into routine care (17,18). This taxonomy provides clarity surrounding the speci c, active ingredients needed to elicit behaviour change and draws on applied research in behavioural medicine, as well as social and health psychology. BCTs likely to in uence key TDF domains have been previously mapped (18). Team members (NI, JP) with training and experience identi ed the most promising behaviour change techniques (BCTs) thought to be feasible to utilize in the implementation and training strategies for the intervention. We used these BCTs and most relevant theoretical domains to create a comprehensive implementation and training plan, which could be tailored to each site if necessary.
Analysis and data collection occurred concurrently; recruitment ceased once thematic saturation was reached. Our threshold for thematic saturation was twofold. Firstly, our initial analysis sample (minimum sample size) included at least one CDE and one manager from each site. Following that, our stopping criterion was a 0% new information threshold in the key theoretical domains (19,20).

Member-Checking
SD conducted member-checking calls with participants to ensure that our interpretation of the barriers and enablers from the original interviews accurately re ected the context of their speci c clinics (21). Since we conducted the member-checking calls during the COVID-19 pandemic, we also took the opportunity to inquire about whether our interview results held true during the context of completely virtual care and understand process clinics initiated to accommodate virtual care.
Member-checking calls were recorded, de-identi ed, and transcribed. Field notes were made during the member-checking call. Two independent researchers (SD, IP) coded the transcripts using a word processor. We used deductive analysis, assigning quotes to the barriers and enablers from the original interviews. We also used open coding for issues other than the barriers and enablers identi ed in the interviews. A member-checking codebook was maintained and updated regularly to ensure inter-coder reliability.

Participants and Sites
We interviewed 35 participants across 12 diabetes clinics in Southern Ontario from February 1 to May 16, 2019. Of the 35 interviews completed, 20 participants were CDEs, 13 participants were managers, one participant was an administrative coordinator, and one participant was a social worker. Two sites declined participation, and three sites did not respond to the invitation email. See Table 1 for the demographic information and process in each participating clinic. We separated our ndings into the two components of the T1ME trial (i.e. high frequency and low touch).
Within the component of low touch (virtual) care, we found two main barriers. Firstly, there was the belief that low touch care would lead to an increased workload. This included double administrative work and increased time and work spent on trouble-shooting technical glitches: "It takes up a very [large] amount of our healthcare practitioners' time to troubleshoot the technology (19)". Secondly, managers reported that there was a lack of nancial resources to obtain virtual care technology, and lack of private clinic space and o ces to offer virtual care.
Alternatively, participants noted that there was an interest and intention to use virtual care. However, this interest and intention varied depending on the CDE, manager, and institution. For example, as one CDE notes, their "organization as a whole wants [low touch care], and [she] knows that part of their strategic direction for the next ve years is to increase virtual visits so this aligns with that (15)". On the other hand, some managers wanted to observe the success of the program before agreeing to participate: "If the feedback is positive, then yes, absolutely (22)". Finally, the CDEs and managers who believed that low touch care would improve patient outcomes and had existing skills and comfort with virtual care (i.e. phone, email, video visits), had more of an intention to participate in the T1ME trial.
We also found barriers and enablers relating to high frequency care. Firstly, the belief that high frequency care would increase the workload of charting and documentation was a barrier to the uptake of the T1ME trial. Secondly, clinic staff reported that there was a lack of resources such as staff and capacity, and time to successfully implement high frequency care. For example, one manager stated she was "very hesitant about [participating in the T1ME trial], just because the volumes that we deal with and the admin support that we have, we just can't handle that (06)".
However, clinic staff who believed that high frequency care would lead to better patient outcomes and increased patient engagement were more likely to participate in the T1ME trial. As one CDE stated, high frequency care would be "wonderful" for patients: "More frequent low touch follow up is probably going to be a wonderful thing for them, a way to check in or get questions clari ed. Now, I'm trying this out. Now, the rubber meets the road. Here's a little hiccup. Being able to troubleshoot that (27)." To further enable successful implementation of the T1ME trial as a whole, engagement of CDEs throughout the duration of the T1ME trial is necessary. "Most of the staff here are very open to trying different things if the patients want it. Again, it has to be something that the patients are willing to do (05)". Positive patient feedback; adoption of the program by a local champion and other clinic staff; and continuing support from the T1ME trial team were suggested as methods to encourage CDE participation and offer ongoing engagement in the program.

Theoretical Domains
We identi ed that responses coded within some theoretical domains related to others in the same domain and to other domains. For example, there were three concepts within the domain of 'beliefs about consequences': 1) improved patient engagement, 2) improved patient outcomes, and 3) increased workload. If participants believed that the T1ME trial would increase their workload, they reported less of an intention to participate. On the other hand, if participants held the belief that the T1ME trial would improve patient outcomes and engagement, they were more likely to participate in the program. Therefore, multiple different beliefs about consequences likely affect the relative intention to participate. Figure 1 maps how constructs within domains may relate to other domains, and ultimately affect clinic staff participation in the T1ME trial. Figure 2 exhibits the map for low touch care, which contains the theoretical domains of knowledge; optimism; belief about consequences; goals; skills; reinforcement; and intention. While, the high frequency component (Figure 3) was informed by the domain of social and professional role and identity, in addition to the domains listed above.

Member-checking
We completed member-checking calls with 20 of our 35 participants, at 8 out of 12 sites from June 23 to September 10, 2020. We spoke to 13 CDEs, 6 managers, and 1 administrative coordinator. Some of the staff we originally interviewed were redeployed to the COVID-19 testing centers at their respective institutions, and therefore were unable to complete the member-checking call.
In general, participants agreed with the barriers and enablers we identi ed above. Additionally, a few opportunities arose during the COVID-19 pandemic. Firstly, even though participants agreed with the barriers, one CDE notes that barriers never impeded offering care to patients who needed it: "I would say that sta ng has always been an issue to some extent…But, I don't know that it necessarily affected our ability to see those patients who really needed to be seen (16)". Additionally, some clinics invested in technology, such as laptops for staff to offer virtual care to patients. Moreover, during the forced switch to virtual care those who wanted evidence of success in high frequency, low touch care before agreeing to participate in the T1ME trial received it: "I worried that accountability wouldn't be there as much, but I think that's proven me wrong very much. Since COVID-19, I think people are more engaged even by phone (19)".
The pandemic provided an opportunity to explore barriers/enablers of current implementation of virtual care (low touch) and distinguish from those of high frequency care. Like clinic staff, patients also had to get accustomed to virtual care visits. This included learning how to log onto and participate in virtual visits; read, interpret, upload, and share blood sugar data from pumps and continuous glucose monitors. Moreover, patients had to change their mindset and understand that the phone call was an actual visit.
One CDE notes that her appointments were longer because patients were not prepared for the virtual visit as they would be for an in-person appointment: "We tell [patients] that we're going to be calling them for care, next thing you know, 'well, wait a minute, my meter is up in my bedroom, oh, wait a minute, my pills are in the kitchen' (20)". However, most CDEs noted that as the pandemic progressed, patients acclimated to virtual care and began to enjoy it, which is a signi cant source of engagement for the CDEs: "Some people are saying, 'so, do I have to come in the next time?' That's kind of been the message. And almost everyone's like, 'I'm really ne if it's this virtual again'. Very rarely do I get the question of, 'when can I come in person?' (33)".

Implementation Plan
We identi ed the antecedent and key domains in our map (Figures 1-3) and linked them to the behaviour change techniques shown to in uence those speci c domains. Additionally, during our member-checking process we learned that clinic staff created a number of workarounds to offer a similar level of care virtually. We integrated these lessons into speci c components of our implementation plan, such as the T1ME manual of operations. Thus, we created a comprehensive implementation and training plan, targeting the key domains and lessons learned during the pandemic (See Table 2).

Discussion
It is estimated that only 20% of health research funding makes a public health impact (22). This can be explained in part by the evidence to practice gap, where there is a disconnect between the care that practitioners know is effective and that which is actually delivered (23). To overcome this gap, implementation science approaches have been developed to understand the contextual factors of the setting in which the healthcare intervention is being implemented (6, 24) In our study, we took an implementation science approach to design and implement a virtual, high-frequency model of care intervention for type 1 diabetes clinics in Ontario, Canada based on site speci c characteristics, semistructured interviews with clinic staff, and behaviour change and implementation literature. Our interviews were completed prior to the COVID-19 pandemic, and therefore we completed a memberchecking exercise during the pandemic to assess if our interview ndings were still relevant within the context of predominantly virtual care.
Prior to the pandemic, the health care providers in the T1D clinics we interviewed reported two main barriers in both the high frequency and low touch components. Firstly, they shared the belief that this model of care would lead to an increased workload and secondly, that they felt that clinics did not have the necessary resources to implement the program successfully. However, during the pandemic when all clinics were utilizing virtual care, these clinics quickly developed strategies to overcome these barriers.
Even though the workload increased due to some clinic staff being re-deployed to COVID-19 testing centers, those we spoke to felt that patients who needed care still received it. Additionally, some institutions invested in virtual care technology during this time, decreasing the barrier that was voiced prior to the pandemic of a lack of nancial resources to obtain technology. These ndings are encouraging, and suggest that existing barriers to participating in virtual healthcare interventions can be overcome with the right support, such as technical training and resource allocation by the organization. However, these needs are not speci c to diabetes clinics. Mohammed et al., showed that technical training and in-house organizational and administrative assistance were also important to primary care physicians and nurses in Ontario when using virtual care during the COVID-19 pandemic (25).
Our study also revealed important enablers to participating in a high frequency, virtual healthcare intervention. During interviews that were conducted prior to the pandemic, participants noted that an interest and intention among staff to deliver high frequency, low touch care, and that continued engagement of staff would encourage long-term success of studies such as the T1ME trial. Patient feedback was reported as being a great source of engagement for staff, and belief that the intervention would result in better patient outcomes was associated with an increased intention to participate in the trial. Similar to observations reported from healthcare providers across Canada during the pandemic, the CDEs we interviewed learned that patients were just as engaged in their care virtually as they were during the pre-pandemic period, when most visits were conducted in-person, and many patients wanted to continue virtual care as their primary means of follow up (26).
Lessons learned during the COVID-19 pandemic helped us to update our implementation and training plan. Staff overcame some virtual care barriers ( gure 2) during the pandemic, and we used these lessons to make our implementation plan more robust. For example, we learned about common IT issues and how staff solved them, and heard about common patient issues and questions. We used these ndings to update our manual of operations and training sessions. Moreover, we were able to collect examples of positive experiences between healthcare providers and patients using virtual care. These experiences will be used to increase uptake of the T1ME trial by staff. Finally, now that clinic staff have become comfortable with virtual care, our team has focused more attention on tailoring our implementation plan to target factors surrounding a high frequency care model. This includes dedicating more time in the modelling and practice portions of training sessions which offer guidance to CDEs on how to offer patient centered care in shorter, but more frequent touch points than is currently used.
This study has a few notable strengths. Firstly, we used a theory-based approach to create our interview guide. The TDF has mainly been used for implementation in healthcare contexts when understanding the behaviours of clinicians (16). Therefore, we were able to ascertain signi cant implementation factors in this context. While other implementation theories have also been used to successfully implement complex interventions in healthcare, they come with limiting factors. For example, the Normalization Process Theory, a theory centered around behaviour rather than belief or intentions (27), has been criticized for focusing on the actions of healthcare providers rather than the experiences of the patients for whom the intervention is supposed to bene t (28). Unlike our ndings using TDF, Ross et al., found that the Normalization Process Theory did not account for the importance that diabetes healthcare providers placed on patient feedback (8), and therefore were not able to include this factor in their implementation strategy. The TDF however, does not come without critiques. For example, a strictly deductive analysis using the TDF will not allow non-TDF elements to be identi ed (29). We overcame this limitation by using open coding when important issues were identi ed that did not clearly t within an existing TDF domain. We also developed themes inductively within domains. Additionally, we will evaluate our implementation strategy which will allow us to further re ne our research plan to include any missing factors. Finally, our data collection prior to and during the pandemic facilitated the creation of a more robust implementation plan that can be applied to a variety of contexts. Feedback we obtained provides insights into what diabetes care will look like in the post-COVID context so we can adjust our research plan to meet these needs.
There are also a few limitations in our study to note. While we were able to interview diabetes staff in diverse clinics, all but one of our clinics were in urban settings. Therefore, experiences of staff in these clinics may not re ect those of clinic staff in non-urban settings. Additionally, we were not able to reach staff from three clinics during the member-checking exercise, and therefore we may have missed some important COVID-19 related barriers. Finally, many members on our team are endocrinologists or research staff in diabetes clinics. Our prior experience and working relationships with some participants could be a potential bias in how we carried out the interviews and in our interpretation of the ndings Conclusion For a complex health intervention to be successful, an implementation science approach is needed to understand contextual factors and identify levers that can support behaviour change. Using site speci c characteristics; semi-structured interviews with clinic staff; and behaviour change and implementation literature, we developed a robust implementation and training plan to successfully implement a high frequency, low touch care model in diabetes clinics in Southern Ontario. Data were collected pre-and during the pandemic to enhance the effectiveness of our implementation strategy. An evaluation of our implementation plan at diabetes clinics in Toronto will allow us to create an improved iteration before applying to other clinics in Ontario in the post-COVID context. Availability of data and materials

List Of Abbreviations
The qualitative data used and analyzed during the current study are available from the corresponding author on reasonable request.
Competing Interests BAP has received speaker honoraria from Abbott, Medtronic, Insulet, and Novo Nordisk; has served as an advisor to Insulet, Boehringer Ingelheim, Novo Nordisk and Abbott, and has received research support to his research institute from Boehringer Ingleheim, Novo Nordisk, and the Bank of Montreal (BMO). NI has received speaker honoraria from Janssen; has provided evaluation consulting from Merck, IQVIA, Centre for Effective Practice, and Diabetes Canada; and serves on the advisory board for Novo-Nordisk. LL has received an honoraria for a presentation by Alberta Diabetes Institute.

Funding
This study was conducted as part of the T1ME trial, which is funded by JDRF and CIHR through their

Map of interactions between theoretical domains
Large black circles show TDF domains important to our project. Smaller circles within the large circles identify concepts that were identi ed during our analysis. Green arrows and circles indicate facilitators, the darker the green the stronger the facilitator; Yellow arrows and circles depict a mixed effect; and red arrows and circles show the barriers Figure 2 Map of interactions between theoretical domains for low touch (virtual) care Large black circles show TDF domains important to our project. Smaller circles within the large circles identify concepts that were identi ed during our analysis. Green arrows and circles indicate facilitators, the darker the green the stronger the facilitator; Yellow arrows and circles depict a mixed effect; and red arrows and circles show the barriers