Application of the FRAME-IS to a Multifaceted Implementation Strategy

Background: Research demonstrates the importance of documenting adaptations to implementation strategies that support integration of evidence-based interventions into practice. While studies have utilized the FRAME-IS [Framework for Reporting Adaptations and Modifications for Implementation Strategies] to collect structured adaptation data, they are limited by a focus on discrete implementation strategies (e.g., training), which do not reflect the complexity of multifaceted strategies like practice facilitation (PF). In this paper, we apply the FRAME-IS to our trial evaluating the effectiveness of PF on implementation fidelity of an evidence-based technology-facilitated team care model for improved hypertension control within a federally qualified health center (FQHC). Methods: Three data sources are used to document adaptations: (1) implementation committee meeting minutes, (2) narrative reports completed by practice facilitators, and (3) structured notes captured on root cause analysis and Plan-Do-Study-Act worksheets. Text was extracted from the data sources according to the FRAME-IS modules and inputted into a master matrix for content analysis by two authors; a third author conducted member checking and code validation. Results: We modified the FRAME-IS to include part 2 of module 2 (what is modified) to add greater detail of the modified strategy, and a numbering system to track adaptations across the modules. This resulted in identification of 27 adaptations, of which 88.9% focused on supporting practices in identifying eligible patients and referring them to the intervention. About half (52.9%) of the adaptations were made to modify the context of the PF strategy to include a group-based format, add community health workers to the strategy, and to shift the implementation target to nurses. The adaptations were often widespread (83.9%), affecting all practices within the FQHC. While most adaptations were reactive (84.6%), they resulted from a systematic process of reviewing data captured by multiple sources. All adaptations included the FQHC in the decision-making process. Conclusion: With modifications, we demonstrate the ability to document our adaptation data across the FRAME-IS modules, attesting to its applicability and value for a range of implementation strategies. Based on our experiences, we recommend refinement of tracking systems to support more nimble and practical documentation of iterative, ongoing, and multifaceted adaptations. Trial Registration: clinicaltrials.gov NCT03713515, Registration date: October 19, 2018

This study contributes to the literature by demonstrating the applicability and value of FRAME-IS for documenting adaptations to multifaceted implementation strategies, when modi ed for that purpose.

Background
Growing evidence demonstrating the importance of improving the t of an evidence-based intervention to the setting in which it is implemented has catalyzed research to identify methods to document and track adaptations.(1) Adaptations are de ned as changes or modi cations to the design or delivery of interventions to improve their t within a given practice context and population.(2,3) Previous studies have shown that planned adaptations to improve intervention t are multidimensional and occur throughout the implementation process, without compromising the level of delity.(1,4,5) Recently, implementation science research has extended the concept of adaptation beyond tracking changes to the intervention itself to also consider the implementation strategies that support its integration into practice.(6-9) Frameworks such as the FRAME-IS [Framework for Reporting Adaptations and Modi cations for Implementation Strategies] have been developed to facilitate the collection of structured data to document and describe adaptations to implementation strategies.(6) The utility of documenting and understanding adaptations to implementation strategies is particularly critical for safety net settings such as federally quali ed health centers (FQHC) that often possess unique nancial and administrative threats to intervention t. (10) FQHCs, which primarily care for medically underserved communities, are often faced with limited resources and sta ng and are subject to lower reimbursement rates for healthcare services.(11,12) These complexities require the adaptation of implementation strategies to maximize intervention t to the outer (e.g., reimbursement policies) and inner (e.g., organizational culture, implementation climate) practice context.
A small but growing number of studies have utilized the FRAME-IS to provide important information on the adaptability of various implementation strategies in diverse settings including FQHCs.(6, 13-16) However, most of these studies are limited by their focus on a discrete implementation strategy (e.g., training), which do not re ect the complexity of multifaceted strategies such as practice facilitation (PF).
To address this limitation, we leverage our ongoing trial, which is evaluating the effectiveness of PF on implementation delity of an evidence-based technology facilitated team care model within a large FQHC, to conduct an ancillary study that characterizes adaptations to our implementation strategy using the FRAME-IS.Based on our experiences, we also share considerations when using the FRAME-IS to characterize adaptations to multifaceted implementation strategies.

Setting and study context
Overview of the Advancing Long-term Improvements in Hypertension Outcomes through a Team-based Care Approach (ALTA) trial We are currently conducting a stepped-wedge cluster randomized controlled trial using a PF strategy to implement remote patient monitoring (RPM) supported by team-based care [herein called ALTA] in 700 patients with uncontrolled hypertension (HTN) receiving care at a large FQHC (clinicaltrials.govNCT03713515).(17) The FQHC where ALTA is being implemented provides healthcare services in southwest and central Brooklyn, capturing a diverse population of mostly Hispanic and Black immigrants (69.6%) facing poverty, and cultural, as well as language barriers (41% are best served in a language other than English).
The trial is being conducted in two phases: a pre-implementation phase where qualitative interviews, work ow analyses and survey data are used to re ne the PF strategy, based on the Consolidated Framework for Implementation Research (CFIR)(18) and (2) a 12-month implementation phase, guided by Proctor's Implementation Outcomes Framework.(19) The primary outcome of the trial is implementation delity, which is de ned as the degree to which: (1) the implementors (in this case clinic providers and staff) adhere to the ALTA intervention protocol, (2) the dose of ALTA delivered, (3) quality of delivery by the implementors, (4) ALTA component differentiation (i.e., as compared to other practice initiatives), (5) patient exposure to ALTA, and (6) patient and FQHC responsiveness (i.e., satisfaction) to ALTA.(17,19) ALTA is an evidence-based intervention that utilizes a RPM platform that is fully integrated into the electronic health record (EHR) in combination with best practices from the Target:BP MAP [Measure Accurately, Act Rapidly, Partner with Patients] protocol to provide practices with a standardized strategy to improve medication adherence and blood pressure (BP) control (Fig. 1).(20 Three data sources are used to document and characterize adaptations to the PF strategy: (1) meeting minutes of the research and FQHC implementation committee, (2) narrative reports completed by the practice facilitators, and (3) structured notes captured on root cause analysis (RCA) and PDSA worksheets.This data is collected on an ongoing basis throughout the pre-implementation and implementation phases.While data on adaptations is collected throughout the trial, coding of the modi cations according to the FRAME-IS was completely retrospectively by the research team.

Implementation Committee Team meetings
Since the beginning of the trial, members of the research and FQHC teams including the project principal investigator, practice facilitators, FQHC leadership, senior practice management, provider champions, and QI team have met biweekly to discuss ALTA implementation at each of the practices.Topics covered include reviewing EHR data on the uptake of ALTA at each of the practices, discussing barriers and facilitators to implementation, and identifying changes that could be made to support implementation with little disruption to the practice work ow.A member of the research team takes structured notes that document the meeting attendees, agenda topics, and action steps.

Narrative reports
During the implementation phase, the practice facilitators conduct site visits to observe delity to the implementation process.During these visits the facilitators shadow practice staff and providers as patients move through the ALTA work ow and conduct informal interviews to gather information about their experiences implementing the model.Practice facilitators capture notes from the visits on structured narrative reports.

RCA and PDSA worksheets
The practice facilitators complete structured RCA and PDSA worksheets during site visits to the practices.The RCA worksheet captures an issue that is identi ed as serving as a barrier to implementing an intervention component(s), the root cause of the issue(s) (e.g., possible reasons why the issue is happening), and possible action items to address the root cause (with information on what, how, when and with whom).The PDSA worksheet complements data captured on the RCA by testing and documenting the effectiveness of the action items to mitigate the identi ed issue(s).

Data management and analysis
Using the FRAME-IS modules as a starting point, the research team created an Excel spreadsheet that captures: (1) the speci c implementation strategy and modi cation(s) being made; (2) modi cation category (e.g., changes to the content of the strategy or the way it is delivered); (3) nature of the modi cation (e.g., removing/adding/re ning strategies); (4) goal of the modi cation and at what level (e.g., practice, provider, staff); (5) when the modi cation was made (pre-implementation, implementation) and whether it was planned; (6) who was involved in the decision; and (7) scope of the modi cation (e.g., one practice team vs. all participating practices).For our coding, we de ne proactive adaptations as changes to implementation strategies in response to anticipated barriers (e.g., a known change in practice sta ng prior to implementation).Alternatively, reactive adaptations are changes to the implementation strategies due to unanticipated barriers during the implementation process (e.g., in response to a PDSA cycle).Planned adaptations are proactive or reactive changes that are decided using a systematic process that includes consulting the EHR data, discussions with the implementation committee, and consideration of their impact on the implementation and intervention outcomes.Unplanned adaptations are changes made to implementation strategies without a systematic process (e.g., originate from practice staff and providers in response to implementation barriers without consulting implementation committee and/or data).
To facilitate tracking of our multifaceted PF strategy, the team also included elds to document the speci c intervention component the implementation strategy targets (i.e., identify, refer, coach, document, monitor), a description of the original and modi ed implementation strategies using the ERIC taxonomy, and any overlap in the categorization of modi cations for each strategy (e.g., what is being modi ed [module 2], the nature [module 3] and goal of the modi cation [module 4]).Categorization of the original and adapted implementation strategies using the ERIC taxonomy served two main purposes: (1) it facilitated discrete capture of the different implementation strategies used to support integration of ALTA into routine care and (2) allowed the team to determine whether changes to the strategies deviated from the core elements of practice facilitation (i.e., a metric of delity).
We followed best practices from previous research to code and categorize adaptations across the three data sources (Fig. 2).(24-26)Text was extracted from the meeting minutes, narrative reports, and worksheets according to the FRAME-IS modules and inputted into a master matrix for content analysis.Two members of the research team independently entered the raw text data into the matrix from each source.A unique identi er was created for each document type (i.e., meeting minutes = MM; narrative reports = NR; Worksheets = WS, see additional le 1) to track the primary source of information as well as capture whether adaptations were represented by multiple data sources.After all data were entered into the matrix, the coders met to resolve any discrepancies in coding and discuss modi cations to the FRAME-IS modules to adequately capture adaptations to the PF strategy.The research team also reviewed the matrix to identify similarities and differences in categorization of the adaptations across the data sources.Member checking was conducted by a third individual on the research team who reviewed the completed matrix, provided input on the proposed FRAME-IS modi cations, and validated coding decisions.

Results
We identi ed 27 adaptations across the pre-implementation and implementation phases, of which 24 (88.9%) were focused on implementation strategies to support the identify and refer components of the ALTA intervention.Most commonly, adaptations were identi ed through meetings of the implementation committee (41.7%), followed by narrative reports by the practice facilitators (30.6%) and notes on the RCA/PDSA worksheets (27.8%).Below and in the supplemental table (additional le 1), we describe the adaptations based on the FRAME-IS modules as well as the unique categorizations that the research team created to account for the multifaceted PF strategy.
Modules 1-3: What was modi ed and nature of modi cation.
Approximately half (52.9%) of the adaptations were made to the way the implementation strategy was delivered (i.e., context).Of these adaptations, about one third (36.3%) focused on changing the format of the implementation strategy from individual focused to group based.For example, the practice facilitators noted on the narrative reports that participating in morning huddles with the entire care team was a more effective strategy for creating practice goals to implement ALTA than targeting single providers during site visits.The remaining context adaptations focused on changes to who delivered the implementation strategy (personnel, 36.3%) as well as who was the target of the strategy (population, 27.2%).A notable change to personnel included the addition of community health workers (CHW) as core members of the PF strategy.The RCA worksheets noted that patients were experiencing several signi cant technology-related barriers that were negatively impacting uptake of ALTA.With support of the implementation team, the CHWs were integrated into the PF strategy to provide one-on-one technical support to patients.The CHW tasks include assisting patients in nding, downloading, and using the BP apps on their phones and navigating the patient portal.
Another notable contextual adaptation was a shift in the population that was the primary target of the PF strategy.ALTA was originally conceptualized as a MA-led model, however, conversations with the FQHC leadership as well as policy-level considerations for reimbursement (i.e., the outer setting) led to nurses being the primary focus of ALTA.The remaining adaptations were categorized as training and content (17.6% and 29.4%, respectively).Training adaptations were mainly done to support asynchronous, selfpaced learning of the intervention content (e.g., proper BP measurement).Content modi cations included adding elements to the original PF strategy to support peer-to-peer learning and the technological infrastructure required to implement ALTA.As shown in the supplemental table (additional le 1), a comparison of the original and modi ed ERIC categories showed that none of the adaptations removed core elements of the PF strategies and were considered delity consistent.

Module 4: Goals of the modi cation and level
Overall, 23 goals were listed across the multiple adaptations, of which 39.1% were made to increase the reach of ALTA to the patient population with the intent of reducing disparities in the delivery of the intervention.The higher prevalence of increasing reach as a goal aligns with more frequent adaptations made to the PF strategy to support the identify and referral components of ALTA.Examples of modi ed strategies to improve reach were tailoring materials so they are offered in multiple languages and expanding the home BP app capabilities to support iOS and android phones.Adaptations were also made to increase acceptability and adoption (13% each) by practice staff, improve delity in the intervention delivery, the t with the practice work ow, and the clinical effectiveness and sustainability of ALTA (8.7% each).Approximately one-third (36.4%) of adaptations were made at the level of the organization (i.e., primary care practices) and the practitioner (i.e., FQHC providers and staff delivering ALTA).Fewer adaptations were made at the level of the patient and the implementors leading the PF strategy (13.6% each).
Module 5: When were the modi cations made and were they planned.
Except for one modi cation, all changes made to the PF strategy were planned.In most cases (84.6%), the modi cations were reactive and occurred within the implementation phase in response to ndings from the RCA and PDSA cycles.The unplanned modi cation was led by practice nurses who developed a tracking system to identify eligible patients for ALTA and reduce missed opportunities for enrollment.While this adaptation was unplanned and reactive to their experiences implementing ALTA, this strategy was eventually spread to other practices due to its effectiveness in increasing enrollment.
Module 6-7: Who participated in the decision to modify and how widespread were the modi cations.
Decisions about the modi cations were mainly made by the FQHC leadership (40.9%) and members of the implementation committee (36.4%), which comprised both the FQHC and research teams.About onequarter of the decisions also involved the FQHC providers and staff implementing ALTA (22.7%).Adaptations to implementation strategies were mainly widespread (83.9%) and shared across the network of participating practices.In some cases (16.1%), adaptations were localized to individual practices due to their unique implementation climate.For example, peer-to-peer learning among staff was an important strategy modi cation for practices that lacked a manager who could lead the implementation effort.
Considerations when using the FRAME-IS to characterize multidimensional implementation strategies.
As shown in Table 1, we originally conceptualized the PF strategy supporting ALTA as distinct and focused on a singular ERIC category.In practice, the strategies are numerous and often interrelated making discrete data capture di cult when applying the FRAME-IS coding system.Moreover, it is often challenging to distinguish between what is within the scope of the PF strategy, which is inherently designed to be exible, and what could be considered an adaptation.Despite these di culties, there is great value in documenting the ongoing re nement of multifaceted implementation strategies that are needed to support implementation of interventions into real world practice.In this sense, we view the FRAME-IS as a helpful tool to track the ongoing optimization of our PF strategy to improve t of the intervention to the practice context.To guide the documentation process, we considered an adaptation to have occurred if any of the following conditions were met: (1) the change in strategy was made in collaboration with the FQHC in response to practice-level implementation barriers; (2) the change represented a shift in focus from the originally planned strategy to optimize t of the intervention; and/or (3) the change was initiated by practice providers or staff (i.e., unplanned).
To keep track of the multiple changes that occurred we also created a numbering system that allowed us to capture the breadth and depth of adaptations across the modules.Using this method, we created part 2 of module 2 (What is modi ed) to add a detailed list of the various modi cations that were made to each PF strategy.For example, as shown in the supplemental table (additional le 1), rather than solely indicating the modi cation type as 'training', our matrix also included the speci c adaptations to the PF strategy that fell within this category (e.g., using a train-the-trainer method, creating online training for asynchronous learning, etc.).Numbering each adaptation allowed the team to map the changes to the other modules and provide data on the most common ways strategies were modi ed, the different goals of each strategy, when they occurred, and how widespread.

Discussion
In this paper, we describe the application of FRAME-IS to track adaptations to a PF implementation strategy.We identi ed over 20 adaptations to our multifaceted strategy, which were mainly focused on supporting practices in identifying patients with uncontrolled HTN who are non-adherent to their medications and referring them for RPM and health coaching for BP control.These adaptations were commonly made to attain the goal of enhancing the reach of our evidence-based intervention, ALTA.Over half of the adaptations were made to modify the context of the PF strategy to include a group-based format, add CHWs as new team members, and to shift the implementation target to nurses.The adaptations were often widespread, affecting all practices within the FQHC network.While nearly all the adaptations were reactive in nature, they resulted from a systematic (planned) process of reviewing data captured by multiple objective (EHR-derived) and subjective (narrative reports) sources.Finally, most of the adaptations to the PF strategy resulted from a discussion between the FQHC and research teams.To enhance intervention t, it is equally important that the decisions around what, when, how, and why to adapt are made in partnership between the research and practice teams.Incorporating both perspectives into the adaptation process can help to align organizational resources, capabilities, and priorities of the practice while also facilitating rigorous evaluation of implementation outcomes.(27) Our paper adds two key contributions to the growing literature.First, this is one of the rst papers to describe the application of the FRAME-IS modules to a PF implementation strategy.We initially experienced challenges in coding the multifaceted strategies that comprise PF using a singular code.For example, in several cases an adaptation could be characterized as a context change to both the format and personnel.Two previous studies described similar experiences when using the FRAME-IS to document adaptations to multifaceted implementation strategies supporting integration of practicebased interventions.(13,14) For example, Mangale et al. (13) found that adaptations to the implementation strategies were not mutually exclusive and t within several categories resulting in 'unavoidable overlap.' Haley et al. ( 9) also described challenges to tracking adaptations to a multifaceted implementation strategy using existing frameworks.Like our approach, the authors addressed these challenges by adding more detailed information on the changes using the ERIC taxonomy and developed new codes to augment the broad categorization of strategies offered by current frameworks.In our study, we also created a numbering system that allowed us to map the strategies across the FRAME-IS modules.Using this method, we were able to document each individual adaptation within a strategy as well as view the bigger picture of how the combination of adaptations supported implementation of ALTA.Moreover, use of the ERIC taxonomy to guide documentation of the original and modi ed implementation strategies enhances the generalizability of our work, as it supports transparency in identifying implementation strategy adaptations as well as the usage of common terminology to describe the strategies.
Second, this research provides key insights into the adaptations that are needed to support implementation of evidence-based interventions into safety-net settings.In this study, all adaptations were made in collaboration with the FQHC to further tailor the PF strategy to the inner (e.g., sta ng models within each practice) and outer (e.g., reimbursement opportunities for health coaching led by nurse practitioners) practice setting.Interestingly, despite inclusion of a CFIR-informed preimplementation phase that was designed to re ne our PF approach, many of the adaptations were not identi ed until transitioning to the implementation phase.This may re ect the dynamic nature of safetynet settings who experience uctuations in their sta ng and resources (e.g., staff turnover) as well as hidden challenges that do not become apparent until the intervention begins, which in our case were technology-driven.Martinez et al.,(15) also used the FRAME-IS to characterize adaptations of an integrated care model in FQHCs.Like our study, they found that adaptations were made to include practice providers and staff as key partners and to re ect service delivery changes experienced by the FQHCs, increase reach of the intervention, and leverage opportunities for reimbursement.
This study also has several limitations.First, while the team captures adaptation data throughout the trial, the coded was completed retrospectively.This may have resulted in missed opportunities to capture real time changes in strategies overtime.Second, most of our data was from qualitative reports completed by the practice facilitators, which may be subject to bias.Future data should include a greater diversity of data sources that combine qualitative and quantitative measures to capture adaptations to the implementation process.Finally, while we documented goals for each adaptation, we did not track whether the changes resulted in the intended implementation outcome (e.g., improving adoption of ALTA).However, once our trial is complete, we plan to publish ndings on the effectiveness of our PF strategy on level of adoption and implementation delity of ALTA.

Conclusion
Applied examples of the FRAME-IS to document adaptations to multifaceted implementation strategies are needed to further re ne the tool and increase its utility for diverse types of implementation science research.This study discusses our experiences using the FRAME-IS to document adaptations to a PF strategy that is supporting the implementation of an evidence-based technology facilitated team care model within a large FQHC.With some modi cations, we demonstrate the ability to code our adaptation data across the FRAME-IS modules, attesting to its applicability and value for a range of implementation strategies.
Based on our experience, we offer the following recommendations for future research and practice.First, tracking systems need to evolve to allow for more nimble and practical documentation of iterative, ongoing, and multidimensional adaptations in real-time.(1,8) While survey tools that embed branching logic can streamline documentation in support of these efforts, it remains essential that the number of questions and list of response options do not become too onerous for implementors to complete.Leveraging emerging best practices in natural language processing and machine learning is one avenue worthy of future exploration.(28)Figures ALTA Team-based Intervention.