Barriers and Facilitators To Implementation and Sustainment of 17 New Medicaid Accountable Care Organizations in Massachusetts: A Study Protocol

Background: Accountable care organizations (ACOs) utilize value-based payment models, which incentivize quality of care and cost reduction through shared nancial risk contracts for payers and providers. The impact of ACOs on cost and quality has been mixed for Medicare and commercial insurers, but the model has yet to be extensively tested in the Medicaid system, which insures a large number of patients in socioeconomically disadvantaged populations. Additionally, despite substantial heterogeneity in ACO models, the majority of ACO studies have examined ACO outcomes without exploring the potential impact of implementation and sustainment on these outcomes. Understanding barriers and facilitators to implementation and sustainment of Medicaid ACOs will help to better understand their impact on patient care, outcomes and costs for a vulnerable population. Methods and Design: The state of Massachusetts (MA) approved 17 new Medicaid ACOs and associated Community Partner (CP) organizations in 2018 as part of a large-scale pragmatic experiment in healthcare reform. The new ACOs will receive $1.8 billion dollars in state and federal funds over ve years aimed at supporting implementation and sustainment the new model. This study aims to identify barriers and facilitators to implementation and sustainment of activities supported by these funds using the Consolidated Framework for Implementation Science (CFIR) as a guiding framework through: (1) review of administrative documents classify organizational characteristics of the ACOs and CPs including plans for innovation; (2) key informant interviews (KII) with ACO and CP leaders, governmental administrative leaders, and patients; (3) case studies of ACOs and CPs; and (4) a survey of front-line providers and staff in the ACOs and CPs. Descriptive quantitative statistics will be used to analyze document and survey data and framework analysis will be used to analyze KII and site visit data. Discussion: The new Medicaid ACOs in MA aim to improve care integration, quality of care, and patient experience while reducing costs through innovations in healthcare delivery and payment. Understanding the barriers and facilitators to implementing and sustaining the ACO model will provide critical context for understanding the overall impact of the Medicaid ACO experiment in MA.

Discussion: The new Medicaid ACOs in MA aim to improve care integration, quality of care, and patient experience while reducing costs through innovations in healthcare delivery and payment. Understanding the barriers and facilitators to implementing and sustaining the ACO model will provide critical context for understanding the overall impact of the Medicaid ACO experiment in MA.

Contributions To The Literature
This study will be one of the rst to systematically identify barriers and facilitators to implementing and sustaining a large-scale statewide Medicaid ACO program across the duration of the 5-year program.
Understanding implementation barriers and facilitators will provide important context for interpreting the overall impact of the ACO experiment on quality and costs of care.
Findings of the study are expected to have utility for policy-makers and health system leaders considering implementation of innovative ACO models in the U.S. and abroad.
Background Page 3/14 Global efforts to address the rising costs of healthcare while maintaining or improving quality of care have increasingly included implementation of accountable care organizations (ACOs) (1)(2)(3). The valuebased payment models used by ACOs incentivize quality of care and cost reduction through payerprovider partnerships in which nancial risks are shared. The ACO model aligns nancial incentives with improved care integration and coordination across health and social service sectors (4), differing from fee-for-service payment models' prioritization of volume and intensity of care. This important shift in nancial incentives is intended to promote prevention and population health, which has the potential to improve healthcare delivery and clinical outcomes for patients at higher risk for experiencing healthcare inequities and health disparities.
The majority of research on ACOs to date has focused on changes in quality of care, costs, and patient outcomes associated with ACO programs (5). Studies of upstream barriers and facilitators to implementing and sustaining the changes in healthcare delivery ACOs are intended to bring about are critical for interpreting the downstream effects. However, prior studies of ACO implementation have focused on Medicare ACOs or been limited to a narrow timeframe or scope (6)(7)(8) and none to our knowledge have examined barriers and facilitators to long-term sustainment over time.
The state of Massachusetts's (MA) Medicaid program (MassHealth) launched 17 new ACOs and 27 associated Community Partner (CP) organizations in 2018 as part of a ve-year experimental demonstration project, subsequently referred to as the Demonstration. The CPs work with ACOs to coordinate and manage care for patients with behavioral health diagnoses or for those who need longterm services and supports (LTSS), an innovative feature of the MA model. This study will aim to identify barriers and facilitators to implementation and sustainment of interventions funded by the Delivery System Reform Incentive Payment (DSRIP) that are intended to support transformation of healthcare payment and delivery to the ACO model for Medicaid bene ciaries in Massachusetts (MA). The protocol for studying the impact of the ACOs on quality and cost measures will be reported elsewhere.

Methods And Design DSRIP Program and ACO Payment Models
The DSRIP program provides funds to support transformation of healthcare delivery through: (1) implementation of payment and delivery system reforms that promote member-driven, integrated, coordinated care and hold providers accountable for the quality and total cost of care; (2) improvement of integration among physical health, behavioral health, long-term services and supports, and healthrelated social services; and (3) support for providers to ensure continued access to care for patients insured by the Medicaid program (9). MassHealth will provide up to $1.8 billion USD in federal and state DSRIP funding through four funding streams over the ve years of the Demonstration: direct payments to ACOs (60%); direct payments to CPs (30%); the Statewide Investment program (SWI) (6%); and administrative costs for MassHealth (4%). The SWI program primarily funds activities related to workforce development and retention, technical assistance, enhanced diversionary behavioral health activities, and increasing access for members with disabilities or for whom English is not the rst language (9). Each ACO was allowed to develop a plan for use of DSRIP funds that were tailored to the ACO's implementation and sustainment needs. The majority of the ACOs (n=13) have a shared risk contract in which they receive capitated payments from MassHealth based on enrollment and member risk scores and take on full insurance risk for the population (Appendix A) (10). As part of these contracts, shared risk payments to/from all 17 ACOs are subject to adjustment based on reporting and performance on 22 quality measures, including two related to patient experience (11). The activities funded by the DSRIP program support ACOs and CPs in achieving their quality and nancial goals.

Study Overview
The study's theoretical framework draws on the Consolidated Framework for Implementation Research (CFIR) (13). CFIR was chosen as the theoretical framework for the study because its ve major domains (Intervention Characteristics, Outer Setting, Inner Setting, Characteristics of Individuals, and Process of Implementation) and the constructs within the domains are pertinent to studying implementation of complex interventions, such as the Demonstration. CFIR is also an appropriate framework for this study because its exible structure is designed to be used across multiple phases of a study, from design through dissemination. Each of the ve domains is explored in at least one of the phases of data collection.
Study procedures will include the following: (1) Document Review to characterize ACOs and CPs; (2) Key Informant Interviews (KIIs) with ACO, CP, and MassHealth leadership as well as MassHealth members in two waves; (3) Site visits conducted with select ACOs and CPs; and (4) Surveys of frontline ACO healthcare providers and CP staff. Methods are described in detail below and the timeline and goals of the elements of the study are outlined in Table 1. The Institutional Review Board (IRB) at the investigative team's institution determined that the overall study did not constitute human subjects research. Each phase of primary data collection will be re-submitted for IRB review.
(1) Systematic Characterization of ACOs and CPs The Demonstration allows for substantial exibility in how the ACOs and the CPs are structured and how they plan to utilize DSRIP funding. Given this heterogeneity, the rst step will be to systematically characterize each ACO's and CP's organizational structure, patient population, and implementation plan through review of the Participation Plans submitted by each ACO and CP prior to being approved to participate in the Demonstration. The Participation Plans vary in format, but all include ACOs' and CPs plans for their governing structures, a description of their patient population, and plans for DSRIP implementation. The Participation Plans provide a framework for how each organization intends to change elements of care delivery -from health information technology to addressing health related social needs and more.
Members of the investigative team will review the Participation Plan for each ACO and CP and extract the data necessary to characterize them, focusing on the following CFIR domains: Intervention Characteristics (speci c plans for use of DSRIP funds); Outer Setting (population characteristics), and Inner Setting (governance structures, partnerships/networks, and prior experience with value-based care models). The data will be summarized in streamlined reports that provide systematic categorization The reports will be made available to the teams conducting KIIs (See section 2) to enable them to tailor interview questions to pertinent aspects of each ACO's or CP's unique organizational structures and DSRIP implementation plans.
(2) Key Informant Interviews Two waves of semi-structured in-depth interviews will be conducted with representatives of four stakeholder groups: (1) ACOs; (2) CPs; (3) MassHealth staff responsible for administering the DSRIP program), and (4) MassHealth patients. Sample sizes for each group are intended to strike a balance between breadth and depth and to achieve theoretical saturation while minimizing respondent burden (Table 2). Interviews will be conducted with each stakeholder group at interim and end-points of the Demonstration; efforts will be made to interview the same respondents in each wave to reduce the chance that any changes reported may be more re ective of change in participant rather than change in implementation.

Sampling and Recruitment
ACOs and CPs will be noti ed by MassHealth that the investigative team will be reaching out to invite them to participate in the interviews. The investigative team will then send a standardized introductory email to the contact listed on the ACO's or CP's Participation Plan document. The e-mail will brie y explain the goals of the KIIs and will ask the contact to identify the appropriate key informants in their organization; the e-mail will include an attachment with a synopsis of the study. The investigative team will then contact the key informants identi ed by the ACO or CP contact via e-mail to address any questions and to schedule the interview.
A Senior Manager at MassHealth will provide the investigative team with contact information for MassHealth leaders responsible for administering the DSRIP program. These representatives will be invited to participate in an interview via e-mail. Sampling will aim to achieve a breadth of experience administering the DSRIP program.
For interviews with MassHealth members, MassHealth leadership will inform contacts at the ACOs and CPs that the investigative team will be reaching out to them to identify members who may be willing to share their experiences with changes in healthcare related to the DSRIP program. To understand the needs of as many members as possible, the investigative team will review the nominations and purposively recruit members who are most likely to have experienced changes in care delivery related to the DSRIP program due to the following conditions: medical complexity (multiple medical conditions, which may involve multiple medications, or high utilization of medical care); living with disabilities; receiving LTSS, receiving behavioral health services through a CP with a particular emphasis on members with substance use disorders (SUD); and parents of children utilizing MassHealth (Table 3). Members who have conditions or life situations that place them in multiple categories (i.e., medical complexity and raising a MassHealth pediatric member) will be recruited based on one of the target conditions. Interviews Interviews will be conducted by trained research staff members using semi-structured interview guides tailored to the interview population (Appendix B). This phase of the study was deemed exempt, but the interviewer will review a study fact sheet with the participant and answer questions prior to beginning the interview. The interview guides for ACO and CP leadership were developed by the investigative team and pilot-tested with one ACO and one CP. For the member interviews, an external stakeholder with extensive experience in this arena reviewed the guides for accessibility. Interviews are expected to last approximately sixty to ninety minutes and will be audio recorded and professionally transcribed.
Interview questions for ACO and CP leaders will elicit perspectives on state actions to support delivery system transformation and the effectiveness of these actions. The interviews will aim to understand the factors that facilitate and impede organizational transformation in relation to three CFIR domains: Inner Setting, Outer Setting, and Process domains. For example, ACO leaders will be asked how prior experience with non-Medicaid ACO and value-based payment models informed implementation (Inner Setting), and CP leaders may be asked what, if any, factors external to the organization and the DSRIP program have facilitated or hindered implementation of DSRIP activities (Outer Setting).
Interviews with MassHealth leaders will similarly focus on implementation of the DSRIP program, but given their high-level administrative roles, interview questions will also explore implementation of the DSRIP program in the broader context of program and policy implications for the future. Interview methods will otherwise follow those described for ACO and CP interviews.
Interviews with MassHealth members and caregivers of pediatric MassHealth members will be conducted via telephone. To ensure member interest, accessibility requirements, and understanding, an initial outreach call will be made to inform members about the project, determine if there are any barriers, such as language or disability, and schedule a time to conduct the interview. Efforts will be made to involve translation services or to accommodate other needs as they arise. Members will be given a $50 gift certi cate as a thank you for their time and candor.

Analysis
Interview data will be analyzed using framework analysis (14,15) and will focus on barriers and facilitators to implementation and sustainment within the three CFIR domains emphasized in the interview guides. Speci cally, interrater reliability will be established among coders (through a process of concurrent coding of an initial set of interviews, comparison of coding approach, and re nement of code de nitions as needed); the remaining interviews will be coded independently by members of the analytic team. Once all interview data are coded, secondary coding (combining codes and creating sub-codes) will be performed and analytic matrices with the nal coded data created to facilitate across-and within-stakeholder group analysis with respect to perceptions of state actions supporting delivery system transformation, barriers and facilitators to care, and the overall member experience. Dedoose software (16) will be used to manage, code and analyze interview data, and calculate Cohen's Kappa coe cients (17) to monitor agreement among coders over time.

(3) Case Studies
Two waves of site visits will be conducted with a subsample of ACOs and CPs at interim and end-points of the Demonstration to inform case studies ( Table 1). The rst and second wave of site visits will aim to achieve a deeper understanding of the speci c delivery system innovations that ACOs and CPs are implementing and the contextual factors that may be facilitating or impeding implementation of the DSRIP program (18). The second wave of site visits will also seek to achieve a thorough understanding of facilitators and impediments to sustainability of the ACO and CP models after the end of the DSRIP program.

ACO and CP Sampling
In the rst wave of site visits, the investigative team will examine up to four ACOs and four CPs that have achieved different levels of success in transforming care delivery for their MassHealth members. ACOs and CPs will be selected based on a combination of: (1) their progress in implementing DSRIP-funded projects and (2) differences in organizational structure and populations served. The timing of the site visits will be determined by what is learned from the other data sources with respect to the two dimensions sampling is based upon. For instance, if the investigative team is able to identify ACOs and/or CPs that excel on implementation of the DSRIP program or are struggling by the second year of the Demonstration, each could be the subject of a site visit. At the same time, it may take until Year 3 of the Demonstration for such patterns to emerge. In sum, we will conduct up to eight site visits between Year 2 and Year 3 (Table 1). For the second wave of site visits, up to four ACOs and four CPs will be selected in Year 5 that represent various levels of performance as de ned by level of change and/or achievement related to accountability scores being used by MassHealth to determine shared risk payments. For both waves, the site visits will focus on the care delivery transformation activities related to DSRIP that the ACOs and CPs have initiated and the barriers and facilitators to effective implementation, performance and sustainability.

Site Visit Procedures and Case Study Development
Semi-structured interviews and focus groups will be conducted with front line providers and staff who are closely involved with DSRIP implementation and who represent a range of functional roles during site visits. Participants in site visit interviews and focus groups will differ from participants in the KIIs, focusing on those responsible for coordinating and delivering clinical care at the ACO and CP practice sites. Participants will include: (1) clinical leads (e.g., medical directors and nurse managers); (2) operational leads (e.g., o ce managers); (3) heads of health information technology (HIT)/health information exchange (HIE); (4) heads of quality improvement; and (5) heads of support services such as case management. In addition, we will interview representatives of ACO governing boards, Patient and Family Advisory Committees, and selected CPs. At CPs, interviews will be conducted with the following functional roles: (1) clinical leads; (2) administrative directors of CP programs; (3) heads of Health Information Technology/Health Information Exchange. Interview guides will cover similar topics/CFIR domains to those used in key informant interviews with leaders, but will explore more pragmatic aspects of implementation experienced by front-line providers and staff, including constructs in the Characteristics of Individuals domain.

Analysis
Analysis of semi-structured interview and focus group data will follow the process described for KIIs to construct a case study for each site. (18) In addition, the site visit data will be triangulated with data collected through the other study data sources to compare and contrast perspectives of those in different roles within the ACO and to explore how the site visit data con rm or con ict with related data from other sources.

(4) Survey of Front-line Providers and Staff at ACOs and CPs
To understand how a large sample of front-line providers and staff (e.g., community health workers, social workers, MDs, DOs, NPs, PAs, nurses) experience changes in care delivery related to the DSRIP program, two waves of front-line provider and staff surveys will be conducted at interim and endpoints of the Demonstration. Surveys will aim to assess the degree to which implemented projects and ACO/CP formation are translating into changes in care delivery from the perspective of front-line ACO providers and CP staff. The survey will provide an opportunity to quantitatively measure and compare these experiences between groups of providers, practice types, and ACOs that differ in important characteristics.

Questionnaire Development
The questionnaire used for the survey will incorporate the previously validated measures of perceptions of care integration from the Provider and Staff Perceptions of Integrated Care (PSPICs) (19) and new questions developed and pilot tested by the investigative team to address speci c aspects of implementation of the DSRIP program. CFIR domains explored will include Inner Setting (care coordination within the practice site and with external providers and community resources); Outer Setting (patient engagement, MassHealth policies and processes, payment and nancial incentives) and Processes (practice site structures and processes). The questionnaire will be pilot tested with a convenience sample of approximately 10-15 ACO providers and 5-10 CP staff with similar roles to those to be included in the survey sample. Pilot testing will include cognitive testing and assessments for clarity, completeness and respondent burden.

Sampling and Administration
The sampling frame for the ACO provider survey will include providers practicing at group practices, community health centers, and hospital licensed health centers participating in the ACO program at the time the program launched (i.e., 2018). Providers at solo physician practices, outpatient hospitals, practice sites located outside of Massachusetts, sites with fewer than 50 MassHealth members, and sites with an unknown number of MassHealth members will be excluded. From within the sampling frame, up to 30 practice sites per ACO (including all sites for those with less than 30 sites and a random sample of practice for ACOs more than 30 practice sites), thereby oversampling the ACOs with fewer practice sites. The providers practicing at the 353 unique practice sites in this sample will constitute the sample frame for the survey of ACO frontline providers.
The investigative team will collect provider contact information from practice and ACO administrators. The questionnaire will then be emailed to a random sample of eligible providers (MDs, DOs, NPs, PAs, RNs, LPNs, and LCSWs) for each ACO. Strati ed random sampling is expected and the sampling fraction will vary by ACO and provider type such that less prevalent characteristics are oversampled. The contact information for CP staff will be collected from administrators at all 27 CPs and the questionnaire will be emailed to a random sample of staff. As with the ACOs, strati ed random sampling is expected and the sampling fraction may vary by CP staff roles such that less prevalent roles are oversampled. The required sample size will be determined based on anticipated response rates and power calculations performed prior to random selection of providers and staff.

Analysis
The results of each survey wave will be analyzed overall, by ACO characteristics, practice site characteristics, and by provider/staff characteristics to explore heterogeneity in provider/staff perspectives of the ACO and CP programs. Changes over time between wave one and wave two of the survey will also be examined overall, by ACO characteristics, practice site characteristics, and by provider/staff characteristics. Findings from the survey will be used to measure provider/staff understanding of the ACO and CP programs, their perceived effectiveness, and the concordance of perceptions between front-line providers/staff and their organizational leaders. Findings will also be used to assess the relationship between providers' perceived experience of transformation and ACO/CP care quality and cost performance. In addition to crude analyses, sampling and non-response weights will be applied to obtain estimates that are adjusted for the multi-stage sampling approach and observed sources of non-response bias.

Dissemination
The study protocol is available to the public on the mass.gov website. Results of the study will be disseminated through multiple channels: (1) peer reviewed journal publications; (2) presentations at national research meetings; (3) publicly available reports to the Center for Medicare and Medicaid Services; (4) publicly available summaries posted on the MassHealth website; and (5) directly to key stakeholders in ACO and CP leadership.

Potential limitations
The Massachusetts ACOs are a large-scale pragmatic intervention aimed at improving quality of care, reducing costs of care, and improving patient experience for Medicaid bene ciaries. As such, the DSRIPsupported interventions to facilitate implementation and sustainment of the new ACOs are taking place in uncontrolled settings. Although this limits the capacity to compare speci c strategies for implementing the interventions and changes each ACO undertakes, it also allows for in-depth study of the implementation in a real-world setting.

Discussion
This study will be one of the rst to identify barriers and facilitators to implementation and sustainment of interventions funded by state and federal governments to support implementation and sustainment of Medicaid ACOs. The population insured by Medicaid is one that experiences socioeconomic and racial/ethnic health vulnerabilities and disparities and healthcare inequities. Traditional models of healthcare delivery are not designed to address the complexity of these issues, but the ACO model may represent an opportunity to better address social determinants of health as part of healthcare. This study is expected to offer important insights into the mechanisms of transforming healthcare delivery and nance to meet the complex medical and social needs of patients insured by Medicaid. This study was reviewed by the Institutional Review Board at the University of Massachusetts Medical School-Worcester and was determined to not meet the criteria for human subjects research. Further review will be requested for each phase of the study, as warranted by participants to be recruited and data to be obtained.

Consent for publication
This is not applicable for this manuscript.
Availability of data and material Data will be made available upon reasonable request.

Competing Interest Statement
None of the authors report a competing interest.

Funding Statement
The study was funded by the Centers for Medicare and Medicaid Services as part of the Medicaid 1115 Waiver Program. MassHealth, the state Medicaid administrative body, provided feedback on the study's design , will have access to data,and will review all materials disseminating study results prior to publication but does not ultimately decide on publication. SG's time was funded in part by an NIH Career Development Award (K23HD080870). MA's time was funded in part by the National Center for Advancing Translational Sciences (TL1TR001454).

Authors' Contributions
SG contributed to development of the protocol to study the implementation of the DSRIP program and led writing of the manuscript. DG led development of the protocol, reviewed the manuscript and approved the nal version. MA, AK, and JN contributed to development of the protocol, revisions of the manuscript, and approved the nal version of the manuscript. JH oversaw development of protocol, reviewed the manuscript, and approved the nal version of the manuscript.