Applying Implementation and Team Science Principles to Facilitate Cancer Program Implementation in Remote, Under-resourced Areas

Background: Failed implementation of health programs is common and may contribute to health disparities in resource-poor communities. We aimed to (1) evaluate barriers and facilitators for implementation of cancer survivorship services in rural communities and to (2) identify optimal strategies for successful implementation of survivorship care programs in these settings. Methods: The study design was guided by the Consolidated Framework for Implementation Research (CFIR) and based on the core principles of effective teamwork. We collected qualitative and quantitative data to assess barriers and facilitators to implementation of rural cancer survivorship care (3 focus groups, size 8, 31, and 77). Data were collected using both in-person and web-based approaches (semi-structured interviews, stakeholder surveys, ThinkTank, project online portal, Google Analytics). Stakeholders included cancer survivors, their families and caregivers, local public services administrators, health providers, and allied health-care professionals from rural and remote communities in Upstate New York. Results: Patients reported preferences for cross-region team-based survivorship care and emphasized the importance of including local providers on regional care teams and networks. Synthesis of results was guided by teamwork principles. Most rural patient stakeholders trusted recommendations received from their local providers (n=6, 86%) and preferred receiving care locally (n=6, 86%). Involvement of rural patient navigators (average rank 1.5 out of 5) and county care managers (1.9) was ranked more favorably than reliance on telemedicine alone (3.3) or visiting practice facilitators (3.3). Rural counties that have existing formal and informal networks involved in cancer care scored higher on the metrics of teamwork and demonstrated more advanced readiness to change in regards to survivorship program implementation. Conclusions: Our analysis identied a unique combination of community socio-economic factors, geographic isolation, and limited provider supply common in rural care settings as barriers to optimal delivery of cancer survivorship care in rural settings. We propose teamwork training and facilitation as an innovative implementation strategy to overcome these barriers and minimize their effect on patient access to care. Merging implementation science and team science frameworks could further assist with program adaptation to minimize implementation failure and improve continuity of care for complex cancer pathway management. Provider availability and care delivery patterns vary greatly among the rural counties. 23 We observed signicant variation in the structure of county health departments, availability of county hospitals, cancer and primary care clinics, cancer services programs, availability of public transportation, and existence of local formal (e.g., provider collaborations) and informal (e.g., patient support) networks. Other studies have noted similar challenges that necessitate extensive tailoring and adaptation of evidence-based programs and care protocols to t the needs of rural community practices and their patients. 124,125 Despite the observed variation in resources and infrastructure, rural patients across all regions were consistent in their preference to involve their local providers, both primary care and oncology, in their cancer care as much as possible. Rural cancer patients and their caregivers also reported signicant challenges with identifying relevant information and making decisions affecting their cancer care, from treatment options to insurance coverage to disability payments and palliative care options. These results present a challenging paradox: patients prefer local care but report distance to care as a barrier. Within the context of the COVID-19 pandemic and the current acceleration of telemedicine roll-out, there may be unique opportunities for enhancing the technology for virtual oncology care and patient receptiveness to telemedicine technology, though patient preferences may change in the post-COVID-19 era.


Contributions To The Literature
Program implementation failures are common and result in inadequate access to services and poor health outcomes. Low-income, rural, and remote communities are most likely to experience implementation failure that could further exacerbate health disparities.
Applying implementation science frameworks and pre-implementation approaches to identify implementation barriers and facilitators could be critically important in areas with limited workforce and resources and help avoid program implementation failure.
In this study we propose a novel implementation approach for adapting complex programs designed for large healthcare systems to the needs and capacity of rural patients and healthcare providers with limited workforce availability.

Background
In the last 20 years, many new clinical and public health programs have been developed to improve quality and continuity of care for geographically isolated populations; however the impact of these programs has been mixed. [1][2][3][4] Limited evidence is available to guide the selection of appropriate implementation strategies in underserved and under-resourced settings. Differences in sta ng, technology, nancing and leadership between rural clinics and large urban care centers pose additional challenges for adaptation, implementation and dissemination of evidence-based practices into rural communities. Implementation science can provide framework and approaches for developing, adapting, and evaluating programs and interventions to t the needs of various populations and settings. 5,6 One special population in the United States that experiences both patient-and regional-level access-to-care barriers are rural cancer patients.
Rural cancer patients and those living in remote regions are signi cantly more likely to experience problems with accessing health care and have worse health outcomes when compared to non-rural populations. [7][8][9] In recent decades, cancer outcomes have improved dramatically due to improved screening and advances in cancer treatments. 2,10−13 As a result, cancer patients are more likely to survive, live longer, but also experience chronic somatic/psychosocial morbidities and socio-economic stresses, making cancer a long-term condition requiring ongoing monitoring and management over the remainder of survivors' lives. 2 Cancer care has also become increasingly complex and requires comprehensive coordination between primary care providers, oncologists, and numerous cancer and ancillary specialists, often across several institutions. [14][15][16][17][18][19][20] While cancer puts a heavy physical, emotional and nancial burden on all patients, rural patients are faced with an additional constellation of challenges related to the uniqueness of their geography and limited access to healthcare . [21][22][23][24][25] Provider and care delivery networks are an exception in rural areas of the US. 26 Rural cancer patients face problems due to long-distance care coordination and challenges of communication among providers from multiple regional systems, limited provider and services availability, and geographic and social isolation. [26][27][28][29][30] Recent studies demonstrated that quality of cancer care depends upon effective coordination between multiple treatment teams and providers. 11-13,31−33 Many cancer patients experience barriers to accessing high-quality cancer programs; this is particularly true for rural patients. 34 Rural patients and their caregivers often face a choice between seeking cancer care at a specialized cancer center hundreds of miles away or being treated by local non-specialized providers. These decisions are often based on insurance restrictions, availability of accessible transportation, and word of mouth recommendations from friends and family and rarely incorporate information about quality of care and outcomes. 35 As a result, a large proportion of cancer patients do not receive coordinated multidisciplinary team-based cancer care, and that proportion is even greater among rural patients. 14-16,36−45 Guided by the Consolidated Framework for Implementation Research (CFIR), in this study we aimed to identify unique barriers and facilitators to implementing a cancer survivorship program in rural communities of Upstate New York and propose targeted evidence-based implementation strategies consistent with our ndings. 6,46 Our analysis is a pre-implementation study essential for successful implementation, maintenance and sustainability of cancer programs in underserved regions. The study was designed to account for the "realworld" complexity of a regional program implementation (limited resources, competing priorities, geographic variation in sta ng availability and stakeholder preferences, among others), with a speci c focus on potential for scale-up/expansion.

Theoretical framework
We used the Consolidated Framework for Implementation Research (CFIR) to identify the WHAT, WHO, HOW and WHERE factors that could facilitate or impede implementation of a regional cancer care program. We speci cally focused on the factors related to unique needs of rural populations, settings and resources. 6 The CFIR is a framework particularly well-suited for implementation of multi-level and multi-component programs. 6 The CFIR is a determinant framework, designed to guide integration of research evidence into practice, adoption, and implementation of evidence-based interventions (e.g., cancer survivorship programs developed for large urban academic cancer centers) by de ning social, behavioral, and economic factors that facilitate or impede implementation (e.g., unique characteristics of rural counties and populations). 6 The CFIR includes ve domains (intervention characteristics, outer setting, inner setting, characteristics of the individuals involved, and the process of implementation), which are subsequently detailed into over 30 different constructs, or "sub-domains". 6 Overview of the study design This study follows sequential exploratory study design. The study took place between January 2016 and December 2017.To identify barriers and facilitators to implementation of evidence-based programs in under-resourced communities, we used data collected during the design, development, and implementation phases of the Patient-Centered Outcomes Research Institute (PCORI)-funded Engagement Award entitled "Virtual Rural Oncology Community" (V-ROC). The overarching goal of the V-ROC was to assess barriers and facilitators to cancer care in rural areas and ultimately to improve access to guideline-recommended cancer care for rural patients and thereby reduce cancer disparities between rural and non-rural communities.
Guided by the stakeholder engagement principles, a core multidisciplinary team of key informants was identi ed in a single rural county (phase I) who helped us identify and recruit a broader patient and provider network in the same county (Livingston County, NY) (phase II). 47,48 Subsequently, the process was repeated in 7 surrounding rural counties (Fig. 1) and then we linked the county networks for information and experience exchange and validation. The V-ROC cancer survivorship stakeholder network included healthcare professionals, cancer survivors, their families and caregivers, and county health department staff. Health professionals included primary care providers, medical and surgical oncologists, and advanced practice providers (APPs; these are registered nurses and physician assistants) working in a variety of clinical settings, as well as care managers and practice administrators.
The University at Buffalo institutional review board reviewed the study and made a Not Human Research determination (UB IRB STUDY00004348).

Data Collection
We rst collected qualitative data through focus groups and semi-structured interviews with the stakeholders to identify the most relevant factors and parameters driving decisions about cancer treatment, and attitudes toward practice change and innovation. Based on the qualitative ndings from the initial period of the study, the stakeholders' preferences and attitudes were subsequently assessed through structured surveys. Finally, we integrated both qualitative and quantitative ndings using a joint display approach to identify stakeholderacceptable and feasible implementation strategies. 49 The study procedures and data collection approaches are described in the sections that follow.

Focus groups:
In total, we conducted 15 focus groups, that included 77 participants and lasted 8hrs 20 min. Three types of focus groups were conducted: rstly, a virtual cross-disciplinary focus group (group 1, n = 8, 90-minute duration). This focus group included physicians, nurses, healthcare administrators, rural APPs, care managers, social workers, and public health o cials. The ThinkTank collaborative software was used to screen share, communicate virtually in real time, express preferences, and con dentially rank participants' responses and yield quantitative data for this focus group.
Secondly, face-to-face focus groups with rural primary care physicians, surgeons, medical oncologists, oncology nurses, care managers, and practice administrators (group 2, n = 15, 30-minute duration; and group 3, n = 16, 20-minute duration). These focus groups were conducted during monthly regional care coordination meetings.
Participants for each focus group were aged 18 years or older and involved in cancer treatment or care that took place in a rural, nonacademic community hospital. These focus groups were conducted at Congregate Meal Program Sites.

Semi-structured interviews:
Potential participants were identi ed for semi-structured interviews (n = 5) from the pool of focus group participants with targeted sampling aimed to include a diverse stakeholder population. Interview questions were informed by the literature, but also included open-ended questions to encourage robust data collection. [50][51][52][53][54][55] Guides for semi-structured interviews were developed and addressed the following themes: awareness of cancer survivorship care guidelines, provider con dence in recognizing/addressing patient survivorship issues, access barriers and facilitators to care in rural settings, and provider beliefs/attitudes regarding their role in clinical decision-making. Interviews with oncologists included an assessment of practice needs and capacity.
Quantitative data collection: Following each focus group, we collected quantitative surveys using web-based reporting tools (focus group 1) and paper questionnaires (groups 2-15) from the focus group participants. The questionnaires were based on the set of questions and priorities identi ed by the participants of focus group 1 and project key informants. To assess regional healthcare provider availability, local provider wages and county public health resources related to cancer survivorship, we used data from publicly available sources. 8,56−63 Data Analysis Qualitative Data Analysis Interviews were recorded and transcribed by the study coordinator and analyzed and interpreted by the V-ROC team. We used the qualitative study checklist and Consolidated Criteria for Reporting Qualitative Research, developed by Tong et al. 64 To interpret meaning from transcribed interviews, we used content analysis and grounded theory, widely used qualitative research techniques. 65 We speci cally focused on barriers and facilitators identi ed by the participants mapped onto CFIR constructs as relevant for implementation of a regional cancer survivorship program. 6

Quantitative Analysis and Data Triangulation
To examine the feasibility of implementing guideline-recommended survivorship care programs in rural communities, we compared economic and socio-demographic characteristics across the participating rural regions. To assess generalizability of the qualitative ndings across the regions, we interpreted the results of the qualitative analysis in the context of local healthcare resources availability and local referral patterns. 8,56− 63 Maps in this paper were generated in part using SAS Software version 9.4 © SAS Institute Inc. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc., Cary, NC, USA.

Results
Regional and Population Characteristics (Structural Characteristics, Implementation Climate) The total population of Livingston County, NY is 63,227; V-ROC county populations range from 17,912 (Schuyler County) to 95,796 (Steuben County); the total population of all V-ROC counties is 445,827 (Table 1). 62,66 In Livingston County, NY, the population has decreased by 1.6%, compared to a 2.2% gain in New York State, overall. 56 All other participating counties also experienced similar population reductions, compared to overall gains in NYS and in the US overall. 56   Aside from retail, education and healthcare, the main industries in the region include mining, quarrying, oil and gas extraction, construction and utility, and farming, . 68 Agriculture is a major income generator in many V-ROC counties; however, there is tremendous variation in agricultural market share (state rank of agricultural products sold at market value in V-ROC counties ranges from 1-48 of 62 NY counties in all of NYS). 60 Outer Setting Hospital availability (barrier): Most V-ROC counties had small community hospitals (5/8). 59,71 There are rural safety net healthcare facilities in most V-ROC counties (75%), 25% of V-ROC counties have a Critical Access Hospital, 63% have Federally Quali ed Health Center Sites Outside of Urbanized Area (12 total in V-ROC region), and there is one Rural Health clinic in the V-ROC region. 58 Most rural counties share public health programs and services with neighboring counties (e.g. shared cancer services programs in Steuben, Allegany, Cattaraugus, and Chautauqua counties). 57 Healthcare and community resources in rural counties are more varied compared to the region's urban counties (38% of counties do not house a YMCA, but there was a visiting nurse service and/or home health care available in all V-ROC counties. [72][73][74] Half of the V-ROC counties do not have a hospital that offers local medical oncology services. 71, [75][76][77][78][79][80][81][82][83][84][85][86][87] Networks and communication & patient needs and resources (barrier): While the vast majority of rural patients and caregivers had access to mobile phones, only 19% of visitors to the study online portal accessed it using a mobile device. While computer-based portal interface generated most of the study-related tra c, all participating counties had lower than average broadband internet coverage ( Figure 1). 61,88 Inner setting Health insurance (facilitator): Approximately 6% of V-ROC county residents were uninsured. 63 In Livingston county, 55% had health insurance coverage through their employer, and 12.8% and 14% were insured through Medicare and Medicaid, respectively (Table 1). 68,70 Implementation champions and competing priorities (barriers): Organizational and individual capacity to support implementation also varied depending on other initiatives and challenges the stakeholders were facing at the time, including the role of opinion leaders (local economic, educational, spiritual, medical and public health leadership), access to knowledge and information, available resources, relative priority, learning climate, as well as how the innovation was perceived by the key stakeholders in terms of the relative advantage it could provide, its complexity and costs. Reported threats to sustainability included hospital, practices and health systems mergers that could disrupt existing formal and informal networks and provider teams. Regional economic stability (e.g., factory or hospital closing) and public health threats (e.g., COVID-19 pandemic, Zika virus epidemic, opioid overdose crisis) are also likely to redirect regional resources and stakeholder attention from what is perceived as low acuity problems (e.g., cancer survivorship).

Individuals involved
Individual identi cation with organization, engaging (facilitator): Stakeholders' identi cation with the organization and perception of membership of a team increased participation in generating community resources and participation in portal activities. Identi cation of key gatekeepers opens tremendous opportunity for engagement, but takes time.
Rural healthcare providers (n=31) emphasized the importance of local cancer care providers but were open to regional cancer networks as long as patients were managed locally. Counties with established processes and infrastructure for multidisciplinary networks (e.g., countywide health provider coalition or business network involving county health and social services agencies) reported greater readiness to change.
Additional patient needs and resources: Most stakeholders trusted recommendations they received from their local opinion leaders (e.g., providers, 86%) and preferred to receive care locally (86%). Rural patients favored in-person, one-on-one care coordination such as patient navigators (average rank 1.5/5) or county care managers (1.9/5) higher than telemedicine (3.3/5) and visiting practice facilitators (3.3/5) ( Figure A1). While patients ranked distance to care as the most important barrier for access to regional cancer services, nancial burden and lack of awareness about cancer treatment guidelines and options were also important obstacles, consistent with previous literature ( Figure   A2). 23,89 Discussion Guided by the CFIR framework, we identi ed several unique barriers and facilitators to implementing a cancer survivorship program in rural communities of Upstate New York. 6 The identi ed barriers included regional variation in infrastructure and healthcare delivery processes,   proportion is even greater among rural patients due to the complexity of rural cancer survivors' needs and scarcity of proximate resources. 14-16, 21,39-43, 94,111-113 Existing implementation strategies in the context of rural cancer care are often challenging to use because they are poorly described, rarely justi ed theoretically, lack operational de nitions or manuals to guide their use, and are part of 'packaged' approaches with poor on-theground training. 6,114 In contrast, evidence regarding the use and effectiveness of team training in healthcare settings has grown rapidly over the last 15 years, which makes it ripe for application to program implementation planning and research. 115 Based on our ndings, below we propose several novel teamwork-based strategies for planning and implementation of healthcare programs in resource-poor communities that could facilitate adaptation of academic care delivery models for rural communities. 94,96,100,103,111 These implementation strategies to overcome identi ed implementation barriers map to the 7 C's of effective teamwork proposed by Salas et al. [93][94][95][96][97][98] Outer setting: Barriers to care integration across practices, providers and systems The challenges: Networks and Communication Unlike many surgical or clinic-based teams where teamwork is dependent on face-to-face interaction, rural cancer survivorship care requires the coordinated work of several dispersed health providers with different clinical expertise who often never meet and are only connected through shared patients. 116 Members of such regional teams may never or rarely meet face-to-face while managing patient care that is fragmented by organizational boundaries, geographic distance, health insurance, information systems, and privacy constraints. 116 Proposed strategy: Enhanced Communication Recent studies in cancer and other elds have provided evidence that quality care depends upon timely information exchanges and regular communication ow between all those stakeholders involved in treatment (including patients, specialist physicians, other specialty disciplines, primary care physicians [PCPs], and support services). 11 In large academic centers, staff efforts on survivorship programs could often be covered through other mechanisms, such as research or administration. In small rural clinics, funding opportunities for health providers are limited to billable revenue. Therefore, without a reimbursement structure directly aligned with survivorship services, such care is prohibitive and unsustainable for rural patients.
Proposed strategies: Top-down planning approach including organization leadership in addition to bottom-up approach; Focus on competencies and skill set instead of licensing and accreditation.
Long-term success of any healthcare innovation is dependent upon its nancial sustainability; therefore, ensuring support and buy-in from the clinical leadership and administration should be one of the implementation strategies. Successful examples include Medicaid nancing models for state-wide hub-and-spoke care delivery models and the Veteran's Administration's regional care management programs for special populations . 4,123 Identifying the new billing codes and appropriate level of staff (e.g., health educator instead of an NP) for performing the necessary activities and adhering to the recommended care pathways is another strategy in making existing reimbursement models work for resource-poor settings. When referring rural patients to services, we suggest focusing on the functions/quali cations necessary to address the needs of rural cancer survivors (e.g., nancial counseling, exercise therapy) instead of the recommended quali cation (e.g., 20% FTE of cancer PT specialist) that may not be available locally. With appropriate coaching and training, the services could be provided by the next best available substitute (e.g., a nurse educator instead of a social worker). In addition, rural practices could push for insurance companies to include survivorship care metrics as part of their quality metrics.
The timing of implementation is critically important when partnering with community stakeholders from resource-limited regions.
Implementation of program steps may need to be postponed or delayed to account for the availability of the local partners. For example, sta ng may be more limited during summer months when practices are more likely to be short-staffed due to school breaks and family Individuals involved: Regional variation The challenges: Patient needs and resources Provider availability and care delivery patterns vary greatly among the rural counties. 23 We observed signi cant variation in the structure of county health departments, availability of county hospitals, cancer and primary care clinics, cancer services programs, availability of public transportation, and existence of local formal (e.g., provider collaborations) and informal (e.g., patient support) networks. Other studies have noted similar challenges that necessitate extensive tailoring and adaptation of evidence-based programs and care protocols to t the needs of rural community practices and their patients. 124,125 Despite the observed variation in resources and infrastructure, rural patients across all regions were consistent in their preference to involve their local providers, both primary care and oncology, in their cancer care as much as possible. Rural cancer patients and their caregivers also reported signi cant challenges with identifying relevant information and making decisions affecting their cancer care, from treatment options to insurance coverage to disability payments and palliative care options. These results present a challenging paradox: patients prefer local care but report distance to care as a barrier. Within the context of the COVID-19 pandemic and the current acceleration of telemedicine roll-out, there may be unique opportunities for enhancing the technology for virtual oncology care and patient receptiveness to telemedicine technology, though patient preferences may change in the post-COVID-19 era.
Proposed strategies: Enhanced care coordination, cooperation and coaching Coordination means synthesizing care goals and decisions across the multiple groups involved in patient care, as well as mutually aligning, timing, and adapting key care tasks among different care teams or team members over time. 20,126 Cooperation is the attitude that team members need to insure coordination. In application to rural cancer care, coordination requires proactive cooperation and readily sharing available resources (e.g., nutrition therapy, pain management), upcoming appointments (e.g., mammogram every 6 months for the next 5 years, chest CT every 12 months) and helping patients prioritize their goals (e.g., what surgical option will allow the patient to go back to work as soon as possible, resource availability to complete physical therapy at home instead of the hospital). Evidence-based implementation strategies supporting coordination include provider tele-coaching via hub-and-spoke models (e.g. Project ECHO), patient-centered models such as individual care navigators, or practice-based quality improvement assistance such as practice facilitators (Table A1). 10,127-131 A huband-spoke model could serve as a platform to coach rural and remote providers to improve provider teamwork and care coordination across the region. Care coordinators such as Community Health Workers are a patient-centered approach that could help patients with making decisions about their care, better communicate with their providers and identify and more e ciently manage available local resources. Such care navigators are often employed by county health departments or social services agencies and are not disease-speci c. 10,124,132 Strengths and Limitations The study was limited by publicly available data that informed our understanding of contextual characteristics. We faced several key barriers to program implementation; however, limitations faced in implementation informed recommendations on how to improve use of implementation frameworks and introduction of novel implementation strategies. In addition, we had limited direct measures of implementation cost, future research should include economic evaluation to assess program sustainability and feasibility. Despite the limited geographic scope, there was a wide variation in population to care provider ratios across V-ROC counties, and thus, the ndings may be generalizable to a variety of rural settings.
Despite these limitations, integrating implementation strategies with the principles of teamwork offered a useful lens and may ll a critical gap in adaptation and dissemination of evidence-based guidelines. Use of team science frameworks is a novel approach to evaluating health innovations with implications for integration in future studies. The CFIR is one of the most commonly used implementation science frameworks. 5 Our application of standardized, validated implementation frameworks allows for comparability across studies and will help researchers to understand gaps and strengths in program development. 6,94,101,114 Conclusions Incorporating teamwork principles and team training when developing survivorship care pathways for community oncology and primary care practices may improve implementation e ciency, stakeholder buy-in and sustainability of guideline-concordant models of cancer care delivery. Further research is needed to evaluate the effectiveness and cost-effectiveness of teamwork training and approaches as an implementation strategy in rural and other settings with limited specialist workforce availability. Award (2481-UB-IC E.N. PB contributed to study conceptualization, data collection, and manuscript preparation. FF, DH, IR, JR, PS and VS guided the development of stakeholder engagement plan and assisted with building the research network. KT helped identify IT platforms for the study data collection and developed the online patient and provider portal. All authors reviewed the paper, provided feedback on earlier drafts and approved the nal manuscript.