RE-AIM Evaluation of a Multilevel Decision Support and Navigation Intervention for Colorectal Cancer Screening in Hispanic Primary Care Patients

Background: Hispanic adults in the United States have low colorectal cancer (CRC) screening rates and are more likely than non-Hispanic adults to be diagnosed with advanced-stage CRC. We evaluated the reach, effectiveness, adoption, implementation and maintenance (RE-AIM) of a novel multilevel decision support and navigation intervention (DSNI) designed to increase CRC screening among Hispanic primary care patients. Methods: The trial enrolled 400 consented participants from a health system sampling frame of 2,720 Hispanic patients eligible for CRC screening in ve primary care practices. We randomized 203 patients to receive a mailed standard intervention (SI Group) that included informational material, a fecal immunochemical test kit, a reminder versus 197 patients who received the SI plus a telephone screening DSNI (DSNI Group). We assessed DSNI effects using health system and study administrative data. Results: 1) Reach: DSNI delivery reached 84% of participants. 2) Effectiveness: The DSNI group produced a screening rate that was signicantly greater (p < .001) than in the SI group (78% and 43%, respectively). 3) Adoption: All participating primary care practices and patients remained in the study. 4) Implementation: DSNI delivery required an average of 4.1 decision support and navigation call attempts with the total call effort averaging 35 minutes. 5) Maintenance: Health system leaders have acknowledged DSNI benets and are actively engaged in determining how to sustain DSNI implementation. Conclusion: The DSNI achieved high levels of reach, effectiveness, adoption, and implementation with a modest investment of resources. The project team is addressing the challenge of DSNI maintenance in the health system. Further work is needed to determine intervention effectiveness in other health systems and in the general patient population. Trial

The RE-AIM framework illustrated the public health impact of a randomized controlled trial to increase colon cancer screening in primary care patients.
These ndings demonstrate effectiveness of a novel decision support and navigation screening intervention for Hispanic primary care patients.
Further study is needed to determine how to reach patients of all eligible age groups more effectively.

Background
Colorectal cancer (CRC) continues to be the second leading cause of cancer deaths for Hispanic adults (Bandi 2014). Hispanic adults are less likely to be screened for CRC and more likely to be diagnosed with advanced stage disease (Bandi 2014, Gonzales 2017). CRC screening rates are low among Hispanic patients despite efforts to increase screening (Pollack 2006). Health equity strategies are needed to increase cancer screening for Hispanic primary care patients (Latino Cancer Report). Prior studies have demonstrated that patient navigation may be an effective method to increase rates of cancer screening (Jandorf 2005 Additionally, mailed SBT (Hoffman 2011) and providing patients with reminder calls (Baker 2014) may increase rates of CRC screening. Further study is needed to determine which of these potential CRC screening interventions are most effective for Hispanic adults (Gonzalez 2012, Robie 2011. The RE-AIM framework (Reach, Effectiveness, Adoption, Implementation, and Maintenance) provides a comprehensive approach for evaluating the population health impact of interventions (Glasgow 2006, Gaglio 2013, Kessler 2013) and has been used to guide translation of ndings into clinical care (Glasgow 2003, Bakken 2009). Reach is de ned as individual level participation. Effectiveness measures the positive and negative outcomes of an intervention on behavior. Adoption assesses participation at the setting-level. Implementation evaluates the delity and replication of the intervention delivery. Maintenance determines long-term changes at the individual or system level (Glasgow 1999). This framework has been applied in other studies to identify the barriers and facilitators of interventions.
The purpose of this study was to conduct a RE-AIM evaluation of a multilevel decision-support and navigation intervention (DSNI) for CRC screening in Hispanic primary care patients. A randomized controlled trial was used to compare a DSNI intervention to a standard mailed intervention (SI) to measure the population health impact.

Study Design
The RE-AIM framework was used to evaluate a randomized controlled trial (Clinicaltrials.gov identi er: NCT02272244) of an intervention aimed at increasing CRC screening rates among Hispanics (Myers 2019). Patients were randomized to an SI group (n = 203) or a DSNI group (n = 197). The SI included a mailed stool blood test (SBT) and instructions for completion in English and Spanish. The DSNI intervention included a SI plus telephone decision support and navigation by a bilingual patient assistant who reviewed the screening materials, veri ed the participant's preferred CRC screening test, and assessed the individual's likelihood of completing his/her preferred test. The goal of this intervention was to increase rates of colon cancer screening for Hispanic primary care patients. The patient navigator assisted DSNI participants to develop and implement a test-speci c screening plan.
Primary care clinicians received documentation of all patient contacts via the electronic medical record in addition to the screening decision, status, and results. All participants' SBT results were routed to the clinician, and the patient was contacted by the practice protocol per usual care. The navigator encouraged and assisted participants with a positive index SBT result in undergoing a diagnostic colonoscopy. A 6-month telephone survey was administered and a 6-month and 12-month medical record review was conducted for all participants. A Patient and Stakeholder Advisory Committee (PASAC) comprised of primary care patients, community leaders, and providers oversaw this process.

Setting
This study was conducted within a large Pennsylvania health system that covers urban, suburban, and rural regions. The system serves 800,000 individuals who receive care in over 160 physician practices and 17 community clinics.

Population
Five participating primary care practices used the electronic medical record to identify potential participants, which included a sampling frame of 2,720 Hispanic patients eligible for screening. Eligibility criteria for the population included an age of 50-75 years, no history of CRC or polyps, and not up to date with CRC screening.

RE-AIM Procedures
Reach. Reach was de ned as the characteristics of the participants as compared to the sampling frame and the percentage of those randomized to the DSNI arm who received navigation contact. Intervention reach was assessed at the patient level in terms of the number/percentage of study participants who received the planned intervention contacts as compared to those who were eligible to receive such contacts. Reach was assessed based on the selection of participants who were enrolled in the rst cohort (N = 2,345) of patients.
Intervention reach was assessed at the provider level and de ned as the number/percentage of providers who received participant screening plans and reports. At the provider level, data was collected to track delivery of the patients' decision summary as well as patient screening status report.
Effectiveness. Effectiveness was de ned as CRC screening adherence rates. Intervention effectiveness was measured at the patient level in terms of impact on de ned primary outcomes (i.e., screening adherence, screening decision stage, and test-speci c adherence). Data from the 6-month survey, 6month medical records review, and 12-month medical record reviews were used to evaluate this dimension.
Adoption. Adoption was de ned as the number of practice participants who agreed to participate and completed the intervention. Characteristics collected include panel size and demographics, sta ng, baseline screening rates, ownership, location specialty, and support services.
Implementation. Implementation was de ned as the patient contacts for those in the DSNI arm. Intervention implementation was examined using quantitative and qualitative methods throughout project development, implementation, and analysis ( Figure 1).
[Insert Figure 1. Implementation examination process] Quantitative data collection. Navigation call metrics were measured. The number, timing, and length of attempted and completed navigator calls were measured, including whether calls were conducted within or outside of regular business hours, in an effort to identify times of increased opportunity and success.
Other metrics evaluated included time from initial mailing to call initiation to call completion and reasons for incomplete calls or navigation refusals. In addition, navigators were asked about their experiences in delivering intervention contacts to participants and providers. Mail and telephone records, along with navigator logs, were used to obtain data to measure this dimension.
Qualitative data collection. Prior to initiation of the randomized trial, qualitative data were collected in two focus groups that were conducted to elicit feedback on implementation strategies for the intervention. Staff recruited focus group participants by working with the health system and local community agencies. Inclusion criteria included individuals who self-reported being of Hispanic ethnicity, ages 50 to 75, having an a liation with a community group whose mission includes attention to Hispanic/Latino community interests and/or being a patient in the health system. Focus group interactions were audio recorded, transcribed, and content analyzed. The focus groups addressed perceptions of ethnicity and race, study recruitment strategies and procedures, and terms and approach for the navigator. Additional attention was allotted to assess the appropriateness of the drafted educational materials and decision support procedures to be used in the intervention. This assessment included cancer screening content, health literacy, and cultural appropriateness. With each item reviewed, participants' perceptions about intervention contact along with screening barriers and facilitators was assessed. Group moderators also solicited feedback on the study process.
Interviews were conducted with staff to elicit perceptions related to intervention content and delivery.
Barriers and facilitators were discussed to facilitate completion of CRC screening.
Following completion of the randomized trial, focus groups were conducted with DSNI participants and were grouped by CRC screening status (screeners and non-screeners). The focus groups were conducted to elicit feedback on the participant experience of the study intervention, study process, and their reasoning for screening or not screening. Both sessions were conducted mainly in Spanish with limited English-speaking. Focus group interactions were audio recorded, transcribed, and content analyzed.
A PASAC was formed as part of the proposal development process and convened throughout the performance of the project. The PASAC was comprised of Hispanic community members and health system members, including non-study patients, providers and administrators. The primary purpose of this committee was to review processes, procedures, and materials; comment on regular reports of study progress delivered by the academic research team; and provide feedback and recommendations in keeping with both the Patient-Centered Outcomes Research Institute (PCORI) and community-based participatory research principles of community and stakeholder involvement in research.
Maintenance.Maintenance was measured at a systems level due to characteristics of the study via interviews with health system leaders. Maintenance at an individual level was de ned as rates of CRC rescreening for those who opted to complete a SBT. Interviews were conducted with select health system leaders to gain insights that would help us to form impressions about the potential for DSNI maintenance in the health system. Thirty-minute interviews were conducted with leaders in the DSNI practices to understand their experience of the intervention and probe its potential continuation after the completion of the research. Three leaders consented to be interviewed: a medical director, community outreach coordinator, and care manager-o ce coordinator at different sites.
Patient level maintenance was examined using electronic health records data for participants who completed SBT. Subsequent patterns of screening beyond the study's 12-month observation period was determined. For individuals with a negative index SBT, guidelines would recommend another SBT or a colonoscopy 12 months later; therefore, the rate of rescreening within 15 months (i.e., 12 months plus a 3month grace period) was analyzed. For individuals with a positive index SBT, rates of a subsequent diagnostic colonoscopy were calculated.

Data Analysis
Descriptive statistics were conducted including frequencies and percentages for categorical variables.
Statistical analyses were performed using Excel.

Human Subjects Protection
This study was approved by the Institutional Review Board at [identifying organization name removed].

Reach
A total of 333 (14%) patients from the 2,345 individuals in the sampling frame satis ed eligibility criteria, were contacted by telephone, consented to participate in the study, and completed a baseline survey. The population was similar to the sampling frame; two exceptions included the enrolled population had more patients aged 50 and 59 and more patients who were members of a speci c payer plan (Table 1). Among those for whom navigation intervention contacts were attempted, 84% were reached for navigation (n=167).
[Insert Table 1. Reach: Characteristics of the sample enrolled in DSNI compared to those invited] Effectiveness At 12 months, CRC screening was signi cantly higher in the DSNI group than in the SI group (78% versus 43%), with an adjusted odds ratio (OR) of 4.83 (95% con dence interval, CI: 3.08-7.58; p = .001). Compared to the SI group, the DSNI group was more likely to complete a SBT (57% versus 37%; OR = 4.20; p = .001) and colonoscopy (20% versus 6%; OR = 8.79; p = .001). More details on the effectiveness of this intervention have previously been published (Myers 2018).

Adoption
All ve (100%) practices agreed to participate and continued throughout the study ( Table 2). The ve practices served approximately 15,792 patients from the health system. Practice 4 had substantially more physicians than the other practices, although the majority were residents. Practice 3 also included residents. Additionally, all practices included an internal care coordinator, while only practice 5 had an onsite social worker with some capacity to support navigation.
[Insert Table 2. Adoption: Characteristics of the ve primary care practices]

Implementation
The screening intervention kit was mailed at an average of 6.4 (SD = 5.2) days after randomization (range: 0-28), and the rst navigation call attempt was made 14.3 (SD = 8.1) days after randomization (range: 2-65). On average, 4.1 (SD = 2.8) calls were required to complete navigation (range: 1-11). Navigation contacts were discontinued at an average of 29.3 (SD = 20.2) days after randomization (range: 3-139). Thirty-one DSNI participants (16%) were not navigated including, 19 (10%) were unable to be reached after ten call attempts, 5 (3%) refused navigation, 5 (3%) moved or transferred out of the network, and 2 (1%) had competing health issues and were unable to complete the call. The time of the call was investigated further, examining success of business hour calls compared to afterhours calls (Table 3). Minimal difference was found in navigation rate or corresponding screening rate.
[Insert Table 3. Implementation: Contacts and outcomes for participants in the DSNI group by contact strategies] Focus group participants were representative of the targeted Hispanic population. The focus groups provided background knowledge to develop training sessions for the study research assistant and patient navigator. Main patient concerns included screening-related costs, fear of a cancer diagnosis, worry about taking the prep needed to under screening colonoscopy, anxiety related to the invasiveness and safety of the colonoscopy procedure, feelings of machismo among male study participants, concerns about the need to take time off work to undergo screening colonoscopy, the challenge of contacting and following participants due to frequent changes in residence and telephone numbers that occur among community members, and the strong desire of community members to take care of a family members' health-related issues for whom they were responsible before (instead of) their own preventive health care.
For the Patient and Stakeholder Advisory Committee (PASAC), meetings were held three times per year with email communications in between in person meetings. The PASAC we most impactful in informing the cultural appropriateness of materials and enrollment and navigation scripting. PASAC members report feeling that they are an extension of their community and have a duty to share their study knowledge with that community. As such, the committee provided strategies on how to disseminate ndings in the larger community via various outlets. The PASAC members report feeling comfortable in the group and that their opinions are both heard and acted upon. While members have provided feedback on study materials and protocols, most felt that their primary role was in disseminating the results to educate the Hispanic community about colorectal cancer screening. Fewer members acknowledged the role of the group in research design and implementation though several described themselves as part of the team. In summary the PASAC contributed to: study promotion, health services access, community education/awareness, core materials and scripting-health literacy, cancer content, cultural terminology, and content reception, nancial and social support barriers, and enrollment and navigation strategies.
Individual interviews with the research assistant and with the patient navigator were conducted. The following four themes were identi ed: 1) How well their training prepared them to interact with patients and the effectiveness of their patient intervention contacts; 2) Adjustments they made to provide studyrelated information to participants and deliver intervention contacts; 3) Barriers to intervention implementation; and 4) Intervention effects on study participants and the Hispanic community.

Maintenance
The three health system leaders identi ed positive bene ts of the intervention: a bilingual navigator addressed language, literacy, and cultural sensitivity needs; navigator involvement facilitated work ow; the DSNI enabled the practice to achieve quality goals. Barriers to intervention maintenance included: providing administrative support for navigation; dedicating a patient navigator to focus on the task of boosting CRC screening rates; determining how to pay for colonoscopy screening for patients who are under-insured or uninsured. Despite these barriers, respondents expressed enthusiasm for the incorporation of the DSNI into the health systems as a centralized approach that could help practices and patients.

Discussion
Increasing CRC screening is critical to reducing CRC morbidity and mortality (Levin 2018), particularly for Hispanic patients who are more likely to be diagnosed with advanced disease (Gonzalez 2017). The RE-AIM framework was applied to a randomized controlled trial of a CRC screening intervention including tailored navigation and decision support for Hispanic primary care patients. This intervention resulted in increased rates of CRC screening via both colonoscopy or SBT (Myers 2018). This trial supports the ndings of prior studies that found culturally tailored approaches increase rates of CRC screening (Myers 2007, Myers 2014, Moralez 2012). Additionally, these results are concordant with prior multilevel interventions, including centralized outreach for mailed SBT (Liles 2015) and combining outreach with navigation (Dougherty 2018), which demonstrated increased CRC screening rates beyond single level approaches.
The RE-AIM framework allowed for additional analysis of the public health implications of this study, including opportunities to modify the study to reach more patients for future implementation (Glasgow 1999). Younger patients (ages 50-59) were more likely to enroll in the study. Previous research demonstrates that almost 40% of those aged 65 and older with a 10-year life expectancy had not received guideline recommended screening (Schonberg 2015). Further discussion with key patient stakeholders within this population is needed to inform strategies to increase reach to older adults eligible for CRC screening.
Primary care practices adopted this intervention and supported use of the bilingual patient navigator to facilitate cancer screening for their patients. The high rate of adoption may have been in uenced by the centralized process, which removed burden from practices to support patients via navigation services. The bilingual navigators also addressed potential language barriers for those with limited English pro ciency and increased rates of screening, consistent with navigation for other types of cancer screening (Genoff 2016). In this study, clinicians received the screening plan and followed up for any abnormal results, but the primary intervention occurred outside of the primary care o ce.
The time for the navigation calls could inform resources other systems would require to implement this intervention. As the healthcare system transitions from fee for service to more value-based payments, the return on investment will continue to increase. Future research should focus on streamlining the phone calls, given that the average time exceeded 30 minutes. This model could be translated to other settings through the utilization of curriculum and development of navigator competencies, particularly to address health disparities (Wells 2018).
While this study was not designed to address health system maintenance, continued outreach and navigation is needed to support maintenance of screening, particularly for SBT. This is a challenge, particularly given that previous research noted increased adherence when the patient received the SBT from his or her clinician (Pornet 2014). Strategies are needed to enhance centralized processes to promote adherence with repeat cancer screening.
The pre-intervention focus groups and the PASAC informed and adapted the intervention, particularly the patient perspectives. Use of this learning health system approach (Greene 2012) facilitated local adaptation to materials and messaging in addition to implementation such as appropriate number of navigation calls and the local referral process. This including script adaptation and cultural tailoring of a brochure and letters. This group informed terminology, colors, graphics. Involving patient stakeholders in development, implementation, and dissemination as part of authentic engagement increased relevance of this study (Woolf 2016).
There were limitations and strengths to this study. There were challenges reaching older patients within the cohort, especially for Hispanic patients; previous research found similar results in a cohort of African American patients (Myers 2014). It is important to note that providers were not included in this analysis as the position of the project was not to impact their work ow; therefore, their knowledge of study procedures was minimal. Strengths of the study included use of a prospective randomized design, the implementation of a novel centralized process, bilingual navigators to address barriers, inclusion of patient perspectives throughout the process, and navigator concordance with the population. Additionally, this process provided a model to support population health within primary care cancer screening.

Conclusions
The DSNI improved CRC screening rates for Hispanic patients. Further study is needed to determine strategies to sustain and scale the intervention within health systems and to more effectively engage patients facing disparities over age 60. The RE-AIM framework may be used to support further dissemination and implementation of DSNI to improve health equity for CRC screening in other settings and populations. Availability of data and material The datasets generated and/or analyzed during the current study are not publicly available but are available from the corresponding author on reasonable request. Further information may be obtained on clinicaltrials.gov.   Figure 1 Implementation examination process