Peace of Mind Program (PMP): Impact of the Dissemination and Implementation of an Evidence-based Intervention (EBI) to Improve Mammography Appointment Adherence in Safety Net Clinics


 Backgroundhe Peace of Mind Program – an adapted evidence-based intervention to improve mammography appointment adherence in underserved women – was expanded to safety net clinics. This study assessed effectiveness of the intervention in improving mammography appointment adherence and implementation of the intervention. MethodsThe intervention was implemented through a non-randomized stepped wedge cluster design with 19 Federally Qualified Health Centers and charity care clinics in the Greater Houston area. Clinics were their own control during the baseline period and conducted at least three mammography drives during the baseline and intervention period. A multivariable generalized estimating equation logistic regression was conducted to examine mammography appointment adherence. To examine adoption and implementation of the intervention, two surveys assessing Consolidated Framework for Implementation Research constructs were conducted with clinic leadership and staff. One-sided t-tests were conducted to analyze mean score changes between the adoption and implementation survey. Results total of 4402 women (baseline period = 2078; intervention period = 2324) were included in the final analysis. Women in the intervention period were more likely to attend or reschedule their mammography appointment (OR = 1.30; p < 0.01). Similarly, for those in the intervention period, women who completed the intervention were more likely to attend or reschedule their mammography appointment than those who did not complete the intervention (OR = 1.62; p < 0.01). The mammography appointment no-show rates for those in the baseline period, in the intervention period, and who completed the intervention were, respectively, 22%, 19%, and 15%. In terms of the adoption and implementation survey, a statistically significant mean score decrease was observed in Inner Setting overall and in two Inner Setting constructs, Culture – Effort and Implementation Climate. Conclusions﻿This study provided a pragmatic approach to translating an evidence-based mammography intervention into practice in safety net clinics. While the intervention improved mammography appointment adherence, there our opportunities to further integrate Consolidated Framework for Implementation Research constructs in future implementation of the intervention. Future research on the effects of implementation moderators particularly Inner Setting constructs would be of value to implementation practitioners.

For the success of EBIs to be translated to different communities, researchers with implementation knowledge must work collaboratively with community and clinical stakeholders in intervention adoption and implementation. [24][25] An example of a clinical stakeholder which can help tailor interventions are mobile mammography providers who can provide screening at no or low cost at clinics close to the underserved communities and neighborhoods, helping to eliminate geographic, cost and transportation barriers. 20 In addition, dissemination and implementation (D&I) practitioners aiming to implement a tailored intervention to intercede across both cost and access to care among underserved communities have partnered with safety net healthcare systems and Federally Quali ed Health Centers (FQHC). FQHCs are community-based health care providers that provide culturally competent primary care and preventative services in underserved communities. 21 The nature of FQHCs lends them direct access to underserved women most likely in need of mammography screening services, but also creates barriers to adoption and implementation of an EBI. [26][27] There is a gap in understanding of effective D&I methods in successful interventions, particularly those aimed at cancer prevention and detection. 28 The current study utilized a theoretically based, D&I strategy to facilitate the implementation of an evidence-based mammography screening intervention focused on reducing mammography screening non-adherence in underserved women with FQHCs and charity care clinics and mobile mammography providers. The Peace of Mind Program (PMP) intervention is an evidence-based mammography screening intervention based on the Transtheoretical Model of Change adapted from a research tested intervention program from the National Cancer Institute's Research Tested Intervention Programs (RTIPs) database. [28][29] In the current study, the PMP intervention was expanded to include implementation components geared toward supporting implementation in safety net clinics, and by doing so, aimed to address the research to practice gap. 28- 29 We hypothesized that the PMP intervention would improve mammography appointment adherence in underserved women compared to usual care. The study objectives were targeted to test the effectiveness of the PMP intervention in improving mammography appointment adherence and to assess implementation of the PMP intervention [See Additional le 1: Standards for Reporting Implementation Studies (StaRI) checklist].

Intervention
The Peace of Mind Program (PMP) is an active listening, tailored telephone reminder call intervention to counsel women through barriers to mammography screening appointment attendance. [28][29] Each woman during the intervention period received up to three reminder call attempts for their scheduled mammography screening appointment at a clinical site. If the woman did not answer on the rst attempt, two additional attempts were made to reach the woman. If the woman answered the phone call, but did not consent to participate in the study, she was reminded about her appointment in the usual care manner of each site. If she answered and consented to participate, a Certi ed Community Health Worker (CHW) who made the call assessed the woman's con dence in attending their scheduled mammography appointment, counseled the woman through barriers to attending the appointment, and recorded the woman's responses in an online interface program designed in RedCap. The results of each reminder phone call, whether the call was completed or a message left, and the woman's resulting mammography appointment attendance or no-show status were also recorded in RedCap. RedCap served as the online platform for both data storage and initial data analysis for this study.

Implementation Strategy
Prior to implementation of PMP, potential clinics were engaged through various adoption strategies including adoption meetings, webinars, and an adoption survey measuring Consolidated Framework for Implementation Research (CFIR) constructs across multiple domains. Those clinics who adopted the program then participated in several implementation strategies, including: stakeholder meetings, a continuing education unit (CEU) certi ed 2-day training for clinic community health workers (CHWs), and a site readiness assessment checklist to prepare for implementation. In addition, the community partner -the Breast Health Collaborative of Texas (BHCTexas) -and research team worked with participating clinics to align goals for mobile mammography drives and facilitated relationships with mobile providers as needed to support implementation. Onsite role modeling and support from BHCTexas CHWs and ongoing technical support from the research team was provided throughout PMP intervention implementation. Financial resources included a $7500 stipend for clinic participation and nancial assistance for women to receive free screening. At eight weeks post-implementation, clinics were invited to complete an implementation survey to assess the same constructs as the adoption survey. A full description of the development, adoption, implementation, and stakeholder engagement components of the PMP intervention has been reported elsewhere. [28][29][30] Study design A non-randomized stepped wedge cluster design was used to assign clinics into two non-concurrent implementation waves with two to three groups in each wave. 28 In order to participate in PMP, clinics must: (1) have been members of BHCTexas within the Greater Houston service area, (2) had a designation as a FQHC by the Health Resources and Services Administration (HRSA) or be a charity clinic which provides free or reduced cost care to underserved populations in their service area, (3) serve women between the ages of 40 to 64 years old who were at or below 200 % of the federal poverty level for a family of four and who lacked health insurance, and (4) engage in provision of mammography screening services at least six times per year (three in baseline; three in intervention) and (5) women at the clinic must have been in need of mammography screening and be scheduled for an upcoming appointment. Patients must have completed a clinical breast exam prior to their scheduled screening appointment per mobile provider requirements. Variation across clinics existed in frequency of mammography drives, number of patients scheduled in each drive, number of staff available to participate in PMP, existing relationships with mobile mammography providers, and available funding and resources which resulted in differences in clinic readiness to start the PMP intervention. Due to these differences, the randomized allocation of the clinics into waves and groups as previously described was not possible. Clinics that had lower levels of readiness were assigned to later groups to bene t from more time to receive implementation strategies from PMP. Each clinic served as its own control during the baseline period and was required to have at least three mammography drives during both the baseline and intervention periods. Clinic, BHCTexas, and research team staff were not blinded in the study. Since each clinic served as its own control during the baseline period, blinding was not possible or necessary for this study.

Dependent variable
The dependent variable was a dichotomous variable indicating mammography appointment adherence with a no-show or cancelled appointment categorized as "0" and an attended or rescheduled appointment categorized as "1."

Independent variables
The independent variables of interest are two dichotomous variables indicating study period, baseline period (categorized as "0") or intervention period (categorized as "1") and for the intervention period, whether the patient did not complete (categorized as "0") or did complete (categorized as "1") the PMP intervention. A patient completed the intervention if they answered the reminder call, consented to the study, received the staging question assessing con dence and barriers counseling. In addition, patient, intervention, and clinic variables examined in the analysis of the data included age, season, wave, group, mobile mammography provider, and clinic racial/ethnic distribution. Age was categorized into three age groups -55 and above, 45 to 54 and 25 to 44 years -to align with the age-based mammography screening guidelines. The season in which the patient scheduled their appointment was categorized by winter (January to March), spring (April -June), summer (July to September), and fall (October to December) to examine a possible seasonal effect. [31][32] The wave was categorized as a dichotomous variable (0/1) for wave 1 and 2 and the group was categorized as a three-category variable (group 1, 2, and 3) based on when the clinic began the PMP intervention. Each of the three mobile mammography providers were categorized as a dichotomous variable based on if the provider assisted with mammography screenings at the clinic in which the patient was scheduled (categorized as "1") or not scheduled (categorized as "0") for a mammography appointment. The provider with an existing reminder call and group education program for usual care was de ned as the reference group. The clinic racial/ethnicity distribution (percentage) of the population served by each clinic from 2015 to 2016 was collected from the Health Resources and Services Administration (HRSA) Uniform Data System (UDS) for FQHCs and the Texas Association of Community Health Centers (TACHC) for charity care clinics. The clinics were categorized based on the racial/ethnicity group with the highest percentage in ve mutually exclusive groups: non-Hispanic Black, non-Hispanic white, non-Hispanic other (another race other than Black or white), Hispanic, and multi-racial/ethnicity group (equal percentage of non-Hispanic Black, non-Hispanic other, and Hispanic women served). Each of the ve racial/ethnicity groups were also categorized as a dichotomous variable indicating if the racial/ethnicity group was the highest reported for the clinic (categorized as "1") or not the highest (categorized as "0").
To assess delity of PMP intervention implementation, implementation variables examined in the analysis of the data for those in the intervention period, included the CHW who made the appointment reminder call, if the patient answered the reminder call, the number of reminder call attempts made by the CHW, and language in which the reminder call was received. The CHW who made the reminder call was a dichotomous variable based on if they were a clinic staff member (categorized as "0") or a BHCTexas staff member (categorized as "1"). A dichotomous variable determined if the patient answered the reminder call (categorized as "1") or not (categorized as "0"). The number of reminder call attempts received by patient in the intervention period was a dichotomous variable for one call (categorized as "0") or multiple call attempts (two or three calls categorized as "1"). Language was categorized by English, Spanish, or Vietnamese.

Data analysis
A descriptive analysis was performed to examine differences in patient, intervention, clinic, and implementation variables across the baseline and intervention periods. To test for statistically signi cant differences in mammography appointment adherence across the baseline period, intervention period, and covariate variables we used chi-square tests. For the primary analysis, we used a multivariable generalized estimating equation (GEE) regression model to examine mammography appointment adherence in two analytical models. In the rst model, we included all patients in the baseline and intervention period (intent to treat analysis). In the second model, we included only patients in the intervention period to examine those who did and did not complete the intervention (i.e., completed the reminder phone call). We modeled clustering across the 19 clinical sites using a logistic GEE regression (logit link with odds ratio) and an independent correlation structure. We analyzed age, season, wave, group, mobile mammography provider, the group variable and the ve dichotomous variables for clinic racial/ethnic distribution, and for those in the intervention period, the CHW who made the appointment reminder call, if the patient answered the call, the number of reminder call attempts, and language independently in each of the models and added each additional variable as a covariate. The Quasi-Akaike information criterion (QIC) value was used to identity covariates to include in the nal models. All analysis was performed using Stata 14.0 (College Station, TX) with α=0.05 as the limit for statistical signi cance.
To measure adoption and implementation factors associated with each clinic, a survey was conducted with clinic leadership and staff with any potential role in PMP prior to adoption of PMP and eight weeks into PMP implementation. A total of 75 survey statement items were used to assess twelve constructs across three CFIR domains using a survey adapted from the Cancer Prevention and Control Research Network for cancer control EBIs with FQHCs [see Additional le 2]. [33][34][35] A mean score for each clinic was created to measure level of agreement with each statement item (5 = completely agree to 1 = completely disagree). Twenty survey statements were recoded to align with level of agreement and scoring direction (E.g., Question in Additional le 2: It will be hard to train providers and staff to implement the PMP). Onesided t-tests were conducted to analyze mean score changes (mean difference < 0) between the clinic adoption and implementation survey responses.

Results
A total of 26 FQHC and charity care clinical locations serving underserved women in the Greater Houston area were approached for study enrollment. Of the 26 clinics, 22 elected to adopt PMP (85%). Two clinics enrolled (three clinical locations), but did not complete the trial yielding a total of 19 clinical sites for analysis. A total of 4408 total patients were recruited for the study as shown in Figure 1. Six patients were excluded from the nal analysis because we were unable to determine their mammography appointment adherence outcome from the Redcap responses. Of the 4402 in the nal analysis, 2078 were enrolled in the baseline period and 2324 were enrolled in the intervention period. The results of the descriptive analysis comparing those in the baseline and intervention periods are reported in Table 1. Women aged 45 to 54 years old made up the largest age group in both periods (baseline -45%; intervention -41%). The patients in the baseline intervention had a higher percentage of mammography attendance in the Summer (baseline -47%; intervention -18%) compared to those intervention period who had a higher attendance in the Spring (baseline -8%; intervention -36%). Over half of patients in both periods were in wave 1 (baseline -59%; intervention -51%). Group 3 was largest in the baseline intervention (baseline -41%; intervention -14%) while group 1 was the largest in the intervention period (baseline -26%; intervention -69%). In terms of mobile mammography provider, provider 2 who had no existing reminder call or group education was the largest mobile mammography provider to women in both periods (baseline -48%; intervention -52%). Hispanic women were the largest served racial/ethnicity group across clinics in both periods (baseline -77%; intervention -70%).
The results of the descriptive analysis comparing those in the intervention period who did and did not complete the intervention are also reported in Table 1. Out of the 2324 patients in the intervention period, a total of 1572 completed the PMP intervention. The characteristics of those who completed the intervention were similar to those who did not complete the intervention in regards to patient age group, season, and mobile mammography provider. In terms of wave and group, the percentage of patients who did not complete the intervention was higher in wave 1 (did not complete -58%; completed intervention -47%) while 71% of those who completed the intervention were in group 1. With Hispanic women being the largest served racial/ethnicity group in the intervention, a higher percentage of those who did not complete the intervention were from a clinic serving predominately Hispanic women (did not complete -76%; completed intervention -67%). In terms of implementation variables, a higher percentage of those who completed the intervention received a call from a BHCTexas CHW compared to a clinic CHW (did not complete -68%; completed intervention -74%). For those who did not complete the intervention, 38% answered the reminder call. For those who completed the intervention, 72% received one reminder phone call compared to 46% of those who did not complete the intervention. In terms of language, a higher percentage of patients who did not complete the intervention received a call in English (did not complete -51%; completed intervention -39%) while a higher percentage of those who completed the intervention received the call in Spanish (did not complete -49%; completed intervention -53%). All patients who received a call in Vietnamese completed the intervention.

Impact on mammography appointment adherence
In the bivariate analysis, multiple statistically signi cant differences in mammography appointment adherence were identi ed as shown in Table 2. A statistically signi cant difference in appointment adherence was observed between periods (p < 0.05), completion of the PMP intervention (p < 0.001), wave (p < 0.01), group (p < 0.001), mobile mammography provider (p < 0.001), and clinic racial distribution (p < 0.001). A marginal trend toward signi cance (p = 0.058) was observed for both age and season. Those who attended or rescheduled their mammography appointment were more likely to be in the intervention period, complete the intervention, be in wave 1 and group 1, be served by a clinic serving predominately Non-Hispanic women who identi ed with a race other than Black or white, and be served by mobile mammography provider 1 who had an existing reminder call and group education program compared to those who did not show or cancelled their appointment. In terms of the implementation variables, the number of reminders call attempts, if the patient answered the reminder call, and language of the reminder call were statistically signi cant (p < 0.001). Those who attended or rescheduled their mammography appointment were more likely to answer the reminder call, receive one reminder call, and receive a reminder call in Spanish or Vietnamese compared to those who did not show or cancelled their appointment. Among the 2078 patients in the baseline period, 448 (22% no-show rate) did not show up to their appointment, whereas among the 2324 patients in the intervention period 438 (19% no-show rate) did not show up to their appointment. Among the 752 patients who did not complete intervention 205 (27% no-show rate) did not show up, whereas among the 1572 patients who completed the intervention 233 (15% no-show rate) did not show up. Table 3 includes the results of the multivariable GEE logistic regression performed to t the covariates to mammography appointment adherence by baseline and intervention period and by completion of the intervention. In the rst model, the intervention period, relative to the baseline period, was associated with higher odds of attending or rescheduling a mammography appointment (OR = 1.30; p < 0.01). The age group of 25 to 44 years was associated with lower odds compared to the 55 years and older age group (OR = 0.73; p < 0.001). Mobile mammography provider 2 was associated with lower odds (OR = 0.60; p < 0.01) compared to mobile mammography provider 1. In the second model, completing the intervention, relative to not completing, was associated with higher odds of attending or rescheduling an appointment (OR = 1.62; p < 0.01). As in the rst model, the age group of 25 to 44 years was associated with lower odds compared to the 55 years and older age group (OR = 0.71; p < 0.05). Relative to receiving one reminder call, receiving multiple reminder call attempts was also associated with lower odds attending or rescheduling an appointment (OR = 0.78; p < 0.05).

Implementation process evaluation
A total of 20 clinics completed the adoption survey prior to implementation, with 15 of these 20 clinics ultimately adopting and implementing the PMP intervention. Of those 15 clinics who completed the adoption survey and implemented the PMP intervention, eight clinics completed the implementation survey at eight weeks post implementation in their site. Table 4 includes the results of comparing the clinic mean scores from the adoption survey to the implementation survey. While we observed a statistically signi cant (p < 0.05) decreases in Inner Setting overall and in Culture -Effort and Implementation Climate (Inner Setting constructs), potential directional trends can be identi ed from the results. We observed a decrease in Intervention Characteristic constructs such as Relative Advantage, Trialability, and Compatibility, but an increase in Complexity (i.e., easier to implement). We observed an increase in the Inner Setting construct of Culture -Stress (i.e., improvements in staff stress and frustration) and in the Outer Setting constructs of Policies and Incentives and Patients Needs and Resources.
The implementation survey included additional questions to assess motivation to participate in PMP, enrollment in PMP, and in uence of the PMP adoption webinar. Across the eight clinics, a total of 16 clinic staff members completed these questions. Clinics staff members reported that their motivation to participate in PMP included participating in an EBI, helping patients to understand the importance of mammography screening, and reducing their mammography no-show rates. Staff members reported being more highly motivated to participate in PMP because of the partnership between BHCTexas and the researchers, compared to just their membership in BHCTexas alone. All but one clinic staff member who participated in the webinar reported that the webinar in uenced their decision to enroll in PMP. All clinic staff members found the enrollment process for PMP easy.

Discussion
This study has important implications for the implementation and scaling of EBIs in clinics serving underserved populations and for cancer prevention in underserved women. First, our stepped wedge design allowed us to accommodate readiness to implement PMP in a real-world setting and to proactively engage practitioners in the planning and implementation of PMP. Our CFIR survey results indicate clinics could describe a concrete need for PMP during implementation. Interestingly, no clinics reported being incentivized by the nancial stipend provided to the clinic. This has been shown in other studies to be a factor which positively in uences implementation. 36 Our survey also found that our engagement approach in the implementation strategy led to readiness to implement and increased readiness in the baseline survey results (adoption). This approach to stakeholder engagement could serve as a model for other programs and is described in detail in elsewhere. 30 Previous studies have shown that a lack of engagement with practitioners and implementers has led to programs that do not represent nor t with the communities they are intended to impact. 28 Within the cancer realm, a wide gulf between research and practice continues to lead to suboptimal EBI implementation. 37 We sought speci cally to address these gaps in the development of PMP and in partnering with BHCTexas as our network partner in this study. 28- 29 We hypothesized that a communityacademic partnership would positively impact adoption and implementation and serve a bridging role to recruit, motivate and support clinics serving underserved women. Bridging factors in implementation science consider relational ties, strength, processes and formal arrangements that connect the Inner and Outer Setting. 38 Using a community-academic partnership, focusing on internal and external incentives, addressing funding gaps, and staging implementation were all applied in the PMP implementation strategy. [28][29] Recent research has shown the urgent priority and current gap in the implementation science literature related to bridging and its impact on EBI implementation. 38 Our CFIR survey indicates this relationship highly motivated clinics to adopt. Our survey also found directional (though not signi cant) increases in Outer setting constructs related to bridging (Patient Needs and Resources) and in Inner Setting constructs related to ease of implementation (Culture -Stress). We also found statistically signi cant decreases in some Inner Setting constructs (Culture -Effort and Implementation Climate). It is possible that eight weeks post-implementation was too soon a time period for staff to feel that su cient systems to support PMP were in place in their clinics and that they had su cient self-e cacy to lead without support. It also is possible that Inner Setting scores decreased at implementation due to staff gaining a more realistic understanding of what successful EBI implementation takes once being exposed to PMP in daily operations. It is also possible that this change re ects the dynamic nature of working in under-resourced, high stress clinical environments. Our previous work in PMP development and results from this study provide an opportunity to expand the knowledge base related to bridging strategies on EBI implementation, highlight practical approaches that can be replicated or built upon by other implementation scientists and identi es opportunities where further study is warranted (such as the effect on Inner Setting constructs).
Our ndings further show that the use of a pragmatic approach to implementation can successfully translate an EBI to practice in underserved clinics. Pragmatically, we implemented across two distinct waves to reach more clinics and women with limited staff. In the GEE model, no statistically signi cant difference in Wave was observed, indicating that PMP was implemented with delity across the trial time periods (two separate enrollments of clinics). This nding could be bene cial for others working in a limited resource environment to maximize reach of EBIs. We also found that standardized training leveled the competencies of staff to implement PMP. Previous studies have shown that inconsistent training and lack of standardized competencies for navigators hinders EBI success. 39 Our implementation survey collected eight weeks post training found no signi cant change in Readiness for Change from adoption to implementation, however, each site completed a readiness assessment checklist indicating they were ready to implement PMP prior to program implementation. It is likely the CFIR survey used in this study did not have enough power to detect a difference in this construct, due to clustering of respondents in clinical sites. Our PMP stakeholder engagement and training approach could be adopted and adapted to other cancers in future studies and further tested for its potential impact on readiness. In the intervention period model, no differences in appointment attendance were observed between patients completing PMP with the BHCTexas CHWs (trainers) and the clinic staff (trainees), indicating that PMP training was successful. Second, through our stakeholder engagement process any program adaptations that may have been needed to implement PMP were considered proactively. 30 In our intervention period GEE model, all patients received the expected multiple phone call attempts, all patients who answered were asked the staging question and received the scripted counseling protocol, which indicates delity to active ingredients of PMP. During our implementation survey clinics reported improvements in their perception of program complexity and that PMP was easier than they expected to implement. Third, PMP improved the no-show rate across the trial and achieved impact across a diverse patient population, which included Hispanic, non-Hispanic Black, non-Hispanic white, and Vietnamese women. Baseline no-show rates varied by clinic, with an overall average no-show rate in participating clinics of 22%. In the intent to treat analysis, women who were in the intervention period were 1.30 higher odds of attending their appointment than those in the baseline period after controlling for known confounders including age, race, language, season, and mobile provider and the no-show rate reduced to 19% (p=0.05). In the intervention period, women who fully completed the PMP intervention (received the staging question and barriers counseling) were at 1.62 higher odds of attending their appointments than those who did not complete the intervention, after controlling for known confounders including age, race, language, season, and mobile provider. Eighty ve percent of women who completed the PMP EBI attended their mammography appointments (15% no-show rate; p<0.001). Previous studies have shown high variability in no-show rates in FQHCs and clinics serving underserved populations, with some sites having no-show rates as high as 45%. 40 Our population-averaged baseline no-show rate for participating clinics was lower than previous studies at 22%. Due to our use of a population averaged approach, we may have missed variability within speci c clinics or operating sites' no-show rates in this analysis. While our impact in the intent to treat model was a roughly two-percentage point reduction, in patients who completed PMP, no-show rates were reduced by seven percent. The impact of no-shows for mammography screening, breast cancer diagnosis and outcomes cannot be overstated. Women who miss appointments are more likely to never screen, have delayed diagnosis, later stage diagnosis and worse outcomes, including survival. 28 Additionally, clinics serving underserved populations operate with limited nancial margins and no-shows signi cantly impact their nancial solvency. 40 No differences in attendance were observed by clinic race in the intervention period model indicating that PMP successfully addressed barriers across these diverse population groups. Further, language in which PMP was delivered, which was statistically signi cant in bivariate analysis, was not signi cant in the intervention period model. This indicates that PMP delivery was effective in multiple languages. Previous studies have found that interventions delivered in a patient's native language facilitates successful implementation. [41][42] Marginal trends observed for season disappeared in the intervention period model indicating PMP successfully addressed structural barriers that have been shown to affect patient appointment attendance during the year (such as lack of time off). Effect of mobile provider's usual care reminder and education practices on attendance also disappeared in the GEE model for women who completed our program, indicating that PMP successfully improved appointment adherence to an equivalent level across providers despite this variation in routine care delivery.
Limitations of the study include not randomizing the start dates of clinics, which could have resulted in bias. We evaluated the effect of wave and group in our GEE models and found no impact. Women of younger age did retain lower odds of attendance even when completing PMP, indicating that there may be unique barriers in this age group which may require additional assessment and tailoring. This nding should be investigated in future studies to understand if younger women face unique barriers to appointment adherence. After the U.S. Preventive Services Task Force (USPSTF) changed the mammography screening recommendations in 2009, studies found a decrease in screening rates among women under the age of 50. [43][44][45] Related to the con icting recommendations, younger women might experience confusion and anxiety in their decision making to get a mammography. Patients with multiple call attempts were also at lower odds of attending their appointments, highlighting the challenges of reaching this population in the short time period prior to their appointment. Underserved women and patients in FQHCs have been shown to be harder to reach, with higher incidence of disconnected phone numbers. 46 We also conducted the implementation survey at eight weeks post PMP implementation for each site. There is no literature to indicate the ideal timing for assessment of these constructs once an EBI has been implemented. It is possible that eight weeks was too soon to observe differences, though we did nd some were statistically signi cant and others were directionally correct. The survey was intended to assess implementation moderators of PMP impact in each clinic and was not intended to in uence program components during delivery, as these had been previously developed. 28 The survey was provided to all clinical site staff, though not all completed it despite reminders. Due to low numbers of staff completing the survey in each site, we were not able to perform more advanced statistical analysis. The survey has been previously tested for validity, however, and we believe there is value in sharing our descriptive ndings. [33][34][35] High rates of staff turnover did occur in the clinics during the study which required ongoing training and support and may impact sustainability of the implementation strategy (e.g., trainings, stakeholder meetings, support) and the PMP intervention. Studies have shown staff turnover to be a challenge in this setting. [47][48][49] Despite challenges in reaching patients and with staff turnover, systematic reviews have shown the positive impact of patient navigation interventions in FQHCs. 39

Conclusions
The PMP EBI improved mammography appointment adherence across a diverse patient population and led to improvement or directional change in implementation constructs across the domains of Intervention Characteristics, Inner Setting, and Outer Setting in clinics serving underserved women. This study extends the knowledge base in multiple areas and could serve as a model for replication for implementation practitioners.

Declarations
Ethics approval and consent to participate The study received IRB approval from the Committee for the Protection of Human Subjects at The University of Texas Health Science Center at Houston (UTHealth).

Consent for publication N/A
Availability of data and material