2.1 Overall Study Design
To achieve our four implementation focused aims, we will use a convergent parallel mixed-methods study design to assess the planned implementation strategies and the effectiveness of In Our DNA SC. This approach will allow us to simultaneously collect quantitative and qualitative data, merge the data, and use the results to provide a comprehensive understanding of the implementation of In Our DNA SC. We will use rapid assessments of the implementation strategies (e.g., fidelity checks, training, and technical assistance) to inform changes to the intervention. Additionally, our design will track planned incidental adaptations made throughout the program.25
We will use the expanded CONSORT diagram to standardize internal reporting and produce rapid, rigorous, transparent, and relevant information.26,27
2.2 Data Collection Strategies
We currently plan to collect data from ten sources. These strategies include a data dashboard, participant interviews, use of training materials, implementation site logs, site readiness surveys, work group logs, adaptation logs, research coordinator logs, and check-in surveys.
Data Dashboard. Monitoring participant recruitment and sample collection will take place through a Data Dashboard in the electronic health record. The Data Dashboard includes summary information about: total number of recruitment messages sent, declined, non-response, express interest, enrolled, samples collected, sample re-collection (original sample was not sufficient), results sent to Helix, results returned to the participant, number of positive individuals who complete genetic counseling, and number who schedule additional screening. Data will also be stratified by demographics, based on information available from the electronic health record, including: age, sex, race, ethnicity, education, income, and area of residence.
Participant Interviews. Qualitative data will also be collected to further probe in areas of significant drop off or where there are discrepancies in the anticipated and actual numbers of individuals based on demographic categories. Current areas identified for qualitative investigation include: a) individuals who do not enroll in In Our DNA SC either because they decline or review the invitation to participate and take no action, b) people who participate in In Our DNA SC and receive negative results, and c) people who participate in In Our DNA SC and receive positive results. Individuals will be invited to participate in brief interviews via phone or email. The interview questions will be guided by a semi-structure interview guide tailored to the type of individual being interviewed. We will focus on recruiting diverse individuals to capture perceptions about genetics. We will conduct interviews every six-months throughout the duration of the program (anticipated to be 48 months).
Use of Training Materials. A major aspect to successful implementation of In Our DNA SC involves saliva sample collection. We will assess the contextual factors and strategies that influence how samples are being collected. These training materials and sample collection occurs only among clinical sites affiliated with MUSC. Data collection about use of training materials includes tracking the use of training materials available sites through MyQuest (internal training site) and Horseshoe (internal website). Data will be captured prior to implementation at a site using quantitative methods.
Use of Community-facing Education Materials. To help support engagement of historically underrepresented populations in genomics research, the In Our DNA SC program will develop educational materials to be delivered in community settings (e.g., via community health workers and other public health workforces). We will assess the impact these materials have on individual’s perceptions and understanding of genetics.
Implementation Site Logs. Qualitative data will also be by tracking weekly technical assistance calls with implementation sites and coding these meetings using CFIR with a focus on: executing, participant needs and resources, implementation climate, and leadership engagement.22 The logs have space to capture open ended notes about these discussions.
Site Readiness Survey. We will assess readiness for implementation and perceptions of the program using quantitative measures of Acceptability of Intervention Measure (AIM), Intervention Appropriateness Measure (IAM), and Feasibility of Intervention Measure (FIM) and open-ended response options.28 These surveys are distributed prior to implementation at new clinical and community sites.
Work Group Logs. Work Group Logs are a RedCap survey used to capture notes at each scheduled work group meeting. Information captured includes: date of meeting, overview of the topics discussed, and option to save key documents provided during the meeting. Immediately following each work group meeting, the study team will use the RedCap survey to identify which components of CFIR were addressed as part of the Work Group Log and whether these elements are considered facilitators or barriers.
Adaptation Logs. Adaptation Logs gather data about the changes being made to the In Our DNA SC program. Adaptation tracking includes a brief description of the adaptation made, whether it was planned or unplanned, when in the program the adaptation was made, what changed (e.g., content, context, training), the nature of the modification (e.g., tailoring, repeating a component of the intervention, changing order of components of intervention), who initiated (e.g., leadership, specific work group, stakeholder), who the change impacts (e.g., patient, implementation teams), the basis on which the changes were made (e.g., based on summary information, financial incentives), why the change was made (e.g., to increase reach, to improve adoption), the impact of the change (e.g., positive, negative), and long term impact of the change (e.g., increase reach, improve participation by teams, improved ability to deliver intervention successfully).23,29-31
Research Coordinator Logs. Research Coordinator Logs will capture questions and technical assistance needs from participants, clinicians and providers, and implementation teams. Details about the type of question and whether follow-up is needed are included in the research coordinator log.
Check-in Surveys. Check-in Surveys will be sent to implementation teams (work group members) prior to the launch of a new phase of the program to capture their experience with In Our DNA SC. These surveys assess how confident implementation teams are in the status of the program using the RE-AIM framework.24
2.3 Qualitative Data Analysis
Participant Interviews. Interviews will be transcribed and quality controlled. We will initially conduct rapid qualitative analysis to provide necessary information to inform ongoing program development. Rapid qualitative analysis involves developing a templated summary table to extract interview data, including illustrative quotes for each interview. Next, the interview summaries are consolidated by participant type in a data matrix to capture themes, sub-themes, and supporting quotes. Data from this step will be used to report to program leadership and optimize implementation.32 Full, in-depth coding will occur using thematic analysis at program completion.
Implementation Site Logs. In Our DNA SC operational staff provides weekly implementation calls where sites implementing the program gather to discuss questions and concerns about the program implementation. A member of the study team participates in these weekly calls and tracks facilitators and barriers to implementation using a RedCap tracking tool that includes space to take notes about the facilitators and barriers identified and CFIR themes. Initial qualitative analyses are conducted in real-time. Themes will be reported in summary across sites. This rapid deductive approach allows for us to capture information in real-time to help inform the process of implementation.33 Once coded, the information from the implementation site logs are used as quantitative data in Aim 2.
Work Group Logs. Similar to implementation site logs, Work Group Logs will be immediately coded in real-time. CFIR facilitators and barriers will be recorded and summarized by work group and in summary across work groups. This rapid deductive approach allows for us to capture information in real-time to help inform the process of implementation. Once coded the work group logs are used as quantitative data in Aim 3.
2.4 Quantitative Data Analysis
Descriptive analyses will be performed prior to conducting statistical analyses for each aim.
Aim 1: Monitor participation in In Our DNA SC and identify factors associated with participant engagement throughout each aspect of the program
The primary outcomes of interest for Aim 1 are at the individual level and include reach (total number of eligible people reached based on recruitment messages, declined, non-response, express interest, and enrollment), implementation (number of samples collected, re-collected, and results sent), effectiveness/efficacy (positive patients who complete program, positive patients who follow-up with genetic counselor).
Predictors for the primary data analysis include information collected through the Data Dashboard described above, including: demographics (age, sex, race, ethnicity, education, income, and area of residence).
All primary outcomes of interest related to Aim 1 are continuous and measured at the individual level. We will use simple linear regression to assess for bivariate associations between all demographic predictors on each continuous outcome of interest. We will also conduct multivariable linear regression to assess the influence of all demographic characteristics and fixed covariates for each collection site as predictors of each outcome. Data will be reported weekly as well as longitudinally to assess for trends in each outcome over time.
Aim 2: Assess contextual factors and strategies that may influence adoption and sustainment of In Our DNA SC among clinical and community sites and ongoing site-specific needs related to program implementation
The primary outcomes of interest include adoption (total number of MUSC and community sites enrolling participants, differences in adoption across sites in South Carolina) and maintenance (sites that continue to promote population-wide genomic screening beyond initial funding period).
Predictors used to assess for outcomes related to Aim 2 include the use of training materials, implementation site logs (number and type of facilitators and barriers identified), research coordinator logs (number and type of questions asked), site readiness (AIM, IAM, FIM), and community-facing materials.28
All outcomes and predictors are assessed at the site level. We will conduct bivariate analyses using simple linear or logistic regression with outcomes of interest. We will also use multivariable linear regression to assess the influence of all predictors on outcomes of interest at the clinical and community site levels.
Aim 3: Describe facilitators and barriers to implementation and perceptions of In Our DNA SC among implementation work groups
We will summarize the implementation barriers and facilitators identified during each work group meeting as part of the work group logs. These will be reported stratified by work group and summarized across all work groups. We will assess changes in facilitator and barriers longitudinally across the program. In addition, we will summarize perceptions of In Our DNA SC using findings from the check-in survey. Findings will be stratified by work group and in summary. Additionally, we will assess change in perceptions longitudinally across the program.
Aim 4: Track adaptations made to In Our DNA SC over time to assess how mechanisms of change impact key programmatic outcomes
All quantitative data captured through the adaptation tracking logs will be regularly assessed through descriptive analyses. Information from each log will be reported in aggregate (e.g., types of adaptations made, key CFIR facilitators and barriers identified in work group logs) on an ongoing basis. The longitudinal tracking will allow the team to assess various mediators and moderators of program outcomes.
The data collected as part of Aim 4 is primarily designed to provide structure to track the roll out of In Our DNA SC and to capture the implementation over time. We will develop additional tracking tools as needed over the course of program implementation.
2.5 Integrating Quantitative and Qualitative Methods
Across all aims, data will be integrated to provide rapid feedback to the implementation teams and leadership overseeing the implementation of In Our DNA SC. We will closely monitor factors that influence DEI and representation of racial and ethnic minorities in the program. Reported barriers and facilitators will be matched with evidence-based mechanisms of change and implementation strategies to quickly resolve implementation barriers and tailor implementation strategies to be suitable for groups that have historically been excluded from genetic research.29 We will continue to use this approach of rapid assessment and intervention modification throughout the stages of program implementation and adjust our approaches to monitoring participation as needed.