Quantitative Data
Individuals Screened and Enrolled
Figure 1 characterizes recruitment flow using Inbound and Outbound strategies. A total of 18,925 potential participants were attempted contact, representing all patients on call lists for Outbound recruitment (18,067) plus individuals who responded Inbound to community-based recruitment (858) across the three study sites. For Inbound callers, 143 screened eligible for baseline while 464 were ineligible during phone screening. Of the 143 Inbound respondents who were referred for baseline, 74 enrolled in the study.
For Outbound recruitment, 154 people were eligible for baseline while 4,220 were ineligible at phone screening. Another 6,000 individuals declined participation and 7,693 could not be reached for screening. Of the 154 respondents who were referred for baseline, 66 completed the assessment and enrolled in the study. The enrollment rate across the population in which contact was attempted was approximately 1% (140 enrolled divided by 18,925 people total).
Differences in recruitment rates were observed between the Inbound and Outbound approaches (Figure 1 and Table 2). Forty-three percent of subjects could not be contacted from the Outbound calling list while 12% of Inbound respondents could not be re-contacted. Of those who were reached, the rate of self-withdrawal from consideration was 3 times higher in the Outbound approach compared with Inbound approach (58% versus 20%). For those who were interested and completed phone screening, ineligibility was frequent in both groups though notably higher in those contacted using the Outbound call strategy (96% versus 76%). Although the numbers of Outbound (18,067) and Inbound (858) pools of people vastly differed, similar numbers of subjects screened eligible for baseline (Outbound = 154 versus Inbound =143). Of those who were reached for phone screening, the success rate of ultimately enrolling the study subjects was 15 times higher in the Inbound calling group compared with the Outbound calling group (9.7% versus 0.6%).
Effort Required to Recruit and Enroll Participants
Recruitment efforts for the study began in May 2012 at the WV and OH sites (see Figure 2). Based on data collected during the pilot study (R34DK71545),(17) we anticipated that it would be feasible to recruit the entire sample from the WV and OH sites. However, because recruitment rates at these two sites were lower than expected, recruitment began at the IN site in December 2013 to expedite enrollment. December 2013 also marked the start of Outbound calling at the WV site. As a result of these changes, the recruitment rate rose from 5.4 participants per quarter to 14.2 participants per quarter. This rate continued February 2015 when recruitment ceased at the OH site. Because WV started recruitment without the use of Outbound calling, the enrollment rate there demonstrates the effectiveness of the Outbound strategy by increasing the average quarterly enrollment rate from 0.6 to 2.9 participants enrolled per quarter.
Several key developments during the recruitment period are evident in Figure 2. These included direct mailings by the OH and WV sites in February 2013, the initiation of Outbound recruitment strategies by the IN and WV sites in January 2014, cessation of recruitment efforts at the OH site in February 2015, and an increase in incentives to participants to enhance recruitment and retention in February 2014 and July 2015.
In order to estimate staff effort needed to achieve recruitment goals, average times for each contact activity were calculated. Group mean values were used to estimate for the missing entries. For all phone screening recruitment efforts, the estimated cumulative time spent on these efforts was 3,567 hours, the majority of which was consumed by Outbound calling (3,209 hours). An average of 25.5 hours of personnel time was spent on all phone recruitment activities for each person enrolled (Table 2).
A total of 1,397 hours of staff time was spent on baseline activities for the 297 people who screened eligible by phone. This included time for scheduling assessments, mailing letters, in-person assessment activities, and the full psychiatric interview. Of the 297, 140 people completed baseline and were eligible for study participation. The average time for eligibility assessments to reach 140 participants was 10 hours per enrolled participant. Adding the baseline time to the phone screening time, a total of 4,964 hours was required for all direct recruitment and enrollment efforts, an average of 35.5 hours per participant enrolled, to obtain the final enrollment of 140 participants.
Costs of Participant Recruitment
Staff labor and advertising costs were calculated to determine the total cost of recruitment per participant. Staff labor rates were figured at $23.59 per hour to include both salary and benefits. The total calculation of all eligibility screening activities was $117,093 (see Table 2). Outbound calls accounted for the majority of recruitment expenditures ($75,709) due to the time involved. The remaining phone screening figure, $8,435, was attributable to Inbound calls. Baseline activities added $32,949 in expenditures. The average amount to screen and enroll each participant into the study was $836.
Additional funds were used for advertising and outreach. Across all three sites an estimated $72,963 was spent on newspaper advertising, printing posters, and postage for direct mailings. This figure is comparable to the Outbound calling efforts. Total sum of all recruitment, screening, and enrollment activities was estimated at $190,056, which represents an average of $1,358 per participant enrolled in the study.
Qualitative Observations
The study team across all three sites met 1-2 times per month throughout the funding period. During these meetings, the study investigators discussed a variety of barriers, giving context to the extraordinary effort required by the study teams to meet recruitment goals. These barriers included geographic factors, institutional factors, study personnel factors, and cultural and social barriers, as we describe next.
Geographic Factors
The differing geographical and cultural landscape at each of the study sites provided opportunities and challenges for recruitment. Physical distance and transportation infrastructure posed challenges to recruitment efforts at the rural OH and WV sites. The lack of public transportation in rural communities presented a unique problem to recruitment s that could have been a factor in motivation to participate in the study. Participants were not compensated for travel to and from assessment sites and some locations for assessments were a fair distance from residences. Ohio participants lived an average of 11 miles and West Virginia participants an average of 13.5 miles from the closest assessment sites, which were located in communities surrounding the study sites in order to be closer to the population areas where the participants lived. However, some participants at these sites lived as far as 20 to 25 miles from the nearest assessment site, for a round trip as much as 50 miles away from the locations were the study procedure would take place. This present a significant burden to the participants, particularly those who were randomized to receive both CBT and exercise interventions, for as many as two appointments per week during the intervention period.
Institutional Factors
The recruitment strategies were influenced by personnel composition and the research climate within each university. Ohio recruitment took place at Ohio University, which is predominantly a teaching institution. While Ohio University has ties to many of the 29 Appalachian counties in southern Ohio, the history and reach of clinical trials has traditionally been on a smaller scale. Formal collaborative partnership agreements with private practices and health systems in the surrounding area did not exist prior to the start of the study and needed to be established on an individual basis. A total of five health care systems partnered with OH over the course of the study. A request to the Ohio University IRB to make Outbound calls to potential participants was denied, limiting this site to the use of Inbound recruitment strategies.
West Virginia University has a long-standing history of clinical research, with the majority being conducted in the basic sciences and T1 translational trials. Institutional infrastructure leveraged for recruitment at this site included partnership with the West Virginia University CTSI, collaborative research agreements between the university and multiple health care systems serving northern WV, and a large-scale electronic medical record system from which eligible patients could be identified for recruitment.
In Indiana, recruitment was conducted at the Indiana University School of Medicine (IUSM), the lead study site. IUSM along with the Indiana CTSI have longstanding collaborations with the Eskenazi Health and Indiana University Health systems to collaborate on participant recruitment for clinical trials. These agreements, in partnership with the Regenstrief Institute’s Data Core, facilitated access to patient lists in order to conduct Outbound recruitment by identifying potentially eligible participants. Additional assistance was provided by the Indiana CTSI Research Network (ResNet), whose research assistants performed the majority of Outbound calling to medically screen interested patients and refer eligible patients to the study team.
Institutional differences across the sites expanded or limited the options available to investigators in reaching potential participants. While all sites were influenced by structural and regulatory changes in health care markets and delivery that occurred during the study period from 2012 to 2016, each site entered this period of change with different characteristics, resulting in more or less dramatic changes during the study period. For example, the communities around OH underwent a dramatic transformation in the consolidation of many small private primary care practices into larger health care organizations. Contractual contingencies on productivity expectations of physicians in these organization placed an implicit and at times an explicit limitation on physician efforts to convey study information to patients. While business managers and physicians acknowledged the benefits of how the study could complement their work and the potential benefits for patients, they were hindered by time and resource restrictions, which is consistent with the finding of other researchers in rural environments with limited resources.(18)
Study Personnel Factors
The organization of the study teams contributed to relative strengths and challenges for each site. For example, at WV, study staffing was initially provided by graduate students. While this provided high quality talent, the nature of the program limited the amount of time students could serve as study employees. This staff turnover contributed to challenges associated with establishing relationships with community organizations. A full-time project coordinator was hired in year 3, which facilitated greater continuity in building relationships with community partners as well as planning and implementation of new strategies. At OH, development of a new research center precipitated changes in study staffing, which ultimately precluded ongoing participation of OH in further enrollment.
Cultural and Social Barriers
Cultural barriers to recruitment included the burden of living with major depressive disorder, limited prior experience with clinical trials, and stigma associated with depression. Initial Inbound recruitment approaches needed potential participants to be sufficiently motivated to call a toll-free number. Since study inclusion criteria required the diagnosis for MDD, it is likely that some potential participants were unable to participate due to the manifest symptoms of depression such as hopelessness, lack of energy, anhedonia, and impaired social interactions. A significant number (296 people or 34.5%) who made the call to the toll-free number did not meet full criteria for MDD, which supports the hypothesis that people with higher burden of depression were less likely to contact the study team. Recruitment strategies such as mailings and fliers posted at clinics, doctor’s offices, churches, and community centers have a lower response rate when compared to recruitment strategies such as the use of a professional interviewer to follow up a mailing or word of mouth.(19, 20)
The target communities at each study site differed in levels of experience with clinical trials. Individuals living in rural areas have limited exposure to clinical trials compared their urban counterparts, which appeared to contribute to greater hesitancy to respond to Inbound recruitment efforts. Perceptions of researchers as outsiders and the perception of participation in research as being a “guinea pig” contributed to hesitancy to engage in screening.
The benefits of participating in a randomized controlled trial may have been viewed with reluctance, especially given that participants could not be promised tangible benefits other than what they may receive by being randomly assigned to a treatment group. As a result, randomization to a specific treatment group without choice may have dissuaded individuals from volunteering. People who are seeking treatment for MDD frequently do not wish to be assigned to a Usual Care group, risking the possibility of receiving no treatment for the three months of the intervention period. This resulted in some eligible participants declining to participate in the study. Additionally, the possibility of being assigned to a counseling group may have influenced participation due to stigma associated with counseling.
At the IN site, clinical trials are more familiar to potential participants but those trials typically involve fewer demands on participant time and engagement. Participants employed in hourly positions working in service industries consistently reported difficulty in knowing their work schedules far enough in advance to be able to keep study appointments. As a result, adaptations were made to accommodate work schedules. Additionally, the target participant population at this site had an expectation of payment for study participation. A decision to institute and increase participant incentives across all sites was made on the basis of these public expectations.