A study within a trial (SWAT) of clinical trial feasibility and barriers to recruitment in the United Kingdom – the CapaCiTY programme experience

doi:10.21203/rs.3.rs-3406175/v1

Download PDF

Research Article

A study within a trial (SWAT) of clinical trial feasibility and barriers to recruitment in the United Kingdom – the CapaCiTY programme experience

https://doi.org/10.21203/rs.3.rs-3406175/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background: Recruitment is the Achilles heel of clinical trials. Failure to recruit the planned sample jeopardises reliability of results and wastes significant time and resources. Site feasibility assessments are a common tool employed by sponsors to assess a site’s recruitment potential and ability to undertake the trial protocol. The purpose of this study was to evaluate site feasibility procedures as a tool for predicting recruitment potential.

Methods: We conducted site feasibility assessments with thirty-nine sites across the UK. Twenty-seven were selected to participate in the CapaCiTY programme: three randomised controlled trials (01; ISRCTN11791740; 16/07/2015, 02; ISRCTN11093872; 11/11/2015, 03; ISRCTN11747152; 30/09/2015) aiming to develop an evidence based adult chronic constipation treatment pathway. We compared site feasibility-based predicted recruitment rates with actual recruitment rates and conducted a telephone survey (n=24) to understand barriers to recruitment.

Results: Sites failed to accurately predict their recruitment potential during site feasibility. Conducting site feasibility did not result in the trials recruiting the required sample. At the time of survey, 17% of sites in CapaCiTY01 were meeting recruitment targets, 83% were recruiting half or less of their target. Of these, 17% recruited no participants. In CapaCiTY02, 13% of sites were recruiting to target, 87% were recruiting half or less; of these, 25% recruited no participants. In CapaCiTY03, 14% of sites were recruiting to target, 86% recruited half or less of the target, of these 43% recruited none. The main barriers to recruitment were lack of resources, high workloads, lack of suitable participants and study design not being compatible with routine care.

Conclusions: Improving the reliability of site feasibility assessment could potentially save hundreds of millions of pounds each year in failed research investments and speed up the time to delivery of new treatments for patients. We recommend 1) conducting site feasibility assessment at the pre-award stage; 2) investment in training researchers in conducting and completing site feasibility; 3) funders to include clinical trial feasibility specialists on peer review and grants panels; 4) development of a national database of sites’ previous trial recruitment performance; 5) data-driven site level assessment of recruitment potential.

Trial registration:

ISRCTN11791740; 16/07/2015, ISRCTN11093872; 11/11/2015, ISRCTN11747152; 30/09/2015.

clinical trial

feasibility

recruitment

barriers

RCT

SWAT

chronic constipation

Recruitment of participants is the backbone of every clinical trial and has long been the Achilles heel of investigators(1). When clinical trials fail to recruit to time or target, they waste time, effort, and money, cause reputational damage, with potentially serious ethical implications(2). Failure to reach the sample size required to achieve the primary outcome causes results to be much less reliable or even untenable. Underpowered trials waste a significant amount of public funds each year. In 2014 the National Institute for Health Research (NIHR) invested £289.6 million in research programmes across the United Kingdom (UK)(3), a substantial proportion being spent on randomised controlled trials (RCTs), which represent the gold standard research design for evaluating clinical interventions(4). This increased to £645.9 million in 2021(3).

A review of the NIHR Health Technology Assessment (HTA) and Medical Research Council (MRC) RCTs in the UK from 2004 to 2016 found that just over half (56%) of these trials achieved their original target sample size and just over three quarters (79%) achieved at least 80% of the required sample size(4). This is only a marginal improvement on 31% of HTA/MRC-funded RCTs achieving the intended sample size in a similar review from 1994 to 2002, with 78% reaching at least 80% of the sample size(5). Over half (53%) of these studies were awarded an extension, requiring more time and sometimes more money to recruit to target(5). These statistics are not much better in the United States where 11% of sites fail to recruit a single patient, 48% of sites underperform, 80% of trials fail to meet their enrolment timelines, with timelines for phase II-IV trials usually doubling and on average only half of all patients screened complete clinical trials(6) .

This phenomenon is nothing new. In 1979 Louis Lasagna, the father of modern pharmacology, observed that the number of patients available to join a study drops by 90% on the day the trial starts and re-appear the day a trial end (22). This is commonly known as the “Lasagna Law”. Given these figures, why are investigators still so overly optimistic about their recruiting potential? Why are funders not doing more to assess the feasibility and validity of the grant proposals they award?

A common tool often employed during the clinical trial set up phase is a site selection and feasibility assessment. The purpose of which is to assist sponsors to: 1) determine whether to work with a particular site and investigator based on their experience, 2) establish site demographics, 3) measure infrastructure, 4) assess readiness, 5) set a realistic recruitment target based on the number of likely suitable participants, and 6) discuss and map potential barriers not only to recruitment but also for trial delivery(7). Thus, site selection and feasibility should allow the sponsor to determine the number of suitable sites required to meet the recruitment target and protocol objectives within the awarded time and budget. Unfortunately, the use of such tools often comes too late in the lifecycle of trial development when the sample size, primary outcome, budget, and timescales have already been set, and funding awarded. Adding additional sites to meet shortfalls in recruitment not only increases costs but places additional burden on trial management, data management and monitoring staff, with potentially significant delays(8). As a result, trial managers are often left with trials that are completely unfeasible within the allocated time and budget and thus are doomed to fail, regardless of the multiple strategies and incentives they may use to bolster recruitment.

Recruitment to RCTs has been identified as a priority by funders and investigators, with research into methods to boost recruitment being nominated as the top priority by UK clinical trials unit directors in a Delphi survey in 2014(9). Although recruitment (and retention) are often hot topics of discussion amongst triallists, the evidence base for barriers to recruitment and successful strategies for overcoming them is still lacking(10).

The UK National Health Service (NHS) is uniquely positioned to perform large scale, multi-centre clinical trials. The clinical trial landscape in the UK consists of specialist clinical trials units (CTUs), which have been set up with a specific remit to design, conduct, analyse and publish clinical trials and other well-designed studies. They have the capability to provide specialist expert statistical, epidemiological, and other methodological advice and coordination to undertake successful clinical trials. A UK clinical research council (CRC) registration process has been developed, recognising CTUs of high quality and experience conducting multi-centre clinical trials(11).

In addition, clinical research networks (CRNs) have been established across the UK with the goal of providing the required infrastructure to support high quality clinical studies across all areas of disease and clinical need, for the benefit of patients. They provide critical resources and support to the NHS, such as clinical research nurses and other staff to match patients with appropriate study participation opportunities and carry out the clinical duties required by the studies. CRN funding also covers costs related to study delivery such as radiological imaging(11). Principal investigators (PIs) are salaried staff, thus trials in the UK generally do not receive a payment for each person recruited to a study (payment per recruit). Rather the CRN provides funding for research nurses to undertake recruitment activities.

Clinical commissioning groups (CCGs) were created in the UK following the Health and Social Care Act in 2012. They are clinically-led statutory NHS bodies responsible for the planning and commissioning of health care services for their local area and getting the best possible health outcomes for their local population(12). The costs of non-commercial research are met by different funders depending on the type of cost. Guidance from the Department of Health and Social Care (DHSC) for the Attribution of Costs for Research and Development (AcoRD) sets out the principles for determining who pays for the different research costs. Treatment costs are the care costs that would continue to be incurred if the patient care service in question continued to be provided following the end of the research study. This cost is usually covered by the service provider or NHS. The difference between the research prescribed treatment costs and the costs of the existing standard treatment is referred to as the excess treatment cost (ETC). ETCs are paid for by CCGs. With a national health service and embedded clinical trial’s infrastructure to support high quality research, why is recruitment still such a major challenge for most UK trials?

The purpose of this study was to evaluate site feasibility procedures as a tool for predicting site recruitment potential. We compare the predicted recruitment rates identified during site feasibility assessments with the actual recruitment rates in a recent major research programme including three RCTs. We describe the strategies used to improve recruitment and whether they were deemed successful.

The UK National Institute of Health Research (NIHR) Programme for Applied Research (PfAR) funded the CapaCiTY programme aimed to develop the evidence base for the management of chronic constipation in adults and develop a Chronic Constipation Treatment pathway (13). The design consisted of a national recruitment programme (n = 808 participants) with standardised methodological framework (eligibility, baseline phenotyping and standardised outcomes) for three randomised trials. CapaCiTY01 (n = 394) randomly allocated adults with chronic constipation to habit training versus habit training with biofeedback. This was a parallel 3-arm trial, with two randomised comparisons, with and without stratification by pathophysiological investigations (14). CapaCiTY02 (n = 300), a pragmatic randomised trial, compared low volume with high volume initiated anal irrigation therapy in adult patients with chronic constipation who did not respond to nurse-led biofeedback or bowel habit training. This was a parallel 2-arm trial (15). CapaCiTY03 (n = 114) used a stepped wedge individual level 3-arm randomised design to evaluate a surgical procedure - laparoscopic ventral mesh rectopexy in adults with chronic constipation(16, 17). The CapaCiTY programme was supported by the UKCRC registered Pragmatic Clinical Trials Unit (PCTU) and received ethics approval from the London City and East research ethics committee (REC) and registered as follows; CapaCiTY01 REC 14/LO/1786 ISRCTN11791740, CapaCiTY022: REC 15/LO/0732, ISRCTN11093872, CapaCiTY033: REC 15/LO/0732, ISRCTN11747152.

We conducted site feasibility assessments [supplementary files 1,2,3] with sites prior to inclusion in the CapaCiTY programme. The feasibility questionnaire was used to inform the coordinating centres (Queen Mary University of London for studies 1 and 3, Durham University Hospital for study 2) which sites were suitable for site selection. Site feasibility assessed principal investigator experience, resource levels (staffing, facilities, and equipment), staff training and acceptability of trial protocol, recruitment potential, methods, barriers and competing trials, ability to comply with planned data management and quality assurance procedures (study 1 & 3), and local irrigation practices (study 2). Feasibility was conducted with the principal investigator and lead nurse or project coordinator or manager at each site and consisted of a series of telephone conversations and completion of site feasibility questionnaire.

Due to poor recruitment performance across most sites, we conducted telephone surveys in 2017, with 24 site staff across the three CapaCiTY trials; 3 PI’s, 5 specialist nurses and 15 research nurses and 1 trial coordinator completed the survey. Surveys consisted of eleven pre-defined questions with fixed responses (Table 1). The aim of the survey was to identify the barriers to recruitment at each site and explore recruitment enablers or strategies to improve recruitment. We then compared actual recruitment numbers at the time of survey (2017) with those predicted at the time of site feasibility assessment and/or at site selection and contracting (2014–2015) with final recruitment figures (Table 2). This study within a trial (SWAT) (18) was approved by the Queen Mary University of London Ethics Committee (ref: QMREC2023a).

Table 1

CapaCiTY programme predicted and actual recruitment rates.
Site Number	Predicted recruitment/ month (feasibility)	Target recruitment/ month (contracted)	Actual recruitment/ month (at time of SWAT survey 2017)	% Monthly Target (at time of SWAT survey 2017)	% Total Target (at end of study recruitment)	*CRN Support
CapaCiTY01
1	2	1.5	2.2	150%	118%	Yes
2	20	1.0	0.2	20%	30%	Yes
3	50	1.9	0.7	36%	34%	Yes
4	-	1.2	0	0%	0%	Yes
5	-	1.7	0	0%	0%	Yes
6	-	1.5	0.4	27%	27%	Yes
7	-	1.5	0.3	20%	33%	Yes
8	2	1.3	1.3	104%	47%	No
9	3	1.2	0.2	17%	23%	No
10	5	1.6	0.2	13%	12%	Yes
11	2	2.1	0.3	15%	16%	Yes
12	10	1.7	1.0	60%	52%	No
CapaCiTY02
13	-	0.7	0.5	71%	67%	Yes
14	-	0.7	1.1	154%	110%	No
15	-	0.6	0.3	50%	30%	Yes
16	-	0.5	0.1	14%	5%	No
17	-	0.8	0	0%	10%	No
18	-	1.5	0	0%	0%	Yes
19	-	1	0.1	10%	3%	Yes
20	-	0.7	0.5	71%	7%	No
CapaCiTY03
21	3	0.5	0.5	100%	92%	Yes
22	2	0.4	0.0	0%	0%	Yes
23	3	0.8	0.0	0%	0%	Yes
24	1	0.4	0.3	75%	70%	Yes
25	1	0.7	0.1	14%	5%	Yes
26	1	0.4	0.0	0%	0%	Yes
27	20	0.3	0.2	60%	50%	Yes

*CRN: clinical research network

Recruitment rates

Table 1 outlines the predicted and actual recruitment rates for the 27 NHS sites participating in the CapaCiTY programme, at the time of the feasibility assessment, site selection (contracted), SWAT recruitment survey, and at the end of recruitment for the programme.

Sites almost uniformly failed to accurately predict their recruitment potential during site feasibility assessments, and in many cases, significantly overestimated. The contracted number (Table 1) reflects target recruitment as agreed in the clinical trial research agreement (CTRA) following discussion with the local team and site selection. The contracted targets were adjusted for site’s overestimation of recruitment potential based on more realistic estimate of eligible participants in each centre and estimated screen failure rates. Conducting a site feasibility assessment did not result in the trials reaching the planned sample size.

Recruitment for CapaCiTY01 commenced on 26th March 2015 and ended 30th June 2018. Twelve sites were assessed for feasibility and opened to recruitment, with ten of these eventually recruiting participants. A total of 182 (target 394, 46%) participants were randomised, of 502 screened from 10 sites (conversion rate, 36%) (Supplementary file 4: CapaCiTY01 consort). At the time of the telephone survey (2017), only two out of twelve (17%) sites were meeting or exceeding their intended recruitment rates, one of which was the lead site. One (8%) site was achieving at least half the intended recruitment rate whilst the remaining nine (75%) sites were achieving less than half the intended recruitment rate with two of these (17%) not recruiting any participants at all.

CapaCiTY02 commenced recruitment on 11th November 2015 and ended recruitment on 30th June 2018. Eleven sites were assessed for feasibility, eight were opened to recruitment and six actively recruited participants. A total of 65 (target 300, 22%) participants were randomised of 150 screened from 6 sites (conversion rate, 43%) (Supplementary file 5: CapaCiTY02 consort). At the time of the telephone survey (2017), only one (the lead) site of eight (13%) was meeting their intended recruitment rate. Three (38%) sites were achieving at least half the intended recruitment rate whilst the remaining four (50%) sites were achieving less than half the intended recruitment rate with two of these (25%) not recruiting any participants at all.

Recruitment for CapaCiTY03 started on 1st March 2016 and ended on 31st January 2019. Sixteen sites were assessed for feasibility, seven were opened to recruitment and four actively recruited. A total of 28 (target 114, 24%) participants were randomised of 81 screened from 6 sites (conversion rate, 35%) (Supplementary file 6: CapaCiTY03 consort). At the time of the telephone survey (2017), only one (the lead) site out of seven (14%) was meeting or exceeding their intended recruitment rate. Two (29%) sites were achieving at least half the intended recruitment rate whilst the remaining four (57%) sites were achieving less than half the intended recruitment rate with three of these (43%) not recruiting any participants at all.

Barriers to recruitment

Table 2

SWAT telephone survey results
	CapaCiTY01 sites (n = 10)		CapaCiTY02 sites (n = 8)				CapaCiTY03 sites (n = 6)
Q1. What have been your barriers to recruitment and are these now resolved?	Yes	No	Yes		No		Yes		No
a) Lack of staff resources	6 (60%)	4 (40%)	2 (25%)		6 (75%)		0 (0%)		6 (100%)
CRN staff	4 (40%)	6 (60%)	1 (12%)		7 (88%)		0 (0%)		6 (100%)
NHS staff for intervention	4 (40%)	6 (60%)	1 (12%)		7 (88%)		1 (17%)		5 (83%)
Radiology	2 (20%)	8 (80%)	1 (12%)		7 (88%)		1 (17%)		5 (83%)
PI/leadership staff	3 (30%)	7 (70%)	0 (0%)		8 (100%)		0 (0%)		6 (100%)
Other	0 (0%)	10 (100%)	0 (0%)		8 (100%)		0 (0%)		6 (100%)
b) Lack of support from PI/research nurse	2 (20%)	8 (80%)	1 (12%)		7 (88%)		0 (0%)		6 (100%)
c) Forgetting to recruit to the trial	2 (20%)	8 (80%)	2 (25%)		6 (75%)		2 (33%)		4 (67%)
d) Not comfortable with trial/protocol	1 (10%)	9 (90%)	1 (12%)		7 (88%)		1 (17%)		5 (83%)
e) Lack of suitable patients	1 (10%)	9 (90%)	5 (62%)		3 (38%)		3 (50%)		3 (50%)
f) Inclusion/exclusion criteria too difficult	1 (10%)	9 (90%)	2 (25%)		6 (75%)		3 (50%)		3 (50%)
g) Patient’s not willing to consent/be randomised	2 (20%)	8 (80%)	4 (50%)		4 (50%)		2 (33%)		4 (67%)
h) Patient’s not willing to travel for study visits	2 (20%)	8 (80%)	2 (25%)		6 (75%)		0 (0%)		6 (100%)
i) Study design not compatible with routine care	3 (30%)	7 (70%)	5 (62%)		3 (38%)		4 (67%)		2 (33%)
j) Busy with other trials?	1 (10%)	9 (90%)	0 (0%)		8 (100%)		1 (17%)		5 (83%)
k) Pressure and workload	6 (60%)	4 (40%)	3 (38%)		5 (62%)		2 (33%)		4 (67%)
l) Lack of funding/resources	3 (30%)	7 (70%)	2 (25%)		6 (75%)		1 (17%)		5 (83%)
m) none	0 (0%)	10 (100%)	0 (0%)		8 (100%)		1 (17%)		5 (83%)
n) unknown	0 (0%)	10 (100%)	0 (0%)		8 (100%)		0 (0%)		6 (100%)
o) Other, specify:	0 (0%)	10 (100%)	0 (0%)		8 (100%)		0 (0%)		6 (100%)
Q2. What are your current recruitment methods?	Yes	No	Yes		No		Yes		No
a) Referral from secondary care	7 (70%)	3 (30%)	4 (50%)		4 (50%)		4 (67%)		2 (33%)
Referral from PI clinic	8 (80%)	2 (20%)	5 (62%)		3 (38%)		5 (83%)		1 (17%)
Referral from other consultant/nurse clinic	6 (60%)	4 (40%)	4 (50%)		4 (50%)		2 (33%)		4 (67%)
Referral from GI physiology clinic	5 (50%)	5 (50%)	1 (12%)		7 (88%)		2 (33%)		4 (67%)
b) Referral from primary care	4 (40%)	6 (60%)	1 (12%)		7 (88%)		0 (0%)		6 (100%)
c) Referral from pharmacy	0 (0%)	10 (100%)	0 (0%)		8 (100%)		0 (0%)		6 (100%)
d) Referral from tertiary care	3 (30%)	7 (70%)	2 (25%)		6 (75%)		2 (33%)		4 (67%)
e) Electronic health records searches	2 (20%)	8 (80%)	2 (25%)		6 (75%)		1 (17%)		5 (83%)
f) Discussion at multidisciplinary team meeting	4 (40%)	6 (60%)	2 (25%)		6 (75%)		4 (67%)		2 (33%)
g) Response to poster	2 (20%)	8 (80%)	1 (12%)		7 (88%)		1 (17%)		5 (83%)
h) Trial website or public registry	1 (10%)	9 (90%)	1 (12%)		7 (88%)		1 (17%)		5 (83%)
i) Newspaper advertisements	2 (20%)	8 (80%)	1 (12%)		7 (88%)		1 (17%)		5 (83%)
j) Social media	0 (0%)	10 (100%)	0 (0%)		8 (100%)		0 (0%)		6 (100%)
k) Patient support groups	0 (0%)	10 (100%)	0 (0%)		8 (100%)		0 (0%)		6 (100%)
l) Other; define	0 (0%)	10 (100%)	0 (0%)		8 (100%)		0 (0%)		6 (100%)
Q3. What level of support are you receiving?	Total	Mean	Total		Mean		Total		Mean
a) Number of staff assigned to the trial?	39	3.9	24		4		18		3.6
b) Total time dedicated to trial activity per week (hrs)?	55.5	7.9	28		7		4		2
c) CRN nurse for recruitment?	7	0.7	2		0.4		5		1
d) CRN nurse time spent on recruitment per week (hrs)?	41	4.5	2.5		0.4		2.5		0.8
	Yes	No	Yes		No		Yes		No
e) Do you know your CRN network coordinator or delivery manager and have you asked for help?	3 (30%)	7 (70%)	3 (38%)		5 (62%)		2 (33%)		4 (67%)
f) CRN support received for sending posters to your GP practices and pharmacy via primary care network	1 (10%)	9 (90%)	2 (25%)		6 (75%)		1 (17%)		5 (83%)
Q4. How can we help you?	Yes	No	Yes		No		Yes		No
a) Business case for funding additional clinics (CRN funding)	2 (20%)	8 (80%)	0 (0%)		8 (100%)		0 (0%)		6 (100%)
b) Request additional CRN nursing support	3 (30%)	7 (70%)	2 (25%)		6 (75%)		0 (0%)		6 (100%)
c) Data entry assistance	3 (30%)	7 (70%)	2 (25%)		6 (75%)		0 (0%)		6 (100%)
d) Posters in clinic, GPs, pharmacies	3 (30%)	7 (70%)	2 (25%)		6 (75%)		1 (17%)		5 (83%)
e) Retraining in trial protocol	0 (0%)	10 (100%)	1 (12%)		7 (88%)		0 (0%)		6 (100%)
f) Protocol amendment	1 (10%)	9 (90%)	3 (38%)		5 (62%)		1 (17%)		5 (83%)
g) Protocol amendment specify	0 (0%)	10 (100%)	0 (0%)		8 (100%)		0 (0%)		6 (100%)
h) Additional site visits	1 (10%)	9 (90%)	1 (12%)		7 (88%)		0 (0%)		6 (100%)
i) Additional investigator meetings	2 (20%)	8 (80%)	3 (38%)		5 (62%)		1 (17%)		5 (83%)
j) Other, specify:	0 (0%)	10 (100%)	1 (12%)		7 (88%)		0 (0%)		6 (100%)
Q5. What can you do to overcome your barriers?	Yes	No	Yes		No		Yes		No
a) Request additional time/support with the project	5 (50%)	5 (50%)	1 (12%)		7 (88%)		0 (0%)		6 (100%)
b) Prioritise the study	1 (10%)	9 (90%)	1 (12%)		7 (88%)		1 (17%)		5 (83%)
c) Place on MDT or research agenda	2 (20%)	8 (80%)	1 (12%)		7 (88%)		1 (17%)		5 (83%)
d) Attend clinics for recruitment	2 (20%)	8 (80%)	2 (25%)		6 (75%)		1 (17%)		5 (83%)
e) Put up posters	3 (30%)	7 (70%)	2 (25%)		6 (75%)		0 (0%)		6 (100%)
f) Liaise with CRN delivery manager	2 (20%)	8 (80%)	1 (12%)		7 (88%)		1 (17%)		5 (83%)
g) Other :	0 (0%)	10 (100%)	0 (0%)		8 (100%)		0 (0%)		6 (100%)
Q6. Do you think the recruitment prizes are fair?	Yes	No	Yes		No		Yes		No
	4 (40%)	6 (60%)	4 (50%)		4 (50%)		2 (33%)		4 (67%)
Q7. Do the prizes make any difference to your recruitment effort?	Yes	No	Yes		No		Yes		No
	6 (60%)	4 (40%)	3 (38%)		5 (62%)		1 (17%)		5 (83%)
Q8. Do you think you will recruit to target?	Yes	No	Yes		No		Yes		No
	4 (40%)	6 (60%)	2 (25%)		6 (75%)		3 (50%)		3 (50%)
Q9. Do you require a revision to your site agreement?	Yes	No	Yes		No		Yes		No
	2 (20%)	8 (80%)	3 (38%)		5 (62%)		2 (33%)		4 (67%)
Q10. Has this experience changed the way you would complete study feasibility in the future?	Yes	No	Yes		No		Yes		No
	5 (50%)	5 (50%)	4 (50%)		4 (50%)		5 (83%)		1 (17%)
Q11. If applicable, do you find running multiple CapaCiTY trials conflicting, beneficial, or neutral? (n = 10)		Conflicting		Beneficial		Neutral		Not applicable
		1 (10%)		3 (30%)		5 (50%)		1 (10%)

During the telephone survey with site staff, the main obstacle to recruitment identified in CapaCiTY01 was a lack of staff resources coupled with high pressure and workload (Table 2). In contrast, the main barriers to recruitment identified in both CapaCiTY02 and CapaCiTY03 were study design not being compatible with routine care, coupled with a lack of suitable participants (Table 2).

Lack of staff and resources

Staffing issues such as long-term staff sickness, maternity leave, retirements, delays in recruiting staff and lack of administrative support for NHS sites were amongst the problems reported. These issues permeated the programme across all three trials, in particular the largest study CapaCiTY01. We tried to overcome some of these issues by providing administrative support to these sites such as data entry, creating and distributing site files and case report form folders, creation of local documents for governance approvals, localised posters for advertising, and assisting with business cases to apply for additional clinical research nurse resource to support participant recruitment. Unfortunately, the CRN support for recruitment model also proved problematic as often research nurses were shared across multiple sites or departments. This meant that a research nurse could spend time in a department recruiting participants for the trial at a time when the principal investigator and other clinical staff were not around. This led to inefficient use of CRN resources and time wasted screening unsuitable participants with lack of access to or guidance from the clinical team. There did not appear to be any direct relationship between the level of CRN support and recruitment success across all three studies. When we investigated this further, we realised that sites with a more integrated model of CRN support (where the CRN nurse was embedded in the one department full time with principal investigator support) were most successful in meeting recruitment targets.

Study design and protocol compatibility

In CapaCiTY02, some UK CCGs stopped prescribing anal irrigation devices due (ironically) to ‘insufficient evidence regarding effectiveness’. Thus, irrigation devices were no longer covered by ETCs. We tried to overcome this challenge by requesting device manufacturers to provide devices to these sites free of charge. Although it helped some sites, it was not an option for some others as the CCG would not allow prescribing of the device. We wrote to the CCGs to explain that the trial was needed to provide the evidence regarding treatment effectiveness that was lacking, but this and letters to the Director of Research and Development at the Department of Health failed to resolve the issue. These sites had to be withdrawn from the CapaCiTY02 trial with detrimental consequences, as sites with large patient populations and potentially high recruitment potential were lost. This counted among the unforeseen hurdles at the time of study design.

The randomisation and delivery of treatment was often contrary to routine practice and in some cases required additional visits or longer waiting times for receiving treatment. For example, in study 2, patients would be randomised to either high or low volume irrigation at the screening visit, and then needed to come back for another visit to receive the treatment. This was because they were asked to complete a two-week baseline diary prior to the start of treatment. If the patient decided not to join the study, they could simply receive their treatment in one visit. Also, in study 2, nurses believed, from their own experience, that the low irrigation therapy was not likely to work for many patients with chronic constipation. Therefore, they found it difficult to encourage certain patients to join the study when there was a 50% chance that they would be randomised to the low volume irrigation therapy group.

In study 3, one of the randomisation arms allocated patients to receive surgery in 4 weeks. This was not practical for a lot of NHS sites, as they would not be able to secure a theatre date for the patient in 4 weeks and required them to be operated on outside of their usual waiting list schedule. Some sites specifically declined joining the study due to this challenge. However, on the other hand, some sites found it positive that patients had a chance of having surgery much earlier than if they were on the routine care waiting list. Whether or not the protocol could be implemented at sites depended on the relationship and persuasiveness of the principal investigator with their local NHS bookings manager and research and development staff.

Long follow-up visits and paperwork

Initially, the follow-up period in all three studies was 24 months, which included questionnaires and diaries to be completed by patients. This was a burden on both patients and staff. This was amended to 12 months, and by making changes to our paperwork we reduced the number of questionnaires that needed to be completed by more than 50%. We also introduced an electronic data capture system using REDCap, which allowed participants to complete their questionnaires online without the need to attend a face-to-face visit. Unfortunately, this did not significantly improve recruitment or retention rates.

Other challenges

Some recruitment challenges were unforeseeable and affected the nation, with no clear solution. For example, in 2016 the NHS started to face legal ramifications due to serious complications from pelvic mesh surgery (19, 20, 21). Adverse press reporting negatively impacted on CapaCiTY03 study recruitment. Further, by 2019, some of our sites had been instructed by their lawyers to put a temporary halt on all their clinical and research mesh procedures and this contributed greatly to lack of recruitment in study 3.

We employed a range of strategies to improve recruitment across the three studies, including (13)

Provided almost all sites with refresher site initiation visits and guidance in reviewing referral letters to identify suitable patients.
Provided all sites with a refresher investigator meeting in December 2017 to get all research staff together from all our sites with the aim to provide them with refresher training on protocol, exchange of experience on overcoming recruitment barriers, encouraging everyone to recruit.
Changed the design of the studies in Jan 2016 to shorten the length of the follow-up visits from 24 months to 12 months to lessen the burden of visits on both patients and research staff and dropped two lengthy questionnaires from secondary outcome measures.
Protocol amendments were made to make the protocol more compatible with routine practice where possible.
Provided sites with support in data entry and administrative work, which would free up the research nurse to do the screening and recruitment.
Launch of an advertising campaign in summer 2016 to find suitable patients through newspapers.
Provided high resolution anal manometry (HRAM) devices to sites on loan to help meet the requirements of study device (Study 1). Undertook competitive procurement and assisted sites with business cases to secure funding for HRAM equipment.
Secured funding from McGregor (irrigation device company) to sponsor those sites affected by the lack of prescription funding (Study 2).
Engaged with NIHR to try and encourage CCGs to meet their obligations to fund excess treatment costs (ETCs) - we had letters from the DOH in this regard but both they and the CRN were unable to mandate ETCs (Study 2).
Recruitment extensions were requested for all three trials to boost the recruitment.
Regular presentation at national speciality meetings e.g., Pelvic Floor Society.
Staff incentives; prizes, newsletter, twitter, professional development.

The variety of strategies employed to overcome poor recruitment performance failed. The final recruitment achieved was 275/808, only 34% of the intended sample size for the CapaCiTY programme. After a 12-month extension from the funder, the three studies were eventually halted by the data monitoring and ethics committee (DMEC) due to futility.

On review of the expected recruitment funnel as outlined in the original grant (Fig. 4) compared with the final consort diagram for each trial (Fig. 1, Fig. 2, Fig. 3), CapaCiTY01 (Fig. 1) had double the rate of screen failures and double the rate of participants declining to take part. This significantly impacted the recruitment potential of sites. It is also evident from the survey results (Table 2) that there was less focus on primary care referrals and low engagement with clinical research networks to facilitate recruitment from primary care, further limiting recruitment. It was not possible to add additional sites to help bolster recruitment as the study budget could only cover ten high resolution anal manometry (HRAM) devices used to deliver the biofeedback. Thus, we had to rely on extending the recruitment period.

CapaCiTY02 and CapaCiTY03 predictions of screen failure and participants declining were accurate. What is evident is that the predicted available population in both primary and secondary care recruitment pathways was substantially overestimated, resulting in a lack of access to suitable participants for screening for all trials, as concluded by some sites in the SWAT survey, and identified in separate qualitative interviews performed as part of the programme of work (13). Both study 2 and 3 were affected by complications related to delivering the intervention. Only six sites actively recruited, rather than the intended ten sites, and it was not possible to recruit additional sites to help with recruitment.

For all three trials, advertising and multi-media approaches were unsuccessful as patients needed referrals from their general practitioner (GP) to secondary care clinics to participate in the trials. The survey revealed that most sites (80%) did not implement electronic records searches to identify potential participants.

The barriers to recruitment faced by the CapaCiTY programme may have been avoided if a more thorough assessment of the existing patient population across the UK had been conducted to confirm the feasibility of the proposed programme of work. This, coupled with a smarter, data-driven approach to site recruitment potential would have revealed the true number of sites and resources needed to recruit the required sample size. A more pragmatic approach to study design, eligibility, and outcome assessments would have further enhanced recruitment and required a better understanding of routine practices.

The CapaCiTY programme experience highlights the importance of understanding trial feasibility from the outset, and preferably before funding is awarded, so that modifications to trial design, timelines and budgets can be made before it is too late.

Of critical importance is the reliability of site feasibility assessment data and the accurate prediction of a site’s recruitment potential. It is clear from the CapaCiTY programme experience that even considering the trial design and eligibility criteria, local site staff and the coordinating sites were not able to accurately predict recruitment potentials, resource constraints or ability to undertake the trial protocol at the time of site feasibility assessments. However, recruitment performance partway through the study (at the time of the SWAT survey) was a reliable indicator of final recruitment performance. Indeed, the results of the SWAT survey confirm that based on their experience with the CapaCiTY programme, most sites (60%) would conduct site feasibility differently in the future. This problem permeates the clinical trial industry with only half of all trials achieving their recruitment targets (4). So, whilst site feasibility assessments are a promising idea in theory, in practice, they are not helpful in their current form, are time consuming and expensive to conduct. However, if the reliability of site feasibility assessments is improved, this will avoid substantial wasted financial resource on trials that fail to recruit and thus fail to reach a conclusion on the primary outcome or provide any reliable results.

Limitations

This study is limited to the results of the CapaCiTY programme of research and the context is specific to investigator-led clinical trials conducted within the NHS in the UK. However, it is clear from the literature that this problem exists for clinical trials (investigator led and industry led) in the UK and abroad. The results and recommendations may therefore be generalisable to all clinical trials. Furthermore, CapaCiTY02 did not assess site recruitment potential at the time of site feasibility assessment, limiting our ability to predict site performance. The SWAT survey represents local rather than expert opinion. In addition, the reason participants declined to take part, even when eligible, remains an unavoidable blind spot.

It is critically important that the reliability of site feasibility assessments is improved, for the benefit of funders, sponsors, researchers, and participants. Based on our experience with the CapaCiTY programme, we recommend:

conducting initial site feasibility assessments at the pre-award stage and provision of resource to do this.

investment in training researchers (principal investigators, trial managers and clinical research nurses) to conduct and complete site feasibility assessments. When conducting trials in the public hospital setting, spend more time at the pre-award stage to understand the feasibility of the clinical trial and compatibility of the study design with routine care. Discuss this carefully with potential site principal investigators, clinical care teams, and research nurses.

funders to include clinical trial feasibility specialists on peer review and grants panels.

development of a national database of sites previous trial recruitment performance

data-driven approaches to recruitment informed by electronic records enquiries that accurately predict number of participants available, expected referrals, participation rates and screen failure rates for different cohorts, removing guess work and uncertainty.

These statistics will provide transparency and help sponsors to assess true site feasibility. By taking a data-driven, analytical approach to site feasibility the clinical trial industry could potentially save hundreds of millions of pounds each year in failed investments and speed up the time to delivery of new treatments for patients. More work is urgently needed in this area.

Ethics approval and consent to participate.

This study within a trial (SWAT) was approved by the Queen Mary University of London Ethics Committee, QMREC2023a. The overarching clinical trials were approved by London City and East Human Research Ethics Committee; REC 14/LO/1786 ISRCTN11791740, CapaCiTY022: REC 15/LO/0732, ISRCTN11093872, CapaCiTY033: REC 15/LO/0732, ISRCTN11747152. Informed consent was provided by all participants in accordance with the ethics approvals. All methods were carried out according to the clinical trial regulations, local governance requirements and quality assurance procedures including; The Medicines for Human Use (Clinical Trials) Regulations 2004, Good Clinical Practice, Data Protection Act, NHS Caldicott Principles, The UK Research Governance Framework for Health and Social Care, Pragmatic Clinical Trials Unit Standard Operating Procedures

Consent for publication

Not Applicable

Availability of data materials

The data generated during the study are included in this published article [and its supplementary information files].

Competing interests

None

Funding

Funding for this study was provided by the UK National Institute of Health Research (NIHR) Programme for Applied Research (PfAR).

Author Contributions

Natasha Stevens: Lead and corresponding author. Lead the design and conduct of feasibility questionnaires and SWAT recruitment survey, applied for ethics approval for the SWAT and drafted the manuscript. Program manager for the CapaCiTY program of research. No conflicts of interests.

Shiva Taheri: Assisted in the design and conduct of feasibility questionnaires and SWAT recruitment survey for CapaCiTY02, ethics application for the SWAT and draft manuscript. Study coordinator for the CapaCiTY02 trial. No conflicts of interests.

Dr Ugo Grossi: Assisted in the design and conduct of feasibility questionnaires and SWAT recruitment survey for CapaCiTY03, review of manuscript. Research fellow for the CapaCiTY programme. No conflicts of interests.

Dr Chris Emmett: Lead the design and conduct of feasibility questionnaires for CapaCiTY02, review of manuscript. Research fellow for the CapaCiTY02 trial. No conflicts of interests.

Sybil Bannister: Assisted in the design and conduct of feasibility questionnaires and SWAT recruitment survey for CapaCiTY01, review draft manuscript. Research nurse for the CapaCiTY programme. No conflicts of interests.

Prof. Christine Norton: Chief Investigator for the CapaCiTY01 trial, provided academic oversight and review of the manuscript. No conflicts of interests.

Prof Yan Yiannakou: Chief Investigator for the CapaCiTY02 trial, provided academic oversight and review of the manuscript. No conflicts of interests.

Prof Charles Knowles: Chief Investigator for the CapaCiTY programme of research, provided academic oversight, review of the manuscript. No conflicts of interests.

Acknowledgements

The authors acknowledge and thank all site staff participating in the study feasibility assessments and phone surveys. We thank all study participants, the Pragmatic Clinical Trials Unit (PCTU), governance committee members and constipation research advisory group (CRAG) who supported the CapaCiTY programme.

Stevens N, Edwards L, Balayah Z, Hooper R, Knowles C. Risk based survey evidence supports electronic informed consent as a recruitment method for UK clinical trials. J Clin Epidemiol. 2016;77:134-6.
Fletcher B, Gheorghe A, Moore D, Wilson S, Damery S. Improving the recruitment activity of clinicians in randomised controlled trials: a systematic review. BMJ Open. 2012;2(1):e000496.
https://nihr.opendatasoft.com/pages/nihr-awards-filters/-value-of-awards#-value-of-awards. Accessed on Aug 14 2022 [
Walters SJ, Bonacho Dos Anjos Henriques-Cadby I, Bortolami O, Flight L, Hind D, Jacques RM, et al. Recruitment and retention of participants in randomised controlled trials: a review of trials funded and published by the United Kingdom Health Technology Assessment Programme. BMJ Open. 2017;7(3):e015276.
McDonald AM, Knight RC, Campbell MK, Entwistle VA, Grant AM, Cook JA, et al. What influences recruitment to randomised controlled trials? A review of trials funded by two UK funding agencies. Trials. 2006;7:9.
Johnson O. An evidence-based approach to conducting clinical trial feasibility assessments. Clinical investigation. 2015.
Rajadhyaksha V. Conducting feasibilities in clinical trials: an investment to ensure a good study. Perspect Clin Res. 2010;1(3):106-9.
Fogel DB. Factors associated with clinical trials that fail and opportunities for improving the likelihood of success: A review. Contemp Clin Trials Commun. 2018;11:156-64.
Tudur Smith C, Hickey H, Clarke M, Blazeby J, Williamson P. The trials methodological research agenda: results from a priority setting exercise. Trials. 2014;15:32.
Treweek S, Pitkethly M, Cook J, Fraser C, Mitchell E, Sullivan F, et al. Strategies to improve recruitment to randomised trials. Cochrane Database Syst Rev. 2018;2:MR000013.
https://www.ukcrc.org/research-infrastructure/clinical-trials-units/registered-clinical-trials-units/. Accessed on Aug 14 2022 [
https://www.nhsconfed.org/articles/what-are-clinical-commissioning-groups. Accessed on Aug 14 2022.
Knowles CH, Booth L, Brown SR, Cross S, Eldridge S, Emmett C, et al. Non-drug therapies for the management of chronic constipation in adults: the CapaCiTY research programme including three RCTs. Programme Grants for Applied Research. Southampton (UK)2021.
Norton C, Emmanuel A, Stevens N, Scott SM, Grossi U, Bannister S, et al. Habit training versus habit training with direct visual biofeedback in adults with chronic constipation: study protocol for a randomised controlled trial. Trials. 2017;18(1):139.
Emmett C, Close H, Mason J, Taheri S, Stevens N, Eldridge S, et al. Low-volume versus high-volume initiated trans-anal irrigation therapy in adults with chronic constipation: study protocol for a randomised controlled trial. Trials. 2017;18(1):151.
Grossi U, Stevens N, McAlees E, Lacy-Colson J, Brown S, Dixon A, et al. Stepped-wedge randomised trial of laparoscopic ventral mesh rectopexy in adults with chronic constipation: study protocol for a randomized controlled trial. Trials. 2018;19(1):90.
Grossi U, Lacy-Colson J, Brown SR, Cross S, Eldridge S, Jordan M, et al. Stepped-wedge randomized controlled trial of laparoscopic ventral mesh rectopexy in adults with chronic constipation. Tech Coloproctol. 2022.
https://www.york.ac.uk/healthsciences/research/trials/swats/ accessed 13 Jan 2023
19. Haskell H. Cumberlege review exposes stubborn and dangerous flaws in healthcare BMJ 2020; 370 :m3099 doi:10.1136/bmj.m3099
Cumberlege J. First do no harm: the report of the Independent Medicines and Medical Devices Safety Review. 2020 Jul 8. https://www.immdsreview.org.uk/Report.html.
European Commission Scientific Committee on Emerging and Newly Identified Health Risks. Opinion on the safety of surgical meshes used in urogynecological surgery. 3 Dec 2015. https://ec.europa.eu/health/scientific_committees/consultations/public_consultations/scenihr_consultation_27_en.
Bogin V, Lasagna Law: A dish best served early, Contemporary Clinical Trial Communications 26 (2022) 100900, https://doi.org/10.1016/j.conctc.2022.100900

Supplementary files 4-6 are not available with this version.

No competing interests reported.

Download PDF

Editorial decision: Revision requested
10 Jan, 2024
Reviews received at journal
18 Dec, 2023
Reviews received at journal
14 Dec, 2023
Reviews received at journal
06 Dec, 2023
Reviewers agreed at journal
05 Dec, 2023
Reviewers agreed at journal
29 Nov, 2023
Reviewers invited by journal
28 Nov, 2023
Editor invited by journal
12 Oct, 2023
Editor assigned by journal
11 Oct, 2023
Submission checks completed at journal
11 Oct, 2023
First submitted to journal
02 Oct, 2023

You are reading this latest preprint version

A study within a trial (SWAT) of clinical trial feasibility and barriers to recruitment in the United Kingdom – the CapaCiTY programme experience

Status:

Version 1

Abstract

Figures

BACKGROUND

METHODS

RESULTS

Recruitment rates

Barriers to recruitment

Lack of staff and resources

Study design and protocol compatibility

Long follow-up visits and paperwork

Other challenges

DISCUSSION

Limitations

CONCLUSION

Declarations

References

Supplementary Files

Additional Declarations

Supplementary Files

Status:

Version 1