New Benchmarks on Protocol Amendment Experience in Oncology Clinical Trials

doi:10.21203/rs.3.rs-3547749/v1

Download PDF

Research Article

New Benchmarks on Protocol Amendment Experience in Oncology Clinical Trials

https://doi.org/10.21203/rs.3.rs-3547749/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 26 Mar, 2024

Read the published version in Therapeutic Innovation & Regulatory Science →

You are reading this latest preprint version

Background

The drug development industry’s focus on cancer-related treatments continues to rise; however, narrow patient populations and complex procedures have increased the complexity of oncology protocols at an accelerated rate compared to non-oncology drugs. Tufts Center for the Study of Drug Development utilized data from a study investigating the impact of protocol amendments to compare how oncology clinical trials differ from non-oncology and identify opportunities to optimize performance in oncology clinical trials.

Methods

Sixteen drug development industry companies contributed data from 950 protocols and 2,188 amendments to a study conducted in 2022 investigating protocol amendments. Analysis compared differences in amendment impact and causes between 249 oncology and 701 non-oncology protocols.

Results

Compared to non-oncology, oncology protocols had a significantly higher prevalence (72.1% and 91.1%, respectively) and number (3.0 and 4.0, respectively) of protocol amendments. Oncology protocols with amendments had significantly lower participant completion rates compared to oncology protocols without amendments, while no significant differences were found among non-oncology. During the COVID-19 pandemic, the study found an increased number of substantial amendments, lower completion rates, and higher dropout rates among oncology protocols compared to before the pandemic.

Conclusions

Efforts to prevent avoidable protocol amendments in the industry have not been effective in oncology, where increasingly complex designs are reflected in difficult to predict cycle times, barriers to recruitment and retention and an increase in protocol amendments.

Protocol amendments

oncology

protocol design

protocol complexity

protocol design changes

Protocol amendments have long been recognized as a necessary evil in the conduct of clinical trials, plaguing investigative sites and industry with unplanned delays and unbudgeted costs[1]. While some amendments are unavoidable and implementation can result in an improvement in clinical trial performance and safety, many amendments can be avoided, particularly those resulting from human error[1].

The therapeutic area of oncology is notable for clinical trials of larger scope, duration, and complexity, which have been shown to have a higher relative prevalence and frequency of protocol amendments[2]. A previous study conducted by the Tufts Center for the Study of Drug Development (Tufts CSDD) found that 50–70% more substantial amendments were implemented in phase II and III oncology trials compared to non-oncology[2]. Protocol complexity in oncology may also be associated with lower success rates, with a success rate of only 3.5% for all oncology drugs in 2022, compared to 20.9% for non-oncology drugs[3, 4].

In 2020, 23% of all drugs in development targeted cancer-related diseases. And between 2017 and 2022, the number of new oncology clinical trials initiated globally grew by 29%[2, 4]. Increasing collection of genetic information to inform targeting more narrowly defined patient populations has also contributed to protocol complexity and patient participation burden in oncology clinical trials[5]. Difficulty recruiting patients is often remedied by expanding the eligibility criteria or altering recruitment methods, both of which require amending the study protocol[1]. Additionally, oncology trials, on average, take 14–18 months longer compared to non-oncology trials, with the longer duration increasing the likelihood of the protocol requiring an amendment[2].

Due to the level of clinical research activity in oncology combined with higher relative protocol design and execution demands – and lower success rates – it is important to capture benchmark data specific to oncology and to compare these benchmarks with those of non-oncology protocols. These benchmarks will inform future protocol design and execution planning.

With respect to benchmark research on the prevalence and impact of protocol amendments, Tufts CSDD conducted two studies—one in 2010 and one in 2015. To update these benchmarks, in 2022, Tufts CSDD conducted a subsequent study. This article presents these updated benchmarks and characterizes trends in oncology protocol amendments and their impact on clinical trial cycle time, recruitment and retention outcomes. The study results also help identify opportunities to optimize oncology protocol performance.

In spring 2022, Tufts CSDD formed a working group of 22 pharmaceutical, biotechnology, and contract research organizations (CROs) to plan and implement data collection activity. Sixteen companies shared protocol design and amendment data. Participating companies were asked to provide data on a randomly selected set of 100 protocols each that had reached database lock (DBL) between 2016 and 2021 and report on the number, characteristics and causes of substantial amendments implemented for the selected protocols. In this study, a substantial protocol amendment was defined as any global amendment that required ethics review. The number of country-specific amendments and major protocol deviations were collected for each protocol, however, characteristics were only analyzed for substantial amendments.

The Tufts CSDD research team developed two standardized data collection workbooks based on previous studies and company input —one for protocol data and one for amendment data. Companies collected data independently using the standardized workbooks, which they then submitted to Tufts CSDD in fall 2022.

The data gathered in the protocol workbook included protocol characteristics and common clinical trial timelines; while the data gathered in the amendment workbook identified amendment characteristics, cause(s) of an amendment, changes made due to an amendment, and amendment cost. The primary cause, or primary driver, of an amendment, determined its classification of “Completely Avoidable”, “Somewhat Avoidable”, “Somewhat Unavoidable”, or “Completely Unavoidable”. Please see Getz 2023 for additional information on data collected and amendment classification[6].

Therapeutic area was recorded by companies using a predetermined list of the most common therapeutic areas provided by Tufts CSDD, including oncology. Any therapeutic area that did not fall within the provided list was categorized under “Other”, and a text entry was used to specify.

Therapeutic area data was collected for all but two protocols; for the two protocols where therapeutic area was not provided by the company, the research team determined the correct therapeutic area using clinicaltrials.gov based on the provided NCT number, protocol identification number, and study title. Therefore, the research team was able to conduct oncology-specific analysis on all protocols in the dataset.

The subgroups pre-pandemic and during the pandemic are based on whether last patient last visit (LPLV) occurred before or after March 1, 2020. This allowed the research team to measure amendments implemented for any protocol that was actively treating patients during the period when restrictions were imposed to prevent the spread of the COVID-19 pandemic.

Excel workbooks submitted by companies were merged and data were analyzed using SAS 9.4.

Participating companies provided data from a total of 950 protocols and 2,188 amendments. Twenty-six percent of all protocols in the dataset were oncology protocols. Oncology clinical trials had a similar distribution of phases and orphan designation compared to non-oncology. However, relative to non-oncology, oncology protocols in this dataset were more likely to have implemented innovative designs, including master protocol designs (defined as a comprehensive protocol created for evaluating multiple hypotheses of sub-studies that are concurrently conducted) or adaptive designs (defined as a clinical trial design that allows for prospectively planned modifications to one or more aspects of the design based on accumulating study data). On average, oncology protocols carried a larger number of endpoints and countries than non-oncology protocols, while non-oncology protocols had a larger average number of investigative sites. The average number of eligibility criteria was comparable across therapeutic areas (Table 1).

[Insert Table 1]

Oncology protocols had a significantly higher prevalence of amendments than non-oncology protocols, with 91.1% of oncology protocols requiring amendments compared to 72.1% of non-oncology protocols (chi-square test, p < .0001). The mean number of amendments per protocol where substantial amendments were present was significantly higher among oncology protocols, with an average of 4.0 amendments among oncology, and 3.0 amendments among non-oncology protocols (ANOVA, p < .0001) (Table 2).

[Insert Table 2]

The most common primary causes of amendments across therapeutic areas were regulatory agency request, change in study strategy, and new data available other than safety data. The most common primary causes of amendments to oncology protocols were similar to those of non-oncology amendments, with only minor differences. For example, oncology amendments were slightly less likely to be primarily caused by an inconsistency or error in the protocol (6.6% for oncology, 9.7% for non-oncology), recruitment difficulty (2.3% for oncology, 4.5% for non-oncology), and the COVID-19 pandemic (1.2% for oncology and 3.8% for non-oncology), and more likely to be primarily caused by new safety data becoming available (10.5% for oncology, 7.3% for non-oncology) and “other” (19.6% for oncology, 10.7% for non-oncology) (Table 3).

[Insert Table 3]

Oncology protocols had a slightly higher proportion of amendments whose primary cause fell under the “Somewhat Unavoidable” and “Completely Unavoidable” classifications (82% for oncology, 74.9% for non-oncology), and lower proportion of amendments falling under the “Somewhat Avoidable” and “Completely Avoidable” classifications, compared to non-oncology protocols (Table 4).

[Insert Table 4]

Any changes applied to the protocol in the process of implementing an amendment were recorded. Oncology amendments implemented more protocol changes overall, including changes related to clinical trial operations; statistical analyses; assessments; patient selection, withdrawal, and treatment; trial design; and trial background, objectives, and purpose. Non-oncology amendments were more likely to carry changes related to general information in the protocol, which included typographical errors (Table 5).

[Insert Table 5]

All amendments, regardless of phase or therapeutic area, were most likely to occur during enrollment (between first patient first visit (FPFV) and last patient first visit (LPFV)). However, oncology amendments were more likely to occur after LPFV (24.2% for oncology, 14.7% for non-oncology) and less likely to occur before FPFV than non-oncology amendments (19.7% for oncology, 26.6% for non-oncology), particularly among phases II and III (Table 6).

[Insert Table 6]

The percent difference between planned and actual participants completing the clinical trial increased by 2.3% for oncology protocols without amendments and decreased by 63.2% for oncology protocols with amendments (ANOVA, p < .001). This trend was consistent with the percent change in completion rate (total enrolled / total completed) with oncology protocols without amendments seeing an increase of 7.9%, while the completion rate of oncology protocols with amendments decreased by 63.0% between planned and actual (ANOVA, p < .001). There were no significant differences between protocols with and without amendments for non-oncology protocol enrollment performance. However, there was a slight decrease between planned and actual participants enrolled, participants completed and completion rates across all non-oncology protocols, regardless of the presence of amendments (Table 7).

[Insert Table 7]

Dropout rate was significantly higher for protocols with amendments overall among both oncology and non-oncology protocols. Significant differences were found among oncology protocols for completion rates (ANOVA, p < .05) and screen failure rates (ANOVA, p < .001), with a completion and screen failure rate of 35.3% and 30.7% for protocols with amendments and 56.6% and 15.3% for those without, respectively. No significant difference was found among non-oncology protocols. Oncology protocols were also found to have a lower completion rate overall compared to non-oncology regardless of amendment status (Table 8).

[Insert Table 8]

The duration of study closeout (LPLV to DBL) among oncology protocols decreased by 100.8% for protocols without amendments and increased by 45.7% for protocols with amendments (p< .01). Decreases of over 100% were possible due to several studies having extended follow-up periods that continued well after DBL, resulting in LPLV occurring after DBL. No other significant differences between changes in cycle time duration for protocols with and without amendments were found. However, on average, protocols with amendments tended to increase in duration between planned and actual cycle times at a higher rate than protocols without amendments across all therapeutic areas and cycle times, save for LP Participation Cycle (LPFV to LPLV) and Study Reporting (DBL to Clinical Study Report (CSR)) (Table 9).

[Insert Table 9]

Oncology protocols conducted during the pandemic had a significantly higher number of substantial amendments than protocols conducted pre-pandemic, 4.1 compared to 3.0, respectively (ANOVA, p <.01). Completion rates decreased and dropout rates increased for oncology protocols before compared to during the pandemic (ANOVA, p <.0001). There were no significant differences in number of amendments, completion rate, or dropout rate, when comparing before to during the pandemic for non-oncology protocols. The screen failure rate among non-oncology protocols decreased significantly during the pandemic compared to pre-pandemic (p <.05); no significant differences in screen failure rate were found among oncology protocols (Table 10).

[Insert Table 10]

The results of this study show that more than 90% of oncology protocols implement at least one amendment with an average of four amendments implemented per protocol. The results also indicate that oncology protocols have a significantly higher likelihood of implementing an amendment and a significantly higher number of amendments than non-oncology protocols. The results also show that protocol complexity and recruitment and retention challenges unique to oncology clinical trials are major contributors to higher amendment frequency and prevalence.

The higher proportion of protocol amendments in oncology protocols is linked to stringent exclusion and inclusion criteria and more narrowly defined patient populations, leading to recruitment difficulties[2, 7]. However, in this study, recruitment difficulties were less likely to be cited by sponsor companies as the primary cause of amending an oncology protocol compared to non-oncology protocols. One potential explanation for this inconsistency is methodological: there are often multiple causes of an amendment. Recruitment difficulties may have been addressed when other causes or changes were “bundled” together and implemented when an unavoidable cause (i.e., regulatory request) was addressed.

Amendments implemented to address recruitment and retention difficulties may have also been classified under “Change in Study Strategy” particularly where the change is related to modifying eligibility criteria, modifying the size of the study arms or changing the size of the target patient population. Additional research is needed to collect more detailed information and insights about the causes of amendments implemented in oncology protocols and why the differences in amendment prevalence and frequency between therapeutic areas exist.

Another common primary cause of oncology amendments was cited as “Other”. While the research team requested that participating companies write in any cause that did not fall under those listed, it is difficult to categorize write-in causes without more context. There is little that can be drawn from the difference in frequency of amendments due to “Other” causes between oncology and non-oncology, however, this may simply indicate that oncology amendments are more complex and may not easily fall under categories created in support of the 2022 study methodology.

These results also show that study conduct (Protocol Approval – LPLV), enrollment completion (FPFV – LPLV), and first visit durations (FPFV – LPFV) were all significantly longer among protocols with amendments compared to those without[6]. The smaller proportion of oncology protocols without amendments still experienced higher cycle times than expected, particularly for last patient participation cycle (LPFV to LPLV) and study reporting (DBL to CSR), with cycle time durations 84% and 58% longer than planned, respectively. This finding suggests that planned cycle times may be unrealistic, or that oncology protocol timelines are more difficult to predict regardless of amendment status. Clinical trials with amendments tended to have larger differences between planned and actual study durations. However, the impact of amendments on planned compared to actual cycle times was not significant for both non-oncology and oncology trials with the exception of study closeout duration. The higher relative volume and diversity of clinical data collected in oncology trials helps explain this finding.

Oncology protocols experienced higher dropout rates and lower completion rates compared to non-oncology. Both oncology and non-oncology protocols had significantly higher dropout among protocols with amendments overall compared to those without amendments, with actual completion rates differing significantly between amendment status for oncology protocols. When amendments were present, a significant difference was observed between planned and actual study completion rates for oncology protocols.

Compared to non-oncology, oncology protocols experienced significantly higher drop-out rates that were unrelated to patient choice (e.g., mortality and adverse events). This finding is not surprising, as oncology clinical trials typically have higher mortality rates, higher likelihood of disease progression, and higher incidence of adverse reactions to the investigational drug[8–11]. The results of this study also suggest, however, that elevated drop-out rates are associated with higher relative patient participation burden and longer study durations (which may include long-term follow-up).

This explains much of the variation between dropout and completion rates among non-oncology compared to oncology protocols. Implementation of amendments late in the clinical trial is also likely contributing to poor study volunteer retention rates. The higher percentage of amendments implemented after LPFV among oncology protocols may be due to efforts to address study participant fatigue and the very low relative success rate of investigational oncology treatments.

This finding is even more important when considering the significant difference in change in completion rate across amendment status indicating difficulty in predicting this subset of participants in oncology study design. Study design typically anticipates a certain number of dropouts and incorporates analyses into study sample size[12]. However, when this calculation is inaccurate and the sample size available for analysis is smaller than the planned sample, the study may be impacted by selection bias, underpowered analysis, or even termination of the clinical trial[12].

One notable finding from this study was the impact of the COVID-19 pandemic on oncology trials compared to non-oncology trials. Oncology protocols faced a significantly increased number of substantial amendments, lower completion rates, and higher dropout rates during the pandemic compared to before the pandemic, while no significant differences were seen in non-oncology protocols. Differences in enrollment performance and frequency of amendments may be partially due to survival bias, as all protocols in this study had reached database lock by 2021, at which time a higher proportion of non-oncology trials had been halted, terminated or withdrawn, compared to oncology[13].

In addition, oncology clinical trials are less adaptable to fully remote models due to the need for specialized equipment and procedures requiring professional supervision (e.g., infusions), leading to a higher likelihood and frequency of in-person site visits. This may have been difficult and potentially unsafe for immunocompromised cancer patients, worsening dropout rates that were already higher than non-oncology trials[14]. Finally, whereas most oncology trials could not shift to entirely remote approaches, some decentralized trial (DCT) elements were implemented to minimize risk of contracting the coronavirus. Since DCT elements were rare among oncology trials prior to the pandemic, implementing these strategies likely required a higher frequency of amendments, while these elements may have already been in place among non-oncology trials[15].

The study had several limitations. Protocols assessed in this research were restricted to those that had reached database lock by 2021. This limited the sample size of protocols that occurred during the pandemic. This limitation is particularly important to consider when oncology trials can last over four years on average, and trials started 2018 or later were likely still ongoing at the time of this study[16]. Subsequent research will be essential to further evaluate the COVID-19 pandemic’s effects on oncology clinical trials. Please see Getz 2023 for further notes on study limitations[6].

Clinical research in oncology remains a highly active and growing area. As the number of oncology clinical trials increases, so too have protocol complexity and regulatory requirements intensified[2, 4]. The goal of implementing protocol amendments is to increase the safety and efficacy of protocol design. However, the associated decrease in completion rates and increase in dropout rates when an amendment is present suggests that amendments have unintended consequences that need to be mitigated.

Based on the results of its 2010 study, Tufts CSDD recommended that research sponsors focus on reducing and preventing avoidable amendments. In response, a number of companies instituted processes and mechanisms to improve design planning and amendment experience. The results of this study indicate that these efforts have not been able to match the impact of evolving protocol designs in oncology[1]. One alternative method to prevent the need to implement amendments is to incorporate patient engagement activity, which has been found to decrease the number of substantial amendments on average and enhance study performance[17, 18].

Another strategy to avoid unplanned costs and delays is to implement novel designs (e.g., adaptive and pragmatic). Adaptive designs allow for flexibility and have pre-determined protocol changes making it less likely that amendments will be postponed due to fear of failure[19]. The FDA published a final guidance on adaptive design protocols in 2019, acknowledging the advantages of the design[20]. However, adaptive designs remain uncommon in oncology, representing only 9% of oncology protocols in this dataset. This may be influenced by the lack of awareness, complexity in statistical models and difficulty with regulatory committees due to low familiarity with adaptive designs[19].

In addition to gathering more granular data on the causes of the need to implement protocol amendments in oncology, future research will also look at determining their economic impact. This latter area has been difficult to evaluate as most sponsor companies are ill-prepared to collect this information. Tufts CSDD also plans to conduct future research on protocol deviation experience in oncology clinical trials.

To continue to provide timely and life-saving research to patients, sponsors and regulatory bodies need to understand the challenges faced specifically in oncology clinical research. Protocol amendments increase the duration and cost of clinical trials[1] and have a higher prevalence and frequency in oncology compared to other therapeutic areas, representing a major challenge for oncology as a significant and growing area of global clinical research activity.

Acknowledgements

Thank you to the many companies that participated and supported this working group study: Amgen, AstraZeneca, Biogen, Bristol Myers Squibb, EMD Serono, Eli Lilly & Company, Gilead Sciences, IQVIA, Janssen Pharmaceutical Companies of Johnson & Johnson, Jazz Pharmaceuticals, Merck, Parexel, Pfizer, Regeneron, Roche/Genentech, Sanofi, Seagen, and Veristat.

Author Contributions

EB: Substantial contributions to the conception and design of the work and the acquisition, analysis, or interpretation of data for the work. Drafting the work or revising it critically for important intellectual content. Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. ZS: Substantial contributions to the conception and design of the work and the acquisition, analysis, or interpretation of data for the work. Drafting the work or revising it critically for important intellectual content. Final approval of the version to be published. KG: Substantial contributions to the conception and design of the work. Drafting the work or revising it critically for important intellectual content. Final approval of the version to be published.

Conflict of Interest Statement

Emily Botto, Tufts CSDD, has nothing to disclose.

Zachary Smith, Tufts CSDD, has nothing to disclose.

Kenneth Getz, Tufts CSDD, has nothing to disclose.

Getz KA, Stergiopoulos S, Short M, et al. The Impact of Protocol Amendments on Clinical Trial Performance and Cost. Ther Innov Regul Sci. 2016;50:436–41.
Protocol complexity and patient enrollment intensify challenges. in oncology trials. 23;3, Tufts Center for the Study of Drug Development.
Wong CH, Siah KW, Lo AW. Estimation of clinical trial success rates and related parameters. Biostat Oxf Engl. 2019;20:273–86.
Global Oncology Trends. 2023, https://www.iqvia.com/insights/the-iqvia-institute/reports/global-oncology-trends-2023 (accessed 1 June 2023).
Garcia S, Bisen A, Yan J, et al. Thoracic Oncology Clinical Trial Eligibility Criteria and Requirements Continue to Increase in Number and Complexity. J Thorac Oncol. 2017;12:1489–95.
Getz K, Smith Z, Botto E et al. New Benchmarks on Protocol Amendment Practices, Trends and their Impact on Clinical Trial Performance. 2023. Epub ahead of print 14 July 2023. 10.21203/rs.3.rs-3168679/v1.
Topaloglu U, Palchuk MB. Using a Federated Network of Real-World Data to Optimize Clinical Trials Operations. JCO Clin Cancer Inform 2018; 1–10.
Lundy JJ, Coon CD, Fu A-C, et al. Collection of Post-treatment PRO Data in Oncology Clinical Trials. Ther Innov Regul Sci. 2021;55:111–7.
Kim J, Kim MG, Lim K-M. Participation in and withdrawal from cancer clinical trials: A survey of clinical research coordinators. Asia-Pac J Oncol Nurs. 2022;9:197–201.
Examination of Clinical Trial Costs and Barriers for Drug Development. ASPE, https://aspe.hhs.gov/reports/examination-clinical-trial-costs-barriers-drug-development-0 (accessed 22 August 2023).
Smith Z, Botto E, Carney C, et al. Insights from a Multi-company Workshop to Apply a Patient Participation Burden Algorithm to Protocol Data. Ther Innov Regul Sci. 2023;57:262–70.
Hui D, Glitza I, Chisholm G, et al. Attrition rates, reasons, and predictive factors in supportive care and palliative oncology clinical trials. Cancer. 2013;119:1098–105.
Upadhaya S, Yu JX, Hodge J, et al. COVID-19 impact on oncology clinical trials: a 1-year analysis. Nat Rev Drug Discov. 2021;20:415–5.
Castelo-Branco L, Awada A, Pentheroudakis G et al. Beyond the lessons learned from the COVID-19 pandemic: opportunities to optimize clinical trial implementation in oncology. ESMO Open; 6. Epub ahead of print 1 October 2021. 10.1016/j.esmoop.2021.100237.
Leyens L, Simkins T, Horst NK. The COVID-19 pandemic as a catalyst for innovation: a regulatory framework to assess fit-for-purpose innovative approaches in clinical research. Trials. 2022;23:833.
Getz K, Smith Z, Kravet M. Protocol Design and Performance Benchmarks by Phase and by Oncology and Rare Disease Subgroups. Ther Innov Regul Sci. 2022;57:49–56.
DiMasi JA, Smith Z, Oakley-Girvan I, et al. Assessing the Financial Value of Decentralized Clinical Trials. Ther Innov Regul Sci. 2023;57:209–19.
Ken Getz MBA. Amplifying Patient Voices in Protocol Design. Appl Clin Trials; 30, https://www.appliedclinicaltrialsonline.com/view/amplifying-patient-voices-in-protocol-design (2021, accessed 23 August 2023).
Pallmann P, Bedding AW, Choodari-Oskooei B, et al. Adaptive designs in clinical trials: why use them, and how to run and report them. BMC Med. 2018;16:29.
Adaptive Design Clinical Trials for Drugs and Biologics Guidance for Industry. Guidance Document, US Food & Drug Administration Center for Biologics Evaluation and Research, https://www.fda.gov/regulatory-information/search-fda-guidance-documents/adaptive-design-clinical-trials-drugs-and-biologics-guidance-industry (December 2019, accessed 23 August 2023).

Table 1. Protocol Characteristics – Oncology vs. Non-Oncology

Characteristic
Oncology
		n	Percent
Phase
	Overall	249	26.2%
	Phase I	99	39.8%
	Phase II	83	33.3%
	Phase III	55	22.1%
	Phase IV	12	4.8%
Orphan Designation
	Yes	12	12.1%
	No	87	87.9%
Master Protocol
	Yes	28	14.1%
	No	170	85.9%
Adaptive Design
	Yes	12	9.4%
	No	116	90.6%
		n	Mean
Number of Eligibility Criteria		429	33.1
Number of Endpoints		429	15.0
Number of Countries		647	7.8
Number of Sites		656	44.8
Non-Oncology
		n	Percent
Phase
	Overall	701	73.8%
	Phase I	259	36.9%
	Phase II	168	24.0%
	Phase III	231	33.0%
	Phase IV	43	6.1%
Orphan Designation
	Yes	29	7.4%
	No	364	92.6%
Master Protocol
	Yes	62	11.5%
	No	477	88.5%
Adaptive Design
	Yes	21	4.8%
	No	417	95.2%
		n	Mean
Number of Eligibility Criteria		429	33.4
Number of Endpoints		429	21.7
Number of Countries		647	6.6
Number of Sites		656	52.0

Table 2. Prevalence and Number of Amendments – Oncology vs. Non-Oncology

Protocol Characteristic	n	Percent with Amendments*	When Amendments are Present*
Protocol Characteristic	n	Percent with Amendments*	n	Mean (CoV)	Median
Oncology	205	91.1%	205	4.0 (0.67)	3.0
Non-Oncology	485	72.1%	485	3.0 (0.76)	2.0

*Chi-square, p < .0001

Table 3. Causes of Amendments – Primary Drivers Oncology vs. Non-Oncology

	Overall		Oncology		Non-Oncology
	n	%	n	%	n	%
Regulatory Agency Request	348	22.8%	109	22.5%	239	22.9%
Change in Study Strategy	330	21.6%	108	22.3%	222	21.3%
New Data Available (Other than Safety)	152	9.9%	42	8.7%	110	10.5%
Inconsistency / Error in Protocol	133	8.7%	32	6.6%	101	9.7%
New Safety Data Available	127	8.3%	51	10.5%	76	7.3%
Protocol Design Flaw	86	5.6%	18	3.7%	68	6.5%
Recruitment Difficulty	58	3.8%	11	2.3%	47	4.5%
COVID-19 Pandemic	46	3.0%	6	1.2%	40	3.8%
Investigator / Site Feedback	21	1.4%	6	1.2%	15	1.4%
Manufacturing Change	12	0.8%	2	0.4%	10	1.0%
Change in SoC	8	0.5%	5	1.0%	3	0.3%
Other	207	13.5%	95	19.6%	112	10.7%

Table 4. Avoidability of Amendments – Oncology vs. Non-Oncology

Avoidability of Amendments	Overall		Oncology		Non-Oncology
Avoidability of Amendments	n	%	n	%	n	%
Completely Avoidable	226	17.1%	52	13.3%	174	18.7%
Somewhat Avoidable	78	5.9%	18	4.6%	60	6.4%
Somewhat Unavoidable	489	37.0%	155	39.7%	334	35.9%
Completely Unavoidable	528	40.0%	165	42.3%	363	39.0%

Table 5. Changes Made During an Amendment – Oncology vs. Non-Oncology

Change		n	%	Oncology		Non-Oncology
Change		n	%	n	%	n	%
General Information		738	64.6%	191	53.7%	547	69.6%
	General Information (Title, Affiliates, etc.)	279	24.4%	95	26.7%	184	23.4%
	Typographical Error	646	56.6%	140	39.3%	506	64.4%
Trial Background, Objectives, and Purpose		637	55.8%	222	62.4%	415	52.8%
	Name and Description of Product	49	4.3%	15	4.2%	34	4.3%
	Findings from Nonclinical or Clinical Studies	196	17.2%	66	18.5%	130	16.5%
	Risks and Benefits	190	16.6%	62	17.4%	128	16.3%
	Route of Administration, Dosage, or Regimen	288	25.2%	128	36.0%	160	20.4%
	Compliance Statement	15	1.3%	3	0.8%	12	1.5%
	Background Literature	118	10.3%	39	11.0%	79	10.1%
	Trial Objective and Purpose	264	23.1%	93	26.1%	171	21.8%
Trial Design		798	69.9%	266	74.7%	532	67.7%
	Endpoints	344	30.1%	93	26.1%	251	31.9%
	Type or Design of Trial	378	33.1%	146	41.0%	232	29.5%
	Measures to Avoid Bias	64	5.6%	8	2.3%	56	7.1%
	Dosage Form, Packaging, or Labeling	136	11.9%	67	18.8%	69	8.8%
	Duration of Subject Participation	187	16.4%	55	15.5%	132	16.8%
	Stopping Rules or Discontinuation Criteria	290	25.4%	107	30.1%	183	23.3%
	Randomization Procedures	66	5.8%	11	3.1%	55	7.0%
	Addition of DCT Methodology	20	1.8%	9	2.5%	11	1.4%
	Source Data	55	4.8%	25	7.0%	30	3.8%
Selection, Withdrawal, and Treatment		764	66.9%	254	71.4%	510	64.9%
	Population Description (Inc. / Exc. Criteria)	693	60.7%	230	64.6%	463	58.9%
	Subject Withdrawal Criteria	70	6.1%	16	4.5%	54	6.9%
	Medication Permitted Before / During Trial	282	24.7%	108	30.3%	174	22.1%
	Procedures for Monitoring Compliance	48	4.2%	3	0.8%	45	5.7%
Assessments		816	71.5%	274	77.0%	542	69.0%
	Efficacy Assessment	191	16.7%	59	16.6%	132	16.8%
	Safety Assessment	449	39.3%	165	46.4%	284	36.1%
	Schedule of Assessments (SoA)	715	62.6%	245	68.8%	470	59.8%
Statistics		495	43.5%	179	50.3%	316	40.2%
	Statistical Methods and Analysis	411	36.0%	150	42.1%	261	33.2%
	Sample Size	236	20.7%	94	26.4%	142	18.1%
	Trial Termination Criteria	72	6.3%	38	10.7%	34	4.3%
Operational		150	13.1%	85	23.9%	65	8.3%
	Quality Control and Quality Assurance	12	1.1%	4	1.1%	8	1.0%
	Ethics	17	1.5%	4	1.1%	13	1.7%
	Data Handling and Recordkeeping	33	2.9%	7	2.0%	26	3.3%
	Financing and Insurance	2	0.2%	1	0.3%	1	0.1%
	Publication Policy	6	0.5%	3	0.8%	3	0.4%
	Supplemental Materials	99	8.7%	76	21.4%	23	2.9%
Other Changes		218	19.1%	102	28.7%	116	14.8%

Table 6. Amendment Timing – Oncology vs. Non-Oncology

	Oncology			Non-Oncology
Phase	Before FPFV	During Enrollment (FPFV – LPFV)	After LPFV	Before FPFV	During Enrollment (FPFV – LPFV)	After LPFV
Overall	19.7%	56.1%	24.2%	26.6%	58.7%	14.7%
Phase I	21.5%	67.2%	11.3%	27.3%	59.5%	13.2%
Phase II	21.0%	50.7%	28.3%	29.1%	56.0%	14.9%
Phase III	16.1%	49.5%	34.4%	24.0%	60.3%	15.8%
Phase IV	27.3%	45.5%	27.3%	28.3%	58.7%	13.0%

Table 7. Percent Change in Enrollment and Cycle Times for Protocols with and without Amendments – Oncology vs. Non-Oncology

	With Amendments			No Amendments
% Change in –	n	Mean (CoV)	Interpretation	n	Mean (CoV)	Interpretation	p-value
Oncology
Participants Screened	110	-1.8% (38.21)	Actual < Planned	7	27.1% (4.55)	Actual > Planned	.3022
Participants Enrolled	182	-11.1% (5.04)	Actual < Planned	19	1.6% (47.05)	Actual > Planned	.3650
Participants Completed	86	-63.2% (0.70)	Actual < Planned	6	2.3% (7.16)	Actual > Planned	.0005
Screen Failure Rate	109	24.8% (5.59)	Actual > Planned	5	-29.4% (1.40)	Actual < Planned	.3865
Completion Rate	84	-63.0% (0.66)	Actual < Planned	6	7.9% (2.05)	Actual > Planned	<.0001
Non-Oncology
Participants Screened	229	13.5% (4.45)	Actual > Planned	53	-2.8% (18.81)	Actual < Planned	.0698
Participants Enrolled	438	-6.5% (4.26)	Actual < Planned	157	-7.0% (4.05)	Actual < Planned	.8272
Participants Completed	243	-5.8% (8.28)	Actual < Planned	50	-4.7% (8.71)	Actual < Planned	.8775
Screen Failure Rate	188	34.7% (10.53)	Actual > Planned	41	-4.1% (20.60)	Actual < Planned	.5003
Completion Rate	243	-4.2% (9.53)	Actual < Planned	48	-1.8% (18.39)	Actual < Planned	.6940

Table 8. Enrollment Performance – Oncology vs. Non-Oncology

	With Amendments			No Amendments			p-value
Performance Measure	n	Mean (CoV)	Median	n	Mean (CoV)	Median
Oncology
Dropout Rate	139	54.2% (0.72)	52.8%	15	25.7% (1.40)	5.7%	.0076
Completion Rate	142	35.3% (1.07)	13.8%	17	56.6% (0.80)	87.5%	.0329
Screen Failure Rate	195	30.7% (0.57)	28.0%	20	15.3% (1.39)	3.5%	.0003
Non-Oncology
Dropout Rate	413	17.9% (1.23)	10.3%	100	11.8% (1.78)	5.1%	.0124
Completion Rate	427	77.2% (0.37)	88.0%	113	74.4% (0.50)	92.9%	.3879
Screen Failure Rate	447	35.3% (0.67)	34.8%	161	32.4% (0.81)	32.6%	.1940

Table 9: Percent Change in Cycle Times for Protocols with and without Amendments – Oncology vs. Non-Oncology

	With Amendments			No Amendments
% Change in –	n	Mean (CoV)	Interpretation	n	Mean (CoV)	Interpretation	p-value
Oncology
Study Startup (Approval to FPFV)	163	24.0% (4.13)	Actual > Planned	17	-0.7% (22.91)	Actual < Planned	.3075
First Visit Cycle (FPFV to LPFV)	139	12.3% (6.97)	Actual > Planned	13	-1.2% (83.51)	Actual < Planned	.5930
LP Participation Cycle (LPFV to LPLV)	130	97.4% (3.52)	Actual > Planned	13	83.5% (3.13)	Actual > Planned	.8874
Closeout (LPLV to DBL)	117	45.7% (2.86)	Actual > Planned	7	-100.8% (2.34)	Actual < Planned	.0072
Study Reporting (Database Lock to CSR)	120	54.2% (3.18)	Actual > Planned	6	58.4% (1.67)	Actual > Planned	.9536
Enrollment Duration (FPFV – LPLV)	149	21.3% (3.45)	Actual > Planned	16	6.7% (5.80)	Actual > Planned	.4360
Total Study Duration (Approval to DBL)	128	21.0% (2.80)	Actual > Planned	7	-4.5% (2.04)	Actual < Planned	.2559
Non-Oncology
Study Startup (Approval to FPFV)	430	16.1% (4.53)	Actual > Planned	109	3.4% (8.99)	Actual > Planned	.0756
First Visit Cycle (FPFV to LPFV)	390	14.7% (5.51)	Actual > Planned	101	2.6% (22.32)	Actual > Planned	.1600
LP Participation Cycle (LPFV to LPLV)	374	13.1% (11.13)	Actual > Planned	97	40.9% (9.98)	Actual > Planned	.2796
Closeout (LPLV to DBL)	323	63.9% (4.15)	Actual > Planned	65	41.4% (5.22)	Actual > Planned	.5213
Study Reporting (Database Lock to CSR)	313	24.1% (12.58)	Actual > Planned	63	31.7% (4.79)	Actual > Planned	.8481
Enrollment Duration (FPFV – LPLV)	418	11.4% (7.11)	Actual > Planned	107	1.8% (18.64)	Actual > Planned	.2318
Total Study Duration (Approval to DBL)	335	11.1% (3.45)	Actual > Planned	70	4.6% (5.92)	Actual > Planned	.1805

Table 10. Pre- and Pandemic Protocols – Oncology vs. Non-Oncology

	Pre-Pandemic			Pandemic
	n	Mean (CoV)	Median	n	Mean (CoV)	Median	p-value
Oncology
Amendments*	84	3.0 (0.90)	2.0	105	4.1 (0.70)	3.0	.0068
Screen Failure Rate	78	27.5% (0.72)	27.5%	102	29.7% (0.60)	25.9%	.4416
Completion Rate	69	52.2% (0.78)	63.2%	84	25.9% (1.32)	2.9%	<.0001
Dropout Rate	60	33.4% (1.09)	16.8%	79	63.7% (0.59)	78.5%	<.0001
Non-Oncology
Amendments*	329	2.0 (1.08)	1.0	303	2.2 (1.03)	2.0	.1155
Screen Failure Rate	305	36.7% (0.67)	37.0%	278	31.9% (0.75)	31.9%	.0172
Completion Rate	281	76.3% (0.42)	90.0%	239	78.8% (0.33)	88.9%	.3290
Dropout Rate	253	15.3% (1.38)	8.2%	232	18.0% (1.24)	10.1%	.1827
*When amendments are present

Download PDF

Journal Publication

published 26 Mar, 2024

Read the published version in Therapeutic Innovation & Regulatory Science →

Editorial decision: Minor revisions
07 Jan, 2024
Reviewers agreed at journal
13 Dec, 2023
Reviewers invited by journal
04 Dec, 2023
Editor assigned by journal
03 Nov, 2023
First submitted to journal
01 Nov, 2023

You are reading this latest preprint version

New Benchmarks on Protocol Amendment Experience in Oncology Clinical Trials

Status:

Journal Publication

Version 1

Abstract

Introduction

Methods

Results

Discussion

Conclusions

Declarations

References

Tables

Status:

Journal Publication

Version 1