This is an observational study for which we aim to retrospectively collect detailed empirical cost and resource use data from 100 investigator-initiated RCTs from three different countries (Switzerland n = 60, Germany n = 20, UK n = 20). This study was registered on Open Science Framework (32).
Eligibility criteria and trial identification
RCTs will be included if they have been completed or are close to completion (statistical analysis ongoing or manuscript in preparation). Trials will be excluded if they are industry sponsored, dose-finding RCTs, RCTs primarily evaluating pharmacokinetics or physiology, include healthy volunteers, or RCTs that published results more than 10 years prior to the start of this investigation. To enhance comparability across countries we will aim to include 10–15 RCTs each from cardiovascular, oncology, surgical and miscellaneous fields. However, it was challenging to find principal investigators willing to share their cost data, as such we decided to expand our scope and include a small sample of pilot studies (n = 10) and RCTs initiated in Switzerland but conducted in low- and lower-middle-income countries (n = 10).
We will identify potentially eligible RCTs using two approaches: Firstly, we will contact Clinical Trial Units (CTUs), trial networks, and individual principal investigators known to the authors in all three countries and ask them about eligible investigator-initiated RCTs they could contribute. In particular, we will approach the Swiss CTU network; UK Registered CTUs Network; Network of Hubs for Trials Methodology Research (UK); UK Trial Managers’ Network, Trial Forge Initiative (UK); Koordinierungszentren für Klinische Studien Network (DE); Oncology network (DE); and the Clinical Trials Centre Cologne (DE).
Secondly, we will screen trial registries for completed RCTs and invite chief investigators to participate in our study. These trial registries will include clinictrials.gov, the ISRCTN registry, the Health Research Authority (NHS) registry, Kofam (Swiss Clinical trial registry) and the Deutsches Register Klinischer Studien (DRKS; the German clinical trial registry).
For each RCT contributed to the project, we will offer the trial teams 200 Euros as a compensation for their efforts.
Data collection procedures
Protocols and publications will be collected for each of the RCTs and the cost data will be collected and stored in the electronic data capture system REDCap (33, 34). Trial teams will enter their cost and resource data directly into the case report form (CRF). Data entry will be followed by a short interview to clarify unclear values and missing data. A cost report will be generated for each trial, which is then reviewed by the trial team.
For the collection of the resource use and cost data, we will use a previously developed and tested cost item list developed by Speich et al. that follows principles of activity-based costing (27, 35). An overview of the collected variables can be found in Table 1. All variables and the CRF can be found in the supplementary material (Supplement-Table 1 and Supplement-Table 2).
In short, we will be collecting two types of cost data: Firstly, the fixed costs associated with the trial, which are costs that are independent of time and human resources, such as ethical approval fees, insurance fees, travel costs and material fees (Table 1). Secondly, we will be collecting variable costs, using a predefined list of study tasks, which are dependent on human resources, time expenditure of the team and salary, such as days needed to write the protocol, days for communication amongst stakeholders, minutes needed for patient screening, recruitment and treatment. (Table 1). Variable costs will be recorded in days or minutes for each study task, for each type of trial team member (principal investigator, site investigator, study nurse/coordinator, senior medical staff, junior medical staff, administrative staff, statistician, IT support and data management, monitor, laboratory staff, or others) and will then be multiplied by their gross yearly income (given by the study team) to calculate billable hours/days. The costs can be entered in five currencies: Euro (€), Swiss Franc (CHF), Pound (£) and United States Dollar (USD).
From protocols and publications, we will extract the relevant RCT baseline characteristics. These will include medical field, type of intervention and control group, type of patient population, specific trial design (e.g. cross-over or factorial design), number of trial arms, number of trial sites, planned and achieved sample size, originally planned total costs and planned budget (if available), primary outcome(s), trial duration, planned and actual recruitment duration, monitoring method, electronic or paper-based data capture/case report forms and funding sources.
Sample size considerations
We simulated data sets of various sizes to explore how many RCTs we will need to achieve a 90% probability (statistical power) for a sufficiently narrow 95% confidence interval (CI) around the estimated mean cost proportions for individual cost items. We assumed that a 95% CI width of 6% (i.e. a maximum of +/- 3%) for the estimated mean would be sufficiently precise. Figure 1 shows how the required sample size depends on the true average proportion. Note that there are two lines: a dark line for a rather wide assumed distribution of proportions (tau = 10; beta distribution with a precision of 10), and a grey line for a plausible, narrower distribution (tau = 20). According to the dark line (the wider distribution), including a total of 60 RCTs would suffice to achieve a power of 90% to find a satisfyingly narrow 95% CI of 6% for a mean proportion below 0.15. Thus, our original plan was to include at least a total of 180 RCTs which appeared appropriate to achieve sufficient precision for estimated proportions (even within subgroups such as countries). However, it proved extremely challenging to find principal investigators willing to share detailed resource use and cost data of their trials rendering the target sample size of 180 RCTs unfeasible. Therefore, we reduced our target to 100 RCTs (Switzerland n=60; Germany n=20; UK n=20). This number is feasible and will still allow for sufficient precision of estimates in the Swiss subsample, rough estimates for the German and UK samples, and to conduct meaningful multivariable regression analyses (36).
Data management
All data will be collected for each of the RCTs and the cost data will be collected and stored in the electronic data capture system REDCap (33,34). For the statistical analysis we will transfer data to the statistical software R version 3.5.1.
The variable costs will be adjusted for employer contributions (%), the overhead of the sponsor institution (%) and for unaccounted time (%). Employer contributions are additional expenses and contributions associated with employing a person, such as social security contributions, pension contributions, health insurance, and other benefits that are part of an employee's compensation package in addition to their base salary; specifically, 23.3% of the gross salary in Germany (37), 15.0% in Switzerland (38) and 21.3% in the UK (39). Unaccounted time is defined as the time spent on study activities not recorded or accounted for by the study team and was determined as 30%, based on results of a study shadowing clinical investigators during their study activities (40). The overhead for the institution was recorded in the CRF by the study team. All costs will be converted into USD using historical exchange rates provided by the European Central Bank (41). We will use the start date of recruitment as the date for the historical exchange rate and convert the costs to values for the year 2022. Values will also be adjusted for inflation to the year 2022 using the World Bank inflation index (42).
In Switzerland a workday was defined as 9.6 hours for medical staff (principal investigator, site investigator, medical staff junior, medical staff senior), and 8.4 hours for other staff members (research coordinator, statistician, IT support and data management, Monitor, laboratory staff), with a total of 226 workdays per year (average number of workdays in Switzerland). In Germany and the UK a workday was defined as 9.6 hours for medical staff and 7.5 hours for other staff members, with a total of 223 workdays per year in Germany and 224 work days per year in the UK.
Statistical analysis
We will calculate proportional contributions to total costs for individual cost items and for the three pre-defined study phases (preparation, conduct, and finalization). Proportions across RCTs will be summarized with descriptive statistics accompanied by 95% CIs. We will summarize total resource use (working hours and costs), as well as resource use and costs for individual cost items, and costs per patient using descriptive statistics (means or medians accompanied by standard deviations (SDs) or 25th to 75th percentile ranges [IQRs] according to the distribution, minimum [min] and maximum [max]), respectively) overall and stratified by RCT over- or under- budget (±10%), medical field, intervention type, country, and single/multi-centre status.
We will categorize reasons for going over- or under-budget as reported by trial investigators and reasons for prolongation of recruitment (43). In sensitivity analyses, we will calculate results without adding 30% of unaccounted time and/or adding overhead costs.
We will conduct multivariable regression analyses with total costs in US dollars and total costs per patient (linear regression) as dependent variables. Medical field, country, number of trial sites, actual sample size, routine data use, intervention type, and actual trial duration will be used as independent variables in both regression analyses. As for the investigation into the heterogeneity of cost proportions of RCTs across medical fields, intervention types, and countries, we will first plot distributions of cost proportions graphically stratified by medical field, intervention type and country. For each RCT, we will compare the originally planned versus actual total costs, the originally planned versus actual recruitment duration and the planned versus actual sample size.
We will calculate excess costs for each RCT (delta of originally planned total costs and actual total costs [Total costs – Planned Costs]). This will help investigate trial characteristics associated with excess costs of an RCT (dependent variable). We will develop a multivariable regression model with delta in recruitment duration, actual sample size, and actual number of trial sites, country, intervention type and medical field as independent variables.
We will not exclude any collected data from planned analyses. Outliers will be reviewed and checked/verified with principal investigators of included trials. Incomplete or missing data will be minimized through additional information requests to principal investigators of included trials. The number of missing data in an analysis will be documented. If income was not provided by the institution to calculate billable hours, we will impute the median income for the respective country, for each specific team role.
Confidentiality
We will inform principal investigators of included trials that only aggregated data will be published and none of the individual RCTs, investigators, staff members or sponsors will be identifiable. The final database will contain coded trial identification numbers