Quantifying Site Burden to Optimize Protocol Performance

Background: The increase in protocol complexity—and the resulting rise in the effort required by investigative sites to implement protocols has been well documented, but existing measures of site burden only offer an incomplete view of the burden experienced by site personnel. The introduction of Decentralized Clinical Trials – trials supported by remote and virtual technologies and services – is expected to impact the burden imposed on sites, but this impact has not yet been systematically measured. Methods: The Tufts Center for the Study of Drug Development conducted an online survey among clinical research sites worldwide and gathered 355 responses assessing the burden associated with distinct activities and procedures related to the implementation of clinical trial protocols using traditional and decentralized approaches. Results:


Introduction
Protocol complexity has been increasing for decades, and its impact on trial performance is well documented. ,Along with, or perhaps more accurately due to, increasing protocol complexity, there has been an increase in the effort required by investigative sites to administer protocols.The Tufts Center for the Study of Drug Development (Tufts CSDD) has demonstrated a dramatic increase in the effort measured in work effort units (WEU) that protocols require of participating investigative sites, with estimated annual increases in WEU of 10.5% from 1999 to 2005, and total increases in site work effort between 61.0% and 82.0% from 2001 to 2015, depending on the phase of the clinical trial. ,Increases in WEU are signi cantly correlated with worsening protocol performance outcomes including longer cycle times, more protocol amendments, and lower recruitment and retention rates.
Although the increase in the work effort required of investigative sites to administer protocols is well documented, it presents an incomplete view of the actual burden borne by site personnel.The work effort units developed by Tufts CSDD are based on Medicare's Relative Value Units (RVUs) methodology.RVUs were designed to provide an objective and useful measure for calculating reimbursement based on the value of time a given medical professional performs a speci c medical procedure.Other factors may impact the burden of performing protocol procedures that are not unaccounted for by WEUs including disease condition, location where a given procedure is performed, as well as patient needs and preferences.In essence, WEUs simplify burden to the time and value of a procedure, and while they are a sound starting point, they are not a comprehensive measure.
Investigative site burden to administer a given protocol now faces a new challenge: the execution of clinical trials supported by remote and virtual technologies and services -also referred to as decentralized clinical trials (DCTs).While some of these solutions have been in use for more than a decade, they became much more commonplace during the COVID-19 pandemic.Although it is generally expected that the use of DCT solutions will ease the burden of participation for clinical trial participants, their impact on the burden for investigative sites to execute a protocol remains unclear.
Although some have speculated that wearable and mobile devices, for example, may help reduce burden by automating data collection activity, recent studies have shown that investigative sites largely perceive additional burden when administering protocols involving remote and virtual solutions.In 2022, for example, the Society for Clinical Research Sites (SCRS) and Medidata conducted a global survey of investigative sites.Among its many ndings, the survey found that sites believe DCT solutions will add executional complexity and complications requiring greater oversight of safety and adherence and adding to uncompensated work provided by study personnel.At the same time, more than half of respondents reported 'somewhat' or 'very' positive experiences using DCT solutions in clinical trials and they reported the same when asked about their participants' experiences.
Given growing interest in, and knowledge about, investigative site burden executing traditional and DCTsupported clinical trial protocols, in 2023, Tufts CSDD -in collaboration and with support provided by ZS -conducted a new study.The purpose of this new study was to both update its Work Effort Unit methodology and adapt and apply a validated patient burden algorithm methodology to determine a more expansive measure of site execution burden per protocol. ,The objectives of the new study were to: (1) quantify the burden of common protocol procedures as perceived by investigative site personnel, and (2) objectively measure the relative impact of DCT solutions on the perceived burden of conducting a clinical trial.

Methods
The Tufts CSDD research team developed a comprehensive survey containing questions to elicit site personnel estimates of the burden of performing common clinical trial activities and procedures, as well as the burden of performing those activities and procedures using a variety of decentralized clinical trial (DCT) solutions.Additionally, the survey included questions to gauge general perceptions about the burden associated with both traditional and DCT approaches to clinical trial conduct.
The survey was reviewed and approved by an ethical review committee and was deemed compliant with European General Data Protection Regulation (GDPR) by an independent data review committee at Tufts University.The survey was administered online among investigative site personnel between November 2022 and February 2023 using a commercially available list of 131,766 physicians, nurses, and other research personnel working at clinical research centers worldwide.The survey instrument contained 42 questions and required approximately 20 minutes to complete.
De nitions of key terms were provided to assist respondents in completing the questionnaire (See Table 1).De nitions were informed by internationally recognized sources and are consistent with those used in guidance documents issued by FDA. ---Insert

Decentralized Clinical Trial (DCT)
A study executed using telemedicine and mobile/local healthcare providers, and processes and technologies differing from the traditional clinical trial model.If a trial uses any one virtual or decentralized component, the trial is a DCT.A DCT can be either 'fully' decentralized or 'hybrid' decentralized.

Fully Decentralized Clinical Trial
A type of DCT in which all operations are decentralized using telemedicine and mobile/local healthcare providers, and processes and technologies differing from the traditional clinical trial model.

Hybrid Decentralized Clinical Trial
A type of DCT in which some operations are decentralized using telemedicine and mobile/local healthcare providers, and processes and technologies differing from the traditional clinical trial model.
To accurately capture data on perceived clinical trial burden, the survey was designed to include three distinct burden assessment sections.In the rst burden assessment section, respondents were asked to estimate the burden of performing common clinical trial activities and procedures using a decentralized clinical trial model compared to performing them using a traditional on-site approach (as de ned in Table 1).These activities and procedures were generally ongoing and performed throughout the duration of a clinical trial.Respondents were asked to provide a relative change in burden on a scale of -100-100%, with 0 representing no change in burden.
In the second burden assessment section, site respondents were asked to rate the burden of performing routine administrative, training, site activation, recruitment, and other operational activities supporting both a traditional and DCT protocol -on a scale of 0 to 100.The activities and procedures in this section were discrete actions, with distinct beginnings and endings.Respondents were asked to compare and rate the burden of performing those activities and procedures in each context compared to a baseline de ned as "scheduling and ordering routine blood work at a clinical trial site"; this baseline was assigned a value of 50 on a scale of 0 to 100.
In the third burden assessment section, investigative site personnel were asked to rate the burden of performing common medical procedures in four contexts so as to capture data to measure the burden of implementing a spectrum of virtual and remote clinical trial approaches-from hybrid to fully decentralized.The four contexts were: (1) administration/performance at site, (2) self-administration or performance by the patient at patient's home, (3) administration/performance at patient's home by a home health professional, and (4) administration/performance at a local clinic/site other than the respondents' own site.
Participants were asked to compare the burden of performing procedures in each of these four contexts in reference to the burden of "measuring and recording vitals at a clinical trial site" and to then rate it on a scale of 0 to 100, with the baseline having a value of 50.Procedures were grouped into seven procedure types: Medication Administration; Routine Procedures; Lab and Blood Procedures; Non-invasive Procedures; Invasive Procedures; Imaging Procedures; and Questionnaires.In context 1, respondents were asked to provide a burden estimate for each individual procedure.For contexts 2, 3, and 4, respondents were asked to provide an estimate of the average burden across all applicable procedures within the procedure type group (i.e., average for all applicable Medication Administration Procedures; average for all applicable Routine Procedures, average for all applicable Imaging Procedures, etc.).
Lastly, participants were asked to share if they had experience with DCTs and to what extent.They were also asked to indicate their general views regarding the conduct of decentralized clinical trials at their site by answering a series of multiple-choice questions, including about the methods they believe are most likely to support successful DCT implementations.
Survey responses were downloaded and saved as comma-separated (CSV) les from Qualtrics, the online survey platform used to deploy the online survey.Raw data were then converted and stored in Microsoft Excel and hosted on Tufts University servers.Completed surveys were evaluated to remove duplicate responses from the same investigative site.Several characteristics (e.g., country, site type, and therapeutic area) were evaluated to identify and remove duplicates and derive estimates for the total number of distinct clinical research sites participating in the survey.
Data cleaning and analysis was conducted using R 4.3.0.Analyses included descriptive statistics, frequency comparisons, coe cients of variation (de ned as the ratio of standard deviation to the mean), comparisons of mean overall response and subgroup response values (where feasible), and signi cance testing (where feasible).Subgroups of interest included site type, therapeutic areas of focus at site, region, site setting, and role.Since some respondents left some questions unanswered, each response was reported and analyzed as a percentage of responses for that question/variable.Data were analyzed to develop and report mean burden scores for common clinical trial activities and procedures using traditional and decentralized solutions or approaches, as well as the perceived change in burden.
---Insert Respondents also provided their assessment of investigative site staff experience required to successfully carry out traditional and decentralized approaches (as de ned in Table 1).More than half of respondents (51.9%; 40/77) indicated that DCT solutions and approaches require "somewhat more" or "signi cantly more" experienced staff compared to a traditional clinical trial.See Table 6.
---Insert In the aggregate, combining all areas where traditional and DCT-supported clinical trials were compared, those involving DCT solutions increased protocol execution burden by 10.4% (Refer to Table 7).This estimate includes the incremental burden associated with DCT solutions when educating patients about study procedures and equipment (25.3%); use of new technologies and devices (24.54%); adverse event monitoring and reporting (19.7%); reporting protocol deviations (18.33%); drug dispensing and shipping (17.8%);trial kit dispensing (13.3%); meeting regulatory requirements (13.2%); record keeping (10.6%); and storing clinical trial kits and materials (6.2%).Drug storage was the only procedure evaluated where a signi cant change was not observed (0.8% decrease).When asked to compare clinical trials involving investigative site visits, home nurse visits, and telehealth visits, investigative sites reported higher burden with in-home nurse visits (17.8% increase), and lower burden with telehealth patient visits (14.9% decrease).See Table 7.
---Insert Note: Burden associated with visits (discrete tasks) are compared between in-person vs. site-based and are therefore presented separately.The upper section of the table compares continuous tasks on the basis of using digital versus analog approaches.The change in burden in all cases included in the table was evaluated on a scale from -100 to 100.
Table 8 shows the perceived burden of performing general management activities in DCT-supported clinical trials was only slightly higher (0.2%) compared to traditional models whereas the burden of performing training activities decreased slightly by 1.6%.In all, the perceived burden of performing most clinical trial management and operations activities-which typically require direct contact with the sponsor or clinical research organization-decreased signi cantly within a DCT implemenattion framework.The estimate of burden for general management activities within both a traditional clinical trial and DCT framework accounted for the following categories of activities: staff training activities, preactivation and site initiation activities, patient recruitment and management activities, data management and trial closeout activities, and general clinical trial operations (e.g., invoicing and collections, regulatory compliance reporting, storing source documents).
The perceived burden of pre-activation and site initiation activities decreased by 20.2% within a DCT framework compared to using traditional approaches.Similarly, the perceived burden of patient recruitment, management, and retention activities decreased by 10.8% when using a DCT framework, as did both data management and closeout activities (a 15.1% decrease) and general site operating activities (a 15.8% decrease).
---Insert Note: Respondents rated the burden of performing activities within a Traditional Clinical Trial, on a scale of 0-100, compared to scheduling and ordering routine blood work (e.g., CBC procedure).The burden level of Scheduling and Ordering Routine Blood Work at Trial Site = 50.A score lower than 50 means the activity is less burdensome than scheduling and ordering routine blood work.A score higher than 50 means the activity is more burdensome than scheduling and ordering routine blood work.Then, they rated the burden associated with the same feasible activities within a DCT, on a scale of 0-100.
Overall, investigative sites perceive increased burden when common protocol procedures are performed outside of the usual clinical trial setting including medication administration, routine procedures, lab and blood procedures, non-invasive and invasive procedures, and imaging procedures (See Table 9).Comon protocol procedures typically require direct study team-patient interaction.
The perceived burden of administering medications at home by the patient was 3.3% higher than that administered at the clinical trial site.The perceived comparative burden of administering medication at a local clinic or at home with the assistance of a home-health professional was 9.5% and 10.7% higher respectively.The medication administration procedures included in this assesment are oral pill, oral liquid, auto-syringe, IV, and topical administration.
The perceived burden of performing routine procedures was incrementally higher when performed virtually or remotely.The following routine procedures were included in our assessment: eye exam, hearing exam, neurology exam, pap smear, physical exam, skin exam, neurology exam, mental health exam, breast exam, testicular exam, and pupil exam.On average, the perceived burden of performing these procedures at the patient's home (when feasible) was 6.8% higher compared to those performed at the clinical trial site.The perceived burden of performing these procedures at a local clinic or at home by a home-health professional (when feasible) was 11.4% and 15.6% higher respectively.
The perceived burden of collecting lab and blood samples (for example, nger prick samples, nasal swabs, sperm samples, stool samples, blood samples, etc.) was higher than those collected at the clinical trial site.When these samples are collected by the patient at home (when feasible), the perceived burden increased by 7.9%; and increased by 9.2% when collected at a local site or clinic.When samples are collected by a home-health professional at the patient's home (when feasible), the perceived burden increased by 10.7%.
The perceived burden of performing non-invasive procedures declined by 4.4% when performed by the patient at home (when feasible) compared to performing them at the clinical trial site.However, when these procedures are performed at a local site or clinic, or by a home-health professional at the patient's home, the perceived burden increased slightly by 1.4% and 2.8% respectively.Examples of non-invasive procedures evaluated include motors skills tests, walking tests, exercise tests and pulmonary tests.
The perceived burden of performing invasive procedures-for example, a urethral catheter, apheresis, a bronchoscopy, a cardiac catheter, an endoscopy or colonoscopy -decreased slightly by 1.1% when performed at a local clinic.However, when performed at home by a home-health professional (when feasible), the perceived burden increased by 3.3%.
The perceived burden of performing imaging procedures decreased by 2.0% when performed by the patient at home using a device (when feasible).However, when performed at a local clinic, the perceived burden increased by 4.8%.
Finally, the perceived burden associated with completing brief questionnaires decreased signi cantly (by 14.5%) when completed by the patient at home compared to at the clinical trial site.However, the perceived burden of completing questionnaires increased by 5.7% when a home-health professional completed the task at the patient's home.
---Insert Table 9--- *N is the total number of ratings provided by site respondents.
Note: Respondents provided an estimate of burden for their site when performing each procedure in their therapeutic area of focus at the investigative site.They also gave a burden estimate for general procedure types performed using a DCT alternative, all on a scale from 0 to 100.To accurately measure burden, respondents compared the burden of each procedure against a central reference point: Measuring and Recording Vitals at the trial site (50).A score lower than 50 means the procedure is less burdensome than measuring and recording vitals at a trial site.A score higher than 50 means the procedure is more burdensome compared to measuring and recording vitals at a site.

Methods to Support Successful Use of DCT Solutions by Investigative Sites
Many respondents indicated that increasing exibility in the use of site-provided versus sponsor-provided tools is a method that may aid in improving DCT implementation (29.0%).Expanding budgets for sites (14.3%) and increasing patient technical support (12.5%) were also frequently cited as methods to support the successful execution of decentralized clinical trials.
---Insert Table 10--- Overall, half of respondents view DCT solutions as 'more' or 'much more' burdensome than traditional clinical trials.A similar percentage (50.6%) of respondents believe that DCTs require 'somewhat more' or 'signi cantly more' experienced staff than do traditional trials.
The results of this study indicate that sites largely perceive greater burden when using DCT methodologies to administer protocol procedures that directly involve the patient and to comply with regulatory and safety requirements.
In all protocol procedure areas assessed, investigative sites perceive a small-to-moderate increase in burden, with select procedures areas having an average perceived relative increase in burden of more than 10.0% and as high as 16.0%.Perceived modest decreases (between 1.0% and 4.0%) in burden were only observed in a few assessment areas with one noteworthy exception: having patients complete questionnaires at home using a mobile device had a perceived decrease in burden of 14.5%.
Sites also reported substantial increases in the perceived burden to educate patients about safety procedures and clinical trial equipment (25.3% increase)-which require direct interaction between the study team and the patient-and to use novel technologies (24.5% increase).Investigative sites perceive increased burden of 15.0% − 20.0% to perform compliance, logistics and safety activities including adverse event (AE) monitoring, reporting protocol deviations, drug dispensing and shipping.The only area where perceived burden modestly decreased was drug storage (0.8% decrease).
In sharp contrast, investigative sites perceive a decrease in burden when DCT solutions support general management, and operating areas (e.g., study start-up, study conduct and study close-out).Most activities-including contracting and budgeting and ethical review-were perceived as less burdensome when conducted remotely or virtually with one exception: feasibility assessments.Virtual investigator meetings were perceived as nearly 25.0% less burdensome; virtual data archiving and storing was perceived as 22.0% less burdensome; and virtual site initiation visits (SIVs) were perceived as nearly 22.0% less burdensome.Despite a general perception that site participation burden increases with DCT solutions, 61.4% of respondents said their preferred clinical trial model was either hybrid or fully decentralized, only 24.0% said they preferred a traditional model, and the remainder (14.7%) said they had no preference.
The study ndings are compelling given growing interest in DCT solutions, and the recent draft guidance issued by the FDA encouraging the use of these approaches.Our results suggest that DCT use is nuanced -sites perceive lower relative burden in general management and site operating areas and higher relative burden in administering procedures, logistical, compliance and safety reporting areasand will likely require more rigorous and thoughtful planning to determine optimal deployment of virtual and remote solutions.
Our method for assessing comparative burden between traditional and DCT-supported clinical trials yielded good and reasonable discrimination between procedural, operating and compliance variables.Future research will focus on expanding our use and validation of a site participation burden algorithm to evaluate protocol design feasibility and performance.This approach has the potential to inform study planning and help reduce site participation burden.
Future research will also look to gather a larger response sample and will examine speci c reasons for the perceived increase in burden, whether there are differences by therapeutic area and the steps that can be taken to mitigate burden.Additional research might also look at balancing site and patient participation burden.
There are several limitations to note in this study.The survey was conducted in English, and this may have restricted participation by site personnel from predominantly non-English speaking communities.
The online survey attracted a disproportionately larger number of respondents in principal investigator/sub-investigator roles, and respondents from government-funded or non-pro t institutions than is typically seen in industry-funded clinical trials of medical interventions.Still, 355 responses from 24 countries/locations provided a comprehensive global perspective from a wide variety of site types.
Lastly, fewer than expected respondents had experience with DCT methodologies.As such, only a hundred or so investigative sites were able to estimate the perceived relative burden of select areas in our survey.

Conclusions
In this study, Tufts CSDD developed and implemented a new method to quantify investigative site burden executing a protocol.This approach and its results can be used to inform and improve clinical trial design and planning.Our study not only sheds light on the burden experienced by site staff during the implementation of clinical trials using both traditional and DCT approaches, but also offers a baseline against which to compare future investigative site burden.
Speci cally, the results indicate that investigative sites largely perceive greater burden when using DCT methodologies to administer protocol procedures and to comply with regulatory and safety requirements, and lower burden when using DCT solutions to support general management and operating areas.The results of this research indicate that exibility and customizability are key when it comes to the use of DCT solutions.No one DCT solution is appropriate for all situations -patient, site, and protocol needs should all be considered when creating a plan for DCT solution deployment. References

Table 2 :
Description of Survey Data Collected , 212 respondents shared their level of exposure to DCT technologies and services and the models within which the solutions were used.Half of respondents (50.5%; 107/212) reported having no experience with DCT (see Table3).Approximately one in ve respondents (46/212; 21.7%)only had experience participating in hybrid DCTs.Another one in ve respondents (45/212; 21.2%) had experience with both fully decentralized and hybrid decentralized trials.
*Respondents were asked to select all therapeutic areas of focus at their site.Many selected more than one area.The number of site respondents providing therapeutic area of focus data was 248.**Ophthalmology,Dermatology,Obstetrics/Gynecology,Pulmonary/Respiratory Diseases, Gastroenterology, Rare Diseases and Disorders, Vaccines, Immunology, Genetic Disease, Rheumatology, Family Medicine, Gene Therapy, Orthopedics/Orthopedic Surgery, Musculoskeletal, Urology, Nephrology, Nutrition and Weight Loss, Plastic Surgery, Dental and Oral Health, Otolaryngology, Podiatry, Sleep.***Countriesrepresented in sample ordered by volume: United States, United Kingdom, Canada, Spain, Japan, Greece, Australia, Italy, Portugal, Croatia, China, Germany, Romania, Slovenia, Belgium, Bulgaria, Cyprus, Denmark, France, Ireland, Latvia, Oman, Slovakia, and Switzerland.Perceptions of and Experience with DCTsBased on the de nitions provided in Table1

Table 3 :
Extent of site personnel experience with DCTs

Table 4
shows general investigative site perceptions of the overall burden associated with performing DCT-supported clinical trials: half (50.6%; 39/77) reported that DCTs are "more" or "much more" burdensome compared to traditional clinical trials; 31.2%(24/77) indicated that DCTs are "less" or "much less" burdensome than traditional clinical trials; and 18.2% indicated that, overall, they found little to no difference between each of these approaches.

Table 4 :
Perceived burden of DCTs compared to Traditional Clinical Trials

Table 5 :
Preferred Clinical Trial Model at Sites

Table 6 :
Site Personnel Experience Required for DCT Implementation

Table 8 :
Burden of Performing Clinical Trial Management and Operations Activities *N is the total number of ratings provided by site respondents.**Traditional Clinical Trial Meetings (conducted in person at site) versus DCT Meetings (conducted virtually)

Table 9 :
Burden Scores -Clinical Trial Procedures

Table 10 :
Methods to Improve DCT Implementation Respondents could select multiple options.Methods are shown in order of most selected to least selected.The number of site respondents providing therapeutic area of focus data was 77.DiscussionThe results of this study indicate that a high percentage of investigative sites (50.5%) have had no experience with DCT solutions and only a small percentage (6.6%)have participated in fully decentralized clinical trials.This nding is consistent with other recent studies documenting low levels of actual DCT use in clinical trials: A recent report from IQVIA, for example, showed that the number of new trials with DCT methodologies registered on clinicaltrials.gov in 2022 was roughly the same as in 2019 and represented only about 1.0% of the total number of new trials initiated in 2022. *