Implementing electronic patient reported outcome data capture for multi-centre oncology clinical trials

doi:10.21203/rs.3.rs-1412445/v1

PDF

Research Article

Implementing electronic patient reported outcome data capture for multi-centre oncology clinical trials

https://doi.org/10.21203/rs.3.rs-1412445/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background:

Traditionally patient reported outcomes (PRO) are collected on paper as key endpoints for clinical trials. With increasing internet usage there is an interest in the use of electronic patient reported outcomes. Although previous studies in the general oncology clinical setting have shown the equivalence of scores in paper and electronic formats, there is a wide ranging level of uptake of electronic patient reported outcome acceptance amongst trial patients.

Implementation plan:

We have chosen to implement the use of electronic patient reported outcomes in our clinical trial population using a study within a trial (SWAT) to assess functionality and acceptability to patients. This will be led by a multi-disciplinary team of project managers, methodologists, clinicians and data scientists. The implementation plan has multiple ethical and regulatory considerations required in system identification, patient and public involvement and the design of a SWAT.

Conclusion:

Implementation of ePRO even in a world of increasing technology use is a complex and multifactorial project requiring careful consideration and adequate resourcing.

Electronic

PRO

QoL

Patient reported outcomes

The Cancer Research UK funded Clinical Trials and Statistics Unit at The Institute of Cancer Research (ICR-CTSU) is an academic clinical trials unit which designs, manages and analyses multicentre phase II and III oncology trials. Trials are conducted in the secondary care setting at hospitals in the UK’s National Health Service and internationally. The ICR-CTSU portfolio includes clinical trials of investigational medicinal products (CTIMPs) and non-CTIMPs investigating radiotherapy and surgery. Key areas of interest are breast, urological, lung, and head and neck cancers.

Assessment of patient reported outcomes (PRO) is a key secondary endpoint for many ICR-CTSU trials and its use as a primary endpoint is increasing. PRO are collected via validated questionnaires completed by trial participants. The questionnaires capture the impact that treatment and health conditions may be having upon their quality of life - defined as “any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else”[1]. These questionnaires are currently completed by trial participants on paper. Questionnaires are either administered to participants by research teams in clinic or sent to patients’ home addresses by ICR-CTSU, after confirmation from site staff that the participant is alive and able to complete the booklet.

In 2012 the ICR-CTSU replaced paper case report forms with electronic data capture (EDC) directly from participating sites and this is now the primary method of clinical data collection. Over the past decade, advances in information technology and improved access to the internet have led to a rapid increase in the use of electronic devices including smartphones, tablets and laptops across the UK population. In 2020 92% of adults in the UK regularly used the internet [2], with usage by the over 75s increasing from 29% in 2013 to 54% in 2020. The effect of the COVID-19 pandemic is likely to have increased internet exposure further and it has already been shown that the proportion of online adults aged over 65 who make a least one video-call each week increased from 22% in February 2020 to 61% by May 2020 [3].

Given the increasing use of the internet and electronic devices in the UK population, following our successful roll out of electronic capture of clinical data we would like to offer our trial participants the option of using an electronic system to complete PRO questionnaires.

Outside clinical trials there has been extensive work conducted across different medical specialities to establish intra-patient equivalence of paper and electronic PRO questionnaires and their validity for data collection. Muehlhausen et al [4] conducted a systematic review and meta-analysis of the equivalence of patient reported outcome measures administered by electronic and paper formats. The review included 72 intra-patient studies, showing overall equivalence between the two formats when completed by the same patients. This was an update of a previous review by Gwaltney et al conducted in 2008 [5], which also showed equivalence within patients. Following these two meta-analyses, further studies have added weight to the finding of within patient equivalence of data following migration from paper to electronic format. Participants of these studies had the equivalence of scores compared between their completion of both paper and electronic questionnaires, with a paper test-retest arm as the control [6, 7] .

Trial participants’ willingness to complete ePROs will be essential to maximise the completeness of data returned for planned clinical trial PRO analyses. One factor which may increase questionnaire return rates and improve the patient experience with ePROs in comparison to paper is a reduction in the amount of time required to complete questionnaires. Park et al showed within the clinical outpatient setting that time taken to complete an electronic questionnaire was significantly shorter than that required for the paper version [8]. Some studies have shown 83% compliance with ePROs in the clinic, with between 76 and 95% of patients finding a system usable and recommending it to others [9, 10]

There are two significant limitations of literature published to date. Firstly, although ePROs are becoming increasingly popular for use in clinical trials, there is limited evidence of patient uptake and compliance in this setting, and none from randomised studies of the mode of PRO completion. One study including rheumatoid arthritis patients within two randomised controlled trials asked patients to fill in electronic diaries. This study showed high compliance of up to 93% of patients over 12 weeks, however there was no control group completing paper questionnaires, meaning it is not possible to be sure whether the compliance was non-inferior to paper diaries [11]. Clinical trials in a surgical setting found poor uptake of ePROs amongst participants offered the choice. In a report of two trials from 2019 only 12% of 642 participants opted for completion of ePRO in one study and 34% of 1296 participants opted for it in another. Overall, 280 of 5700 expected questionnaires were completed electronically (5%), with the remainder completed on paper [12].

The second limitation is the lack of information about whether completeness of data is equivalent or superior in the electronic format. One recent study in a healthy university undergraduate population [13] assessed data capture in electronic and paper versions with participants of a prospective study being given the opportunity to choose the format for completion of food intake questionnaires at baseline and 10 year follow up. The results were mixed, with increased missing data in some subsections in the electronic version with improved data levels in other subsections. The study concluded that the number of questions correctly filled in was equivalent between electronic and paper questionnaires. However, these results may not be applicable in a patient population, particularly in oncology where patients can be unwell and are more likely to be an older cohort of the population [14]. There is a limited number of other studies but these again are largely in either the mental health, general healthy or paediatric population [15, 16] and therefore not directly applicable to oncology patient population.

Here we explore the considerations, potential challenges and benefits of ePRO implementation within the cancer clinical trial setting.

Some of the key considerations when implementing ePRO are the ethical and regulatory requirements of this method of data capture. Although the UK’s Medicines and Healthcare products Regulatory Agency (MHRA) are yet to issue specific guidance with respect to ePROs there have been critical findings in recent MHRA GCP inspections [17, 18] The GCP inspections metric report of 2018–2019 reported that “There was incorrect data in the eDiary that could not be changed, but was used for the analysis” and that “The eDiary devices used by subjects did not have an audit trail.” Similarly, in the 2017–2018 MHRA GCP inspections metric report there was concern about insufficient documentation of user acceptance testing for electronic patient diaries.

Outside the UK the FDA guidance for industry: Electronic Source Data in Clinical Investigations published in 2013 [19] noted that the subject of the PRO when electronic should be listed as the originator and the eCRF should be the source. Further guidance has not yet been published. The European Medicines Agency (EMA) published a “Reflection paper on expectations for electronic source data and data transcribed to electronic data collection tools in clinical trials”[20]. This highlighted the importance of an instrument being “an accurate representation of the protocol ensuring that the data … can be captured correctly and that the …subject response is not biased by default values present within the instrument” and that “The clarification process for data entered by trial subjects should be documented and it should be clearly stated where changes to data entered by subjects will not be made.”

Further ethical challenges need to be considered in the use of ePROs within oncology trials, particularly those in cancer sites occurring in older patient populations. Although internet use is becoming increasingly widespread, as discussed above, there remains a substantial proportion of over 75s who do not regularly use the internet. Although we wish to offer our trial participants the option to complete questionnaires online, we do not wish to exclude people from contributing to our PRO studies for lack of internet access or disinclination to complete questionnaires electronically. We therefore are planning to implement ePRO in parallel to our existing paper PRO collection systems in order that we do not disenfranchise any of our trial participants. This adds an additional layer of complexity to the implementation but we believe this is crucial not only in terms of avoiding the inadvertent introduction of bias, but also to ensure that possibly underrepresented groups can continue to contribute their experience.

We have developed an implementation plan to systematically introduce and assess use of ePROs into ICR-CTSU trials. Figure I outlines the key steps required for implementation and considerations for each step.

1. System Identification

A crucial element of the roll out of ePROs in our clinical trials will be the identification and implementation of a suitable database system. The first step in our implementation plan was to develop a full system requirements specification (table I). In brief, the system is required to have a user-friendly interface with an identification verification system and password management. Secure access, data storage and usability of data are paramount and it must comply with all applicable regulations.

The system currently used by ICR-CTSU for EDC did not have ePRO functionality therefore an alternative needed to be sought. Finding ePRO systems meeting the requirements which are also affordable to an academic trials unit has been challenging, with 17 systems having been assessed to date. A system which appears to meet the essential requirements has now been identified and is currently undergoing host institution review for information governance and information security prior to implementation.

2. Patient and public involvement

Patient and public involvement (PPI) in the design of health research is integral to ICR-CTSU’s work and can enhance enrolment and retention by improving trial design, recruitment and retention strategies and patient facing material [21]. As we are implementing a new system for use by our trial participants PPI is an integral part of this project.

We have developed a public survey investigating attitudes to completing health questionnaires online and capturing views on introducing epos. A key aim is to reach people with diverse demographics and a range of IT experience in order to obtain as broad and representative review of attitudes to ePRO as possible.

Following the completion of a survey small focus group sessions will be conducted with PPI representatives to discuss the acceptability and potential barriers to ePRO, the usability of the chosen system and the design of a pilot study of ePRO.

Participants of the focus group will be invited to join the pilot study steering committee to assist with study oversight and contribute to reviewing and disseminating outcomes.

3. SWAT/pilot study

Following consultation with our PPI partners we intend to conduct a randomised pilot study, in the form of a study within a trial (SWAT) to assess the feasibility of ePRO implementation within ICR-CTSU trials and obtain evidence to fill some of the gaps in the literature described above. The intention is to develop a study which can sit across multiple ICR-CTSU host trials to assess ePRO uptake, completion rates and impact on data reported across a number of patient populations.

At the ICR-CTSU we currently have extremely high return rates of paper PRO questionnaires. On review of 10 trials there was a median questionnaire return rate of 76% at the first post-trial intervention time point. In trials where the PRO was the primary endpoint this increased to > 90%. As such we will be aiming to assess whether ePRO completion rates are non-inferior in comparison to paper PRO. Additional aims of the SWAT are to develop ICR-CTSU procedures for ePRO collection, to develop patient supporting documentation (including system use guidance as needed) and acquire high quality data on the use of ePRO within clinical trials. Challenges include designing a SWAT that will have a sufficient sample size whilst retaining the option for participants to complete their questionnaire on paper to ensure that no PRO data is lost within the host trial and that results are not biased by being undertaken by only those who are able to complete ePROs.

Our SWAT, SPRUCE (A Study within a trial of electronic versus paper Patient Reported oUtcome CollEction), is in set up (REC:21/WM/0223). The initial study outline was developed with input from several PPI representatives and the design will be adjusted as needed to reflect the input of the PPI focus groups described above. It is currently anticipated that the study will be a partially randomised patient preference trial which will allow trial participants the option to choose to fill out the PRO questionnaire either on paper or electronically if they have a strong preference, preventing the inadvertent exclusion of participants and damage to QoL data capture within the host trials. Further details of the SWAT will be published in due course once the protocol is finalised.

Identification of a user friendly and workable system has proved to be a complex process requiring the review of multiple systems. The resource required to implement this system to ensure its acceptability to our trial participants will be substantial and requires a multi-disciplinary approach. The ICR-CTSU is fortunate to have a dedicated team of clinical trial IT and data management professionals embedded within the unit, as well as PPI and trials methodology expertise to allow the robust assessment of ePRO implementation within a randomised setting.

Implementation of ePRO even in a world of increasing technology use is a complex and multifactorial project requiring careful consideration and adequate resourcing to ensure that both end users and clinical trials units benefit from its introduction and no trial participants are inadvertently disenfranchised.

Ethics approval and consent to participate

Not applicable

Consent for publication

Not applicable

Availability of data and materials

Not applicable

Competing interests

The authors declare that they have no competing interests

Funding

Not applicable

Authors' contributions

All authors contributed to the work outlined in this commentary. The input of all authors was required to form the implementation plan. LP and RL drafted this manuscript with review from all authors.

Acknowledgements;

The ICR-CTSU is supported by a Cancer Research UK core grant (C1491/A25351). The patient and public involvement in study development is supported by NHS funding to the NIHR Biomedical Research Centre at The Royal Marsden and the ICR. Lara Philipps is supported in her PhD by a CRUK clinical trials fellowship (A30384) and the BRC. She is supported in her PhD by her supervisors including Professor Robert Huddart. This report represents independent research supported by the National Institute for Health Research (NIHR) Biomedical Research Centre at The Royal Marsden NHS Foundation Trust and the Institute of Cancer Research, London. The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care.

FDA. Guidance for Industry use in Medicinal product developement to support labeling claims. Clinical/medical Federal register; 2009.
ONS. Internet usage statistics. 2020 [cited 2021 September 15th]; Available from: https://www.ons.gov.uk/businessindustryandtrade/itandinternetindustry/datasets/internetusers.
Ofcom. 2021 [cited 2021 September 15th]; Available from: https://www.ofcom.org.uk/about-ofcom/latest/features-and-news/uk-internet-use-surges.
Muehlhausen W, et al. Equivalence of electronic and paper administration of patient-reported outcome measures: a systematic review and meta-analysis of studies conducted between 2007 and 2013. Health Qual Life Outcomes. 2015;13:167.
Gwaltney CJ, Shields AL, Shiffman S. Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review. Value Health. 2008;11(2):322–33.
Byrom B, et al. Measurement Equivalence of Patient-Reported Outcome Measure Response Scale Types Collected Using Bring Your Own Device Compared to Paper and a Provisioned Device: Results of a Randomized Equivalence Trial. Value Health. 2018;21(5):581–9.
Lundy JJ, et al., Agreement Among Paper and Electronic Modes of the EQ-5D-5L. Patient, 2020. 13(4): p. 435–443.
Park JY, et al. Comparison between an electronic version of the foot and ankle outcome score and the standard paper version: A randomized multicenter study. Med (Baltim). 2019;98(40):e17440.
Basch E, et al. Patient online self-reporting of toxicity symptoms during chemotherapy. J Clin Oncol. 2005;23(15):3552–61.
Ali FM, et al. Comparison of the paper-based and electronic versions of the Dermatology Life Quality Index: evidence of equivalence. Br J Dermatol. 2017;177(5):1306–15.
Bingham CO 3. Use of daily electronic patient-reported outcome (PRO) diaries in randomized controlled trials for rheumatoid arthritis: rationale and implementation. Trials. 2019;20(1):182. rd, et al, , ( .
Culiford L, Hopkins GE, Maishman E, Mazza R, Walker Smith G T. Do study participants complete electronic health questionnaires? in Trials. 2019. CTEU Bristol, BTC, Translational Health Sciences.
Zazpe I, et al. Paper-Based Versus Web-Based Versions of Self-Administered Questionnaires, Including Food-Frequency Questionnaires: Prospective Cohort Study. JMIR Public Health Surveill. 2019;5(4):e11997.
UK CR. 2021 [cited 2021 September 15th]; Available from: https://www.cancerresearchuk.org/about-cancer/causes-of-cancer/age-and-cancer.
Raat H, et al. Feasibility, reliability, and validity of adolescent health status measurement by the Child Health Questionnaire Child Form (CHQ-CF): internet administration compared with the standard paper version. Qual Life Res. 2007;16(4):675–85.
Richardson CG, et al. The Influence of Web- Versus Paper-based Formats on the Assessment of Tobacco Dependence: Evaluating the Measurement Invariance of the Dimensions of Tobacco Dependence Scale. Subst Abuse. 2009;3:1–14.
GCP inspections metrics report. 2018–2019 [cited 2021 September 15th]; Available from: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/961531/GCP_INSPECTIONS_METRICS_2018-2019_final_12-02-21.pdf.
GCP inspections metrics report. 2017–2018 [cited 2021 September 15th]; Available from: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/961531/GCP_INSPECTIONS_METRICS_2018-2019_final_12-02-21.pdf.
FDA, Electronic Source Data in Clinical Investigations: Guidence for industry. 2013.
Reflection paper on expectations for electronic source data and data transcribed to electronic data collection tools in clinical trials. 2010.
Crocker JC, et al. Impact of patient and public involvement on enrolment and retention in clinical trials: systematic review and meta-analysis. BMJ. 2018;363:k4738.

Table I. System requirements specification

Requirement	Status
Make the appropriate questionnaires available to the participant via a secure internet site and/or via a mobile application that could be downloaded onto the participant’s device(s).	Essential
Be compatible with various internet browsers	Essential
Be compatible with various mobile devices	Desirable
Be user friendly – easy to navigate and enter responses for trial participants	Essential
Be accessible for use 24/7	Essential
Allow the participant to change their password and provide a forgotten password facility	Desirable
Require two step confirmation of identity before the participant is allowed to proceed E.g. Username and password plus date of birth and initials.	Desirable
Allow questionnaires to be submitted only once	Essential
Remove previously submitted questionnaires from participants’ view	Essential
Remove access to uncompleted questionnaires after a specified time period has elapsed	Essential
Provide a scheduling facility Allowing clinical trial unit staff to view the participants’ schedule (all forms and visits required) Allow trial management staff to set the questionnaire due dates and reminder dates Allow disablement of future reminders and questionnaires if applicable by manual addition of milestones e.g. Upon death, withdrawal from study Allow the facility for participants to follow different schedules within the same study dependent upon participant attributes.	Essential
Allow disablement of future reminders and questionnaires if applicable by automatic addition of milestones e.g. Upon death, withdrawal from study using automatically imported data from other systems e.g. EDC	Desirable
Provide an automatic electronic reminder facility allowing reminders to be sent to participants when questionnaires are due to be completed	Essential
Provide validated compliance reports for Overdue questionnaires (not started) Incomplete questionnaires (started but not submitted/completed) Submitted/completed questionnaires OR the ability to create the above reports. Reports should be able to display the above items by centre, patient, visit and questionnaire type	Essential
Allow creation of bespoke ePRO questionnaires	Essential
Provide pre-validated ePRO questionnaire/instruments	Desirable
Provide capacity to manage paper-based questionnaires within the same system	Desirable
Allow clinical trial unit staff to schedule automatic electronic participant health checks to be sent to site contacts prior to questionnaire reminders being sent to participants	Desirable
Provide a system API to allow new participants to be created automatically in the ePRO system at trial randomisation using the ICR-CTSU Randomisation system	Essential or alternative method provided
Be approved for use by the ICR-IT IT Security Manager, Enterprise Architecture and Information Governance	Essential
Provide appropriate evidence of business continuity, disaster recovery and back up schedules	Essential
Provide appropriate evidence of systems validation	Essential
Hold all data securely in accordance with GCP and general data protection regulatory (GDPR) requirements	Essential
Provide data in a readable format for statistical analysis i.e. Excel, csv or other input readable by statistical programs e.g. Stata. Preference would be for one row per patient per visit.	Essential
Have been GCP validated	Essential

PDF

Reviews received at journal
21 Apr, 2022
Reviewers invited by journal
12 Apr, 2022
Editor assigned by journal
04 Mar, 2022
First submitted to journal
02 Mar, 2022

You are reading this latest preprint version

Implementing electronic patient reported outcome data capture for multi-centre oncology clinical trials

Status:

Version 1

Abstract

Figures

Background

Ethical And Regulatory Requirements

Implementation Plan

1. System Identification

2. Patient and public involvement

3. SWAT/pilot study

Conclusion

Statements And Declarations

References

Table

Status:

Version 1