Letter on “Sharing trial results directly with trial participants and other stakeholders after the SARS-CoV-2 pandemic hit the UK – experience from the ActWELL trial”

doi:10.21203/rs.3.rs-317781/v1

Download PDF

Letter

Letter on “Sharing trial results directly with trial participants and other stakeholders after the SARS-CoV-2 pandemic hit the UK – experience from the ActWELL trial”

https://doi.org/10.21203/rs.3.rs-317781/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

After the SARS-CoV-2 pandemic took hold in the UK the ActWELL trial team’s plans to present the trial results to participants and other stakeholders had to change. Instead of face-face events, three online events were planned and hosted successfully. In this article we describe the choices made in planning and organisation of the online events including things we would do differently if we were to do it again. We think that online events are a useful platform when informing participants and other stakeholders of the results of your trial, even beyond the SARS-CoV-2 pandemic and we hope this article can help other trial teams to plan their own online events.

Infectious Diseases

Drug Discovery, Design, & Development

Epidemiology

Randomised trials

Research Transparency

Results Dissemination

Trial Conduct

Online Events

COVID-19

In 2015 the United Kingdom’s (UK) Health Research Authority (HRA), whose core purpose is ‘to protect and promote the interests of patients and public in health and social care research’, published guidance¹ recommending that all researchers offer trial results to trial participants. Following this guidance, the ActWELL (ISRCTN11057518)² trial team planned to share the trial’s results with participants. The ActWELL trial results were published in March 2021³.

The ActWELL multi-centre RCT evaluated the effect of a theory based lifestyle intervention (delivered via two, face-to-face visits and up to 9 telephone calls with a BCN lifestyle coach) on changes in weight and physical activity over a one year period. Women were offered the chance to participate when they attended their routine National Health Service (NHS) Breast Screening appointment in four Scottish health boards. The intervention itself was delivered by Breast Cancer Now (BCN) volunteer lifestyle coaches who were recruited and managed by the charity.

As the trial drew to a close, a researcher was appointed in July 2019 to work on a dissemination package, which included preparing written summaries of results and organising face-to-face dissemination events in each of the four Scottish Health Boards where recruitment had taken place. The SARS-CoV-2 pandemic changed these plans.

Here we present the practical considerations we faced when switching from face-to-face meetings to three online events for feeding back trial results to participants and other stakeholders during the SARS-CoV-2 pandemic. We also consider aspects we would likely do differently next time based on attendee feedback.

We hope the article will provide a practical guide to conducting online dissemination events for trial participants, an activity that will remain important beyond the SARS-CoV-2 pandemic.

Purpose of the events

The purpose of the online events was two-fold:

1) To inform trial participants and other stakeholders of the trial findings

2) To enable attendees to ask questions about any aspect of the trial, its results and implications arising from the findings

Event programme

The programme for the events was the same for all three events and each event lasted for about 90 minutes.

Running of events

We decided to plan three events with the intention of having more if the dates we proposed left many participants still unable to attend. The events were arranged for different days and different times– one lunchtime (12.30–14.00) and two evenings (18.30–20.00).

Each event was limited to 100 attendees in order to allow the sessions to be interactive and manageable from the trial team’s perspective. Registration and waiting lists were managed through Eventbrite (https://www.eventbrite.co.uk/), which is free to use for free events such as ours. The Eventbrite event page could be kept private so only people with a link were able to register.

Stakeholders/attendees

All women taking part in the ActWELL trial and who had not withdrawn or moved away from Scotland were offered an invitation to the online events. Other stakeholders including BCN lifestyle coaches, breast screening centre staff and leisure centre staff were also identified by the trial team and invited by email. Email addresses were available for most trial participants, those without an email address were sent postal invitations.

Invitations and permissions

One invitation was designed for all stakeholders and for both email and postal distribution (see supplementary file 2).

Dissemination of trial results to participants had been included in the initial NHS REC review. Anyone interested in attending was asked to register for the event on Eventbrite.

Platform and event management

Two hosting platforms were considered for the events: Microsoft Teams (Meetings/Live; https://www.microsoft.com/en-gb/microsoft-teams/group-chat-software) and Zoom (Large meeting/Webinar; https://www.zoom.us). The platform had to enable an interactive session and we chose the Zoom Large Meeting add-on. Many members of the public are now familiar with Zoom, there is evidence that research participants prefer it to other platforms⁴ and with the large meeting add-on it was possible to let attendees ask questions both via chat and audio. Further benefits of using Zoom were that it integrates well with Eventbrite and enables you to communicate with your registered attendees, as well as send out an Evaluation survey using Survey Monkey (https://www.surveymonkey.co.uk). There was some initial resistance from our institution (the University of Aberdeen) to give approval for Zoom because its institutional platform is Teams and because of data security fears. However, concern about Zoom security fears in October 2020 were reduced compared to earlier in 2020, especially with Zoom’s use of passwords and Waiting room. Evidence that research participants preferred Zoom was also persuasive. Permission to use Zoom for these events was therefore granted by the University.

Event management and technical assistance was hired for the events so that the team had someone who was very familiar with the platform and who was able to ‘produce’ the event and help it run smoothly as wells to record it (https://stauntonmedia.ie/). The sessions were recorded mainly for internal learning purposes and with a view to possibly make it available to participants who could not attend events if requested. Otter.ai (https://otter.ai) was used for live captioning of the events as it integrates with Zoom to provide captions in a separate browser window.

Event management also provided help for attendees with joining and other technical issues, which meant they could contact a member of the team behind the scenes via email.

Evaluation

To get feedback from attendees and to help us make suggestions for improvements for future events, we sent attendees an evaluation survey after the event (see supplementary file 3). The survey included seven questions with a mixture of closed and open-ended text responses.

Invitation distribution was therefore managed through HIC, which charged the trial team to do this as it was not part of the original contract between the trial team and HIC. Other stakeholders were sent an invitation via email by members of the trial team.

Table 1

Stakeholder groups and the numbers invited, registered to attend, attending the events and responding to the event evaluation survey.
Stakeholder group	Invited	Registered (%)	Attended (% of those registered)	Responded to evaluation survey (% of those attending)
Trial Participants [1]	438	79 (18)	53 (67)	30 (56)
Breast Cancer Now (BCN)coaches	45	22 (49)	21 (95)	12 (57)
NHS staff [2]	12	8 (67)	8 (100)	3 (38)
Leisure Centre staff	8	2 (25)	2 (100)	1 (50)
BCN staff	5	5 (100)	4 (80)	0
Public representatives	5	1 (20)	1 (100)	1 (100)
Funder staff	2	0	0	0
Trial team	3	0	0	0
Total	519	117 (23)	89	47
[1] All participants within ActWell who are alive, have not withdrawn, have not left Scotland and have the status “Completed Study”.
[2] Twelve NHS staff members were invited directly (investigators and their team) while staff at breast cancer screening centres were sent invitations via a generic email address and the number of recipients is therefore unknown.

The invitations were emailed (n = 400) and posted (n = 38) out 15 days before the first event. We were fortunate that most participants (91%) had provided their email address and by proxy could be expected to have internet access. Out of 519 stakeholders invited a total of 89 attended of which 53 were trial participants and 21 BCN lifestyle coaches. The lunchtime event was the most popular with highest number of registrations (n = 50), including most NHS Staff attending and a total of 39 attendees.

A number of attendees were in contact via email before and after events. One of the trial participant attendees sent a list of questions providing us with a useful set of questions to start off each Q&A session. Other email contacts were mainly about how to access the results other than at the events, issues around joining the events and thanking the trial team for holding the events. Demand was met with the three events planned and therefore no further events were scheduled.

Experience of the event

All three events were delivered as planned and there were no technical problems. All three were delivered as per the program shown in [Supplementary 1]. All three started and finished on time.

The majority of survey respondents indicated that they had had all their questions about the ActWELL study and its findings answered. Five of the 47 attendees who responded said that their questions had not been answered and these questions were around future plans for the rollout of the intervention, which the team could not yet answer (n = 2); making the intervention more accessible to all socio-economic groups (n = 1); specific risk (n = 2). In addition, one attendee also requested access to the paper when published in their response.

All survey respondents indicated that the information was presented in a way that was easy to understand (n = 47). One did add a comment that there was not enough time to look at the results on the slides.

All responses to the survey are summarised in [Supplementary file 4].

From our perspective as speakers (ASA, AH, JM and ST), the online events were a great success and hugely rewarding.

We were asked many excellent and probing questions in over 90 minutes of questioning across the three events and these both made us think and gave us confidence that the ActWELL intervention⁵ is worth pursuing. It is hard to now imagine not including online dissemination events in future trial dissemination plans.

Costs

The total cost for the three events was £1,624 GBP (€1,842 EUR; $2,220 USD) with trial participant invitations and event management and technical assistance at the events being the highest expenditure items (see supplementary file 5). Staff hours spent planning and organising the events have been counted separately (see supplementary file 5) as cost depends on local circumstances. A total of 40 hours, over two months, were spent by the research team on preparing the events. Having now done these events, we would expect somewhat less time to be needed for future events.

There is clearly an appetite among trial participants and other stakeholders for events of this type and attendance after registration was better than is usual (in personal experience) for other academic online events. Overall, all the events were well received and there are only a few things we might do differently next time.

Although the evaluation was overwhelmingly positive, we think it would have been useful to include Patient and Public Involvement (PPI) partners in all of the planning process for online events. Unfortunately, none of the PPI partners who had been involved in ActWELL were available for ours and it did not seem fair to expect someone new to the project to get involved this late in the trial and under considerable time pressure.

For future events we would consider changing the programme to give a little more focus on the results themselves and less on the background to the trial. We would also consider offering a slightly later event start for at least one event, so 19.00 rather than 18.30 start. Considering the number of professional stakeholders who attended the events, also in their own time, we would in future consider offering a ‘quick’ lunchtime session specifically aimed at them. Alternatively, by providing more timing detail in the joining instructions, attendees could choose to join the event at the point they were most interested in (e.g. when the results were presented) if they were less interested in other parts of the session.

We would recommend use of an event management and technical assistance company, or your own organisation’s event management staff if available. Having professional support completely removed any concern about the smooth running of the event and allowed speakers to concentrate entirely on presenting and discussing the trial with attendees. The cost of £770 was modest when compared to the cost of organising three face-to-face events (e.g. the original main face-to-face event was costed at £8000 (€9252 EUR; $11,154 USD) in total including travel for study team and attendees, overnight accommodation for the study team, venue and catering during the event).

After several queries from participants, we will now offer the results in three different modes, not just the two that were planned – dissemination events; written plain English summary and now also a recording of dissemination events. All three will be available on a trial specific website, which we will tell participants about once it becomes live (www.actwellstudy.org).

Providing trial results direct to trial stakeholders online was a success and should also be considered post-pandemic. Face-to-face events will still be important but we suggest that online events should always be offered because they remove the need to travel (which may be a problem for some trial participants), are easily recorded, and are efficient in terms of time and cost. However, trial teams will have to consider how to communicate with and disseminate to stakeholders who do not have internet access, or who have access but are less familiar with internet-based tools.

Lastly, there is a considerable workload associated with planning dissemination events, whether face-to-face or online, which should not be underestimated. We hope this article will give pointers as to what trial teams need to plan and budget for when designing their trials. This work is worth it.

We will give the last word to a participant from the ACTWELL comparison group who attended the online event of 26th November 2020:

“It was tremendous to have the Feedback Day, far beyond what I would have expected (so much more than an A4 sheet mailed out or even less). It was also a valuable reminder to me as to why I should maintain the changes and some of the techniques for achieving them.” (S. MacAskill, with permission).

ActWELL – Working together to support ACTive living and WELL being (ActWell) in the health promoting health service - feasibility trial to reduce breast cancer risk factors.

BCN – Breast Cancer Now

HRA – Health Research Authority

RCT – randomised controlled trial

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Availability of data and materials

Data sharing is not applicable to this article as no datasets were generated or analysed during the current study.

Competing interests

The authors declare that they have no competing interests.

Funding

This work was supported by The Scottish Government, grant number BC/Screening/17/01.

Authors' contributions

HB and ST conceived the initial design of the online events and both, especially HB, were involved in the practical aspects of putting together the events. All authors approved the operational side of the events, designed the event program, took part in technical tests and spoke at all three events (ASA, AH, JM and ST) or provided technical and organisation support (HB). HB wrote the first draft of this paper and all authors subsequently contributed to it. All authors approved the final version.

Acknowledgements

We would like to thank our ActWELL participants who attended the events for their interest and enthusiasm in making the events a success. We would like to thank Susan MacAskill for giving us permission to use a quote from her email to us. We are also grateful to the technical support who got us through all events, Cormac Staunton (of stauntonmedia.ie). We would also like to acknowledge and thank Dr Katie Gillies and Dr Seonaidh Cotton for their contribution to the design of the evaluation survey. HSRU is core funded by the Chief Scientist Office of the Scottish Government Health and Social Care Directorates.

Health Research Authority. Publishing your research findings. 2019. Available: https://www.hra.nhs.uk/planning-and-improving-research/research-planning/publishing-your-research-findings/ [accessed 21.05.2020].
Anderson AS, Craigie AM, Gallant S, et al. Randomised controlled trial to assess the impact of a lifestyle intervention (ActWELL) in women invited to NHS breast screening. BMJ Open. 2018;8:e024136. doi:10.1136/bmjopen-2018-024136.
Anderson AS, Chong HY, Craigie AM, et al. A novel approach to increasing community capacity for weight management a volunteer-delivered programme (ActWELL) initiated within breast screening clinics: a randomised controlled trial. Int J Behav Nutr Phys Act. 2021;18:34. https://doi.org/10.1186/s12966-021-01099-7.
Archibald MM, Ambagtsheer RC, Casey MG, Lawless M. Using Zoom Videoconferencing for Qualitative Data Collection: Perceptions and Experiences of Researchers and Participants. Int J Qual Methods. 2019 Sep;11;18:160940691987459–8.
Anderson AS, Craigie AM, Gallant S, McAdam C, Macaskill EJ, McKell J, et al. Optimisation of the ActWELL lifestyle intervention programme for women attending routine NHS breast screening clinics. Trials; 2020 Jun 3;21(1):1–12.

Download PDF

Editorial decision: Minor revision
22 Apr, 2021
Review #3 received at journal
10 Apr, 2021
Reviewer #3 agreed at journal
26 Mar, 2021
Reviewer #2 agreed at journal
24 Mar, 2021
Reviewer #1 agreed at journal
24 Mar, 2021
Review #1 received at journal
24 Mar, 2021
Review #2 received at journal
24 Mar, 2021
Reviews received at journal
24 Mar, 2021
Reviewers invited by journal
23 Mar, 2021
Editor assigned by journal
18 Mar, 2021
Submission checks completed at journal
18 Mar, 2021
First submitted to journal
10 Mar, 2021

You are reading this latest preprint version

Letter on “Sharing trial results directly with trial participants and other stakeholders after the SARS-CoV-2 pandemic hit the UK – experience from the ActWELL trial”

Status:

Version 1

Abstract

Background

Approach

Implementation

Recommendations

Summary

List Of Abbreviations

Declarations

References

Supplementary Files

Status:

Version 1