Implementing Germ Defence digital behaviour change intervention via all primary care practices in England to reduce respiratory infections during the COVID-19 pandemic: an e�cient cluster randomised controlled trial using the OpenSAFELY platform.

Background: Germ Defence (www.germdefence.org) is an evidence-based interactive website that promotes behaviour change for infection control within households. To maximise the potential of Germ Defence to effectively reduce the spread of COVID-19 the intervention needed to be implemented at scale rapidly. Methods: With the approval of NHS England, we implemented an e�cient two-arm (1:1 ratio) cluster randomised controlled trial (RCT) to examine the effectiveness of randomising implementation of Germ Defence via GP practices across England, UK, compared with usual care. GP practices randomised to the intervention arm (n=3292) were emailed and asked to disseminate the Germ Defence intervention to all adult patients via mobile phone text, email or social media. GP practices randomised to the usual care arm (n=3287) maintained standard management for the 4-month trial period after and then asked to share Germ Defence with their adult patients. The primary outcome was the rate of GP presentations for respiratory tract infections (RTI) per patient. Secondary outcomes comprised rates of acute RTIs, con�rmed COVID-19 diagnoses, suspected COVID-19 diagnoses, COVID-19 symptoms, gastrointestinal infection diagnoses, antibiotic usage, hospital admissions. The impact of the intervention on outcome rates was assessed using negative binomial regression modelling within the OpenSAFELY platform. The uptake of intervention by GP practice, and by patients were measured via website analytics. Results: Germ Defence was used 310,731 times. The average satisfaction score after using the website was 7.52 (0-10 not at all to very satis�ed, N = 9933). There was no evidence of a difference in the rate of RTIs between intervention and control practices (rate ratio (RR) 1.01, 95%CI 0.96, 1.06, p=0.70). This was similar to all other eight health outcomes. Patient engagement within intervention arm practices ranged from 0-48% of a practice list. Practices with high levels of intervention uptake (>11%) had a lower proportion of minority ethnic groups. Conclusions: We demonstrated that rapid large-scale implementation of a digital behavioural intervention can be evaluated with a novel e�cient prospective RCT methodology analysing


Page 4/22
The COVID-19 pandemic has highlighted the importance of behavioural strategies for controlling infection transmission. Effective implementation of good hygiene practices as public health measures (including social distancing, self-isolation, handwashing, mask wearing and ventilation) was vital to control the spread until a vaccine was developed (1).
Supporting behaviour change is a complex process that requires an in-depth understanding of why people do/do not engage in target behaviours (2), and tailored support to facilitate engagement. In line with this, there have been calls for major research to develop and evaluate behavioural, environmental, social and systems interventions (3). Shortly before the H1N1 pandemic, 'Germ Defence' was developed to reduce virus transmission within homes. Germ Defence is a digital behaviour change intervention that provides accessible, tailored advice using behaviour change techniques to improve infection control behaviours such as handwashing, social distancing and ventilation in the home.
Reducing within-household transmission pathways is important in contexts where inter-household contact is reduced (i.e. during lockdowns and self-isolation (4,5)) as well as within more freely mixing circumstances where infectious individuals can expose cohabitants inadvertently to high virus levels leading to increased risk of infection and possibly more severe disease (6). In a previous trial of Germ Defence, 20,066 adults from UK primary care practices were randomised to be given access to Germ Defence reported fewer respiratory tract infections (mean 0.84 vs 1.09 in control group), fewer family member infections and less severe infections (7); primary consultations and antibiotic prescriptions were also signi cantly reduced in the intervention group.
Rapid co-participatory methods were used to update and optimise Germ Defence for the COVID-19 pandemic https://www.germdefence.org/ (8). After initial dissemination via clinical and public health networks, social media and press coverage, an analysis of 28,825 users showed an effect size on intentions to improve infection control behaviours similar to that observed in the original trial, and recommended wider promotion through primary care and public health networks (9). It is vital that such digital behaviour change interventions can be implemented effectively, rapidly and at scale. Therefore, we aimed to examine whether the Germ Defence intervention: 1. could be disseminated to patients via GP practices Historically, NHS (National Health Service) electronic health record data has been accessed by researchers via a process of pseudonymisation (replacing explicit identi ers such as name and address with a pseudonymous identi er) followed by dissemination of a subset of patients' records for local analysis. Recently published UK (United Kingdom) Department of Health and Social Care policy (10) states that detailed electronic health records data should instead be analysed within a secure Trusted Research Environment (TRE). Concerns have been raised that TREs are not suitable for following up patients in an RCT (11).
We used a novel e cient trial design to evaluate the effectiveness of implementing Germ Defence through all GP practices in England being asked to send the intervention to their adult patients. The e cient trial design meant that no GP practice or patient recruitment was required, and GP practices were not required to send the research team any data, with all outcomes assessed using anonymous patient record data in situ via the OpenSAFELY platform trusted research environment and website analytics.

Design
As detailed in our protocol (12), this was an e cient pragmatic two-arm (1:1 ratio intervention versus usual care) cluster randomised trial, disseminating Germ Defence to all GP practices in England to reduce respiratory tract infections (RTI). Randomisation was conducted by the independent Bristol Trials Centre (BTC). The 133 NHS Clinical Commission Groups (CCGs: NHS bodies responsible for the planning health care services for their local area) in England were divided into blocks according to region, and equal numbers in each block were randomly allocated to intervention or usual care. The randomisation schedule was generated in Stata statistical software by a BTC statistician not otherwise involved in the enrolment of general practices into the study. The principal investigators, the study statistician and research team remained blinded to the identity of randomised practices until the end of the study.

Setting and participants
All GP practices in England registered with NHS Digital (N = 6579) were included to ensure that the intervention was rolled out across demographically and geographically diverse regions.

Sample size considerations
To detect a relative risk reduction of 0.14 with 90% power (alpha 0.05), based on the previous Germ Defence implementation PRIMIT trial (7) was calculated to require 11,124,176 participants from approximately 1,484 practices (accounting for clustering). We randomised all GP practices in England, aiming for at least 25% of GP practices of those contacted successfully disseminating the intervention to their patients.

Intervention
Germ Defence (https://www.germdefence.org/)content was rapidly adapted throughout the pandemic using state-of-the-art evidence, theory and the person-based approach (13), in order to ensure the advice remained up to date and appropriate. Content, design and structure were iteratively optimised via co-participatory approaches with the general public in order to ensure the intervention was as accessible, credible and motivating as possible (8). On the rst page participants reached, they could access content in 25 languages as well as infographics (which were also translated into other languages) that they could share with people who were not able to access digital content.
The single session intervention sought to improve users' awareness of risks of infection and transmission, increase skills and con dence to reduce risks and use behaviour change techniques (such as making if-then plans) to support behaviours. The Germ Defence content was tailored such that a user selected one of four streams that was relevant to the user's situation: 1. to protect themselves generally.
2. to protect others if the user was showing symptoms.
3. to protect themselves if household member(s) showed symptoms. 4. to protect a household member who is at high risk.
Clear and detailed advice was provided for self-isolating, social distancing, cleaning, wearing face-coverings, ventilation and handwashing.

Intervention Implementation
An initial email to practices was drafted by the research team that contained a unique weblink to the Germ Defence website and asked practices to disseminate this to all their adult patients (aged 16+) via mobile phone text, email or social media. This email was iteratively optimised in pilot interviews with nurses, GPs and administrative staff from 6 practices to ensure it was acceptable and engaging. Reasons that practices might not engage were discussed (e.g. not enough time, did not perceive bene ts, did not typically engage in research, concerns around privacy) and email content was re ned to address these barriers.
To further support engagement, the email linked to a trial information website that addressed key concerns and frequently asked questions in more detail (14). Practices did not need be a research active practice or to sign up to take part in the study; the practice-unique Germ Defence weblink allowed the study team to detect their involvement once patients accessed the Germ Defence intervention. The email also contained suggested text for patient mobile phone message and email. This was also made available in Bengali, French, Polish, Portuguese, Punjabi and Urdu.
On 10 November 2020, intervention arm practices were emailed (see Supplementary File 1) with a practice speci c weblink to the Germ Defence website and asked to disseminate this to all their adult patients (aged 16+) via mobile phone text, email or social media. Two reminder emails were sent on 25 November and 10 December 2020 to intervention arm practices (see Supplementary File 2).
Data suggest that 16% of the GP practice email addresses forwarded by NHS Digital to the study team did not work, with a total of 613 'undelivered' e-mails recorded in response to Germ Defence's initial approach to intervention practices in England. This was usually because registered email addresses were out of date. During the intervention delivery phase, all invalid email addresses were investigated further via a series of manual internet searches and telephone calls to practices, replacing invalid emails with new information as appropriate. This follow-up effort improved the data quality by around a third.
Patients at GP practices randomised to the usual care arm received standard management for the 4-month (17 week) trial period. On 10 March 2021, usual care arm practices were emailed a generic weblink to Germ Defence and asked to disseminate it to all their adult patients.

Measures and outcomes
The primary outcome was the rate of GP presentations for respiratory tract infections (RTI) per registered patient. Secondary outcomes comprised the rates of acute RTIs, COVID-19 diagnoses, COVID-19 symptoms, gastrointestinal infection diagnoses, antibiotic prescriptions and hospital admissions. COVID-19 symptoms were de ned using two different code lists, one designed for high sensitivity and the other for high speci city.
Each outcome was de ned using SNOMED-CT codelists (see Supplementary le 3 and github repository). A consultation for a speci c outcome was identi ed if a patient had a code from the codelist recorded on a given day. If a patient had multiple codes from the same codelist on the same day, this was counted as one consultation. The number of such consultations divided by the number of patients formed the consultation rate.
All health outcomes were analysed using routinely recorded clinical and patient information in GP practice data.
All data were linked, stored and analysed securely within the OpenSAFELY platform, https://opensafely.org/, a trusted research environment (TRE) enabling secure, transparent analysis of electronic health records. Data included pseudonymised elds such as coded diagnoses, medications, physiological parameters, patient age, patient ethnicity and deprivation score of the practice area. No free text data were included. All code used in this study is shared openly for review and re-use under MIT open license: https://github.com/opensafely/GermDefence. Detailed pseudonymised patient data is potentially re-identi able and therefore not shared. Primary care records managed by the GP software provider, TPP, were linked to Admitted Patient Care (APC) data through OpenSAFELY. Practice allocations were ingested into the OpenSAFELY platform and linked to pseudonymised practice IDs by TPP and made accessible to the study team by OpenSAFELY A further secondary outcome, uptake of the intervention by GP practices, was monitored using embedded code in a unique Germ Defence website link given to each practice. When practices communicated the unique weblink to their patients, the study team were able to record usage of the weblink. Uptake was measured using website analytics such as number of users per practice, average time spent on the Germ Defence website and pages visited, monitored using Matomo to ensure privacy (15). (13). In line with MRC (Medical Research Council) Guidelines for evaluating complex interventions (16) we also sought to understand mechanisms of action by aggregating individual self-report measures of infection control behaviours (social distancing, selfisolation, wearing masks, handwashing, cleaning/disinfecting, ventilation) collected by the Germ Defence website, and combined this with metrics of engagement with key intervention behavioural components (e.g. pages viewed, amount of time spent on intervention).

Patient and public involvement
Patient and public involvement (PPI) feedback was a key part of the co-participatory approach of the development of Germ Defence, in which members of the public were invited to feed back about the website and study in order to optimise and update it. A public contributor (CR) was a co-investigator on the study team, and contributed to writing the research proposal, updating and optimising the content of the intervention (including optimising intervention communications sent to patients by practices) and co-authoring the papers. Study

Data analysis
Summary of baseline data This cluster randomised controlled trial was analysed at the practice level. Randomisation was carried out at practice level and we did not have direct feedback on whether practices distributed the Germ Defence information to all, some or potentially no patients, nor whether individual patients were offered the information and made use of it. We, therefore, conducted all analyses using aggregated data at the practice level and considered each practice as a unit for the purpose of analysis. Outcome and covariate data were aggregated in weekly time-series prior to analysis, covering the period from 17 weeks prior to randomisation until 17 weeks after randomisation (14th July 2020 to 15th March 2021) to achieve a target minimum of 15% infection rate.
Primary care data in the OpenSAFELY system at the time of analysis represented approximately 40% of practices in England (17). Analyses of health outcomes were applied to 2498 practices.

Intention-to-treat analyses
The primary analysis used a standard intention-to-treat approach. For each of the eight health outcomes, rates of consultations per registered patient were compared at practice level between intervention and control groups for the 17-week post-intervention period. This was done using negative binomial regression with the consultation count as the outcome, the number of registered patients as the offset and the binary indicator of intervention/control group as the only independent variable.

Controlled interrupted time series analyses
An additional analysis was performed for the same eight health outcomes using a controlled interrupted time series (CITS) approach to understand temporal changes related to the intervention as distinct from the timeagnostic intention-to-treat approach. This was implemented within a generalised linear mixed modelling framework by applying negative binomial regression to weekly-level data spanning pre-and post-intervention periods for both the intervention and control groups. Data was also disaggregated by practice, allowing random intercepts at practice level. Variables included in the model were: consecutively numbered weeks to capture a log-linear trend, intervention-control indicator, pre-post intervention indicator and all two-and three-way interactions between these. Additional covariates included calendar month to capture seasonal effects and practice-level indicators such as area-level socioeconomic status, median patient age and sex distribution represented as the proportion of females.

Process analysis
Implementation process Germ Defence website usage recorded from the unique identifying website links sent by each practice was used to examine whether intervention engagement (i.e. a practice effectively communicating link to patients) was predicted by practice characteristics (such as indices of deprivation, NHS Quality and Outcomes Frameworks).

Individual intervention usage
A range of additional behavioural mechanisms, overall patterns of practice and user engagement were described using website analytics. Analytics included number of users per practice, average time spent on the Germ Defence website and pages visited.

Association with health outcomes
To understand the mechanisms of action in the intervention, we examined the association between the rate of website usage within a practice (number of users divided by number of registered patients) and the rate of each health outcome (consultations per registered patient). A negative binomial model was applied to practice level data and the association of interest was adjusted for decile of deprivation, proportion of patients from an ethnic minority and median age. This was done for all practices and then separately for a subset of practices that had greater than one percent uptake.
Information governance and ethical approval NHS England is the data controller for OpenSAFELY-TPP; TPP is the data processor; all study authors using OpenSAFELY have the approval of NHS England. This implementation of OpenSAFELY is hosted within the TPP environment which is accredited to the ISO 27001 information security standard and is NHS IG (Information Governance) Toolkit compliant (18,19). Patient data has been pseudonymised for analysis and linkage using industry standard cryptographic hashing techniques; all pseudonymised datasets transmitted for linkage onto OpenSAFELY are encrypted; access to the platform is via a virtual private network (VPN) connection, restricted to a small group of researchers; the researchers hold contracts with NHS England and only access the platform to initiate database queries and statistical models; all database activity is logged; only aggregate statistical outputs leave the platform environment following best practice for anonymisation of results such as statistical disclosure control for low cell counts (20). to require organisations to process con dential patient information for the purposes of protecting public health, providing healthcare services to the public and monitoring and managing the COVID-19 outbreak and incidents of exposure; this sets aside the requirement for patient consent (21). This was extended in November 2022 for the NHS England OpenSAFELY COVID-19 research platform. In some cases of data sharing, the common law duty of con dence is met using, for example, patient consent or support from the Health Research Authority Con dentiality Advisory Group (22).
Taken together, these provide the legal bases to link patient datasets on the OpenSAFELY platform. GP practices, from which the primary care data are obtained, are required to share relevant health information to support the public health response to the pandemic and have been informed of the OpenSAFELY analytics platform.

Results
The initial 10 November 2020 email to 3,292 intervention arm practices from 133 CCGs reached 2,679 GP practices. The subsequent reminder emails on 25 November and 10 December 2020 reached 2,870 GP practices ( Fig. 1).
Data show that the Germ Defence website was viewed by patients from 16% of the intervention arm general practices approached as part of the trial. This is based on analysis of website analytics for the usage data which suggest that 10 + clicks were registered for 459 of the 2,870 general practices offered the intervention.
A full consort diagram is presented in Fig. 1.

Intention-to-treat analysis
Using data available within OpenSAFELY, we assessed health outcomes in 1246 intervention practices and 1252 control practices, representing 11.9 million and 12.3 million registered patients, respectively. There was no evidence of a difference in the rate of RTIs (the primary outcome) between intervention and control practices (rate ratio (RR) 1.01, 95%CI 0.96, 1.06, p = 0.70) ( Table 1). This was similarly the case for all other health outcomes, where rate ratios ranged from 0.98 to 1.11 but with no evidence of a difference for any outcome with p-values ranging from 0.15 to 0.92. Table 1 Intention-to-treat analysis: Comparison of consultation rates between intervention and control groups for eight health outcomes.  Table 2). Although rates did uctuate over time due to the covid-infection spikes, seasonal variations and other factors, any such changes affected both intervention and control practices similarly on average. While there was some evidence of an intervention-related trend change for Covid diagnoses (p = 0.02), in the context of the many p-values in the table there is considerable potential for this to be a false positive. Defence website) to engagement from 48% of a practice list. There was no association between practice list size, and proportion of uptake (r = 0.00, p = .998). Full details of practice engagement are in Table 3.

Individual intervention usage
The trial intervention link was used 310,731 times, of which 163,991 'bounced' i.e. did not engage beyond rst the page. Access to the Germ Defence website using the generic ('non-trial') link also increased substantially during the trial period. This is likely due to trial users sharing the website (e.g. with their family or via social media), but these additional users were not included in our analysis. 97.29% (298,752) of users on the trial website were from patients who had been sent a text message from an intervention arm GP practice, with remaining visits via practice social media or GP websites.
Average satisfaction score after using the website was 7.52 (0 meaning not at all satis ed, 10 meaning very satis ed, N = 9933). The mean number of page views was 5.2 (standard deviation (SD)7.2). In 'engaged' users who did not bounce, the mean page view was 9.8 (SD 8.4), which included all key content targeted at reducing transmission.
While using the intervention, users reported their intentions to improve all infection prevention behaviours

Practice-level usage and health outcomes
There was no clear evidence of an association between website usage rates and health outcomes either among all intervention practices or among those with a user rate greater than one percent (Table 4). While there was modest evidence of higher usage rates among those with covid symptoms when looking at all practices, this effect direction was reversed for practices where over 1% of patients used the website. Note: Rate ratios indicate change in consultation rate for every 10% increase in user rate. Results are shown separately for 1) all intervention practices, and 2) for a subset where the user rate was greater than 1% of patients in a practice. All estimates were adjusted for median age, deprivation percentile and the proportion of patients from an ethnic minority.
Place here - Table 4. Process evaluation E cient trial design and intervention implementation The trial was endorsed by Chris Whitty, the then Chief Medical O cer (CMO) for England, as a national priority project and adopted by the CRN as an Urgent Public Health portfolio study. These endorsements facilitated the novel design by allowing access to email addresses of all GP practices via NHS Digital. However, despite extensive piloting of the process by which practices and patients could be contacted, several practical barriers were encountered: i) some email addresses were 'inactive' due to organisational name changes or practice closure, ii) practices expected to be contacted by CRNs to take part in research projects rather than directly from study teams, iii) the intervention practice individualised Germ Defence weblink being perceived as spam by staff and patients, particularly where patients had never previously received a text message from their practice).
Whilst these concerns were all identi ed before the trial began (and addressed via the 'FAQ' for patients and clinical staff) (14), some practices may not have had su cient time to engage with this content due to the operational pressure of the pandemic.
Overall, the study team received 61 enquiries/concerns from primary care staff, patients/members of the public and staff within local Clinical Research Networks during and immediately after the 4-month implementation period ( Table 5). The reasons for these ranged from checking that the study and/or the text from practices was legitimate and the unique Germ Defence weblink was not some kind of scam (e.g. https://www.unknownphone.com/phone/07800007089). Place here - Table 5: Reasons for feedback about study intervention or implementation Because the primary aim of the Germ Defence team was to implement Germ Defence as rapidly and widely as possible (based on the previous evidence of effectiveness) extensive implementation was also undertaken outside the de ned trial context. For example, the Germ Defence website and the key messages from Germ Defence were publicised on numerous occasions (via national and local radio, TV, online and print media) and was directly linked to from online government advice for Covid infection control (23).
We encountered no material barriers to importing and linking the trial randomisation schedule into a TRE, or to evaluating outcomes using patients' data in situ through the OpenSAFELY platform rather than via data dissemination.

Discussion
We implemented a novel e cient trial design which was also the rst RCT where follow up was conducted entirely within a TRE. We did not nd any consistent overall evidence that the intervention impacted rates of respiratory tract infections or other health outcomes. Similarly, we found no evidence that higher user rates were associated with changes in health outcome rates at a practice level, although there was evidence that more relevant symptoms within a practice was associated with more website uptake.
Although by comparison to many other trial interventions the reach of the Germ Defence intervention was extremely large (it was accessed more than 300,000 times across England during the trial period), we could only con rm that it was accessed by patients from 16% of the general practices approached, below the 25% of GP practices assumed in the sample size considerations. Therefore, it is hard to draw conclusions from our trial data. While we expected that using Germ Defence would improve infection control behaviours and reduce household virus transmission (24), this did not lead to a determinable reduction in health-related outcomes at GP practice level. It is likely that, with such a complex behavioural intervention in a rapidly changing contextual environment, determining a signal would require a larger sample of practices that disseminated the intervention effectively to their patients. Our ability to recruit practices was hampered by 16% of the GP practice email addresses forwarded by NHS Digital to the study not working. We recommend NHS England consider how they can better enable e cient, pragmatic trials by having e cient communication channels to contact all GP practices.
However, these ndings should not downplay the importance of the e cient trial design that allowed us to conduct a large-scale prospective randomised controlled evaluation of an active behavioural intervention during a pandemic. Our design allowed us to safely recruit GP practices in England during a national pandemic, and we used several novel techniques to minimise practice burden: i) We recruited practices en-masse via email, removing the lengthy process of contacting individual practices, ii) we set up intervention access links that were individualised to practices (meaning that practices merely had to use the intervention and we could remotely track their 'enrolment'), iii) we used national routinely collected patient record data (accessed through a secure TRE, OpenSAFELY) to analyse electronic health records, iv) we recorded anonymous digital intervention analytics to understand how many individual patients used the intervention, and how they used it.
There was no indication that the use of Germ Defence was impacted by any factors related to deprivation.
Generally, this is encouraging news that digital interventions have potential to support healthcare across the socioeconomic spectrum. However, despite substantial effort to ensure accessibility (such as translation, using infographics etc), we did see that practices with more patients from minority ethnic groups were less likely to have high levels of use. This needs to be examined in more detail to ensure that scalable digital solutions do not lead to digital exclusion for some groups.

Limitations
This 'low burden' design comes with risks. Evaluating an accessible behavioural intervention that can be passed on via multiple communication pathways means we could not prevent people from using the intervention who were not from our randomised intervention group, nor could we control broad contextual factors that may have led to increased contamination (such as the frequent public health communications about Germ Defence during the study period). The nature of behavioural interventions means their core behavioural functions can be communicated through means other than the intervention (for example, word of mouth) which would have been outside of our randomisation procedure.
Despite the unusually rapid set-up and implementation of this study, Germ Defence was rolled out through primary care over six months after the start of the Covid-19 pandemic. It is likely that by this point in time most people who were concerned about infection control in the home may have already obtained all the advice they wanted and needed. Additionally, the lack of lengthy individual practice enrolment processes may have meant that many practices did not have inclination or time to properly engage with the study, given the need to maintain enhanced infection control measures reduced capacity that were increasing pressure on practices during the study period (25). The combination of delayed implementation of Germ Defence and reduced enrolment of patients may explain why we were unable to provide evidence that disseminating Germ Defence to patients via GP practices improved health outcomes in GP practices.
However, this means our design accurately re ects 'real-world' uptake of such interventions outside of usually tightly constrained trial environments. Further research should use implementation frameworks to understand how to further improve rapid adoption and implementation (26) and encourage healthcare providers to recommend digital health interventions to their patients (27).

Conclusions
In this study we used a novel, e cient prospective randomised controlled trial methodology to examine the effectiveness of an active digital behavioural intervention across every GP practice in England. We showed that it is possible to link intervention usage with individual practice health outcomes, and to determine the effects of behavioural intervention engagement with health outcomes. Further work should explore how to improve rapid implementation, and how this design can be applied to other types of intervention.