Evaluation of the Scale up of Remote Monitoring in Rheumatology Outpatients Across Three NHS Trusts in South East London, UK: Study Protocol


 Background

Modern treat-to-target approaches to rheumatoid arthritis (RA) involve frequent monitoring of disease activity with the goal of disease remission or a low disease activity. The Rheumatoid Arthritis Impact of Disease (RAID) is a multidimensional, validated patient-reported outcome measure that covers seven domains, which has been found to discriminate between active and non-active disease. Applying smartphone apps to monitoring of RA is described as an innovation which has been implemented in the UK and USA. The proposed study will evaluate the feasibility of scaled implementation of a remote monitoring service based on RAID for eligible patients with RA at three NHS organisations (trusts) in south east London, UK.
Methods

Pragmatic formative service evaluation study informed by implementation theory and incorporating the perspectives of RA service users throughout. The study will follow a multi-method approach. Rapid evidence review will be carried out to identify implementation approaches used in similar services. Quantitative data will be collected from a cross-sectional sample of service users through a web-based questionnaire assessing patient satisfaction, as well as service-level data routinely collected by trusts and from the remote monitoring system and documentation produced in developing and implementing the remote monitoring service. Qualitative data will be collected from approximately 30 clinical and non-clinical staff and 20-30 patients purposively sampled to conduct semi-structured interviews to explore their perspectives on remote monitoring in RA. The evaluation will be supported by established implementation frameworks, including EPIS (Exploration, Preparation, Implementation and Sustainment) and COM-B (Capability-Opportunity-Motivation-Behaviour), which will be used to guide data generation and to inform the framework analysis of qualitative data.
Discussion

This pragmatic study will enhance the understanding of implementation process and outcomes and will explore the potential to scale up the remote monitoring system in RA. A larger scale hybrid study can be designed based on the dataset the current study will produce to offer definitive clinical and implementation evaluation.

The study presents an opportunity to monitor how a mobile version of disease speci c outcome (RAID) performs and to support the improvement of patient-centred care in patients with rheumatoid arthritis.
Well-established implementation and evaluation frameworks will provide the basis for the pilot evaluation and a larger effectiveness-implementation hybrid study can be subsequently designed to follow up this evaluation.

Background
Rheumatoid arthritis (RA) is a chronic, in ammatory joint disease affecting around 1% of the UK population (1). RA is a uctuating condition in which patients can experience exacerbation of symptoms, known as ' ares'. Without appropriate treatment, persistent disease can lead to progressive joint damage and irreversible disability (2). Modern treat-to-target approaches involve frequent monitoring of disease activity with the goal of disease remission (or at the very least a low disease activity) (2). In the UK, NICE guidelines recommend patients should be involved in decisions about their care and have rapid access to specialist care for ares (3). Similar guidelines exist in other countries and the European Alliance of Associations for Rheumatology (EULAR) has recently published recommendations supporting the inclusion of self-management advice and resources in the routine management of people with RA (4).
For RA, patient reported outcome measures (PROMs), when used effectively, can support patient centred care, by identifying more active disease or when patients can no longer self-manage their condition (5). One example is the Rheumatoid Arthritis Impact of Disease (RAID), a multidimensional, validated PROM covering seven domains which has been found to discriminate between active and non-active disease (5). The utility of RAID in real-world routine care settings and its strength in identifying patients with unmet needs has also recently been demonstrated (6). RAID produces ratings that correlate with the 28joint Disease Activity Score (DAS28), a composite score, which captures and characterises disease manifestations activity and is used to determine eligibility treatment thresholds in clinical practice (6, 7).
Incorporating a PROM such as RAID into an electronic system offers additional potential bene ts in terms of integration with the patient electronic health record (EHR) (5). In recent UK-based research, the Remote Monitoring of RA (REMORA) study designed and tested a smartphone app which enabled people living with RA to monitor and report daily symptoms, with data integrated into their EHR and summarised graphically to inform clinical consultations (8). The study demonstrated the acceptability and feasibility of the REMORA system. A USA study tested a similar system which was not integrated into the EHR (9).
Although they found patients and physicians reported positive experiences with the app, they found mixed use of data from the app in consultations, and this lack of integration was suggested as one possible reason for this.
Embedding such remote monitoring systems into clinical practice in outpatient settings provides further potential in terms of e ciencies, for instance saving patients the time and money expended attending unnecessary appointments, as well as savings to the NHS gained from improved patient ow by making better use of capacity. Patients who need to be seen by a specialist could be reliably identi ed, and treatment delivered in a timely way. At the start of 2020, the need to achieve such economies in outpatient clinics was already widely recognised (10,11), but the COVID-19 pandemic has brought this to the fore, in order to reduce face to face care as well as manage the backlog of appointments resulting from the associated disruption to services due to the pandemic.
Evidence highlights the need to understand the factors in uencing implementation of remote monitoring technologies in routine practice, along with the concurrent implications for adoption at scale (12). The ndings of the recent USA-based trial described above led to plans for further work to test implementation strategies for integrating the app into the patient electronic health record and routine care, including educating clinicians in using the data to support patient centred care (9).
Remote monitoring service for rheumatoid arthritis in south east London, UK In this study, we will evaluate pragmatically the implementation at scale of a remote monitoring service for eligible patients with RA across six sites at three NHS organisations (trusts) in south east London, UK ( Figure 1). The service has been designed by and with clinicians, managers and patients. The model of care involves patients or their carers submitting PROMs and text comments via an online form accessed through a link sent to their mobile phone every four weeks. Returns are monitored by a digital pathway coordinator (DPC) working closely with service teams, and following a protocol developed by clinicians across the three trusts. An automated agging system alerts the DPC to an incoming text message or elevated RAID score. They then manage the ow of patient data in order for clinicians to make a triage decision about whether patients need to be booked for a clinic or a telephone appointment or receive advice and guidance remotely. The patient data is uploaded to each site's hospital electronic record by the clinical team. This approach was developed by and piloted with 129 patients at Guy's and St Thomas' NHS Foundation Trust (GSTT) from January 2019 to March 2020. Early service evaluation ndings showed high patient engagement with 87.6% (n=113) patients engaged with the service, of which 72.6% (n=82) completed ≥50% of PROMs. In addition, 77.5% (n=62/80) of patient-initiated contacts led to escalation of and change to clinical management; thereby, potentially offsetting contact elsewhere in the system.
The intended south east London-wide scale-up of the service builds on this approach. The remote monitoring system is managed by a DPC supported by the clinicians, with each local site managing the process of onboarding eligible patients into the service. Patient are deemed eligible if they meet the following criteria: DAS28-CRP score <3.2 indicating disease remission or low disease activity and RAID <4 When DAS28-CRP not available, clinician agreed disease remission or low disease activity and RAID <2 Patient has a smart phone Patient and/or carer has su cient written English language skills Patients are invited to join the remote monitoring services during consultations. The details of consenting patients are sent by the clinician or service manager to the DPC to be added to the remote monitoring service. Appendix 1 outlines the different roles and responsibilities across the service.

Study aims
The primary aim of the study is to evaluate in a rapid and pragmatic manner the feasibility and acceptability of the novel remote monitoring model in RA outpatient services scaled across three hospitals in south east London. Speci cally, the evaluation will explore (i) patient and clinician acceptability, (ii) initial clinical, service and estimated cost-related outcomes, and (iii) a potential implementation support package to help embed and scale the model.
The secondary aims of the evaluation are to: Conduct a rapid review of evidence of remote monitoring in rheumatoid arthritis to inform the implementation of the remote monitoring service and a future impact evaluation or larger scale trial.
Undertake mapping and secondary analysis of existing evidence collected to-date during the development and piloting of the remote monitoring services at a single Trust in south east of London against the evaluation objectives Produce prospective qualitative and quantitative datasets assessing service providers and users' perspectives on the remote monitoring service and its scalability The tertiary aim of the study is to explore the feasibility of undertaking a subsequent larger-scale study to evaluate clinical, service and health economic impacts and implementation effectiveness.

Study design
This is a pragmatic, formative, mixed methods service evaluation study to help support the implementation and sustainability of remote monitoring in rheumatology services. The study will follow a simultaneous mixed methods approach to generate both quantitative and qualitative data, which will be synthesised in the light of implementation and programme theory. Table 1 outlines the detailed evaluation framework and metrics for the evaluation. The evaluation framework is structured based on the established generic logic model proposed by Proctor et al (13), in which patient-level outcomes are impacted by service-level outcomes, which in turn are in uenced by implementation outcomes.

Stakeholder engagement and patient and public involvement
The evaluation is designed in a partnership with the clinical product owner and digital pathway coordinator (MM, ES), clinical lead (TG), a team of implementation/improvement scientists and evaluators (NS, AW, HS, OB, KW). The draft evaluation protocol was shared with stakeholders including the project clinical steering group and local commissioners, who offered inputs. Service user perspectives were obtained working with the patient and public involvement (PPI) coordinator at King's Improvement Science, King's College London (LD).
Seventeen GSTT service users involved in the original pilot study have consented to ongoing engagement in the evaluation. Developing plans include connecting with patients at the two other trusts involved, and with a wider group of public, service user and carers, friends and family recruited through the National Rheumatoid Arthritis Society (NRAS) database (nras.org.uk). A tiered approach is planned to engage people who would like to be involved, with three levels re ecting individual preference: participating in project meetings and contributing to project outputs, joining workshops and/or one to one discussions, and receiving updates via an electronic bulletin.

Theoretical frameworks
The study will be informed by a number of established, evidence-based implementation frameworks. These will used to guide data collection, analysis and provide a theory-informed lens for interpreting the implementation process. The following complementary frameworks will be applied: The Proctor et al implementation outcome taxonomy and logic model, which links the outcomes of implementation efforts to observable improvements in services, which ultimately lead to improved patient outcomes. This taxonomy has been used to conceptualise the levels of evaluation that is being conducted (Table 1) (13).
The Expert Recommendations for Implementing Change (ERIC) framework of implementation strategies will be used to identify early approaches taken to implement the remote monitor service across sites, and to develop a potential implementation support package in the future (14).
The Exploration, Preparation, Implementation, Sustainment (EPIS) framework provides a temporal lens to explore the different stages of the implementation process; thereby, allowing the remote monitoring model to be understood from its development and initial piloting at one trust, to the current implementation and scale-up across an Integrated Care System (ICS), and sustainment (15). The framework also incorporates an analysis of barriers/drivers to implementation, as well as local and wider system-level contextual factors that may impact on them.
The evaluation of patient experience will be informed by the COM-B theory enabling a systematic evaluation of behavioural aspects. The COM-B theory is based on three interacting domains for behaviour change: capability, opportunity and motivation (16). Capability is de ned as the individual's psychological and physical capacity to engage in the activity concerned; this includes having the necessary knowledge and skills, e.g. self-e cacy. Opportunity is de ned as all factors outside the individual that enable the behaviour or prompt it, e.g. support from a physician.
Motivation is de ned as brain processes that direct behaviour, e.g. goals, conscious and analytical decision-making, habitual processes and emotional response. The COM-B theory is increasingly used in evaluation research which involves patients' behaviour and intentions in their engagement with new technologies. For example, the model has been deployed in the evaluation of eHealth, selfmanagement interventions in patients with cardiovascular disease and persistent physical symptoms (17,18). The model was successfully applied to the examination of decision aids in the context of shared decision-making in care for patients with rheumatoid arthritis (19).

Setting
The project will take place in rheumatology outpatient services in three National Health Service (NHS) trusts comprising six sites that collectively provide all rheumatology outpatient services across South East London, UK. The sites are Guy's and St Thomas' Hospital (GSTT), King's College Hospital (KCH), Princess Royal University Hospital -Orpington (PRUH), Queen Elizabeth Hospital (QEH), Queen Mary's Hospital (QMH), and University Hospital Lewisham (UHL). Following the 18-month pilot at GSTT in January 2019, the remote monitoring service was launched at the three trusts in October 2020. Remote monitoring will be offered to all patients with rheumatoid arthritis who meet the eligibility criteriaestimated as between 1,500 to 2,000 in total within the catchment areas of the six study sites. During the period of the evaluation 350-400 patients are estimated to be registered on the remote monitoring system.

Participants and recruitment
Both staff and patients involved in the remote monitoring service will be invited to take part. This evaluation will involve interviews with staff employed in rheumatology outpatient services across the three NHS trusts delivering the remote monitoring service (e.g. doctors, nurses, allied health professionals, administrators and service managers). Patients with rheumatoid arthritis accessing care via the remote monitoring service will be invited to participate in interviews and surveys.
Purposive sampling will be used for qualitative data collection to ensure the evaluation considers a range of perspectives that are representative of the population and implementation process. We will ensure clinical and non-clinical staff are represented from a range of sites, disciplines, and seniority. All eligible patients of different backgrounds (age, gender, ethnicity, disease longevity) will be approached. The recruitment will continue until the sample of approximately, 20-30 patients and 30 staff (estimated 10 per trust) is achieved, which is estimated to allow saturation to be met.

Data collection methods and sources
The evaluation will use (i) rapid evidence review and synthesis, (ii) service-level data routinely collected by Trusts and from the remote monitoring system, (iii) documentation produced in developing and implementing the remote monitoring service; (iv) data collected from staff via interviews, (v) data collected from patients via interviews and survey.
The rapid evidence review will involve database searches of Medline and Web of Science, combining the following Search/MESH terms: Rheumatoid Arthritis AND remote monitoring OR mobile app OR mobile applications OR smartphone app OR smartphone application OR telehealth OR tele-health OR telecare OR tele-care OR electronic health OR mhealth OR ehealth. Inclusion criteria will be research papers and protocols published in English; exclusion criteria will be articles focussing on 'telemedicine'. Abstract and full text screening will be undertaken by two researchers (OB, NA), with any discrepancies discussed with a third reviewer; the data will be synthesised as a narrative review.
Interviews with staff and patients will focus on the overall experience of using the remote monitoring service, what works well and what does not; staff interviews will also explore implementation and feasibility aspects of introducing the service as an organisational innovation within their respective trusts. Interview topic guides have been developed incorporating the early experience of the service development team, the view of patients and service users involved in the study, and theoretical perspectives from implementation theory (see Appendix 2 for the draft topic guides). We anticipate that the topic guides will be ne-tuned after an initial one or two interviews and nalised.
The patient survey will be based on survey developed in the original pilot at GSTT to provide a comparison. The evaluation will use the RAID score (6) and Health Con dence Score (HCS) (20) as standardised PROMs for the condition. The RAID score is a validated 7-item PROM assessing pain, functional, disability, fatigue, sleep, coping, physical and emotional well-being. It will be collected every 4weeks via the remote monitoring system (i.e. using an SMS link) for all registered users of the system.
The Health Con dence Score is a 4-item validated measure for self-care health con dence, which provides self-reported information on patients' con dence about knowledge about their health, selfmanagement, access to help, and involvement in decision making. The HCS will be collected as part of a one-off online patient survey, which will be sent via SMS link to all registered users from the remote monitoring system. The predicted number of patients registered on the system during the evaluation is 350-400 people across all sites.
Initially, staff and patient data will be collected remotely in line with COVID-19 pandemic guidelines. However, the evaluation may use face-to-face approaches (e.g. interviews) if permitted by national and local guidelines relating to COVID-19 infection control and social distancing. Table 1 outlines the data collection methods and data sources in more detail.

Data analysis
For the rapid evidence review, relevant studies will be identi ed and relevant data extracted and tabulated in pre-formulated data extraction tables. We anticipate extracted data will include the study design and methods, the range if implementation outcomes assessed and how the assessment was carried out, the core ndings and, nally, the limitations of the study. Study identi cation and data extraction will be undertaken by a minimum of two researchers, who will reach consensus on the process of identifying and extracting data from included studies; and resolve any disagreements via discussion (including the wider project team as required).
For the quantitative part of the study, we will use descriptive statistics (mean, standard deviation, range, percentages) to summarise patient satisfaction with the service. Administrative data (budget information, telephone helpline data, other service-level data) will be extracted in a form of descriptive summary tables and charts.
For the qualitative part of the study, we will follow ve stages of framework analysis outlined by Ritchie and Spencer (24): familiarisation, identifying a framework, indexing, charting, mapping and interpretation. Interview recordings will be transcribed professionally, identi ed information will be removed and transcripts will be coded in NVivo. 10% of the interview data will be double coded and consensus will be reached through a dialogue. Emerging frameworks will be reviewed in light of the EPIS and COM-B theories and discussed among the researchers.

Discussion
This pragmatic, mixed method evaluation aims to assess the feasibility and acceptability of a novel remote monitoring model in RA outpatient services across south east London. Speci cally, the evaluation will explore acceptability in patients and clinicians, initial service outcomes and providers and patients' experiences with the service, as well as barriers and drivers to scaling up the service outside the organisation where it was originally conceived and implemented to a wider region.
The evaluation approach is pragmatic in that it assesses the use of the RA remote monitoring system in the context of usual clinical practice, rather than in the context of a research study. A pragmatic approach was selected to ensure the study was able to rapidly identify and incorporate lessons learned, in order to maximise bene ts to patients and other stakeholders within a complex setting and with the constraints of limited time, budget and other resources. However, such an approach is not without limitations, the main one being that without a robust control group it will be unable to draw de nitive conclusions about the relationship between any observed or self-reported outcomes and the intervention. For this reason, the current evaluation is regarded as a stepping-stone which will provide a base for developing a randomised controlled evaluation of the novel remote monitoring service across several NHS sites should the ndings support such a development.
The study takes place in a world which is still very much affected by the COVID-19 pandemic. This has signi cant implications for clinical services and creates a unique set of circumstances, limiting the extent to which the ndings can be regarded as transferable to a COVID-free context. The pandemic introduces other limitations to the study such as those associated with its impact on sta ng levels. With outpatient clinic personnel absent in high numbers due to sickness or managing their disease risk, either to self or others, as well as being redeployed to help with the COVID-19 response, the pressure experienced by remaining staff has increased. The evaluation design has responded to this situation pragmatically by minimising the demands made of staff in terms of inputting to the evaluation, but this does bring costs in terms of the quality of data gathered. The pandemic also affects the methods of data collection, placing an emphasis on electronic formats rather than on face-to-face approaches which are typically regarded as preferable in terms of establishing rapport with the interviewee (21).
The evaluation also has considerable strengths. As described above, the pragmatic approach allows assessment of the implementation and scale-up of the RA monitoring system in real world settings. It will provide rapid feedback which can be incorporated into practice to maximise bene ts to the users. Furthermore, the approach has been co-designed by a group of stakeholders, including the clinicians, managers and patients who are the users of the service. The study was approved by Guy's and St Thomas' NHS Foundation Trust as a service evaluation. The study used anonymised routine monitoring data collected under GDPR article 6 1 (e) and article 9 2 (h). Informed consent will be obtained from participants for all other data (i.e. collected via interview and survey) prior to any data collection carried out.

Consent for publication
As part of the written informed consent, all participants will be asked to give permission for information they provide in interviews to be published in an anonymised form.

Availability of data and materials
The datasets used and/or analysed during the current study will be available on reasonable request.