The trial had a recruitment target range of 264-300. The lower range of this target had already been reached (n=267) at the time that national COVID-19 lockdown restrictions in the UK commenced on 23 March 2020 and in-person research activity was suspended. The end of the scheduled recruitment period coincided with the height of the first wave of the pandemic in the UK in April 2020. The pandemic response had little impact on follow up data collection. It has always been planned to collect the six and 12-month outcome measures for the trial remotely (via telephone, mail or online form), and data collection continued unimpeded during lockdown restrictions as per protocol.
The main impact of COVID-19 on the trial was the interruption to the delivery of the intervention and control treatments (outpatient physiotherapy). Suspension of non-essential physiotherapy treatment had occurred at all trial sites by mid-March 2020. At this time, 89 participants (33% of the total recruited) had been randomised but were awaiting their allocated treatment. The suspension was then lifted at different times for the trial sites, based on regional variations to the pandemic response. Most sites were given permission to resume physiotherapy treatment with precautions after six to nine months but some sites did not reopen. When physiotherapy treatment restarted, trial participants from both randomised groups were subjected to additional delays in receiving treatment due to the impact of the pandemic on NHS waiting times. As a result, a proportion did not receive their allocated treatment within their trial follow up period. Including these participants in the final analysis as part of an intention to treat (ITT) analysis plan is likely to dilute a potential treatment effect and may result in a type 2 error (a negative trial despite an effective treatment). Other consequences of the pandemic response on the trial are reported later in this paper.
Mitigating the Impact of COVID-19
Guidance for minimising the impact of COVID-19 on clinical trials has been produced by a number of organisations, including the Medicines and Healthcare products Regulatory Agency[9], European Medicines Agency[10] and Food and Drug Administration[11]. The main message is to evaluate the changes that have been necessary as a result of COVID-19 and develop a plan to mitigate them. In terms of statistical analysis, this is interpreted as collecting as much data as possible, prioritising the more important outcomes, logging protocol deviations as appropriate and planning prior to the database lock to undertake additional analyses to explore the effects of the COVID-19 interruption on trial outcomes.
Recommendations about what to do with follow-up data collected from participants who did not receive their trial allocated treatment due to the pandemic have been based on the ICH E9(R1) addendum and the role of intercurrent events[12, 13]. The first step is to determine the estimand (treatment effect) of interest. In the case of COVID-19 the potential estimands of interest are (i) the treatment effect in a hypothetical ‘pandemic free world’; or (ii) the treatment effect in a ‘world including a pandemic’. In the case of Physio4FMD, the pandemic-influenced scenario will estimate an average treatment effect when a proportion of the population remain untreated. Although COVID-19 may continue to cause disruption to the delivery of physiotherapy treatment and future pandemics are possible, we argue that the initial COVID-19 pandemic response was unique and outcomes that are heavily influenced by these events will not be generalisable. Therefore, the estimand of interest to Physio4FMD is the treatment effect in the absence of the initial pandemic response.
The Physio4FMD trial management group sought external advice from independent statisticians in developing a COVID-19 mitigation plan. In consultation with the Data Monitoring and Ethics Committee, the Trial Steering Committee, Patient and Public Representatives, and the research funder, the following mitigation plan was agreed:
1. Extend the trial and increase the sample size
The funder approved a request to extend the trial to recruit an additional 90-120 participants. The lower limit of 90 will allow additional people to cover for those who did not receive timely treatment due to the pandemic. The upper limit of 120 will mitigate against further possible disruption due to COVID-19 after restart and is a feasible upper target based on previous recruitment rates.
2. The primary analysis
Data from participants who were randomised but did not receive treatment before 23 March 2020 (n=89) will be treated as missing data for the primary analysis. This group either did not receive any treatment during the trial period (12 months post randomisation) or their treatment was substantially delayed and it occurred close to the primary outcome assessment point. We justify this decision in the discussion section of this paper.
3. Complete additional sensitivity analyses
A series of sensitivity analyses will be conducted to determine the impact of COVID-19 on the trial outcome. The main sensitivity analysis will include all randomised participants, irrespective of whether COVID-19 prevented them from receiving treatment.
Below we describe in detail our approach to the analysis of the clinical and health economic data.
Trial Status
Participant recruitment began on 16 November 2018 and was stopped on 16 March 2020 due to COVID-19 restrictions. The total recruitment was n=267. Follow up of all participants continued remotely as planned, whether or not they received treatment. An extension to the trial was granted by the funder in April 2021. Recruitment restarted in July 2021 and is scheduled to continue until 31 January 2022 or until 120 participants have been recruited, whichever happens sooner. The anticipated trial end date is September 2023.
Ethics
Ethical approval was obtained from the London-Surrey Borders Research Ethics Committee, reference number 18/LO/0486, on 28 March 2018. An amendment to encompass changes due to COVID-19 was approved on 15 July 2021.
Trial Design
The Physio4FMD trial is a pragmatic, multisite, single-blind, parallel group, randomised controlled trial in adults with FMD. The trial will compare a specialist physiotherapy protocol with treatment as usual (referral to community physiotherapy). Participants will be recruited from trial sites in England and Scotland and will be assessed at baseline, 6 and 12 months. Follow ups will be conducted remotely either online, by post or over the telephone (according to the participants’ choice).
Research Objectives and Outcome Measures
The primary objective of Physio4FMD is to evaluate the effectiveness of specialist physiotherapy compared to treatment as usual (TAU) in reducing disability, measured by the Physical Function domain of the Short Form 36 questionnaire (SF36-PF)[14] at 12 months post randomisation[5]. The secondary trial objectives are to evaluate the effect of specialist physiotherapy compared to treatment as usual on a range of health outcomes and to undertake an economic evaluation to assess the cost-effectiveness of the intervention compared to treatment as usual. The secondary objectives and the corresponding outcome measurement tools are listed in Tables 1 and 2.
Sample Size
The sample size was calculated using data from the preceding single centre feasibility study[4]. The sample size was calculated to assess a difference for the Physical Function domain of the SF36 questionnaire between randomised groups of 0.41SD, using the ANCOVA method. The following factors were included in the calculation: (i) 90% power; (ii) clustering by therapist in 8 centres; (iii) an anticipated 20% dropout rate; (iv) a design effect of 1.4; and (v) a correlation between baseline and outcome score of 0.55. The resulting sample size was 264 (132 in each arm)[5]. This figure was subsequently reviewed, and considered as a minimum sample size, with a more ambitious sample of 300 as the new target, allowing for a more conservative retention rate. As described above, the sample size was increased by between 90 and 120 as part of the COVID-19 mitigation strategy. The final target range is between 357 and 387 participants.
Randomisation and Blinding
Randomisation is at the level of the participant, stratified by site. Participants are randomised with a 1:1 ratio to a novel specialist physiotherapy treatment protocol and TAU groups. Block randomisation with random block sizes was used to ensure even allocation of intervention and control participants within sites. The randomisation protocol will continue as before the COVID-19 pause.
The researchers collecting outcome data, the health economists and statisticians will be blind to treatment allocation. The Trial Manager, participants and treating clinicians will not be blinded due to the nature of the intervention under investigation.
Statistical Analysis Plan
General Considerations
All statistical tests and confidence intervals will be two-sided. Statistical significance will be considered at the 5% level and estimates will be presented with 95% confidence intervals. Analyses for the primary outcome and main secondary outcomes will involve only groups A, B and D as discussed below and will follow Intention to Treat (i.e. participants will be analysed in the arm to which they were randomised, irrespective of treatment withdrawal, noncompliance or crossover between trial arms). Group C will take part in a sensitivity analysis only. The analysis of the primary and secondary outcomes will be conducted following a complete-case approach. Presentation of all findings will be in accordance with the latest CONSORT statement[24]. The impact of missing data on the primary outcome will be explored in a supplementary analysis.
Descriptive Statistics
Both the covariates and the outcomes will be summarised using descriptive analysis. Categorical variables will be reported as frequencies and percentages. Reports of continuous variables will include mean or median and standard deviation or interquartile range as appropriate. The number of missing observations will also be reported. Summary measures for the baseline characteristics will be presented overall and by treatment arm. No formal statistical tests will be performed to compare baseline characteristics; hence any observed differences between the treatment arms will be due to chance rather than randomisation bias. A CONSORT flow chart will be provided[24]. This will include the number of participants: agreeing to enter the trial; continuing through the trial by randomised arms; withdrawing at each follow up time point; lost to follow-up at each time-point and excluded/analysed. The reasons for exclusion or withdrawal when known will be reported.
Analysis of Primary Outcome
The primary outcome will be analysed using random effects modelling, with either therapist or individuals as the random effect, for all participants in the specialist physiotherapy and TAU group respectively. This model will control for baseline SF36-PF values and it will also adjust for the randomisation stratification factor, that is, site, using fixed effects. The model will be:
where the i subscript denotes the ith therapist, the j subscript denotes the jth participant and
The model for the primary outcome analysis assumes that the residuals are normally distributed and homoscedastic. These assumptions will be checked using residuals plots. If substantial departures from normality occur, a transformation of the outcome variable will be considered. Hausman specification test[25] will be used to assess whether the random effect model is superior to the fixed effect one. In case of poor model convergence, we will explore the use of a random effect term to adjust for “site” and fit a three-level mixed-effect model.
Analysis of Secondary Outcomes
For Hospital Episode Statistics (HES) and Information Services Division (ISD) digital data, we will report descriptive statistics for each type of service (i.e., outpatient, Accident and Emergency (A&E), inpatient). Suitable descriptive statistics and statistical tests will be selected depending on the distribution of the variables. Mixed effects Poisson regression models or suitable alternatives (such as Negative Binomial or Zero Inflated models) depending on the distributions of the relevant outcomes will be used to explore the difference between the randomised groups.
The CGI-I scale will be dichotomised into (i) good outcome, and (ii) poor outcome. Good outcome will be defined as ratings of “much improved” or “improved” and poor outcome will be defined as a rating of “same”, “worse”, or “much worse”. The 5-point Fatigue State scale will be dichotomised into (i) extreme and severe fatigue; and (ii) moderate, slight or no fatigue. Analysis for the dichotomised scales will use mixed effects logistic regressions, adjusting for baseline values (when collected, i.e., not for CGI-I scale as it is not collected at baseline because it is a measure of change) and site using fixed effects, if possible. Other clinical secondary outcomes, measured using continuous scales, will be analysed similarly to the primary outcome, using linear mixed models and adjusting for baseline values.
Adverse events (AE), and serious adverse events (SAE) will be summarised, by both number of events and number of participants.
Sensitivity Analysis to Evaluate the Impact of COVID-19
Descriptive statistics of baseline characteristics for 4 groups of participants will be tabulated by randomised treatment. The groups have been identified to reflect the manner in which the COVID-19 pandemic impacted the trial. The four groups are as follows:
Group A (n=24): Participants who received their treatment as described in the protocol and completed their 12-month follow up by 23 March 2020, that is, the date when national lockdown restrictions were imposed in the UK.
Group B (n=131): Participants who were recruited and received their treatment as described in the protocol before 23 March 2020, but were still in follow up when lockdown came into place and completed their 12-month follow up after 23 March 2020.
Group C (n=89): Participants who were recruited and randomised before 23 March 2020, but could not complete their treatment before this date due to the pandemic.
Group D: Participants recruited after July 2021 as part of the trial extension. The target recruitment for this group is n=90 to 120.
The potential impact of the pandemic response for each group is described in Box 1. The primary outcome analysis will be fitted using data from participants belonging to groups A, B, and D.
To account for the impact of possible delays in starting treatment due to suspension of non-essential hospital activities in relation to the COVID-19 pandemic, a sensitivity analysis will be conducted and will include all participants (groups A, B, C and D). We will repeat the primary outcome analysis, adding a supplementary fixed effect to the model and its interaction with the assigned treatment, which will thus become
where COVij is the patient-level indicator of whether insufficient or no treatment has been administered due to the outbreak of the COVID-19 pandemic (i.e. group C).
Further sensitivity analyses to evaluate whether there is indication of a different treatment effect in a post COVID-19 world will be conducted. This will be explored by fitting the model for the primary outcome analysis on two different cohorts, one only using data from groups A and B and the other only from those in group D.
Box 1. Potential impact of the pandemic response by COVID-19 affected groups
|
Potential Impact
|
Description
|
|
Treatment delays
(Group C)
|
Suspension of research and non-essential NHS services led to substantial delays for participants waiting to receive their trial treatment. In many cases, treatment occurred shortly before the final 12-month post randomisation assessment. The impact of this on outcome is unclear. If, after treatment, there is a loss of treatment effect over time, the reduced follow up period may inflate a treatment effect size. Conversely, delays may have reduced a treatment effect if longer symptom duration and/or living with FMD during lockdown restriction is associated with worse outcomes. It is possible that delays had different effects on the intervention and control groups.
|
|
Allocated treatment not received
(Group C)
|
A proportion of participants who had not received their allocated treatment by 23 March 2020, did not receive any treatment during the trial follow-up period. This is likely to dilute a potential treatment effect and may lead to a type 2 error (a negative trial, despite an effective intervention).
|
|
Altered patterns of health and social care utilisation impacting on the health economic analysis
(Groups B & C)
|
Suspension of non-essential NHS services and discouragement from attending A&E for non-life-threatening ailments is likely to have caused reduced health and social care utilisation at 6 and 12 month follow ups. Rates of unemployment may have also been affected, as many workers were placed on furlough.
|
|
Influence on how participants answered outcome assessments
(Groups B & C)
|
The outcomes reported by participants who completed follow-up during the pandemic response that may be negatively influenced include the measurement domains of anxiety, depression, social interaction, work/employment and physical activity.
|
|
Potential influence on the primary outcome measure
(Groups B & C)
|
The primary outcome measure (SF36-PF) asks participants if their health limits their ability to complete a range of physical activities ranging from vigorous activity such as participating in sports, to climbing several flights of stairs and walking various distances. We do not expect lockdown restrictions to have had a substantial impact on the primary outcome. However, it is possible that for some participants the pandemic “stay-at-home” orders resulted in reduced levels of physical activity and ultimately lower physical function scores.
|
Other Sensitivity and Supplementary Analyses
The following sensitivity and supplementary analyses are planned for the primary outcome measure only:
- We will conduct a Complier Average Causal Effect (CACE) sensitivity analysis[26]. Participants who have been offered and could participate in at least five sessions in the intervention group will be deemed as being compliers.
- To examine the effect of missing data, we will identify predictors of missingness and add them into the primary outcome regression model.
- We will describe the impact of treatment withdrawals using descriptive statistics to summarise the primary endpoints for participants who have withdrawn from treatment but have continued in follow-up.
- A dose response analysis will be performed. We will fit an alternative version of the primary outcome model adding the interaction term between the number of sessions attended and the randomised treatment, to evaluate whether the finding in those who attended more sessions in the treatment groups differed.
Exploratory Analysis
Analyses aimed at exploring moderators, mediators and predictors of outcome will be performed. The outcome will be determined by the dichotomised CGI-I scale or a 10-point increase in SF36-PF score at 12 months. The two exploratory analyses of prognostic indicators will use random effects logistic regression modelling. Potential prognostic factors examined will be baseline demographic and clinical characteristics. This will include an exploration of the impact of fatigue on outcome, using both the dichotomised fatigue state scale and the SF36 Energy/Vitality domain; and an exploration on the impact of the number of somatic symptoms reported on the Extended Patient Health Questionnaire-15[27] (these data are collected at baseline assessment only). Results will be exploratory, and any factors which appear to be associated with the outcomes will need further investigation in a study that is powered for the purpose.
Health Economic Analysis Plan
The primary aim of the health economic analysis is to calculate the mean incremental cost per QALY gained (using the EQ-5D-5L) of specialist physiotherapy compared to TAU at 12 months from a health and social care cost perspective. The secondary aim is to calculate the mean incremental cost per QALY gained of specialist physiotherapy compared to TAU at 12 months from a wider cost perspective.
Outcomes
EQ-5D-5L
A preference-based measure of health-related quality of life, EQ-5D-5L, will be collected at baseline, 6- and 12-months’ post randomisation. The responses to these questions will be converted to utility weights where the maximum possible score for perfect health is 1, death is anchored at 0, and scores less than 0 are possible using the UK tariff set published by Devlin et al 2018[28].
Client Service Receipt Inventory (CSRI)
Resource use will be collected using a modified version of the CSRI[20] previously developed and tested for use in patients with FMD as part of the feasibility trial[4]. The CSRI collects information about community and secondary care services, out of pocket costs, help received from family and friends, the cost of transport associated with FMD appointments, any equipment and adaptations made due to the illness and medication costs. This will be completed at baseline, 6 and 12 months post randomisation asking about the previous 6 months.
Hospital Episode Statistics (HES) & Information Services Division (ISD) data
HES/ISD data will be used to validate the results of the incremental costs calculated from the CSRI by: (i) Checking the reliability of patient reporting; (ii) Applying more specific secondary care costs based on reason for attendance; and (iii) Using HES/ISD data to calculate mean incremental health and social care costs of specialist physiotherapy compared to TAU at 12 months.
Work Productivity & Activity impairment – Specific Health Problem (WPAI-SHP)
The WPAI-SHP will be used to calculate the cost impact of improved engagement with employment due to specialist physiotherapy.
Costs
Cost of Specialist Physiotherapy
The cost of the physiotherapist delivering the specialist physiotherapy will be calculated by multiplying the time spent delivering the intervention to each participant by the average cost per hour of hospital-based physiotherapy from the Personal Social Services Research Unit (PSSRU) to calculate the individual level cost per participant. Time spent delivering the intervention will be calculated at 1.5 hours per session to account for organisation and administration. Treatment costs will include 30 minutes of clinical supervision per intervention-participant (as per trial protocol). We will cost intervention-group physiotherapists’ attendance at a five-day (37.5 hours) training programme and divide by the number of participants in the specialist physiotherapy arm to calculate the cost per participant. The cost of training will be a conservative estimate of the cost per participant given physiotherapists may have more patients than this on their caseload in practice.
Cost of Treatment As Usual (Community Physiotherapy)
Participants randomised to the TAU arm are asked to report physiotherapy appointments received as part of the trial in a telephone interview[5]. Physiotherapy appointments will be costed based on the unit cost for a community physiotherapy appointment[29], with an uplift for average travel time for home appointments.
Cost of Health and Social Care Resource Use
The cost of health and social care resource use for the specialist physiotherapy group versus TAU will be calculated using resource use reported in the modified CSRI[20]. These will be calculated for each participant using the unit costs from the most recent version of the Unit Costs of Health and Social Care published by the PSSRU[29] and NHS Schedule of Reference Costs[30]. Medication will be costed using the British National Formulary (BNF) and online sources when not available from the BNF[31]. NHS Schedule of Reference Costs[30] will also be applied to HES/ISD data as part of the secondary analysis using this data.
Wider Societal Costs
Wider societal costs include out-of-pocket costs and impact on carer time collected by the modified CSRI and the cost of losses to productivity due to FMD collected as part of the WPAI-SHP. Productivity will be costed using the human capital approach. Participant wages will be based on the median wage of reported professional group from the most recent version of the Office for National Statistics Annual Survey of Hours and Earnings[32]. Carer time will be costed as the unit cost per hour for a social care worker[29].
Quality Adjusted Life-Years (QALYs)
QALYs will be calculated using the area under the curve method[33] using utility values calculated from responses to the EQ-5D-5L collected at baseline, 6 and 12 months, and the EQ-5D-5L valuation study by Devlin et al[28]. The cross-walk algorithm with the EQ-5D 3 level (EQ-5D-3L) by van Hout et al[34] will be included as a sensitivity analysis. QALYs will also be calculated using the Short-Form Six Dimension health index (SF-6D), using the algorithm from Brazier et al[22] applied to SF36 data.
Discounting
As the analysis is for 12 months no discounting will be included.
Analysis
In line with the statistical analysis, the primary health economic analysis will exclude participants whose treatment was affected by COVID-19 (Group C).
Descriptive Statistics
Descriptive statistics for the percentage of participants using a type of contact, and mean number of contacts for those with non-zero contacts, for each type of health and social care contacts collected by the modified CSRI will be reported at baseline, 6 and 12 months by group. Information on data completeness will also be reported.
We will report the mean cost per participant and standard deviation for the cost of specialist physiotherapy and referral to community physiotherapy (excluding private physiotherapy).
Mean cost per participant will be reported for specialist physiotherapy versus TAU as total cost per participant and type of service use. The difference in health and social care costs and wider societal costs between the two groups will be calculated using regression analysis, adjusting by baseline values and centre with therapist as a random effect. Bootstrapping will be used to calculate 95% Confidence Intervals.
Mean utility per participant for each time point and mean unadjusted QALYs from baseline to 12 months will be reported for specialist physiotherapy and TAU. The incremental mean difference in QALYs between specialist physiotherapy and TAU adjusting for baseline and centre with therapist as a random effect using regression analysis will be reported[33] for both specialist physiotherapy and TAU. Bootstrapping will be used to calculate 95% CIs.
Incremental Cost-Effectiveness Ratio (ICER)
We will report mean incremental cost per QALY gained between specialist physiotherapy and TAU at 12 months adjusting for baseline and centre with therapist as a random effect. Costs will be as specified above and will include the cost of health and social care resource use and the cost of specialist physiotherapy. 95% confidence intervals will be calculated using two-part bootstrapping.
Cost-Effectiveness Acceptability Curve (CEAC) and Cost-Effectiveness Plane (CEP)
The bootstrapped means and 95% CIs for costs and QALYs will be used to calculate the probability that specialist physiotherapy is cost-effective compared to TAU for a range of cost-effectiveness threshold values. We will also report a cost-effectiveness plane showing the bootstrapped results[35].
Missing Data
Data will be analyzed using a complete-case analysis, excluding group C in line with the statistical analysis. The number of missing observations for each outcome at each time point will be reported. Patterns of missingness will be explored, predictors of missingness will be assessed, and the suitability of missing data assumptions considered. Depending on the level and pattern of missing information, we will consider performing multiple imputation as appropriate, in consultation with the statistician, to ensure that any assumptions are consistent across analyses.
Validating HES/ISD data
We will report the level of agreement between CSRI and HES/ISD data on matching variables (inpatient, outpatient and A&E attendances). Agreement will be tested using the paired t-tests for normally distributed variables and the Wilcoxon Signed Rank Test for skewed variables.
Sensitivity Analysis to Evaluate the Impact of COVID-19 on Cost-Effectiveness Results
In line with the statistical analysis we will report mean utility at baseline 6 and 12 months, mean QALYs at 12 months and mean health and social care resource use costs at baseline, 6 and 12 months for each of the four levels of COVID-19-affected groups specified above. We will report the ICER, CEAC and CEP for specialist physiotherapy versus TAU at 12 months separately using participants who were treated before the pandemic (groups A & B) and then using only participants from group D to evaluate the cost-effectiveness of receiving treatment after service availability has been impacted by COVID-19.
To evaluate the implications of any dampening of the treatment effect due to reduced access to care, we will include an analysis where a covariate will be included for the time point data was collected (before, during or after lockdown). We will explore the impact of lockdowns in particular for (i) EQ-5D-5L utilities; (ii) routine secondary care appointments; and (iii) emergency secondary care contacts.
Other Sensitivity Analyses
To explore the uncertainty around costs used in our analysis we will test the impact of changing assumptions used to calculate the cost of specialist and community physiotherapy.
Secondary Analysis
- We will report the ICER, CEAC and CEP for specialist physiotherapy versus TAU at 12 months from a wider cost perspective.
- We will report the ICER, CEAC and CEP for specialist physiotherapy versus TAU at 12 months using HES/ISD data to calculate costs.
- The Devlin et al[28] value set for England has been chosen as our primary analysis of cost-effectiveness given that it has been shown to be more responsive to changes in anxiety and depression[36]. NICE recommends the use the EQ-5D-3L mapping algorithm by van Hout et al[34] for technology assessment submissions. As a result, we will conduct a secondary analysis using the EQ-5D-3L mapping algorithm.
- The ICER, CEAC and CEP comparing specialist physiotherapy to TAU from a health and social care cost perspective will also be reported using the SF-6D to calculate QALYs.