Trial Design
This was a two-by-two factorial randomised SWAT to evaluate two strategies to improve retention embedded within a randomised prevention trial. A brief protocol for the SWAT can be found on the SWAT repository store (SWAT ID 25) (8). Participants in the SWAT were parents who had given consent for their infant to be randomised into the BEEP host trial. Screening for eligibility and the consent process for the host trial was carried out during pregnancy or shortly after delivery. Randomisation took place at the same time for both the host trial and SWAT, and was within 21 days of the baby being born.
Follow up in the host trial was at 3, 6, 12 and 18 months by questionnaire and a visit at 24 months. A web-link to the questionnaires was sent in an email for parents to complete the questionnaires online. For parents that did not wish to complete questionnaires online, paper copies of the questionnaires were provided by post with pre-paid return envelopes. Where questionnaires were not completed or returned, a reminder was be sent by email after 2 and 3 weeks of non-completion (or in the post for paper copies). The link to the online questionnaire remained active for 4 weeks after the initial email invitation was sent. Due to the lower than expected completion of questionnaires, the protocol was amended in May 2016 to allow members of the host trial team to telephone participants where questionnaires had not yet been completed but were still active. Text messages or emails were also sent by the trial team when they were unable to reach participants by phone.
Host trial primary outcome data was collected during a face to face visit with a researcher at 24 months. If a face to face visit at 24 months was not possible then researchers attempted to collect the data via telephone, text, email or post. If contact could not be made the coordinating centre attempted to collect key minimal data, to be used in sensitivity analysis for the host trial, from the child’s general practitioner.
The following small gifts were sent to all parents by the coordinating centre:
- BEEP branded muslin or bib at randomisation
- Birthday card and BEEP branded plastic cutlery set or storybook at the child’s first birthday
- BEEP branded cloth shoulder bag sent at 18 months
Parents were also sent trial newsletters every 6 months (from January 2016).
Interventions
Intervention 1: Contact by short message service (SMS) versus no contact, prior to sending questionnaires at 3, 6, 12 and 18 months.
For participants allocated to receive prior notification, a SMS message (text message) was sent the day before the email with the link to the questionnaire (or letter with the questionnaire) with the following wording:
- “Your BEEP study questionnaire will be ready soon. Please check your email tomorrow. Contact [email protected] if you have any problems completing it. Thanks!”
- For participants completing questionnaire on paper: “Your BEEP study questionnaire is on its way to you in the post. Contact [email protected] if you have any problems completing it. Thanks!”
Intervention 2: Compensation for parent’s time in the form of £10 high street shopping voucher sent to parents either before or given at the 24 month visit.
At around 22 months, parents were sent a letter by the coordinating centre to ask them to get in touch with the research nurse to schedule the host trial 24 month visit. For participants allocated to be given the voucher before the visit, the letter included the following text:
“As a thank you for your ongoing participation with the study, we are enclosing in this letter a £10 voucher”.
For participants allocated to be given the voucher at the visit, the letter included the following text:
“As a thank you for your ongoing participation with the study, your BEEP study nurse will give you a £10 voucher at the visit.”
Research nurses rang parents to book appointments for the 24 month visit if they did not get in touch following the invitation letter.
Randomisation
Research nurses randomised participants to the BEEP host trial by accessing an online system provided by the coordinating centre. A second randomisation was then automatically performed for the SWAT to each of the retention strategies (1:1) described above using an allocation schedule created by the Nottingham Clinical Trials Unit. SWAT allocation was stratified by recruiting site and host trial allocation and used a fixed block size of 4. Participants were informed in the host trial information sheet about the SWAT for SMS notification for questionnaires and timing of the voucher for the 24 month visit but were not informed at the time of their randomisation of their allocated groups for the SWAT.
The trial management team and the research nurses were not blinded to the allocations for the SWAT. The sequence of allocations for the SWAT was concealed from the statisticians (LB and AAM) until the database was locked for the host trial.
Outcomes
There were two co-primary outcomes:
- Collection of data via the chosen method of questionnaire (electronic or postal) at host trial interim follow up times (3, 6, 12 and 18 months).
- Collection of the BEEP host trial primary outcome at 24 months during a home or clinic visit with a research nurse.
Secondary outcomes were time to questionnaire completion and the number of reminders required to obtain questionnaire completion.
Sample size
The sample size for the SWAT was determined by the sample size of the host trial. Based on the target of just under 1300 and assuming collection of outcome data from around 80-85% of participants (based on previous similar studies), a between-group absolute difference of ≥7 percentage points (equivalent odds ratio 1.7) could be detected with 90% power and 5% two-sided alpha.
Statistical analysis
Analysis was conducted according to SWAT allocations for all randomised participants in the host trial, regardless of whether the allocated intervention was received. Between group estimates are presented as odds ratios and absolute differences in percentage completion with 95% confidence intervals.
Collection of data via the chosen method of questionnaire was defined as completing either of the first two questions on the questionnaire. Collection of questionnaire data at interim follow up times according to SWAT allocation for the prior SMS notification was analysed using Generalized Estimating Equations with the binomial family and logit link with an exchangeable correlation matrix to account for multiple observations per participant including allocated group in the host trial and questionnaire time point as covariates. An interaction term was also included between the intervention and questionnaire time point to explore whether there was any evidence that the intervention effect changed over time. The difference in the percentage of participants completing the questionnaire was also estimated with Generalized Estimating Equations using the binomial family and identity link. SWAT allocation for the timing of the voucher for the 24 month visit was not included in the models as this could not have had an effect on questionnaire completion as the questionnaire time points were prior to 24 months.
Collection of the host trial primary outcome at 24 months during a home or clinic visit with a research nurse for the two SWAT interventions was analysed using a multivariable logistic regression model including allocated group for the host trial as a covariate. The possibility of an interaction between the two SWAT interventions was investigated by inclusion of an interaction term in the model. SWAT allocation for the prior SMS notification intervention was included in the model for completeness although was not expected to have an effect on the collection of host trial primary outcome data at 24 months.
Pre-planned sensitivity analyses for the co-primary outcomes were performed to explore: the extent of questionnaire completion according to allocated group for the prior SMS notification intervention, collection of host trial primary outcome via any method as well as the timing of the 24 month visit completion for both SWAT interventions and further adjustment for baseline variables with an observed imbalance between the groups (based on comparison of summary statistics only).
When the statistical analysis plan was written, it was thought that allocation to the SWAT was stratified by host trial allocation only therefore the SWAT analysis plan specified that primary analyses would adjust for host trial allocation only. At the final analysis of the SWAT, it was discovered that trial site was also included as a stratification variable. Therefore a sensitivity analysis using mixed effects logistic regression was conducted to also include recruiting site as a random effect so that both variables used for stratifying allocation were included in the analysis.
Time to questionnaire completion was presented in Kaplan Meier curves according to allocated group for the prior SMS notification intervention for each questionnaire time point. The effect of the prior SMS notification intervention on time to questionnaire completion was estimated using a Cox proportional hazards model including allocated group in the host trial as a covariate and using a shared frailty to account for inclusion of four questionnaire time points for each participant. Questionnaires that were not completed were censored at 28 days. The number of reminders required to obtain questionnaire completion at each questionnaire time point was tabulated by allocated group for the prior SMS notification intervention along with the reasons that questionnaires were not completed.
Interim analysis
A separate interim analysis of the data for each of the two co-primary outcomes for the first 400 participants in the trial was planned to allow implementation of any strategy found to be superior for the remainder of the trial. Stopping boundaries for the interim analysis were calculated using the O’Brien and Fleming spending function for a total type 1 error of 0.05 for each co-primary outcome with one interim analysis. The interim analyses were conducted by statisticians at the coordinating centre independent of the trial and used the analysis methods described above.
Due to resource constraints, the interim analysis for the prior SMS notification for questionnaire completion was conducted later than planned and therefore included all questionnaires that had reached the end of the questionnaire completion window by the end of 2016. The stopping boundaries for the interim analysis for the prior SMS notification were therefore calculated based on 700 participants included in the interim analysis.
Full details of the analysis are documented in the statistical analysis plan for the SWAT, which was finalised prior to the first interim analysis (available on the host trial website https://www.nottingham.ac.uk/research/groups/cebd/projects/1eczema/beep-maintrial.aspx).