Study setting {9}
The trial will be conducted remotely (using phone calls and digital methods), with researchers based in three trial centres: London, Southampton, and Nottingham. Across all trial centres, participants will access the intervention (the STEPS app) using their mobile device in their preferred setting.
Eligibility criteria {10}
Inclusion criteria:
- Parents (aged 18 years or older) of children aged 5 to 11 years, who have been accepted onto the assessment waiting list in selected services in England and their child referred for a mental health assessment. Given the differences in the referral pathways existing in the organisations supporting OPTIMA recruitment, all mental health and neurodevelopmental referrals will be initially accepted as eligible and screened. To be eligible, the child should have been on the waitlist for no longer than nine calendar months.
- The child scored ≥ 8/10 on the hyperactivity/inattention and ≥ 4/10 on the conduct problems subscale of the Strengths and Difficulties Questionnaire (26). These cut-offs each identify the top 10% of the population (27).
- Self-perceived English language competence.
- The parent has access to a mobile phone using the iOS or Android operating system that connects to the internet.
Exclusion criteria:
- A child living under local authority care.
- The child already received a clinical diagnosis of ADHD and/or received treatment for ADHD (pharmacological or non-pharmacological).
- Children with other pre-existing diagnoses will not be excluded.
If two children from the same family are referred during the trial at the same time and both meet inclusion criteria, then only the older of the two will be included. If they are referred at different times the first child will be included.
Who will take informed consent? {26a}
Participants in the study will be parents of children, who screened positive for high levels of hyperactivity/impulsivity, inattention, and conduct problems, children themselves (parent-child interaction task only, optional), and clinicians (qualitative interviews only). The method for taking consent will be different for each of these three participant groups:
Parents
All parents will provide written informed consent to take part in the study. The informed consent form will be provided in electronic format using Red Pill ePRO system and will be completed by the participant before they enter the study. A link to the electronic consent form will be emailed or texted to parents who wish to enroll in the study. The electronic consent form will include a link to the Parent Information Sheet. On the consent form, there will be a separate question asking parents to consent to a qualitative interview. Parents, who agree to take part in the interviews via the written consent form, will be asked to confirm their consent verbally before the interview takes place.
Children
Consent statements regarding a child’s participation in the study will be included in the electronic consent form completed by parents via the Red Pill ePRO system. Parents, who consented to their children’s participation, will be emailed The Child Information Sheet. Children’s verbal assent will be obtained before each child-parent task session and will be documented in the trial database.
Clinicians
The informed consent form for clinicians will be provided in electronic format using Qualtrics. A link to the electronic consent form will be emailed to clinicians who wish to take part in the interviews. The electronic consent form will include a link to the Clinician Information Sheet.
Additional consent provisions for collection and use of participant data and biological specimens {26b}
Not applicable.
Interventions
Explanation for the choice of comparators {6b}
No alternative evidence-based intervention is currently recommended, available, or implemented in the United Kingdom during the post-referral waiting period for patients waiting for a child health services assessment. WAU, therefore, is the most appropriate comparator. Those randomised to WAU will receive STEPS after they have completed the 12-month OPTIMA follow-up. In the sites across our three centres, given the current length of waiting lists in the UK, we do not expect any children to receive their clinical assessment and initiate treatment within the first three months of randomisation or few families, if any, to engage in self-initiated treatment during this period. Parents will not be stopped from initiating access to services over any of the 12 months of the trial.
Intervention description {11a}
STEPS (Structured E-Parenting Support; Figure 1) is a mobile phone application (app) that provides a set of tools to support parents to manage their 5-11 years old children’s behavioural problems. Its content has been shaped by research about parenting and child behaviour and many years of clinical experience. The content is delivered using short videos and audio clips. There is no personal clinical support for parents using STEPS. The app also includes downloadable resources and space to record audio or written notes. The app aims to be flexible and convenient; parents can move through the eight modules (steps) at their own pace and in their own time. However, the order in which parents progress through the modules is fixed. It takes about 20 minutes to complete each module. For more detailed information about the content and structure of each module see Kostyrka-Allchorne et al. (28). STEPS works on most smartphones using the iOS or Android operating systems. It is available to download through the public app stores; however, access is controlled through the use of study IDs required to register on the app.
Frequency and duration of intervention
Participants in the treatment arm will have access to STEPS for three months after which they will no longer be able to use the app. STEPS is a self-guided intervention and the time needed to complete each module (step) will depend on the pace of the individual user. Completion of the two first modules: ‘Make a fresh start’ and ‘Look after yourself’ will constitute adherence to the intervention.
Intervention records
Parents’ use of the intervention will be recorded automatically within the app. Usage data will include the number of modules started, the number of completed modules – that is the number of modules where the parent progressed through all the mandatory elements contained within that module, the number of recorded reflections, the number of accessed resources, the number of items added to favourites and the amount of time parents spent on watching videos or listening to audio skills.
Criteria for discontinuing or modifying allocated interventions {11b}
The risk of participants experiencing any adverse events during this trial because of using STEPS, and therefore the need to discontinue the intervention, is very low. If there is any indication that a participant has experienced harm due to taking part in the trial or the intervention, a decision of whether they should be withdrawn from the study will be taken by the Chief Investigator in consultation with the Programme Steering Committee (PSC). Participants may withdraw from the trial or the intervention at any time.
Strategies to improve adherence to interventions {11c}
The app has an attractive design and is easy to use. Its usability has been tested with the OPTIMA Patient and Public Involvement and Engagement Panel (PPIE). To promote engagement, automatically programmed digital reminders (text messages) will be sent from the app to participants reminding them to use STEPS. Levels of engagement will be monitored by evaluating the number of completed modules (steps) – data that are collected automatically by the app while it is being used.
Relevant concomitant care permitted or prohibited during the trial {11d}
There will be no restrictions on concomitant care, which will be monitored carefully during the trial through the service use questionnaire (CA-SUS; 29).
Provisions for post-trial care {30}
Not applicable
Outcomes {12}
Child outcomes
Parent-rated behaviour problems (primary outcome at 3 months post-randomisation)
The primary outcome will be measured with the oppositional and defiant disorder (ODD) subscale of the Swanson, Nolan, and Pelham Rating Scale –MTA version (SNAP-IV; 30). The subscale consists of eight items that are rated on a 4-point scale (not at all, just a little, quite a bit, very much). The subscale score is obtained by averaging responses across the eight items. This SNAP-IV ODD subscale is a valid outcome measure for use in clinical trials (31).
Child disruptive and defiant behaviour was selected as the primary outcome because for many parents of children referred to clinical services, this is likely to be the most urgent treatment target at the time of their initial referral.
Parent-rated hyperactivity/impulsivity and inattention
This will be measured with the respective subscales of the SNAP-IV (30). Each of these two subscales consists of 9 items that are rated on a 4-point scale (not at all, just a little, quite a bit, very much). The subscale scores are obtained by averaging across the 9 items associated with the subscale and can be further combined into a single ADHD scale score by deriving an average across the two subscales (31, 32). The SNAP-IV ADHD scale has confirmed validity for use as an outcome measure in clinical trials (31).
Parent-rated emotional problems.
This outcome will be measured by the relevant SDQ subscale (25). This subscale measures fearfulness, anxiety and low mood and consists of 5 positively phrased items rated on a 3-point scale (not true, somewhat true, and certainly true). Individual items’ scores are summed to derive an overall emotional problems subscale score.
NB: This outcome has been added via a substantial amendment after the start of recruitment, therefore, we have added it is as an exploratory outcome. Baseline scores (T1) for all participants will be extracted from the medical records, using information from the SDQ completed nearest to the baseline assessment due date. Three- (T2) and 12-month (T5) data will be collected for most participants online via the Sealed Envelope platform. For any participants already enrolled in the trial and past the T2 timepoint by the time this measure is added, we will aim to extract T2 data from the medical records if a participant completed the SDQ as part of the routine clinical follow-up within the T2 visit window.
Independent observer-rated behaviour problems.
Parents and children will be invited to jointly complete an online drawing task, Etch-a-Sketch Online. It is a newly developed and validated online tool that allows remote observation of parent-child interactions at home (33). During the 5-minute task, parents and children will take turns drawing a simple picture on a mobile phone screen. The task will be audio recorded and the parent-child verbal exchange will be rated using the Child Oppositional and Defiance Speech Sample (CODSS), which has been developed specifically to be used in the present study. CODSS captures the level of four problematic aspects of child behaviour (i.e., being argumentative, defiant, easily annoyed, angry) by an independent researcher using a 5-point scale (not at all, a little, moderately, very, extremely). The overall rating score will be derived by averaging across the items.
Parent and family outcomes
Parenting style: laxness and over-reactivity
Laxness captures a lack of consistent responding; over-reactivity captures overly emotional or harsh responses (34). These outcomes will be measured with the respective 5-item subscales from The O’Leary Parenting Scale (35). The probability of using specific parenting strategies in response to child misbehaviour is rated on a 7-point scale and are anchored by one effective and one ineffective response strategy. Responses to individual items are summed up to derive an overall subscale score.
Parenting satisfaction and efficacy
Parenting satisfaction will be measured with an 9-item subscale from the Parental Sense of Competence Scale (PSCS; 36), which has good validity (37). Parenting efficacy reflects parents’ perceived competence, capability and problem-solving abilities as a parent and will be measured with a 7-item subscale from the PSCS (36). The subscale has established validity (38).
The respective subscale items are positively framed (efficacy) or negatively framed (satisfaction), and parents make responses on a 6‐point scale, with options ranging from “strongly disagree” to “strongly agree”. The respective subscale scores are calculated for each participant by summing up individual item scores.
Parenting-related strain
This will be measured with the global score obtained on the Caregiver Strain Questionnaire (CGSQ; 39). CGSQ consists of 21-items with responses made on a 5-point scale ranging from “not at all” to “very much”. Three indices are calculated for each participant by averaging individual item scores for the three subscales: Objective Strain (11 items), Subjective Internalised Strain (6 items) and Subjective Externalised Strain (4 items). Global Caregiver Strain Score is determined by calculating the sum of the three subscale scores. The scale is a reliable and valid measure of parenting-related strain (Brannan et al., 1997).
The closeness of the child-parent relationship
This will be measured with the closeness subscale of the Child-Parent Relationship Scale – Short Form (CPRS; 40). This 7-item subscale scale measures the extent to which parents feel that their relationship with a child is characterized by warmth, affection, and open communication. It has good validity for measuring child-parent closeness (41). Responses are made on a 5-point scale ranging from “definitely does not apply” to “definitely applies”. The items are summed to obtain a single subscale score.
Other measures
Characterisation of the child’s behaviour problems
This will be measured with the Child’s Challenging Behaviour Scale version 2 (CCBS; 42). The CCBS is a 9-item measure of challenging behaviours for children aged 5-18 years. Parents rate agreement with statements about their child's behaviour on a 4-point scale ranging from “strongly agree” to “strongly disagree”, and the total score on the CCBS is calculated by summing the scores.
Autism spectrum disorder (ASD) symptoms
This will be measured with the Social Communication Questionnaire – lifetime version (43). The SCQ is a 40-item screening measure for autism spectrum disorder and is validated for use with children ages 4 years and older. Questions focus on behaviours that are likely to be observed by the primary caregiver and concern the following domains: reciprocal social interactions, language and communication and repetitive and stereotyped patterns of behaviours. The presence of autism behaviours is coded as 1 and its absence as 0. The first question concerns the level of current language and is not included in the total score. For children with language all questions apply, and the maximum score is 39; for children without language, the first 6 questions do not apply so the maximum score is 33 (language items are inapplicable).
Family characteristics and demographic measures
Parents will provide information about their child’s age and sex, their own sex, their own and their child’s ethnicity, parental education and employment and relationship status and whether there are other children with neurodevelopmental difficulties living in the household. We will estimate family socioeconomic status based on total household income, coded as < £16,000 | £16,000 - £29,999 | £30,000 - £59,999 | > £60,000 | Prefer not to say.
App usage data
To establish intervention adherence, the number of completed modules will be measured (min = 0; max = 8), with the completion of the first two modules constituting adherence to the intervention. Other collected app usage events will include: the number of started modules, the number of videos watched, the time spent watching videos (in seconds), the number of audio clips listened to and the time spent listening to audio clips (in seconds), the number of reflections recorded, the number of items saved to favourites, and the number of accessed text resources. These will be used to provide descriptive information about app usage patterns. The format of the app usage data is being finalised as part of the further app development, so we are not yet able to specify the app usage variable(s) up to T2 that will be used as the effect modifier. Likely options include the number of modules started or the total time engaging with the app. We will aim to specify this in the statistical analysis plan.
Clinical diagnosis
Information about ADHD diagnosis and any pharmacological treatment for neurodevelopmental and mental health disorders will be extracted from medical records at 12-month follow-up. In case it is not possible to access a child’s medical record, information will be collected directly from a parent (via a phone call). The presence of an ADHD diagnosis will be coded as 1 and absence as 0. In addition, if another diagnosis has been given, information about the type of diagnosis will be recorded for descriptive purposes. Similarly, pharmacological treatment for mental health disorders will be coded as 1 and its absence as 0. If pharmacological treatment has been prescribed, information about the name of the drug and dose and a reason for prescribing it will be recorded for descriptive purposes.
Trial expectations
Information about the participants' expectations about parent training in general as well as the specific expectations about the STEPS app will be collected using a questionnaire developed specifically for the study (see Supplementary Information). Parents will be asked to rate the statements on the Likert-type scale as well as to provide open-ended text responses.
Experience of parenting
This will be measured with open-ended questions: “Think of a memorable interaction that you have had with your child within the last 24 hours. Tell us about that interaction. For example, what went well and why or what went badly and why? What might you do differently next time?”
Adverse events
This will be measured with the Medical and Psychological Events and Difficulties Questionnaire (MAPED), which has been developed for the study (see Supplementary Information). Parents will be asked to report retrospectively on any physical and/or mental health difficulties they or their child have experienced in the last three months. They will also be asked to report any difficulties they or their child have had with daily activities. These will be used to monitor safety and to provide descriptive information about adverse events experienced during the trial.
Economic measures
Resource-use data
This will be measured using the Child and Adolescent Service Use Schedule, a measure that has been applied in a range of populations of young people with mental health problems (44, 45) The CA-SUS collects information on the use of all hospital and community-based health and social care services, including health and social care services provided within education settings, service-provided accommodation (for example, Local Authority foster or residential care), and prescribed medications for mental health conditions.
Child’s health-related quality of life
This will be measured with the Child Health Utility measure (46). The CHU9D is a paediatric preference-based quality of life measure for use in healthcare resource allocation decision-making. The CHU9D has been designed for self-report by children aged 7 to 17, but with an interviewer’s help, can also be used in children as young as 6-years-old (47) and guidance is available from the developers for proxy completion by parents for children aged 5 and under. The current study, however, does not involve collecting questionnaire data from children and thus the CHU9D will be proxy completed by the parents/carers of all young participants using the proxy version of the measure. The questionnaire includes 9 items, each with a 5-level response category. Each item focuses on a different domain of children’s present functioning: worry, sadness, pain, tiredness, annoyance, school, sleep, daily routine, and activities.
Parental health-related quality of life
This will be measured with the EQ-5D-5L (48). This questionnaire captures general health across five domains: mobility, self-care, usual activities, pain/discomfort and anxiety/depression. Each domain has five levels (no problem, slight problem, moderate problem, extreme problem and severe problem/unable to complete activities).
Participant timeline {13}
Measures will be taken at baseline (T1), scheduled within one month before randomisation and then at three months (T2 primary outcome, primary timepoint), six months (T3), nine months (T4) and 12 months (T5) post-randomisation. Figure 2 details screening and assessment details at each timepoint.
Sample size {14}
A total of 352 parents and their children will be recruited into the study. This power calculation is based on projected effects at the primary endpoint (3 months post-randomisation). As behaviour problems measured with the ODD subscale of the SNAP-IV questionnaire (30) is our primary outcome, we estimated the smallest difference of clinical importance between STEPS and WAU to be equivalent to an effect size d = 0.4 standard deviations based on the NICE guidance supporting the use of PT for the treatment of ODD or conduct disorder (49). A within-trial drop-out rate of 25% is assumed. This is higher than in most previous trials of face-to-face PT programmes because of the unsupported nature of STEPS in OPTIMA. We will monitor drop-out rates during the internal pilot and, if necessary, recalculate the sample size if it is higher than the 25% estimated. Using Stata (version 14.0) command sampsi, ANCOVA analysis with a conservative zero correlation assumed between baseline and primary endpoint SNAP-IV ODD score, a two-sided test, and an alpha of 0.05, we estimated that 176 individuals will be needed per trial arm (total n = 352) to provide 90% power to test the hypothesis that STEPS is superior to WAU.
Recruitment {15}
Initially, recruitment of parents will take place across five sites delivering secondary (community paediatrics) and secondary/tertiary (child and adolescent mental health and behavioural support services) care and support for children’s behavioural and mental health problems. These sites are in urban areas with catchment populations from a range of socioeconomic backgrounds. However, more sites will likely join during the trial. A complete up-to-date list of study sites will be available on the OPTIMA website www.optimastudy.co.uk and will be reported in the main trial publication.
Identification of potentially eligible parents will occur mainly via myHealthE, a Caldicott Guardian approved, General Data Protection Regulation (GDPR) compliant, online portal for the automated screening of referred families using NHS CAMHS data (50, 51). Using this method will allow quick and efficient screening and obtaining of consent-for-research-contact from families on the waitlist that have very limited contact with the clinicians, who usually act as gatekeepers to referring families to research studies. Other digital (i.e., Interactive CAMHS Assessment Network, ICAN) and non-digital methods of identification (i.e, manual clinical records review) will also be used. All these approaches have been described in detail in the previous publication (28).
Clinicians taking part in the qualitative interviews (n ~ 10;) will be recruited from the services involved in the study. Service managers will be approached with a request to circulate the Clinician Information Sheet to the members of the team and the clinicians interested in taking part will be asked to contact the research team directly.
Assignment of interventions: allocation
Sequence generation {16a}
Once the parent has consented and completed the baseline assessments, they will be randomised to either the STEPS or WAU. Randomisation will be carried out online via a randomisation platform provided by Sealed Envelope in a 1:1 ratio and stratified by trial centre location (London, Nottingham, Southampton) using random permuted blocks procedure with varying block sizes.
Concealment mechanism {16b}
To ensure allocation concealment a centralised service provided by Sealed Envelope will be used. It will not be possible for researchers responsible for randomisation to know the allocation sequence in advance.
Implementation {16c}
The designated trial administrators will access the randomisation service, complete an electronic form and the randomisation allocation will be released. The parent will be notified of the treatment allocation via email or by phone by the researcher who performed randomisation.
Assignment of interventions: Blinding
Who will be blinded {17a}
Parents taking part in the study will remain unblinded throughout the trial; they will be informed about their group allocation after randomisation. Research assistants will remain blinded throughout the trial. The senior statistician and senior health economist will remain fully blinded until a review of the first draft of the final statistical/health economic reports for checking when they will become fully unblinded. Similarly, the Chief Investigator and Principal Investigators in each trial centre will remain fully blinded until they review the finalised statistical report when they will become fully unblinded. The junior statistician and junior health economist will be partially blinded until sign-off of the statistical and health economic analysis plans, after which they will be fully unblinded so they can inspect and utilise app usage/therapy-related data. The trial manager and the trial administrators will be unblinded. The only individuals that will be able to summarise/see data by arm before the review of the statistical report are the junior statistician, junior health economist and the members of the data monitoring committee.
Procedure for unblinding if needed {17b}
No serious harms associated with taking part in the intervention are expected, therefore a formal procedure for unblinding any blinded staff during the study is not needed.
Data collection and management
Plans for assessment and collection of outcomes {18a}
All OPTIMA trial data will be collected remotely: online, by phone and via video/audio chat. For each participant, assessments will take place over 12 months and will be conducted at five timepoints as described above [baseline (T1), T2, T3, T4, T5].
Four categories of data will be collected in the study:
(1) Participants’ reports of the outcome measures and information about adverse events, which will be collected online using Red Pill – a secure electronic data capture and management system provided by Sealed Envelope - or over the phone by a trained researcher. Red Pill will also be used to record data entered directly by researchers, for example, the CODSS ratings or information about diagnosis and treatment.
(2) A speech sample will be audio-recorded during a remote parent-child drawing task (ESO) conducted via Microsoft Teams.
(3) The STEPS app usage data will be collected within Google Firebase and will be stored with Google servers.
(4) Video and audio recordings of qualitative interviews. Whenever possible remote interviews will be conducted and recorded via Microsoft Teams. In case this is not possible, participants will be given an option to take part in a telephone interview, which will also be recorded with Microsoft Teams. Alternatively, a researcher will email a participant a list of questions, and they will be asked to respond in text form.
Plans to promote participant retention and complete follow-up {18b}
In the first instance, participants will be notified by email and/or a text message when a scheduled assessment requires them to complete the questionnaires online using the participant entered forms (ePRO) system provided by Red Pill. They will be able to complete the questionnaires using the browser on their computer or phone. Researchers will be able to track questionnaire completion within Red Pill using a display listing both complete and incomplete questionnaires for each participant. Up to three emails/text messages will be sent as reminders to complete online forms. Non-responders will be followed up with phone calls throughout the assessment window. Participants will be informed that, if they wish, they can also complete the questionnaires or provide information about adverse events over the phone.
As a thank you for taking part, parents will receive shopping vouchers: £10 at baseline, £25 voucher at 3-month (T2) and £10 voucher after completing 6- (T3), 9- (T4) and 12-month (T5) assessments. Children will receive a £5 voucher and a certificate for each completed task.
Data management {19}
The main trial database for the study will be provided by Sealed Envelope, which will ensure there are robust processes for quality management, security and backup of data https://www.sealedenvelope.com/security/. Access to the database and electronic data capture forms will be restricted by user identifiers and passwords to a limited number of researchers (i.e., trial manager and trial administrator, junior statistician, research assistants). Blinded researchers will not be able to view randomisation information within the database and will not have access to the forms that may provide unblinding information (e.g., experience of parenting or MAPED).
Sealed Envelope uses Red Pill, an online application for collecting and managing case report form (CRF) data on participants recruited to a clinical trial or other research studies. In this study, Red Pill will be used to record data collected offline (e.g., phone or medical records) as well as collect data directly from parents themselves [electronic patient-reported outcomes (ePRO)]. The system used in the OPTIMA trial will be specifically configured for the study.
Handling of all data collected in the OPTIMA trial is described in detail in the OPTIMA Data Management Plan (available from the study team). In brief, during the study, extracts of research data and the app usage data will be periodically downloaded from the respective databases (i.e., Red Pill and Google Firebase). These data will be de-identified and will be stored separately from any identifying information on King’s College London cloud storage infrastructure (OneDrive for Business or Microsoft SharePoint). The data stored on KCL cloud storage infrastructure are encrypted and access to the data will be restricted only to those who need to have it. All data will be handled in line with the institutional information governance policies and will follow the GDPR guidelines.
Confidentiality {27}
The Chief Investigator and all members of the research team will take every effort to preserve the confidentiality of participants taking part in the study. To de-identify the data, each participant will be assigned a study ID. Participants' identifiable data required for administrative purposes (e.g., name and contact details) will be stored in a separate file from the data files. These will be accessed only by those members of the research team who are responsible for contacting participants (e.g., to email a link to the online survey) and will be password protected. No individual participant’s data will be identifiable in the publications or reports that may result from this study.
Plans for collection, laboratory evaluation and storage of biological specimens for genetic or molecular analysis in this trial/future use {33}
Not applicable
Statistical methods
Statistical methods for primary and secondary outcomes {20a}
The main analysis will follow the intention to treat (ITT) principle (52, 53). A mixed-effects linear analysis of covariance (ANCOVA) model with repeated measures will be used with the SNAP-IV ODD scores at three (primary outcome), six, nine and 12 months (secondary outcomes) post-randomisation as the dependent variables, and intervention group, time point, intervention group by time point interaction, baseline SNAP-IV ODD score, and the trial centre stratification variable as independent variables, with the interaction term used to extract the primary and secondary outcome STEPS vs WAU mean differences and associated 95% confidence intervals at the four time points. Similar statistical models will be used to test for intervention effects on the other continuous secondary (parent-rated hyperactivity/impulsivity and inattention, parenting style, satisfaction and efficacy, child-parent closeness, and parenting-related strain) and exploratory (average levels of directly observer-rated child behaviour problems (CODSS), parent-rated emotional problems) outcome scale scores. For the binary exploratory diagnosis and medication outcomes, we will use modified Poisson regression with robust standard errors to estimate STEPS vs WAU relative risks (and associated 95% confidence intervals) for getting a diagnosis and being prescribed medication (54).
In addition to ITT analysis, we will undertake a complier average causal effect (CACE) sensitivity analysis (55, 56) on the ODD outcome primary outcome at 3 months (T2) and ODD secondary outcome at 12 months (T5), to estimate treatment effects in those who completed at least two modules of the STEPS app (i.e., those that complied with the intervention).
Further detail of the analyses described in this section will be provided in the statistical analysis plan.
Interim analyses {21b}
The study does not have a formal interim analysis planned but will include an internal pilot study (the first 9 months of recruitment). The objective of the internal pilot is to determine whether recruitment, engagement with the intervention and retention to the trial are sufficient to allow the trial to progress and provide a definitive answer on the effectiveness of the intervention. Progression rules regarding recruitment, treatment engagement and attrition during the internal pilot are specified in Table 1. Briefly, green means that the trial will continue, amber – the research team will review ways of improving with the PSC and the OPTIMA PPIE panel, and red – the trial will be stopped unless there are exceptional mitigating circumstances. The decision to continue or stop the trial will be made independently by the PSC taking advice from the Data Monitoring Committee.
Table 1. OPTIMA RCT internal pilot progression rules.
RECRUITMENT
|
GREEN: > 70% of recruitment target for that period.
AMBER: 50-70%. Lower rate would be also acceptable, if there’s evidence of an upward recruitment trajectory and/or a clear plan for further improvement (e.g., new sites, or ways of recruiting).
RED: < 50% and no evidence of an upward trajectory.
|
INTERVENTION ENGAGEMENT
|
GREEN: > 90% of participants providing primary outcome data at primary endpoint (3 months post-randomisation) will have engaged with the two first modules of the intervention.
AMBER: 60-90%. Lower rate also acceptable if evidence of improvement or/and a clear plan for improving engagement.
RED: < 60% and no evidence of improvement.
|
ATTRITION
|
GREEN: > 75% of participants who have reached the relevant time window will have completed primary endpoint assessments to provide primary outcome data.
AMBER: 50-75%. Lower rate is acceptable if evidence of improving retention and/or a clear plan for enhancing retention.
RED: <50% and no evidence of improving retention.
|
Methods for additional analyses {20b}
Exploratory post-randomisation effect modifier analysis
Two exploratory post-randomisation effect modification analyses will be conducted. First, we will analyse the impact of the app usage (likely in terms of a number of completed modules or total time spent in the app) collected up to T2 on the effects of STEPS on the primary ODD outcome measured at 3 months post-randomisation (T2) and the secondary ODD outcome measured at 12-months (T5). Second, the impact of clinical contact/care: up to 3 months (T2) on the effects of the STEPS app on the ODD primary outcome at 3 months post-randomisation, and up to 12 months post-randomisation (T5) on the effects of the STEPS app on the ODD secondary outcome at 12 months. Clinical contact/care will include any treatment during the trial up to the two specified time points. The clinical care variables used as post-randomisation effect modifier will be extracted from the service use form (the CA-SUS) and will likely be somewhat post-hoc as we are not currently sure which variables are most appropriate; this approach will be made clear to the reader in the subsequent publication. We plan to use appropriate methods for post-randomisation variables to explore whether the effects of interventions differ by these variables, such as principal stratification analysis (56). Further details will be provided in the statistical analysis plan.
Health economic analysis
The economic evaluation will adopt the National Health Service (NHS)/personal social services perspective preferred by NICE, including health and social care services provided in education settings services, given the age of the population. Resource-use data collected using the CA-SUS will be costed using nationally applicable unit costs (e.g., Personal Social Services Research Unit Costs of Health and Social Care, NHS Reference Costs, British National Formulary for medications). The STEPS app will be costed in consultation with the application developers. Patterns and potential mechanisms influencing missing data will be explored to inform appropriate methods for handling missing data such as multiple imputation.
The primary economic evaluation will be a cost-utility analysis carried out at 3-months post-randomisation with outcomes expressed in terms of quality-adjusted life years (QALYs), using the proxy version of the CHU9D completed by a parent. Secondary economic analyses will include: (i) a cost-utility analysis at 12-months post-randomisation to assess cost-effectiveness after formal assessment and treatment; (ii) a cost-effectiveness analysis undertaken at both 3-months and 12-months using the primary clinical measure of outcome (SNAP-IV ODD score) and (iii) a cost-utility analysis undertaken at both 3-months and 12-months combining QALYs for both the young person (using the CHU9D) and the primary parent/caregiver (using the EQ-5D-5L). Appropriate sensitivity analyses will be carried out, dependent on any assumptions made about the costing of the intervention, the method of measurement of outcomes or the approach to combining QALYs for young people and their primary carer.
Costs and outcomes will be compared at the 3-month and 12-month follow-up points and presented as mean values by the trial arm with standard deviations. Mean differences in costs and 95% confidence intervals will be obtained by non-parametric bootstrap regressions to account for the non-normal distribution commonly found in economic data. To provide more relevant treatment-effect estimates, analyses will include an adjustment for baseline covariates (57), which will be pre-specified and in line with the clinical analyses. Cost-effectiveness will be assessed using standard net benefit approaches (58). A joint distribution of incremental mean costs and effects for the two groups will be generated using non-parametric bootstrapping to explore the probability that each of the treatments is the optimal choice, subject to a range of possible maximum values (ceiling ratio) that a decision-maker might be willing to pay for an additional QALY (or unit improvement in behaviour problems). Cost-effectiveness acceptability curves will be presented by plotting these probabilities for a range of possible values of the ceiling ratio (59). These curves are a recommended decision-making approach for dealing with the uncertainty that exists around the estimates of expected costs and expected effects associated with the interventions under investigation and uncertainty regarding the maximum cost-effectiveness ratio that a decision-maker would consider acceptable.
Process evaluation
The process evaluation will follow the Medical Research Council guidelines for evaluating the implementation of complex interventions (25) and will be described in more detail in a separate protocol. Briefly, the process evaluation will use baseline data and post-intervention quantitative trial data (e.g., app usage data, including counts and time), qualitative data from semi-structured interviews with parents and clinicians (e.g., expectations, perceptions of impact, barriers to engagement), and textbox responses to open questions to explore the mechanisms for intervention engagement and impact (child behaviour change).
All parents will be invited to respond to questions about: (1) their expectations of the trial and (2) accounts of their experience of parenting during trial participation. Post-intervention, researchers will conduct remote (telephone, video call or email) in-depth, semi-structured interviews with a subgroup of parents in the intervention group (n ~ 50). Maximum variation, purposive sampling will be used to ensure that a full range of views and experiences are captured (taking account of demographic factors and levels of app engagement). In addition, telephone interviews with clinicians (n ~ 10) will explore perceptions of the app and its perceived impact on preparing families for the formal clinical assessment. The interview schedules will be developed with Patient and Public Involvement (PPI) panel input. Interview data will be audio-recorded and analysed using thematic analysis (60), applying a framework approach (61).
Methods in analysis to handle protocol non-adherence and any statistical methods to handle missing data {20c}
Missing data will be dealt by using maximum likelihood methods to fit the mixed models described in the Statistical Analysis section (20a), with baseline variables predicting missing data included in the models. We will consider performing multiple imputation (MI) for primary and secondary outcomes only if there are post-randomisation variables that are predictive of missingness for these measures, in particular the measure of adherence to the intervention described in the Intervention Description section (11a), and the proportion of participants with missing values for any of the outcome variables is equal to or greater than 10%.
Plans to give access to the full protocol, participant level-data and statistical code {31c}
The full protocol is available in Supplementary Information. Requests for data and statistical code should be directed to the corresponding author.
Oversight and monitoring
Composition of the coordinating centre and trial steering committee {5d}
The Programme Steering Committee (PSC), a body independent of the research team, chaired by Professor Tamsin Ford, will provide formal oversight and expert advice for the overall OPTIMA programme, which includes oversight over the present trial. Its role is to ensure that the trial is conducted in a rigorous and timely manner and consider any proposed changes to the agreed programme of research. The PSC consists of an independent chair, statistician, digital health expert, digital mental health interventions expert, two parents of a child with ADHD, and a health economics expert.
Composition of the data monitoring committee, its role and reporting structure {21a}
The Data Monitoring Committee (DMC) is an independent body of experts, chaired by Professor Chris Metcalfe, that has been established to monitor the quality of trial data and the safety of participants. The DMC will have access to unblinded data if they wish. The DMC will be responsible for monitoring the overall conduct of the study, including recruitment, protocol compliance, accuracy and completeness of data collection. Based on this information, the DMC will make recommendations to the PSC, the Funder and the Sponsor on whether the study should continue or whether there are any ethical or safety reasons why the study should be modified or terminated. Any key changes to the study design and methodology will be reviewed by the DMC. The members of the DMC are completely independent of the trial and consist of an independent chair, statistician, and clinical expert. The DMC will agree a DAMOCLES charter, and the PSC will agree terms of reference to outline their tasks and responsibilities.
Adverse event reporting and harms {22}
The risk of participants experiencing any adverse events during this trial as a result of using STEPS is very low. Adverse events concerning parental or child physical and mental health will be monitored throughout the trial. Any physical or mental health difficulties spontaneously disclosed by a parent in their communication with researchers will be entered into the Red Pill database. Moreover, at each timepoint, participants will be asked to complete a formal questionnaire on adverse events (MAPED) that happened to them and/or their child. This information will be collected online using ePRO. The forms will be reviewed regularly, and all adverse events will be recorded on the OPTIMA adverse events form in the Red Pill database. All participants that experience a serious adverse event will be followed-up by the researchers, who have completed Level 2 safeguarding training, until the event is resolved. Where necessary, the participant’s clinical service which accepted the referral will be informed about the event.
Frequency and plans for auditing trial conduct {23}
Not applicable.
Plans for communicating important protocol amendments to relevant parties (e.g. trial participants, ethical committees) {25}
In case of new information becoming available, which may result in significant changes to the risks and benefits of taking part, the Participant Information Sheet and informed consent form will be reviewed and updated accordingly. All participants actively enrolled in the study will be informed of the updated information and will be given a revised copy of the Participant Information Sheet and informed consent form to confirm their wish to continue taking part.
Dissemination plans {31a}
A complete account of the trial results will be published in a high impact peer-reviewed scientific journal and a full report for the funder. Authorship will be determined according to COPE standards based on the individual contributions. There will be no dissemination of the primary trial results until they are accepted for publication in peer-reviewed journal. In addition, the findings will be disseminated through oral and poster presentations at a range of conferences and seminars in the UK and overseas. There will also be a general dissemination programme for clinicians, commissionaires, and parents through the OPTIMA website.