Protocol for the Process Evaluation of the Online Remote Behavioural Intervention for Tics (ORBIT) Randomised Controlled Trial for children and young people

doi:10.21203/rs.2.10297/v1

Download PDF

Study protocol

Protocol for the Process Evaluation of the Online Remote Behavioural Intervention for Tics (ORBIT) Randomised Controlled Trial for children and young people

https://doi.org/10.21203/rs.2.10297/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 02 Jan, 2020

Read the published version in Trials →

You are reading this older preprint version

Read the latest preprint version →

Background

Process evaluations are an important component in the interpretation and understanding of outcomes in trials. The ‘Online Remote Behavioural Intervention for Tics’ (ORBIT) study is a randomized controlled trial evaluating the effectiveness of an internet delivered behavioural intervention (called BIP TIC) compared to an internet delivered education program aimed at children and young people with tics. A process evaluation will be undertaken alongside the main trial to determine precisely how the behavioural intervention works and ascertain whether, and if so, how, the intervention could be successfully implemented in standard clinical practice. This protocol paper describes the rationale, aims, and methodology of the ORBIT trial process evaluation.

Methods

The process evaluation will use a mixed-methods design following the UK Medical Research Council’s 2015 guidelines, comprising of both quantitative and qualitative data collection. This will include: analyzing data usage of participants in the intervention arm; purposively sampled, semi-structured interviews of parents and children, therapists and supervisors, and referring clinicians of the ORBIT trial, as well as analysis of qualitative comments input into the online therapy platform by participants at the end of treatment. Qualitative data will be analyzed thematically. Quantitative and qualitative data will be integrated in a triangulation approach, to provide an understanding of how the intervention works, and what resources are needed for effective implementation, uptake and use in routine clinical care.

Discussion

This process evaluation will explore the experiences of participants, therapists and supervisors, and referring clinicians of a complex online intervention. By contextualising trial efficacy results, this will help understand how and if the intervention worked and what may be required to sustain the implementation of the treatment long-term. The findings will also aid in our understanding of factors that can affect the success of complex interventions. This will enable future researchers developing online behavioural interventions for children and young people with mental health and neurological disorders to gain invaluable information from this process evaluation.

Integrative & Complementary Medicine

Process evaluation

complex intervention

mixed methods

tics

Tourettes

children and young people

randomized controlled trial

protocol

There is growing interest within healthcare as to how advances in technology can be used in developing effective treatments for people with psychological and neurological disorders [1]. Although children and young people (i.e. individuals up to the age of 18; CYP) make up a large proportion of the population with psychiatric and neurological conditions [2, 3], there is limited access to evidence-based treatments aimed at reducing symptoms in this population. Access to services for CYP is the lowest amongst all demographics [4] with only 25% of CYP receiving appropriate treatments [5]. Behavioural treatments, in particular, are desirable and highly recommended by healthcare professionals as a first line treatment in reducing symptoms in CYP due to the limited side effects relative to pharmacotherapy [6, 7]. However, these treatments are often difficult to access and CYP may avoid face-to-face therapy due to stigmatization [8]. Due to their affinity for technology, a promising development that may benefit CYP are online or digital health interventions (DHI). Randomized controlled trials (RCTs) have shown that DHI can be effective in treating psychological and neurological symptoms for CYP [9-11] but they can also be ineffective [12, 13]. Hence, before any new DHI is introduced, clinicians, patients and commissioners need robust research to determine efficacy. However, data on efficacy alone is insufficient to inform effective implementation and uptake in routine health care. Data are also required on acceptability, uptake, and use of the intervention, including any apparent impact of the digital divide on health inequalities, as well as on the resources and activities required to achieve effective implementation.

Despite this, little is known about how, and for whom in particular, they work and what makes them effective in one context and not another and the barriers to effective implementation [14, 15]. The UK Medical Research Council (MRC) has developed specific guidelines for conducting a process evaluation of complex intervention to assess quality of implementation (fidelity), dose, reach, adaptations, analyze causal mechanisms, and identify any contextual factors [16]. Process evaluations can therefore aid interpretation and understanding of trial outcomes as well as informing future refinements of the intervention under study.

Grant et al. [17] have identified the importance of outlining process evaluation methodology a priori and consider the publication of process evaluation protocols as ‘best practice’ in order to improve trial quality. Despite the increasing popularity in conducting process evaluations of complex interventions [15, 18] and the aforementioned importance of publishing protocols, explicit guidelines for publishing process evaluation protocols are limited [19].

Using previously published process evaluations of complex interventions protocols as a guide [20, 21], here we outline the methodology and describe the planned process evaluation of the Online Remote Behavioural Intervention for Tics (ORBIT) trial.

The ORBIT trial and its BIP TIC intervention have been described in detail previously as part of the main trial protocol [22] (03/01/2019; version 3.0), a brief summary is given to provide context to the process evaluation design. The ORBIT trial is a 10-week, parallel group, single blind, RCT with an internal pilot. ORBIT aims to evaluate the efficacy of an online, remote, therapist supported and parent-guided behavioural intervention for tics, initially developed and piloted in Sweden called BIP TIC [23]. The comparator is an online, remote, therapist supported and parent-guided psychoeducation program for tics. Participants will be recruited from clinics, Patient Identification Centres (PICs) across National Health Service (NHS) trusts, or from the two study sites involved in the trial (Queen’s Medical Centre (QMC), Nottingham and Great Ormond Street Hospital (GOSH), London), or via a tic disorder charity (Tourettes Action), the ORBIT study website, or social media. Participants need to be aged 9 to 17 years old and be suspected or confirmed as having Tourette syndrome (TS) or chronic tic disorder (CTD) and must not have had any form of behavioural treatment for tics in the last 12 months or a change in medication for tics in the previous two months. Participants will be followed-up mid treatment, and at 3-, 6-, 12-, and 18-months post-randomization.

Participants will be randomized to one of two groups. The intervention group will receive 10 self-help modules of behavioural therapy delivered over a period of 10-12 weeks, which will be accessed via a secure online platform [23]. The behavioural therapy will follow evidence-based Exposure and Response Prevention (ERP) therapeutic principles, whereby patients learn strategies for managing their tics through allowing premonitory urge sensations to come to the fore and actively tolerate the premonitory urges and suppress their tics. In doing so, the child masters their ability to tolerate the urge, control their tics, and is able to do so for an increasing amount of time in a hierarchical manner. The child also receives education about tics for the family and others, such as teachers, friends and family. The parent components contain information about how to support their child and various coping strategies for themselves. Previous studies have shown that ERP is effective in reducing tics [24, 25], with European clinical guidelines [24] and a National Institute of Health Research Health Technology Assessment Evidence Synthesis [26] recommending that behaviour therapy should be offered as a first-line intervention for tics in CYP. The primary outcome measure is the severity of tics as measured on the Total Tic Severity Score (TTSS) subscale of the Yale Global Tic Severity Scale (YGTSS) [27].

Overall, the ORBIT trial aims to evaluate the clinical effectiveness of an online behavioural treatment for CYP with tics compared to online tic-related education in reducing tics, as measured by the YGTSS TTSS. Furthermore, the trial aims to evaluate the cost-effectiveness of the online treatment and to estimate the longer-term impact on patient outcomes and health service costs.

The aim of the ORBIT process evaluation is to understand the causes of the observed efficacy data obtained from the RCT, and in particular, to explore the fidelity of intervention delivery, acceptability of the intervention, reasons for observed variation in uptake and use, and consider the resources and implementation processes required.

Specific objectives are:

To assess the fidelity, reach, and dose of intervention delivery.

To explore whether any of the intervention features were adapted for individual needs enabling potential recommendations for adaptations.

To explore BIP TIC from the perspective of children, parents, therapists, and clinicians in order to gain a deeper understanding of potential mechanisms underlying participant behaviour change whilst probing for any unexpected consequences.

To evaluate any factors external to BIP TIC that may have affected delivery (i.e. the environment and its characteristics) or whether its mechanisms of impact worked as intended.

To consider the resources and implementation processes required for effective implementation, uptake and use of the intervention.

The design of this process evaluation is guided by MRC directives on the process evaluation of complex interventions [16]. The MRC outline three essential components in understanding how outcomes are achieved: implementation, mechanisms of impact, and context. The application of these guidelines in the context of the ORBIT trial will be as follows:

Implementation: an exploration as to how delivery of BIP TIC was achieved by examining quality (fidelity) and quantity (dose) of what was implemented. The structures and processes through which BIP TIC was delivered as intended, any adaptations made, and establishing the extent to which BIP TIC reached its intended audience (reach).

Mechanisms of impact: an examination of the causal mechanisms through which BIP TIC produces change by understanding how participants interact with the intervention. This also allows for an identification of any unexpected pathways and consequences.

Context: an exploration of any factors external to BIP TIC, which may have influenced its implementation (e.g. home life for the family, school life for the child, system factors in health services).

MRC guidance on the development and evaluation of complex intervention notes that identifying and developing a theoretical understanding of the likely process of change is a key early task for developing a complex intervention or evaluating one that has already been developed. MRC guidelines stipulate an important component of a process evaluation is to outline the processes of the intervention and the outcomes it aims to achieve by means of a logic model. The logic model for the study is shown in Figure 1.

Fig 1. Logic model for the BIP TIC intervention

The overall design of the ORBIT process evaluation is a mixed-methods study using purposively sampled qualitative data together with quantitative data from the trial. This will involve semi-structured interviews with children, parents, therapists and supervisors and clinicians as well as analyzing online feedback from participants together with data from the online platform, such as total therapist time, number of chapters viewed, and number of log-ins.

The schedule of the ORBIT process evaluation procedures is displayed in Figure 2. In Additional File 1 a populated Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) checklist is provided.

Ethical approval for the process evaluation was obtained from North West - Greater Manchester Central Research Ethics Committee as part of the ORBIT trial (REC: 18/NW/0079).

Fig 2. Schedule of ORBIT and process evaluation procedures

Qualitative data collection

Qualitative data will be collected by interviewing participants in the BIP TIC intervention (both CYP and parents, either separately or as a dyad), therapists and supervisors and referring clinicians. Interviews with therapists and supervisors involved in the ORBIT trial will be conducted early in the trial and near the end of recruitment in order to gain an understanding of their experience at different time points.

Sampling and recruitment for interviews

Children and parents

In line with previous literature [28, 29], four semi-structured interview schedules were developed (see Additional File 2). The child and parent interview schedules were drafted and underwent revision from the main researcher and three academics. Questions include: (a) how they found out about the ORBIT trial; (b) why they took part; (c) their initial expectations; (d) their views of the content, structure, and the different chapters of the online program; (e) what impact the therapy had, if any, on their tics; (f) what they found most and least helpful; (g) barriers to participation; (h) how they felt about communicating with their therapist; (i) if they would alter anything about the program; (j) their recommendations for improvement of the interventions and their overall experience of participating in the trial.

The revised drafts were sent to two dyads of the Patient and Public Involvement (PPI) group — including two children with tics — for feedback and were revised accordingly.

All interviews will be carried out with CYP and parents of CYP following completion of the intervention at the three-month (primary end-point) follow-up assessment in the main trial. Recruitment for the interviews began in August 2018 through the following methods:

Following completion of the primary end-point, the researcher conducting the follow-up assessment asks participants if they are willing to be contacted about taking part in an interview. If the participant agrees, the researcher informs the process evaluation researcher who makes contact with the family.

Researchers at both QMC and GOSH arranges a convenient date, time, and method for interview to participants who agree following their primary end-point follow-up assessment.

A proportion of the participants are to be contacted by telephone following their primary end-point assessment by the main researcher of the process evaluation.

Participants will only be contacted if they gave explicit written consent to participate in an interview for the ORBIT trial and, for a child under 16; assent was obtained with parental consent (see Additional File 3). Participants will be purposively sampled with the intention of collecting data from a diverse cohort to obtain varying views on the intervention. This will include ensuring perspectives from a range of ages; gender, ethnicity, and level of interaction with the intervention are voiced. We anticipate that this sampling strategy will result in sufficient heterogeneity to provide examples of both relatively poor and relatively good adoption, delivery and maintenance, and will allow us to identify barriers and facilitators to implementation and to generate hypotheses about factors that may be associated with differing outcomes for CYP in the intervention arm.

The target for participant interviews is CYP (n=>20) and parents of CYP (n=>20). This will ensure that data will reach a level of saturation [30] and to enable a diversity of views.

Therapists

The therapist interview schedules were drafted and underwent revision from the main researcher and three academics, as well as input from a therapist and clinical researcher with specific expertise in the field. Therapist questions include: (a) their role on the ORBIT trial; (b) how they found out about ORBIT and why they got involved; (c) what specific skills they felt a therapist needed for the program; (d) any training needs identified; (e) how they managed ORBIT around other commitments; (f) their experiences of receiving/giving supervision sessions; (g) if the therapy is being delivered as planned; (h) their experiences of interacting with participants; (i) their views on the two trial arms; (j) and their recommendations for future use.

Therapists will initially be interviewed individually early into the trial (halfway through the study) and then interviewed again near the end of the trial. This will allow for a range of experiences at different time-points to investigate trial progression. The target for therapist interviews is n=>5, of which two are supervisors.

Clinicians

The clinician interview schedules were drafted and underwent revision from the same team and were guided by normalization process theory (NPT) [31, 32]. As the purpose of the clinician interviews were to explore their views about the feasibility of integrating the intervention into everyday practice, including any potential barriers to or facilitators of this, NPT framework approach seemed the most appropriate. The clinician interview schedule questions aim at eliciting information on how they got involved in the ORBIT trial and why, their experience of recruiting for the trial including factors that affected recruitment, and how the NHS could incorporate the intervention into everyday practice. Clinicians will be purposively selected from the PIC sites involved in recruiting for ORBIT and the target for clinician interviews is n>5.

Quantitative data collection

Online data will be collected and recorded from participants throughout the trial. This includes the following measures: total therapist time; therapist time specific to each therapist; therapist time specific to each child and parent; total number of characters submitted by child and parent (as part of communication messages via the online system); total number of logins for child and parent; average time between each login (in days) for child and parent; average pages visited per login for child and parent; and the five most frequently visited pages per child and parent. This data will be amalgamated and entered into a centralised online database whereby the main researcher will then extract this data for analysis as part of the process evaluation.

Trial data

As part of the quantitative measures for the process evaluation, we will also be extracting and analyzing change in YGTSS TTSS from baseline to primary end-point, demographic data, overall symptom improvement as measured on the Clinical Global Impressions Scale (CGI) improvement [33], depressive symptom change from baseline to primary end-point as measured on The Mood and Feelings Questionnaire (MFQ; Child completed version) [34], and service use data as measured by the modified Child and Adolescent Service Use Schedule (CA-SUS) [35].

Table 1 presents a summary of the data sources that will be used to inform each component of the process evaluation.

Table 1. Process evaluation components, areas of research, measures and sources of data

Data analysis

Qualitative data will be exported and analyzed in QSR International’s NVivo 12 Software [36] and quantitative data will be exported and analyzed in SPSS (version 25.0) [37]. Process evaluation data will be analyzed autonomously of the main outcome data of the ORBIT trial.

Qualitative data analysis

All interviews will be recorded either by the WebEx videoconferencing application or by Dictaphone and then transcribed verbatim. Transcripts will be checked for accuracy against the recordings with any corrections made as appropriate. Prior to the transcripts being imported into QSR NVivo 12, any reference to places, clinicians, therapists, and/or family members that may reveal participants’ identity will be redacted, and all participants’ names will be anonymized. The interviewer will take notes during all interviews.

As the process evaluation is a combination of exploration and description, thematic analysis will be used to identify, analyze and report patterns within the transcribed interviews. Thematic analysis is widely used within the field of psychology and is considered the most flexible qualitative analytical process [38]. More broadly, the Framework Method [39] of analysis will be employed, as it is most commonly used for the thematic analysis of semi-structured interviews [40]. Moreover, Ritchie and Spencer [39] outline four types of research questions that they believe framework analysis can helpfully address: 1) Contextual - identifying the form and nature of what exists (e.g. what is the nature of people’s experience?); 2) Diagnostic - examining the reasons for, or causes of, what exists (e.g. why are services or programmes not being used?); 3) Evaluative - appraising the effectiveness of what exists (e.g. what affects the successful delivery of programmes or services?); and 4) Strategic - identifying new theories, policies, plans or actions (e.g. how can systems be improved?). As the process evaluation covers all of these questions, we feel this is the appropriate methodology to use.

Ritchie and Spencer [39] suggest five key stages of framework analysis: familiarization, identifying a thematic framework, indexing, charting, and mapping and interpretation. During the familiarization stage, the main researcher will immerse himself in the data by listening and/or watching back the interviews, reading transcriptions, and studying observational notes whilst listing key ideas and recurring themes. The data will then be analyzed to identify key issues, concepts, themes, and sub-themes drawing on both a priori and emergent issues. Next, the transcripts will be coded and indexed into framework categories by systematically applying the thematic framework to each interview. The indexed data will be summarized for each category and organised in chart form. This process will involve working through each framework category, summarizing all data that have been indexed to that category, and then providing a summary for each category, for each participant, using headings and subheadings. Consequently, key characteristics of the holistic data set will be mapped and interpreted. A subset of transcripts will be double coded by an independent coder to identify emergent patterns and themes relating to participants’, therapists and clinicians experiences of the ORBIT trial. Charted data will be annotated independently with discussions taking place on these findings, which will allow for a refinement and amendment of data in an iterative process. Once confidence in the congruity and meaningfulness of interpretation is established between researchers, we will review the remaining interviews to establish whether our understanding has reached acceptability.

The large amount of data collected for the process evaluation encouraged us to use computer assisted qualitative data analysis software (CAQDAS). One CAQDAS package, QSR NVivo 12, is fully integrated with framework analysis and this will be used to categorise data and document any themes and sub-themes. Online feedback given by participants at the end of therapy will also be analyzed using the aforementioned methodology.

Quantitative data analysis

Quantitative data from the online platform will be subject to descriptive statistical analysis with total numbers and percentages and mean with standard deviation or median (range), if not normally distributed, being presented. This will provide information on intervention delivery, including the implementation of different components and fidelity. Independent samples chi-squared and t tests will be calculated to explore any significant differences within the intervention group. For data not normally distributed, non-parametric alternatives will be used (i.e. Kruskal–Wallis H and Mann–Whitney U tests), using a significance level of P<0.05.

Mixed methods analysis

Qualitative and quantitative data will be analyzed separately and then mixed during analysis in a methodological approach known as triangulation [41]. Both qualitative and quantitative data will be given equal importance, as both sets of data are central to addressing the research questions posited by the process evaluation. In Additional File 4 a Good Reporting of A Mixed Methods Study (GRAMMS) checklist has been provided.

Coding of qualitative data and preliminary qualitative analysis will be conducted synchronously with the analysis of descriptive statistics of participants’ online data. Thus, the descriptive data will aid in the refinement and amendment of questions central to qualitative data collection. In other words, key themes may emerge from the quantitative data, which could then be further explored or clarified from qualitative data, and vice versa. The main researcher will integrate and compare outcomes from the various data sets guided by triangulation protocol. The aim of this is to create a matrix of converging data sets to assess outcomes where there is agreement, dissonance, and where themes or outcomes emerge in one dataset but not another. Once the matrix of outcome synthesis from the various datasets is finalized, it will be used to emphasise the mechanisms of impact, implementation fidelity, and, more broadly, explain the outcomes of the trial.

Integration of findings

The process evaluation data will be analyzed prior to knowing the main ORBIT trial results with the two analyses being independent of each other. The ORBIT trial team will be unaware of the findings of the process evaluation until the primary outcomes from the main trial have been analyzed. Once both trial and process evaluation analyses are complete, combined qualitative and quantitative data may aid in the development of hypotheses about the potential successful implementation in one context over another and how and why some components were delivered successfully and others were not. Furthermore, the analysis of different components may aid in the identification of causal mechanisms and how and why individual intervention components were more effective than others were. Following quantitative analysis of ORBIT trial data, qualitative data from the process evaluation can potentially be used to help explain outcomes of the trial. Additional analyses can then be conducted to test hypotheses emanating from integration of process evaluation data with trial outcomes, drawing together the findings to understand why the intervention worked (or not), context, and implications for further dissemination to improve provision of care for CYP with tics.

This protocol outlines the rationale, design and methodology for the planned mixed methods process evaluation of BIP TIC, a complex online intervention for CYP with tics. The process evaluation is designed to explore the implementation of the online intervention and provide a holistic view of trial outcomes. By explicitly outlining our process evaluation methodology, guided by MRC framework of complex intervention trials [16], this paper adds to the literature of process evaluation protocols using a mixed methods design. In doing so, this will improve the integrity of this process evaluation and, as mentioned, there is growing emphasis on the importance of publishing process evaluation protocols in advance to improve overall trial quality and reporting [17].

The combined qualitative and quantitative process evaluation data will support the homogenous interpretation of the main outcome data from the ORBIT trial. By providing an illumination of how and why BIP TIC was effective or not, the process evaluation will help elucidate a holistic view of the intervention. Moreover, understanding the mechanisms of impact and any contextual factors, this data will augment the dissemination plan and may support the long-term implementation of the intervention. The process evaluation will also offer insight into digital interventions and may inform future development of such health technologies.

Strengths and limitations

Conducting the process evaluation will contribute to explaining the overall findings of the main RCT: the factors underlying positive and negative effects of different aspects of BIP TIC. For example, if there were certain negative outcomes from using BIP TIC, the process evaluation will be an invaluable resource in elucidating whether the intervention was inherently inadequate, if there was a failure of implementation, and if this was related to participants (e.g. lack of motivation) or contextual factors (e.g. pre-existing beliefs of online therapy). This would help to improve the intervention progressively.

In contrast, if there were positive outcomes from using BIP TIC, the process evaluation will identify the core components that made the intervention a success. For example, if it was determined that an essential component for promoting participants’ adherence to the intervention was the use of parental support and therapist encouragement, these findings will be crucial to the development and implementation of future digital programs aimed at CYP with tics.

By collecting data from a range of relevant stakeholders (e.g. parents, children, therapists and supervisors, and clinicians) and combining quantitative and qualitative data, we will gain a holistic understanding of the mechanisms underlying the impact of the intervention. Furthermore, the proposed sample size is adequate to capture a comprehensive overview of perspectives, generating rich data and analytical depth.

One potential limitation arising from this is that the majority of participants who drop out of treatment are more likely to refuse to be interviewed, which could lead to a more positive overall evaluation of the intervention. We will attempt to overcome this by making a more concerted effort to recruit participants who drop out of treatment or, if this is not possible, those who complete fewer modules. The main limitation in terms of future implementation is that the environment/context will be heavily influenced by this study being an RCT. It would arguably be more appropriate to conduct a parallel implementation study, however lack of resources prohibit this.

Trial status

This protocol is version 3.0; 24/04/2019. Recruitment for the process evaluation began on 22^nd August 2018 and we aim to complete recruitment for the interviews by 31^st December 2019. The ORBIT protocol is version 3.0; 03/01/2019. Recruitment for the main ORBIT trial began on 8^th May 2018 and we aim to complete recruitment on 1^st October 2019.

CTD: chronic tic disorder; CYP: children and young people; DHI: digital health intervention; ERP: Exposure and Response Prevention; MRC: Medical Research Council; NHS: National Health Service; ORBIT: Online Remote Behavioural Intervention for Tics; PIC: Patient Identification Centres; PPI: Patient and Public Involvement; RCT: randomized controlled trial; SPIRIT: Standard Protocol Items: Recommendations for Interventional Trials; TS: Tourette syndrome; TTSS: Total Tic Severity Score; YGTSS: Yale Global Tic Severity Scale.

Ethics approval and consent to participate

Ethical approval for the conduct of the study was gained from the North West - Greater Manchester Central Research Ethics Committee (REC: 18/NW/0079). We sought written parental consent, and written informed assent/consent to participate in the study from children and young people. This covered process evaluation measures.

Consent for publication

Not applicable.

Availability of data and material

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Funding

The Ph.D. is funded by NIHR MindTech MedTech Co-operative and NIHR Nottingham BRC Mental Health & Technology Theme. The Ph.D. explores the process evaluation of the Online Remote Behavioural Intervention for Tics (ORBIT) Trial, which is funded by the NIHR Health Technology Assessment (HTA) (Ref 16/19/02).

CG, EBD, and CH acknowledge the financial support of the NIHR Nottingham Biomedical Research Centre and NIHR MindTech MedTech Co-operative. This research was supported by the National Institute for Health Research Biomedical Research Centre at Great Ormond Street Hospital for Children NHS Foundation Trust and University College London

All research at Great Ormond Street Hospital NHS Foundation Trust and UCL Great Ormond Street Institute of Child Health is made possible by the NIHR Great Ormond Street Hospital Biomedical Research Centre. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.

The funding sources had no role in the design, collection, analysis and interpretation of data, or in the writing of the manuscript.

Authors' contributions

CH is PI on the ORBIT trial and wrote the original grant application. KK drafted the manuscript and edited multiple drafts. CG is leading the design and implementation of the process evaluation along with KK, with input from CLH and EBD. CG, CLH, EBD, DMC, PA, TM, BB, and EM read and provided critical feedback on multiple drafts. All authors read and approved the final manuscript.

Acknowledgements

The authors thank Tourettes Action for their ongoing support with the ORBIT trial and particularly acknowledge Dr Seonaid Anderson for her help and advice. They thank the PPI members for their help in revising the interview schedules, including James Bungay, Claire Bungay, Sandra Wang, and Marco Wang.

1. Fairburn CG, Patel V. The impact of digital technology on psychological treatments and their dissemination. Behav Res Ther. 2017;88:19-25.

2. Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62(6):593-602.

3. McGorry PD, Purcell R, Hickie IB, Jorm AF. Investing in youth mental health is a best buy. Med J Aust. 2007;187(7):S5-S7.

4. Gibb SJ, Fergusson DM, Horwood LJ. Burden of psychiatric disorder in young adulthood and life outcomes at age 30. Br J Psychiatry. 2010;197(2):122-7.

5. Sanci L, Lewis D, Patton G. Detecting emotional disorder in young people in primary care. Curr Opin Psychiatry. 2010;23(4):318-23.

6. Cuenca J, Glazebrook C, Kendall T, Hedderly T, Heyman I, Jackson G, et al. Perceptions of treatment for tics among young people with Tourette syndrome and their parents: a mixed methods study. BMC psychiatry. 2015;15:46.

7. National Institute for Health and Care Excellence: Clinical Guidelines. Depression in children and young people. Identification and management in primary, community and secondary care. Clinical Guideline, 28. London: National Institute for Health and Clinical Excellence; 2015 [Available from: https://www.nice.org.uk/guidance/cg28/chapter/1-Recommendations#/care-of-all-children-and-young-people-with-depression.

8. Gega L, Marks I, Mataix-Cols D. Computer-aided CBT self-help for anxiety and depressive disorders: experience of a London clinic and future directions. J Clin Psychol. 2004;60(2):147-57.

9. Rice SM, Goodall J, Hetrick SE, Parker AG, Gilbertson T, Amminger GP, et al. Online and social networking interventions for the treatment of depression in young people: a systematic review. J Med Internet Res. 2014;16(9):e206.

10. Conaughton RJ, Donovan CL, March S. Efficacy of an internet-based CBT program for children with comorbid High Functioning Autism Spectrum Disorder and anxiety: A randomised controlled trial. J Affect Disord. 2017;218:260-8.

11. Ricketts EJ, Goetz AR, Capriotti MR, Bauer CC, Brei NG, Himle MB, et al. A randomized waitlist-controlled pilot trial of voice over Internet protocol-delivered behavior therapy for youth with chronic tic disorders. J Telemed Telecare. 2016;22(3):153-62.

12. Fletcher-Watson S, Petrou A, Scott-Barrett J, Dicks P, Graham C, O'Hare A, et al. A trial of an iPadTM intervention targeting social communication skills in children with autism. Autism. 2016;20(7):771-82.

13. Whitehouse AJO, Granich J, Alvares G, Busacca M, Cooper MN, Dass A, et al. A randomised controlled trial of an iPad-based application to complement early behavioural intervention in Autism Spectrum Disorder. J Child Psychol Psychiatry. 2017;58(9):1042-52.

14. Hawe P, Shiell A, Riley T. Complex interventions: how “out of control” can a randomised controlled trial be? BMJ (Clinical research ed). 2004;328(7455):1561.

15. Oakley A, Strange V, Bonell C, Allen E, Stephenson J. Process evaluation in randomised controlled trials of complex interventions. BMJ (Clinical research ed). 2006;332(7538):413.

16. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258.

17. Grant A, Dreischulte T, Treweek S, Guthrie B. Study protocol of a mixed-methods evaluation of a cluster randomized trial to improve the safety of NSAID and antiplatelet prescribing: data-driven quality improvement in primary care. Trials. 2012;13:154.

18. Moore G, Audrey S, Barker M, Bond L, Bonell C, Cooper C, et al. Process evaluation in complex public health intervention studies: the need for guidance. J Epidemiol Public Health. 2014;68(2):101-2.

19. Grant A, Treweek S, Dreischulte T, Foy R, Guthrie B. Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting. Trials. 2013;14(1):15.

20. Jong ST, Brown HE, Croxson CHD, Wilkinson P, Corder KL, van Sluijs EMF. GoActive: a protocol for the mixed methods process evaluation of a school-based physical activity promotion programme for 13–14year old adolescents. Trials. 2018;19(1):282.

21. Mann C, Shaw A, Guthrie B, Wye L, Man M-S, Hollinghurst S, et al. Protocol for a process evaluation of a cluster randomised controlled trial to improve management of multimorbidity in general practice: the 3D study. BMJ Open. 2016;6(5):e011260.

22. Hall CL, Davies EB, Andrén P, Murphy T, Bennett S, Brown BJ, et al. Investigating a therapist-guided, parent-assisted remote digital behavioural intervention for tics in children and adolescents—‘Online Remote Behavioural Intervention for Tics’ (ORBIT) trial: protocol of an internal pilot study and single-blind randomised controlled trial. BMJ Open. 2019;9(1):e027583.

23. Andrén P, Aspvall K, Fernández de la Cruz L, Wiktor P, Romano S, Andersson E, et al. Therapist-guided and parent-guided internet-delivered behaviour therapy for paediatric Tourette’s disorder: a pilot randomised controlled trial with long-term follow-up. BMJ Open. 2019;9(2):e024685.

24. Verdellen C, van de Griendt J, Hartmann A, Murphy T. European clinical guidelines for Tourette syndrome and other tic disorders. Part III: behavioural and psychosocial interventions. Eur Child Adolesc Psychiatry. 2011;20(4):197-207.

25. Rizzo R, Pellico A, Silvestri PR, Chiarotti F, Cardona F. A Randomized Controlled Trial Comparing Behavioral, Educational, and Pharmacological Treatments in Youths With Chronic Tic Disorder or Tourette Syndrome. Front Psychiatry. 2018;9:100.

26. Hollis C, Pennant M, Cuenca J, Glazebrook C, Kendall T, Whittington C, et al. Clinical effectiveness and patient perspectives of different treatment strategies for tics in children and adolescents with Tourette syndrome: a systematic review and qualitative analysis. Health Technol Assess. 2016;20(4):1-450, vii-viii.

27. Leckman JF, Riddle MA, Hardin MT, Ort SI, Swartz KL, Stevenson J, et al. The Yale Global Tic Severity Scale: initial testing of a clinician-rated scale of tic severity. J Am Acad Child Adolesc Psychiatry. 1989;28(4):566-73.

28. Partridge SR, Allman-Farinelli M, McGeechan K, Balestracci K, Wong ATY, Hebden L, et al. Process evaluation of TXT2BFiT: a multi-component mHealth randomised controlled trial to prevent weight gain in young adults. Int J Behav Nutr Phys Act. 2016;13(1):7.

29. Young DR, Steckler A, Cohen S, Pratt C, Felton G, Moe SG, et al. Process evaluation results from a school- and community-linked intervention: the Trial of Activity for Adolescent Girls (TAAG). Health Educ Res. 2008;23(6):976-86.

30. Dworkin SL. Sample Size Policy for Qualitative Studies Using In-Depth Interviews. Arch Sex Behav. 2012;41(6):1319-20.

31. Murray E, Treweek S, Pope C, MacFarlane A, Ballini L, Dowrick C, et al. Normalisation process theory: a framework for developing, evaluating and implementing complex interventions. BMC Med. 2010;8(1):63.

32. May CR, Mair F, Finch T, MacFarlane A, Dowrick C, Treweek S, et al. Development of a theory of implementation and integration: Normalization Process Theory. Implement Sci. 2009;4(1):29.

33. Guy W, National Institute of Mental H. ECDEU assessment manual for psychopharmacology. Rockville, Md.: U.S. Dept. of Health, Education, and Welfare, Public Health Service, Alcohol, Drug Abuse, and Mental Health Administration, National Institute of Mental Health, Psychopharmacology Research Branch, Division of Extramural Research Programs; 1976. 603 p. p.

34. Angold A, Costello E, Messer S, Pickles A, Winder F, Silver D. The Development of a Questionnaire for Use in Epidemiological Studies of Depression in Children and Adolescents. 1995. 237-49p.

35. Byford S, Barrett B, Roberts C, Wilkinson P, Dubicka B, Kelvin RG, et al. Cost-effectiveness of selective serotonin reuptake inhibitors and routine specialist care with and without cognitive–behavioural therapy in adolescents with major depression. Br J Psychiatry. 2018;191(6):521-7.

36. QSR International Pty Ltd. NVivo qualitative data analysis Software. Version 12 ed2018.

37. IBM Corp. IBM SPSS Statistics for Windows. Version 25.0 ed. Armonk, NY: IBM Corp.; 2017.

38. Braun V, Clarke V. Successful Qualitative Research: A Practical Guide for Beginners: SAGE Publications; 2013.

39. Ritchie, J. & Spencer, L. 1994. Qualitative data analysis for applied policy research by Jane Ritchie and Liz Spencer in A. Bryman and R. G. Burgess [eds.] ‘Analysing qualitative data’, (pp.173-194). London: Routledge.

40. Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol. 2013;13(1):117.

41. Jupp V. The SAGE Dictionary of Social Research Methods. London, SAGE Publications, Ltd, 2006.

Table 1. Process evaluation components, areas of research, measures and sources of data

Process Evaluation components	Research questions	Measures	Data sources
Implementation	· Fidelity of implementation · Dose of intervention delivered · Adaptations · Reach	· Therapist contact · BIP TIC adherence · Time spent on BIP TIC · Therapist time · Number of logins · Demographic characteristics	· Intervention metrics · Usage metrics · Interviews with participants · Baseline data
Mechanisms of impact	· Mediators and moderators · Unexpected pathways and consequences	· Participant responses to BIP TIC · BIP TIC resources · Unintended outcomes	· Interviews with participants · Interviews with therapists and supervisors · Usage metrics
Context	· Factors related to improvement in YGTSS TTSS, strong fidelity of delivery	· Change in YGTSS TTSS scores over time · Demographic data · Participant and therapist perceptions · Service use · Comorbidities	· Baseline data · YGTSS TTSS at 3-months · 3-months follow-up data · Interviews · Usage metrics
	Implications of process evaluation for future implementation into health services		Synthesis of above data sources with interview data from clinicians

*YGTSS – Yale Global Tic Severity Scale; TTSS – Total Tic Severity Score

Download PDF

Journal Publication

published 02 Jan, 2020

Read the published version in Trials →

Editorial decision: Major revision
30 Oct, 2019
Review #2 received at journal
05 Oct, 2019
Review #1 received at journal
27 Sep, 2019
Reviewer #2 agreed at journal
01 Sep, 2019
Reviewers invited by journal
30 Aug, 2019
Reviewer #1 agreed at journal
30 Aug, 2019
Editor assigned by journal
11 Jul, 2019
Submission checks completed at journal
12 Jun, 2019
First submitted to journal
20 May, 2019

You are reading this older preprint version

Read the latest preprint version →

Protocol for the Process Evaluation of the Online Remote Behavioural Intervention for Tics (ORBIT) Randomised Controlled Trial for children and young people

Status:

Journal Publication

Version 1

Abstract

Figures

Background

The ORBIT intervention

The ORBIT process evaluation

Specific objectives are:

Overall design

Qualitative data collection

Sampling and recruitment for interviews

Children and parents

Therapists

Clinicians

Quantitative data collection

Trial data

Data analysis

Qualitative data analysis

Quantitative data analysis

Mixed methods analysis

Integration of findings

Discussion

Strengths and limitations

Trial status

Abbreviations

Declarations

References

Table 1

Supplementary Files

Status:

Journal Publication

Version 1