Study design
This feasibility study has a mixed methods design with two phases:
Phase I: The first phase of the study will use a co-design methodology to determine how the impact of TINT is measured in the trial. This methodology has been chosen to increase relevance of the research in line with a broader objective of considering how we can improve outcomes for service users in CAMHS. Co-design has proven value in other health settings in New Zealand (21–23). In co-design, service-users are viewed as experts alongside clinicians and researchers who work together to integrate their experiences. A representative group of parents and 10 to 14-year olds from CAMHS in the Wellington region will be recruited to participate in a co-design workshop. Workshops will be facilitated by the research team with the aim of co-designing an outcome measure to be included in an evaluation of TINT. The co-design process will be guided by a multi-stage approach incorporated in other New Zealand health research projects: engage, plan, explore, develop, decide and change(24) Following exploration and development of themes in the workshops, a measure will be drafted for each group (parents and 10–14 year olds) for inclusion in the trial.
Phase II: The second and final phase of the study will be a feasibility study using an experimental methodology of a randomised control trial (RCT). There will be two arms to the trial:
-
Usual care + TINT; and,
-
Usual care only.
The comparison of TINT plus usual care with usual care only (described later) gives a pragmatic and ethical comparator in a real-world clinical setting. This design also gives an accurate representation of practice in CAMHS where different types of interventions are often layered together. If a larger scale trial is feasible, this design will allow for testing of the hypothesis that adding TINT to usual care improves outcomes for young adolescents in CAMHS.
Participants will be randomly allocated to a study arm. Those allocated to receive TINT in addition to usual care will complete a TINT group programme at CAMHS (described later). TINT groups will be facilitated by at least two CAMHS clinicians and involve eight sessions of two hours each week. Those allocated to control will be given the option of completing TINT after the study period. Outcome measures will be collected prior to starting groups (T0), immediately after completing TINT/matched for control arm (T1) and eight weeks after completing TINT/matched for control arm (T2). See Fig. 1, Flow chart of phase II RCT. The trial will follow the Standard Protocol Items for Clinical Trials (SPIRIT) statement (25). See Fig. 2 for Schedule of enrolment, interventions and assessments for phase II RCT and SPIRIT checklist in additional files (Additional file 1: SPIRIT checklist).
Figure 1: Flow chart of phase II RCT
Figure 2: Schedule of enrolment, interventions and assessments for phase II RCT
Study Setting
The setting for this study is CAMHS services in the Wellington region of New Zealand. New Zealand is a high-income country with a population of around 5 million people of mixed ethnicities. Its history and culture have been affected by European colonisation with ongoing impacts on the indigenous population of Māori. Wellington is the capital city, and the greater Wellington region is home to around 550,000. Based on 2018 census data, the regional population was 74.6% European/Pākehā, 14.3% Māori, 8.4% Pacific peoples, 12.9% Asian and 3.3% other ethnicities (26). CAMHS form part of the public health service in New Zealand which provides free universal care to eligible citizens, funded by general taxes (27). Public health services have been administered by 20 regional District Health Boards (DHBs), which as of 2022 are in the process of integration into one national administrative body. In the greater Wellington region there are five mainstream CAMHS and two speciality services: one for Māori families and one for Pacific peoples. Those from Māori or Pacific communities may choose whether to access mainstream or speciality services.
Participants
Participants will be young adolescents aged 10–14 years who have been referred to CAMHS with a presenting difficulty related to anxiety or depression, and their parents or guardians (including non-birth parents e.g. step-parents, other family members in a parenting role or legal guardians). Participant criteria will be the same for both phases so that those designing the measure are from the same population intended for the evaluation.
Inclusion criteria
Inclusion criteria for adolescents are:
-
Aged 10 to 14 years at time of recruitment.
-
Accepted into a CAMHS in the Wellington region.
-
Primary presenting difficulty related to anxiety (including any DSM-5 Anxiety Disorder) or depression (including any DSM-5 Depressive Disorder).
-
Sufficiently competent and fluent in English language to complete outcome measures.
-
Have at least one parent/guardian who meets study inclusion criteria.
Inclusion criteria for parent/guardian are:
-
Aged over 18 years.
-
Have care of a child in CAMHS who meets the inclusion criteria.
-
Sufficiently competent and fluent in English language to complete the intervention group and outcome measures.
-
Available and willing to complete the intervention group (determined prior to randomisation).
Exclusion criteria
-
Presenting difficulty is not anxiety/depression.
-
Communication difficulty (e.g., lacking fluency in English, or, having a learning difficulty which impacts ability to give consent or complete outcome measures).
Recruitment And Target Sample Size
All participants will be recruited through participating CAMHS. General information about the study will be provided to these services and advertised via printed material displayed in waiting rooms and other public areas of these services. Clinical staff members may also send out individual invitations to current service users who may meet the study criteria. Interested participants will be invited to contact the research team directly.
Recruitment for phase I (co-design) will aim to recruit six to ten parent participants and six to ten 10 to 14-year olds. Recruitment for phase II, the RCT feasibility study, will aim to recruit 50 participants (25 per arm) as an approximate upper-limit: the total numbers recruited in the study period will form part of the feasibility assessment.
Consent And Eligibility
The principal investigator will provide further information to those who express interest in the study, screen them for inclusion/exclusion criteria and obtain informed consent. All potential participants will be provided with written information about the study and provided with options to discuss the study based on their personal preference (phone, face to face, email, text). For those aged 10 to 14, an assessment of their ability to provide informed consent will be made by the principal investigator who has expertise working with young people in mental health settings. This may include obtaining additional information from family members and/or the care team at CAMHS as required, for example, to determine the presence of any learning or cognitive difficulties that may impact ability to consent. For those requiring support to provide consent, this will be sought from a parent/guardian with assent sought from the young adolescent. Participants may withdraw at any time on their own request without providing a reason.
Randomisation, Allocation, And Blinding
Randomisation and allocation apply to phase II (RCT). A pre-determined allocation sequence will be created by the team statistician (JS) and uploaded to REDCap (Research Electronic Data Capture) trial management software(28) Allocation will be on a 1:1 basis (i.e. equal size groups for intervention and control arm).
As separate TINT groups will be run in each participating service, separate randomisation schedules will be constructed for each centre to allow allocation on this basis. Randomisation schedules will be constructed using a permuted block randomisation (29) with a mean block length of 4 allocations (2 intervention, 2 control). Participants will be recruited during one school term (approximately 10 weeks) with the intervention run in the following term. This randomisation schedule will allow for accrual over the term and participants to be notified of allocation as they enter the study.
Following confirmation of eligibility and informed consent, participant data will be entered in REDCap which will be used to draw the next allocation from the random allocation sequence for the appropriate service (the allocation schedule is not visible). The research team member responsible for this step will communicate these allocations to relevant staff members to coordinate running the TINT groups. Due to the nature of the intervention, participants and clinicians will not be blind to intervention status; however, the researcher collecting outcome data will be blinded to allocation when recording outcomes to reduce risk of bias. Analysis will be conducted blind to outcome status, including leaving analysis of responses regarding perception of the TINT sessions until after the main analysis is complete and the study subsequently unblinded for reporting.
Intervention
TINT is designed to be delivered in a group format over weekly sessions of two hours each. An 8-week format will be used for the study as the recommended format for clinical settings. The focus of sessions is teaching emotion coaching skills. Parents are taught key steps to emotion coaching with a focus on awareness, listening and acceptance and then, if necessary, problem-solving or negotiating boundaries with their adolescent.
Sessions also include information about adolescent development, teaching of emotion regulation skills such as mindfulness (for parents and adolescents), and supporting parents to reflect on their own beliefs about emotions based on their life experiences. Sessions are clearly structured, and a manual is provided to facilitators to guide sessions. Each session includes a mix of teaching, shared reflection and practicing skills. As much as possible, parents are encouraged to practice emotion coaching skills with each other using ‘live’ examples from home life, and to practice at home in between sessions. Each session has a specific focus, for example, managing fear, or managing anger and conflict. Handouts are provided for each session to allow parents to review material. A fidelity checklist for each session is used for facilitators to complete to check adherence to programme content.
Training
Prior to the start of phase II RCT, participating clinicians not already trained in TINT will attend training run by the programme creators from the University of Melbourne. Regular trainings for TINT are run online and up to four clinicians from each participating team will be trained. This training is being provided for the study by the programme creators.
Comparator: Usual Care In Camhs
CAMHS are multidisciplinary teams that may include mental health nurses, social workers, family therapists, psychiatrists and psychologists, who typically have additional training and expertise in working with young people and families. Based on the most recent national stocktake of this workforce, most staff in community CAMHS are in a clinical role (84%) and are composed of nurses (23%), social workers (18%), psychologists (16%), occupational Therapists (9%) and Psychiatrists (7%) (5). Between 2018 to 2020/21 there was a 15% staff turnover rate nationally (5)
A case manager is typically assigned to each referred young person to coordinate care. The case manager may also be a treating clinician. Care plans are discussed and reviewed within the multidisciplinary team. Wait time to begin treatment varies depending on service capacity, with variation in wait times to receive different interventions services. For example, a young person may have to wait to start medication due to low psychiatry resource but may be able to start talking therapy sooner, or vice versa. For those with primary difficulties related to anxiety or depression, usual care may include any or a combination of case management, antidepressant medication, individual talking therapy (e.g. CBT), other therapies including group or family based sessions, and/or liaison with school and other agencies (5). Data will be collected from all participants about the frequency and form of contact with CAMHS during the study period.
As clinicians providing usual care may also be trained in TINT, potential contamination will be assessed by determining number of total trained clinicians in a team, how many provided usual care during the study period and where possible, the amount of TINT content that may have been included in usual care sessions.
Feasibility Outcomes
Recruitment and retention feasibility outcomes include:
-
Service participation: assessed by the number of services involved across the region.
-
Recruitment rates: assessed as the numbers of participants expressing interest in the study, number of eligible participants and number who consent to participate.
-
Retention rates: assessed by the number of participants who consent and:
-
complete the first session of either TINT or usual care
-
remain in the study till the first follow-up point (T1)
-
remain in the study till the final follow-up point (T2)
Acceptability and feasibility of the intervention includes:
-
Clinician adherence to TINT: assessed by session fidelity checklists (provided in TINT manual).
-
Participant engagement: assessed by attendance records at group sessions.
-
Participant and clinician experience: assessed by post-programme questionnaires, including barriers to attendance and learning.
-
Depending on results of other feasibility outcomes, interviews may be conducted with participants and/or clinicians e.g., if a particular issue is identified that warrants further exploration.
Acceptability and feasibility of outcome measures
Outcome Measures
The final measures included in the RCT will depend on the outcome of the co-design with service-users. It is anticipated that at least one co-designed outcome measure will be included alongside one or more previously validated measures (noted below). This co-designed measure will not be externally validated but will be intended for use with this trial, and may take the form of an additional questionnaire, scale or input into selection of pre-existing standardised measures. Pending the outcome of the co-design, the following standardised quantitative measures will also be included based on previous research with TINT and this age group(4). Parents and 10- to 14-year-olds will complete each measure independently with parent and youth forms used where available and appropriate. Participants will receive a gift voucher on completing measures as a thank you and to promote retention. All outcomes will be administered and recorded through REDCap.
Emotions as a Child Scale (EAC)
The EAC is a 45-item measure assessing parent responses to anger, sadness, and anxiety, with 15 items per emotion (30). There are parent and youth forms which are identical other than the phrasing of perspective, with the parent form asking how the parent responds to their child’s emotions; and the youth form asking how the youth perceives their parent’s responses. The EAC has five subscales: parental encouragement, parental punishing responses, parental neglect, parental matching or magnifying and parental overriding of emotion. There are no clinical cut-off scores for this measure. Previous studies have shown acceptable internal consistency, test–retest reliability, and validity metrics for various versions of the EAC with both adolescent and adult samples (31–33).
Difficulties in Emotion Regulation Scale, Short Form (DERS-SF)
The DERS-SF is an 18 item self-report questionnaire and is a shortened form of the original 36-item version (34). There are six subscales: Nonacceptance of emotions, Difficulties engaging in goal-directed behaviour when experiencing negative emotions, Impulse control difficulties, Lack of emotional awareness, Limited access to emotion regulation scales and Lack of emotional clarity (34). Items are rated from 1 (almost never) to 5 (almost always). There are no clinical cut-offs for this scale and higher scores indicate greater difficulties in emotion regulation. The shortened form has been chosen to minimise burden for participants and has comparable psychometric properties to the full version in both adult and adolescent samples (35). Parents and 10–14-year-olds will be given the same self-report form and asked to respond about their own emotional regulation.
The Revised Child Anxiety and Depression Scale 25 (RCADS-25)
The RCADS is a questionnaire with self- and parent-report forms assessing symptoms of anxiety and depression (36). The parent form asks about their child’s experiences whilst the child self-report form asks the child about their own experiences. In its original form, the RCADS contains 47 questions and can be interpreted using its composite subscales which assess different types of anxiety and depressive disorders. As the objective of the current study is to look at overall anxiety and depression symptoms, and to minimise demands on participants, the shorter form RCADS-25 will be used. This provides an overall depression score, an overall anxiety score and a total internalizing score. In interpreting results, meaningful improvement has been suggested as at least a 50% reduction in the raw score or at least a 25% reduction for youth with treatment resistant depression or remission defined as a T-score under 65 (37, 38). The RCADS-25 has good psychometric properties for both overall anxiety and depression score (39, 40).
Other Data
Demographic information will be collected from participants including age, ethnicity, and socio-economic status. Participant addresses will be collected to calculate the New Zealand Index of Deprivation as an estimate of socioeconomic status (41). The New Zealand census questions will be used for ethnicity, gender and education (42).
For those allocated to receive TINT during the study, pre- and post-programme questionnaires will be administered. This will include set questions e.g. ‘How would you rate your ability to respond effectively to your child’s emotions?’ as well as space for open-ended feedback.
To assess components of usual care, all participants will be given a checklist to complete to identify what services they received from CAMHS during the study period.
Data analysis
Analysis for the feasibility trial will be largely descriptive. Variables related to recruitment and retention, acceptability and feasibility of the intervention, and acceptability and feasibility of the outcome measures will be described. This will include appropriate summary statistics (e.g., percentages reporting aspects as acceptable).
The statistical analyses for preliminary effect sizes from outcome measures will depend on the final measures chosen but will likely follow common methods for continuous outcomes (e.g., random-effects models). A formal statistical analysis plan will be produced prior to the start of data analysis. Analyses of trial outcomes will be based on intention-to-treat principles. Analysis will include interim measurements to allow inclusion of participants with partial follow-up. Methods allowing for more complex handling of missing data will not be feasible within this initial study.
A summary of any participant and clinician feedback will be reported.
Data Monitoring
The primary investigator will monitor outcome measure data. Any concerns or adverse events will be reported to the research team and responded to according to need for example, communicating back to the treating team.
Protocol Amendments
It is anticipated that the co-design phase will lead to amendment of outcome measures. There may also be amendments in response to feasibility findings as the trial progresses for instance related to recruitment. Any substantial amendments will be submitted to the overseeing ethics authority (Health and Disability Ethics Committee New Zealand).