Study design
The current study aims to investigate whether staff training in PBS delivery is clinically and cost effective compared to treatment as usual in adults with ASD+, in a trial described previously (40). In summary, this was a multicentre, two-arm cluster randomised controlled trial in which 246 adults with ID (with and without ASD) were recruited from 23 community intellectual disability services across England.
Sample size
The sample size for the main trial was calculated to detect a difference of 0.45 SD in the primary outcome (Aberrant Behaviour Checklist-Community total score) using 90% power and 5% significance level indicating that a minimum of 19 clusters and 246 participants were required. Recruitment of services and participants took place between June 2013 and January 2015. Ultimately, 11 services (n=109 participants) were allocated to the intervention arm and 12 (n=137) to TAU arm.
Ethical approval, trial registration and consent
Ethical approval was issued by the NRES Committee London - Harrow (reference
12/LO/1378). The trial was registered on ClinicalTrials.gov, identifier: NCT01680276.
Easy read information sheets and consent forms were used to obtain informed
written consent from participants. Where a participant lacked capacity, a consultee
acted on their behalf, in accordance with UK law.
Clusters and participants
The 23 community intellectual disability services were randomly assigned to staff
training in PBS and TAU or TAU alone. Participants were eligible to take part if they
were at least 18 years old, had received a diagnosis of ID and displayed challenging
behaviour (with an ABC-CT score of no less than 15 indicating a weekly experience of any type of challenging behaviour). Participants were excluded if they had a primary clinical diagnosis of personality disorder, substance misuse, relapse of a pre-existing mental disorder, or the clinical team decided that participation to the study would not be appropriate (e.g. participants with an acute illness episode or where there were significant issues around safeguarding of the individual at the time of the screening for inclusion in the study). Any services which had embedded PBS therapists or specialist behaviour teams were also excluded. Participants were assessed at baseline, and at 6 and 12 months post randomisation.
Intervention
Therapists drawn from a variety of health professions, e.g. psychology, psychiatry,
occupational therapy, speech and language therapy, received six-day manual
assisted training in delivering PBS over 15 weeks. All professionals were employed
by specialist community ID services and had experience in working with this patient
group although some may have had limited experience of delivering behavioural
interventions such as PBS. However, the training intervention was designed to be
delivered by these health professionals rather than experienced behaviour
specialists, in a real-world community setting.
In brief, the training was designed around the following topics:
a) Functional Behavioural Assessment and formulation skills
b) Primary Prevention
c) Secondary Prevention and Reactive Strategies
d) Periodic Service Review and Problem Solving.
All behavioural strategies taught via direct and didactic instruction are well established and depend on teaching the adult with ID alternative positive skills via his/her family or paid carers. The therapists were given a manual which described the rationale and treatment strategies and included printed and electronic materials and other resources, received individual feedback on their cases and associated paperwork, e.g. reports and formulations and received post-training mentoring by the trainers. They were further supported through monthly teleconferences with peers, the Chief Investigator, the trial manager and the trainers. Further details can be found in the full trial report (40).
Treatment as Usual (TAU)
TAU comprised a variety of behavioural and/or psychosocial and pharmacological treatments as those being available in the services taking part in the study at that time (39).
Allocation and blinding
An independent Web-based randomisation system (Sealed Envelope) was used, with random permuted blocks (1:1 allocation). Randomisation was stratified by the staff:patient ratio for each cluster using a binary factor indicating whether a cluster was below or above the median ratio. Whereas clusters, participants and carers were informed about group allocation, the Research Assistants (RAs) and Clinical Study Officers (CSOs) completing the study assessments were blind to treatment allocation. Where unblinding occurred, RAs from other locations who had remained blinded to arm allocation, carried out the study assessments.
Measures
Primary outcome
Challenging behaviour was measured with the total Aberrant Behaviour Checklist-Community total score (ABC- CT) (41) at three assessment points, baseline, 6 and 12 months. The ABC-C consists of 5 dimensions of behaviour (Irritability, Agitation, Crying; Lethargy, Social Withdrawal; Stereotypic Behaviour; Hyperactivity, Noncompliance; Inappropriate Speech), each measured on a four-point rating scale (0-3). The total score is attained by summing up all the domain scores. The scale has very good psychometric properties and is widely used for monitoring of treatment effects (41).
Secondary outcomes
Mental health status of participants was measured with the Mini-Psychiatric Assessment Schedules for Adults with Developmental Disabilities (Mini PAS-ADD) (42). This scale measures active mental health symptoms (over the past 4 weeks) and consists of 86 items that provide thresholds for 6 mental disorders: Depression, Anxiety, Obsessive Compulsive Disorder, Hypomania/Mania, Psychosis, Unspecified disorder. For the purpose of the analyses, those disorder categories were grouped into severe mental illness (SMI; psychosis and hypomania/mania) and into common mental disorders (CMD; depression, anxiety, obsessive-compulsive disorder).
Service use for the preceding 6 months based on patient or proxy responses was captured with the study adapted version of the Client Service Receipt Inventory (CSRI) (43). Health related quality of life was measured by the EuroQol EQ-5D Youth version (EQ-5D-Y) to calculate Quality Adjusted Life Years (QALYs) in line with accepted guidance (44). The EQ-5D-Y is a 5 dimension (usual activity, self-care, mobility, pain and anxiety/depression), 3 level (no problems, some problems and extreme problems) questionnaire. Family and paid carers completed it as proxies at baseline, 6 months and 12 months.
Other measures
Autism screen (carried out once at baseline): The Mini PAS-ADD includes 17 relevant questions which cover the following domains: impairments in social interaction (threshold=4); impairments in communication (threshold=1); repetitive stereotyped patterns of behaviour, interest and activities (threshold=3). These items were used to describe the extent of autism symptoms. Participants’ level of functioning as proxy measure of intellectual disability was assessed with the short form of the Adaptive Behaviour Scale (SABS) (45) at baseline. Current medication prescriptions were also collected at each assessment point categorised into ‘any medications’ (i.e. for both physical and mental health), ‘antipsychotics’ and ‘other psychotropics’ (e.g. antidepressants, mood stabilisers).
Fidelity assessment
All documentation around treatment (i.e. PBS plan, goodness -of fit checklist, functional assessment and observational data) that was submitted by the therapists were assessed by an independent reviewer by means of the Behaviour Intervention Plan Quality Evaluation Scoring Guide II. The quality tool enabled the assessment and classification of the intervention plans as weak, underdeveloped, good or superior (46).
Ascertainment of ASD+ group
This group comprised all participants who were reported by paid or family carers to have had an established diagnosis of autism and which was recorded in the Case Report File.
This categorisation was further validated by comparing the Mini-PASADD autism subscale scores between the two diagnostic groups: ASD+ and ASD-; the ASD+ group scored significantly higher in all subscales as hypothesised (mean difference = 9.7 95%CI (2.5, 16.8); p = 0.008). The mean (SD) ABC- CT score in the ASD+ group was 70.7 (SD 29.5) compared to the ASD- group 61.0 (SD 27.1). The details are shown in figure 1.
Statistical analysis
Before analysing the unblinded data, a statistical analysis plan was developed and agreed by the Trial Management Group, the Data Monitoring and Ethics Committee and by the Trial Steering Committee. All analyses were based on intention to treat. Differences in categorical and continuous variables according to ASD status were assessed using Chi-square, t-test or non-parametric equivalent. We used a three level random effects linear regression model adjusting for baseline ABC-CT measurements, the staff:patient ratio (low/high), time period and effects of clustering by services and repeated measures within subjects using random intercepts to examine the difference in the primary outcome, ABC-CT over 12 months for ASD+ participants randomized to the intervention compared to those randomised to TAU arms. The normality assumptions of the residuals were investigated using residual plots and were satisfied. A significance level of p<0.05 was used. All analyses were by intention to treat using STATA version 14 (47).
Analysis of service use and QALY
We calculated descriptive statistics for baseline, 6 and 12 months for community health and social care inputs, general medical and specialist mental health services use, comparing ASD+ participants in the control group versus those in the intervention group. Participants who reported using a service without specifically reporting the number of visits were included in the proportion using the service but could not be included in the calculation of means or standard deviation for number of visits. We provide total mean health and social care costs over twelve months for ASD+ participants allocated to the intervention and to TAU arms. Costs include the cost of the intervention.
Mean participant QALYs are calculated as the area under the curve adjusting for baseline imbalances (44).
Missing data for costs and QALYs was assumed to be missing at random and imputed using multiple imputations with chained equations. Variables identified as predictive of missingness across the trial were ID level, current living situation (alone or with others, and accommodation type (supported or independent) were included in the imputation model, with a percentage of missing data equal to 20%.
The mean incremental cost per QALY was calculated as the mean incremental cost of the intervention divided by mean incremental QALYs for the intervention. Mean incremental costs and QALYs were calculated using regression analysis, adjusting for baseline costs and staff to service user ratio and accounting for clustering by site as random effects. Ninety-five percent confidence intervals for health and social care costs and QALYs were calculated using bias corrected bootstrapping with 7,000 draws using imputed data. The costing perspective is health and social care costs only and over 12 months. Units costs for care are reported in a previous publication by the authors (40). Given the 12-month time horizon no discounting of costs and QALYs is included. All costs are in 2014/2015 British Pounds. A Cost-effectiveness plane of costs and QALYs for PBS training and delivery compared to TAU has been reported in the additional file 1.