Participants
Participants were 76 children, 8 to 13 years of age (Mage = 10.58, SD = 1.52), who participated in a randomized controlled trial. This trial investigated the effectiveness of YourSkills, a newly developed CBT (Alsem, van Dijk, et al., 2022). Children were recruited at fifteen clinical centers in the Netherlands providing mental health care for children who have severe problems that impair daily functioning and require treatment. Recruitment began in September 2019 and the study ended in July 2021. Therapists working at the clinical centers were asked to approach parents of boys whose casefiles met the inclusion criteria: age 8–13 years, aggressive behavior problems, estimated intelligence level above 80, no severe autism spectrum disorder, and no epilepsy or severe visual or auditory limitations.
The randomized controlled trial included 115 children, randomized to three groups: YourSkills virtual reality, YourSkills roleplay, or care-as-usual (for the flowchart see Alsem, van Dijk, et al., 2022). In this study, we only included children who were randomized to the two YourSkills groups, because our aim was to examine the mechanisms of change of interventions for children’s aggression. We excluded children in the care-as-usual group as we had no information about what mechanisms were targeted in the care-as-usual treatments. In addition, we only included children who participated in at least five of the intervention sessions, because children should have had the opportunity to learn the expected mechanisms of change.1 Of the included children, 36 participated in YourSkills virtual reality and 40 in YourSkills roleplay. Most children (96.1%) were born in the Netherlands and in most families, both biological parents were born in the Netherlands as well (73.7%). In 14.5% of the families only one parent was born in the Netherlands and in 11.8% of the families both parents were born elsewhere. Parents attained middle levels of education (44.7%; ISCED 3–4), high education (39.5%; ISCED 5–8), or low education (15.8%; ISCED 0–2; UNESCO, 2012).
We obtained written informed consent from parents and 12- and 13-year-old children. Participation was voluntary and children and parents were assured of confidential use of their data. Children received a small gift after filling out the last assessment (e.g., a multicolor pen). This study was approved by the Ethics Committee of the University Medical Center Utrecht (NL67139.041.18) and the trial was registered in the Dutch Trial Register (NTR; https://www.trialregister.nl/trial/7959).
Procedure
The current study consisted of eleven weekly assessments scheduled at each of the eleven YourSkills treatment sessions; one introduction session with parents and ten sessions with children. Therapists scheduled all sessions and invited researchers to conduct the first assessment during the introduction session (i.e., our baseline assessment). This assessment took place in another room with the first author or a trained research assistant. Children first completed a short task and then filled out the weekly assessments used in this study, as well as some other questionnaires (20–30 minutes in total). Directly after the session ended, the researcher asked parents to fill out questionnaires on a laptop. The ten other assessments were integrated within the treatment sessions with the child: all sessions started with children filling out the brief measures of anger regulation, hostile intent attribution, and aggression in a paper booklet (5 minutes). Parents or caregivers who brought the child to the treatment session were asked to fill out a brief measure of aggression in the clinical center’s waiting room.
Yourskills Treatment
YourSkills is a manualized CBT, developed based on evidence-based treatments for children with aggressive behavior problems. The effectiveness of YourSkills is examined in a previous study (Alsem, van Dijk, et al., 2022). Results of this study showed that both the virtual reality and roleplay versions of YourSkills were more likely to reduce aggressive behavior than care-as-usual for six out of seven aggression outcomes, suggesting that YourSkills successfully reduced children’s aggressive behavior. When we directly compared virtual reality to roleplays, results favored virtual reality above roleplays on four of seven aggression measures (Alsem, van Dijk, et al., 2022).
The aim of YourSkills is to reduce children’s aggressive behavior problems by enhancing emotion regulation and social information processing skills. Children practice anger recognition, anger regulation, nonhostile intent attribution, and other social problem solving skills in social interactions (for an overview of the treatment sessions, see: Alsem et al., 2021). To enable children to practice their skills or new cognitions whilst being emotionally engaged, therapists create challenging social situations for children in virtual reality or roleplays. In each session, therapists first explain a new skill, then model the skill using roleplay, and then—depending on the YourSkills version—use virtual reality or roleplays to let children practice the skill in anger-provoking social situations, such as being disadvantaged, having authority conflicts or being rejected by a peer. The details of the virtual reality and roleplay version of YourSkills are described in detail elsewhere (Alsem, van Dijk, et al., 2022).
Measures
The psychometric quality of the weekly measures used in the current study were investigated in an earlier study. Results showed adequate internal consistency, convergent, and concurrent validity for the child-report version (Alsem, Keulen, et al., 2022).
Aggression
To assess children’s aggression over the weeks, we asked both parents and children to rate three items (e.g., “This week my child/I fought with someone”) on a scale from 1 (never) to 5 (very often). Ratings were averaged across items. The internal consistency in the current study was adequate over the weeks for both children (αmean = .79, αrange = .75-.87) and parents (αmean = .79, αrange = .69-.83).
Adaptive Anger Regulation
We assessed weekly adaptive anger regulation by asking children to rate three items (e.g., ‘This week I managed to do something against my anger’) on a scale from 1 (never) to 5 (very often). The internal consistency in the current study was adequate over the weeks (αmean = .64, αrange = .58-.71).
Hostile Intent Attribution
To assess children’s weekly hostile intent attribution, we asked children to rate three items (e.g., ‘This week people were mean to me’). Children rated the items on a scale from 1 (never) to 5 (very often). The internal consistency in the current study was adequate over the weeks (αmean = .71, αrange = .52-.83).
Analyses
We ran all analyses in Mplus 8.4. Before we tested our hypotheses, we explored whether there was variability in children’s individual scores on mechanisms and outcomes over the weeks. We ran four random intercept models (i.e., for anger regulation, hostile intent attribution, and child- and parent-reported aggression) to estimate the amount of variance at the within- and between-person level for each variable. We expected substantial individual within-person variation over the weeks, reflecting intervention effects on children’s adaptive anger regulation, hostile intent attribution, and aggression. We did not expect much variation at the between-person level because of our homogenous sample (i.e., participants all had aggressive behavior problems).
To test our first research question, whether children’s adaptive anger regulation increased and their hostile intent attribution and aggressive behavior decreased across the course of the intervention, we estimated four univariate latent growth models (LGMs) using an intercept factor and a linear slope factor. We expected that slope factors would significantly increase for adaptive anger regulation and decrease for hostile intent attribution and child- and parent-reported aggression. We modelled linear slopes because we expected change over the whole treatment period, and this approach is more parsimonious, preventing power problems in our relatively small sample.
Next, we investigated our second research question, whether individual-level increases in adaptive anger regulation and decreases in hostile intent attribution were associated with decreases in child- and parent-reported aggression. We estimated four bivariate LGMs, each estimating the slope factors of two variables at a time: 1) one of the two mechanisms and 2) children’s aggressive behavior, either child- or parent-reported. We expected significant correlations in between slope factors in each of the four models. As LGMs estimate individual-level change trajectories, these models are suitable to examine mechanisms of change (Laurenceau et al., 2007).
Last, as a control analysis, we checked whether children’s change trajectories differed between the virtual reality and the roleplay group, using Wald tests. If slopes differed between these two intervention groups, we corrected for this difference by adding intervention group as predictor to the slope factors. Results are presented in Appendix A.
For all LGM models, we examined the model fit using the Chi-square test statistic (χ2), the Comparative Fit Index (CFI), and the Root Mean Square Error of Approximation (RMSEA). However, we did not necessarily expect a good model fit because we had many parameters in the model (i.e., eleven weeks of assessment) and a relatively small sample size. We hoped to obtain adequate model fit: CFI above .90, RMSEA below .10 and the ratio between the χ 2 test statistic and the degrees of freedom below 3 (this was not based on the χ2 significance value to avoid problems with sample size; Schermelleh-Engel et al., 2003).
We inspected missingness in our measurements over the weeks. Most children completed questionnaires in all eleven weeks (78.9%) and all children completed questionnaires in at least six weeks. We compared children who completed all weeks (n = 60) with children who missed at least one week (n = 16) and found no significant differences in levels of adaptive anger regulation, hostile intent attribution, and aggression. To check for missing data patterns on the item-level across assessments, we conducted Little’s test. This test yielded a normed χ2 (χ2/df) of 0.10, indicating that data was missing completely at random (Bollen, 1989). We therefore used default settings for multilevel data in Mplus to estimate missing data (i.e., maximum likelihood; Muthén & Muthén, 2007).
1 We conducted sensitivity analyses including the five children that participated in less than half of the treatment sessions and results revealed similar conclusions.