Implementation of Virtual Reality to Parent-Child Interaction Therapy for Enhancement of Positive Parenting Skills: The Study Protocol

Background: Disruptive behaviour is a common reason that young children are referred to child and adolescent mental health care services worldwide. Research indicates that treatments for child disruptive behaviour where parents are the primary agents of change are most impactful. Parent-Child Interaction Therapy (PCIT) is an effective parent management training program, currently implemented in therapeutic settings within the Netherlands. If parenting skills have been mastered, PCIT has shown to make impactful long-term improvements on parental warmth, responsiveness and effectiveness. The current study aims to increase the proven effectiveness of PCIT by increasing accessibility and impact of the treatment with a focus on strengthening positive parenting skills. To enhance the skills from the therapy, the current study will implement a Virtual Reality (VR) element for additional practice in the home-setting. We expect that if the positive parenting skills are trained by implementing the VR-element, skill mastery will be achieved sooner, treatment completion rates will increase and the parent-child relationship will better, whilst parental stress and child disruptive behaviour will decrease. Methods: Fifteen children (ages 2 to 7 years old) with disruptive behaviour and their parents will followed throughout their PCIT-VR treatment. Using a non-concurrent multiple baseline Single-Case Experimental Design (SCED) with three phases, fteen families will ll out questionnaires weekly, in addition to having pre-, and post-treatment and follow-up measurements to monitor their positive parenting skills, child disruptive behaviour, parenting stress and VR progress. Moreover, the quantitative information and qualitative interviews will be analysed visually and statistically and summarised to provide a complete picture of experiences from both parents and therapists. Discussion: The SCED - with its phased design, randomisation and the opportunity to replicate and assess both individual and group treatment effects - and adaptability of the VR technology are the strengths of the study. The risks of increased type I errors, maturation effects or technological failure will be mitigated with the right statistical support. The current study aims to enhance the scope of the treatment through the additional skill training, ultimately in support of routinely implementing VR at scale within PCIT.

disruptive behaviour, parenting stress and VR progress. Moreover, the quantitative information and qualitative interviews will be analysed visually and statistically and summarised to provide a complete picture of experiences from both parents and therapists.
Discussion: The SCED -with its phased design, randomisation and the opportunity to replicate and assess both individual and group treatment effects -and adaptability of the VR technology are the strengths of the study. The risks of increased type I errors, maturation effects or technological failure will be mitigated with the right statistical support. The current study aims to enhance the scope of the treatment through the additional skill training, ultimately in support of routinely implementing VR at scale within PCIT.

Contributions To Literature
The research aims to enhance the working elements of PCIT by further engraining positive parenting skills, by creating an additional practice platform through Virtual Reality to innovatively maximise the effect and scope of PCIT.
The implementation of Virtual Reality gives parents the unique opportunity to practice taught skills in the comfort of their own home, whilst still receiving a form of coaching feedback.
By using multiple baseline Single-Case Experimental Designs, this research is able to closely monitor change in positive parenting skills and child disruptive behavioural problems on a weekly basis.

Page 3/21
Disruptive behaviour is one of the most frequent reasons that young children are referred to child and adolescent mental health care services worldwide (1). Statistics show that of the population in the US, 7-9% of children have a diagnosed behavioural problem (2), and in the Western culture, there is an increase in prevalence of behavioural problems and disorders (3). Disruptive behaviour disorders (DBD) include oppositional de ant disorder (ODD) and conduct disorder (CD). Children with DBDs show patterns of de ant and uncooperative behaviour and their behaviour can often be characterised as stubborn, disobedient, irritable or even (physically) aggressive (4). Displaying such behaviour has an effect on the child, as well as on his or her surroundings and society at large (5,6). Long-term effects of untreated externalising, disruptive behaviour include school dropouts, peer rejection, developing antisocial personality disorders, higher public costs for health care and education, and both non-violent and violent delinquency and criminality in adulthood (7)(8)(9)(10)(11). However, protective factors include enhancing prosocial skills for children and fostering the mental wellbeing of the parents (12). Additionally, according to a longitudinal study, when parental supervision and involvement is high in the early years, it reduces risk of developing delinquency during adolescence (13). Moreover, addressing problem behaviour at a young age diminishes the risk of the problems getting worse and interfering with child's social and emotional development (14). Offering early intervention for children who exhibit behavioural problems, is bene cial for multiple reasons. Firstly, the problem behaviour has not yet been fully manifested, and children are still somewhat malleable at a young age. Secondly, the child's cognitive skills are still in development, meaning that they have fewer resources to challenge or question interventions. Thirdly, parents are still the factor with the most in uence on a young child, as peers and society still have limited impact on a pre-schooler (14). Therefore, to prevent child disruptive behaviour or intervene at an early stage, children need evidence-based programs that focus on ameliorating parental skills and diminishing child disruptive behaviour. Research concludes that treatments for child disruptive behaviour where parents are the primary agents of change have the most substantive evidence for effectiveness (15).
There are currently a few parent management training (PMT) programs, that focus on improving a parentchild interaction, that are being implemented worldwide (16). Within these PMT programs, research shows that the programs based on the social learning theory are most effective (15), of which the Parent-Child Interaction Therapy (PCIT) is one (17). PCIT is an evidence-based treatment that focuses on diminishing externalising child behaviour disorders by strengthening parenting skills (18). The therapy focuses on increasing positive parenting skills, whilst reducing child disruptive behaviour (19).
PCIT has been well-researched pan culturally, where effect sizes vary between 49% and 59% (20). Speci cally in the Netherlands, PCIT has also shown to be effective with higher effect sizes in an RCT comparing PCIT to another Dutch treatment (M = 45.4, SD = 3.6 for PCIT, versus M = 34.0, SD = 4.93, t(23) = 6.25, p < .001) (21). Although these effect sizes indicate that PCIT is an effective treatment, the current study aims to focus on the gains that can still be achieved by letting more families bene t from PCIT by increasing the effect sizes, the accessibility, and the impact of the treatment by focusing on strengthening positive parenting skills.
Worldwide, drop-out rates vary between 12-67% (22), and speci cally in the Netherlands, studies indicate that there is a high attrition and that parents have a hard time to grasp the different parenting skills. In that study, fty-two percent of parents who received PCIT dropped out (21). Attrition rates worldwide seem to be attributed to parenting stress levels and treatment barriers (22). This indicates that treatment should focus on making parents feel more con dent in their parenting, thus potentially diminishing parenting stress and -with that -dropout rates. Additionally, positive parenting skills appear to be crucial for bettering the parent-child relationship (8). A meta-analysis on PCIT shows that when skill mastery is required for completion of the treatment, child externalising behaviour was signi cantly lower than when this was not a requirement (20). If the parenting skills have been mastered, PCIT has shown to make impactful long-term improvements on parental warmth, responsiveness and effectiveness (19). Additionally, studies show that performance and motivation of a skill can be improved when deliberately practicing that skill (23). Moreover, parenting programs that incorporate the opportunity to practice new skills with their child, are considered more effective interventions than those where this is not the case (24). This suggests that it could be bene cial for parents to be able to practice the skills they learn in the therapy more intensely, hence potentially reducing the dropouts and magnifying the scope of the treatment. In addition, research indicate that a current obstacle in PCIT is that parents may have a hard time to grasp parenting skills, which leads them to drop-out due to the inability to fully understand and translate the parenting skills to their home-setting (21). Therefore, to bridge the gap between practicing the skills in the therapy, and applying the skills in a home-setting, the current study will implement a Virtual Reality (VR) element. Over the past years VR has become increasingly popular within the mental health care sector, as it creates opportunities that would otherwise not be possible (25).
VR offers multiple advantages in the eld of psychological research. Firstly, a stimuli can be presented in three dimensions (25). Secondly, a virtual environment can be manipulated and controlled to create scenarios for participants, whereas in a real-world environment, there are always variables present that cannot be controlled (26). Thirdly, VR technology is designed for immersion, meaning that the created virtual world becomes the real world for a moment (27). Clearly, VR is an experience generator that opens up possibilities to create experiences that would otherwise not be possible (27). Thus far in psychology and psychiatry, VR has been an effective tool in the treatment of anxiety disorders, PTSD, schizophrenia, eating disorders, and substance abuse disorders (28). It is most commonly used in exposure-based therapy and for behavioural skills training (29). Considering the current study, we will focus on the use of VR for the behavioural skills training of parents. The goal of a behavioural skills training is to teach certain skills in order to apply them to multiple environments (30). When using VR for behavioural skills training, the goal is to create an environment where skills that are currently found di cult to master, are practiced in VR. A study regarding developing mindfulness skills for borderline personality disorder has shown that practicing mindfulness within the virtual environment creates the opportunity to generalise the skills to the natural context outside of therapy (31). Additionally, it has also been shown that practicing and training skills for dangerous situations, such as re hazards in a virtual environment was also effective (30). These studies are precedents of how it is possible to apply VR to practice and train skills in a virtual environment. Thus, this study aims to implement VR as a low-threshold opportunity for parents to practice the skills they are taught in the therapy, on their own time, within the context of their own home.
To date, no study has previously implemented VR to improve positive parenting skills in PCIT or other PMT programs. However, considering that studies indicate that a current attrition obstacle in PCIT is that parents may have a hard time to grasp the parenting skills, the addition of VR as a tool to practice skills, which can henceforth be generalised to a person's natural context, seems to be a good addition to the therapy. Nonetheless, research suggests that VR should and could in no way replace the clinician, but rather it be integrated in the therapy in order to enhance the intervention (32). Therefore, the implementation of VR within PCIT creates an accessible platform for parents to further grasp the positive parenting skills, thus increasing the potential for PCIT to have an impactful and lasting effect. The research group involved in this project, have previously tested this VR-concept for PCIT with a test 360degrees video showing the PCIT-skills (33). This was consequently evaluated by PCIT-therapists and the researchers. All therapists and researchers were enthusiastic about the prospect of adding this VRelement to the therapy, creating the additional opportunity for parents to practice (33). For example, if parents are divorced and do not always have their child with them, they can practice the parenting skills nonetheless through VR. Therefore, the purpose of implementing PCIT-VR is to magnify the scope and give parents an easy opportunity to solidify the positive parenting skills in the comfort of their own home.

Aims
The primary goal of this research project is to evaluate the implementation of VR to PCIT. We expect to nd that PCIT-VR will ameliorate positive parenting skills, leading to faster skill mastery. We believe that if the positive parenting skills are trained more frequently by implementing the VR-element, additional effects will also take place, such as achieving Child-Directed Interaction (CDI)-skill mastery sooner than when not using VR and increasing treatment completion rate. We expect that by implementing a VRelement, it will innovatively magnify the scope of families (for example for split families with separated parents) who can bene t from PCIT. In addition, parental stress, child disruptive behaviour and analytics of VR will be secondarily measured.
As a whole, PCIT-VR is developed as an enhanced version of PCIT, where focus lies on ensuring that parents practice and learn positive parenting skills intensely. This is done by incorporating practicing with VR as an integral part of the therapy. The threshold to practice skills at home using VR is low, as parents are simply able to use their mobile phones. PCIT-VR create the opportunity for parents to practice and become familiar with the skills they have been taught in the therapy sessions, in the comfort of their own home. The implementation of VR functions as an additional 'skill training'. Providing families with PCIT-VR at an early age and early stage, can potentially prevent them from needing more intensive help at a later stage and can possibly minimise the impact of child disruptive behaviour -not only on the long run, but also during the therapy itself.
Secondary objectives are effects that we expect as a result of the overall amelioration of positive parenting skills that cause both parental stress and child disruptive behaviour to diminish. Therefore, we secondarily expect that the total amount of PCIT sessions will diminish, and we believe that the overall completion of treatment will increase. Secondary objectives also include positive VR-experiences and general therapy satisfaction. Furthermore, we expect that PCIT-VR will secondarily lead to a better quality of parent-child relationship. We expect that the effects of PCIT-VR are further maintained and engrained on the long-term due to the additional skills training provided by the VR scenarios.

Method
Participants Children (ages 2 to 7 years old) and their parents are referred through community channels after seeking help or support. The research will be conducted in clinical practice in the Netherlands, where families rst follow a standard intake procedure to establish their need for help. After this, families can be included in the study, and receive PCIT(-VR) if: Disruptive behaviour problems of the child are a reason for referral, Children are aged between 2 and 7 years old, Parents speak Dutch or English.
A total amount of 15 families will be included, preferably families who nish the entire trajectory.
As the design is a Single Case Experimental Design (SCED), the calculations for the amount of participants is based on the What Works Clearinghouse (WWC) (34) and on information gained from previous PCIT research. The WWC Standards dictates that there are ve criteria that should be acknowledged in a SCED. One of these criteria states that the minimum number of participants necessary in order to demonstrate an intervention effect is three (35). Additionally, the American Psychological Association (4) also states that to display effectivity of an intervention, a minimum of nine replicated single-case studies should be done. Furthermore, the attrition rates according to the standard PCIT have ranged between 34-77% (36). Speci cally, in the Netherlands, a previous study shows a 40% attrition rate (21). The current study will therefore strive to include 15 families, preferably who nish the entire trajectory to: (1) adhere to the minimum of three participants according to WWC standards, (2) be able to state that the intervention is effective in accordance with the APA guidelines and (3) compensate for the potential of a 40% attrition rate.

Inclusion criteria
Families that are referred for treatment to the clinical practice, will be screened with a questionnaire for behavioural problems, the Eyberg Child Behaviour Inventory (ECBI; (37)) and a questionnaire about parental stress ('opvoedingsbelasting vragenlijst' [in Dutch, OBVL], (38)). All families that are referred are required to ll out both questionnaires online to check their scores and whether they meet the inclusion criteria. The clinical cut-off score for the ECBI is > 132, meaning that all scores above 132 will be included. For the OBVL, a T-score of 60 or higher is problematic, thus counting as the cut-off score.

Exclusion criteria
A potential subject who meets any of the following criteria will be excluded from participation in this study: The child has a severe physical impairment, such as deafness, The child has a mental disability (IQ < 60), An unsafe home situation, where home displacement is indicated, Problems with the child or parent that require personal health care, such as suicidality, parents with problems with aggression regulation or an addiction, Parents are known to have severe problems with motion sickness.
Note IQ will not be tested during the screening, but rather a clinical judgment is made by the therapist or professional during intake

Study design
The study will use a non-concurrent multiple baseline Single-Case Experimental Design (SCED), across fteen participants. This means that fteen individual cases will be analysed, following the same design structure. SCEDs are often implemented to determine whether a causal relation exists as a result of introducing an independent variable to the dependent variable (39). Additionally, it is frequently used when attempting to answer a question regarding the addition of a component to an intervention (40). In terms of this study, this means that we will evaluate whether the introduction of a VR-element will increase positive parenting skills through multiple baseline SCEDs.
SCEDs can be used to both evaluate treatment effects for individuals, and to assess the e cacy of individualised treatments (39). Although SCEDs are often compared with a case study design, there is a clear distinction: SCEDs require the deliberate manipulation of an independent variable and results from SCEDs are usually both visually and statistically analysed (41,42). Another clear distinction from other designs is that SCEDs differ from experimental designs that are based on comparing groups: the manipulated variable in a SCED refers to repeated measurements of an individual that is assigned to different treatments, rather than the manipulated variable is being evaluated by assigning multiple individuals to different groups (41).
Strong SCEDs can be characterised by three dimensions: 1) the design is divided into phases, 2) the design contains random assignment, 3) the design should be replicated. Incorporating these dimensions improve the internal validity of the study and minimise the possibility of history and maturation affecting treatment effects (41). These dimensions are incorporated in the current study (see Figs. 1 and 2). Firstly, the building blocks of a phase design can be divided into a baseline phase (A) and a treatment phase (B), with each phase containing multiple measurements. This can be made more complex by, for example, adding another treatment phase (C). Secondly, a phase design can be randomised by having different starting points for the treatment phase. In addition, a minimum of three measurement points is required per phase. In our current study randomisation of the length of phases is used, which is increasingly considered to be important to make valid inferences about results. Thirdly, replication can occur two-fold: simultaneous replication entails conducting multiple phase designs at the same time, and sequential replication entails conducting multiple SCEDs in order to test generalisability of the results. When both forms of replication are used, we speak of a hybrid design, which is called a non-concurrent multiple baseline design, where partial temporal overlap is implied (41). To adhere to these three dimensions to maximise the internal validity, the current study will use a non-concurrent multiple baseline design.
The strength of SCEDs when designed with phases, randomisation and replication, is that it allows for comparison of an individual in different phases of treatment, thus demonstrating progression, whilst also being able to compare the treatment effects of multiple individuals at different times (41). Analyses are conducted both visually and statistically, thus giving an aggregated overview of individual and group treatment effects. SCEDs allow for a unique perspective where the clinical practice lends itself to research, rather than the other way round. An individual is personally being followed and data points are manipulated to give results, rather than their treatment condition being manipulated. Subsequently, group comparisons can be made on the basis of an individual's own trajectory. Considering the fact that PCIT has already widely been researched, employing the current design allows for an in-depth analysis of the newly implemented VR-element to the treatment.
The multiple baseline SCED in the current study is set up as follows (see Fig. 1): rstly, informed consent is signed after families have received verbal and written information about the study. Secondly, participants are dually randomised, to determine their baseline period (4, 5 of 6 weeks of baseline measurements; phase A) and the addition of their VR-element (VR at the start of CDI, after 3 or 6 sessions of CDI; phase B). After randomisation, participants start with phase A, the baseline period. Parents will electronically ll out questionnaires on a weekly basis regarding child disruptive behaviour and parental stress, using the ECBI and the short version of the OBVL. At the end of phase A, the pre-treatment assessment is done. In this assessment, parents are required to electronically ll out more questionnaires regarding both child and parental factors (see Fig. 1 and assessment instruments). In addition, the researcher will also tape a video recording of the parent and child playing in three play situations (Child-Led Play, Parent-Led Play and Clean Up), which will consequently be coded using Dyadic Parent-Child Interaction Coding System (DPICS; (43)). Following this, phase B, the intervention phase starts. Dependent on their random allocation, families start off with PCIT with or without the VR-element. They will once again weekly electronically ll out the ECBI and the OBVL-K, they will also register their homework completion sheet and (if applicable) their VR completion sheet. During the PCIT-sessions, PCITtherapists will weekly register the parental progress by coding 5-minutes of playtime with the DPICS. Once families reach CDI completion, they move on to phase C, which is the PDI phase of PCIT.
Measurements in phase C are identical to phase B. After treatment completion, a post-treatment assessment is done, which is identical to the pre-treatment assessment. Phase D is the follow-up assessment, which takes place 6 months after treatment completion. The assessment is also identical to the pre-and post-treatment assessments (see Fig. 2).
See 'Additional le 1' for Fig. 1 and insert here.
See 'Additional le 1' for Fig. 2 and insert here. Treatment PCIT focuses on changing dysfunctional parent-child interactions in two phases, and teaches authoritative parenting (19), which implies rm control, warmth to the parent-child relationship and a balance between discipline and stimulating independence (44). Parents are coached by therapists from behind a one-way screen through an earpiece (19). In the rst phase of PCIT, which is the Child-Directed Interaction (CDI) phase, emphasis is put on building up the quality of the parent-child relationship and increasing positive parenting skills. Additionally, in this phase, children begin to build the ability to regulate their behaviour (17). This rst phase lays a foundation for effective behaviour change. In the CDI-phase, parents follow their child's lead in play and are coached on using positive parenting skills, speci cally, praise, re ective statements, behavioural descriptions, imitation and enthusiasm (19). The number of CDI-sessions is dependent on the parents' skill mastery (10 behavioural descriptions, 10 re ective statements, 10 labelled praises and less than three questions, commands or negative verbalisations during a 5-minute observation period).
However, the CDI-phase alone, is typically insu cient for children with disruptive behaviour to return to displaying normal behaviour (45). After mastering the CDI-skills, the second phase of PCIT, which is the Parent-Directed Interaction (PDI) starts. The goal of the PDI-phase is to further enhance parents positive skills in order to set consistent, predictable, and age-appropriate boundaries for their children (19). In the PDI-phase, parents are coached on using clear commands, and how to apply consequences for compliance (praise) and non-compliance (timeout). Additionally, they are taught and encouraged to keep using the positive parenting skills from the rst phase. Once parents demonstrate mastery of both CDIand PDI-skills, and their children's behaviours according to the ECBI Intensity Scale raw is < 114, the treatment is completed, as standard protocol dictates (19).
In October 2020, twelve therapists were trained to become PCIT-therapists in the clinical treatment centre in the Netherlands. They were trained by PCIT-International certi ed trainers, thus ensuring the quality of the treatment. Therapists follow PCIT-protocol and have bi-weekly consultations with their trainers to keep track of their progress. In addition to that, the clinical centre already had multiple PCIT-therapists that were trained around ten years ago, who also still give PCIT to families. Both newly trained and other PCITtherapists will help collect data for current study.

Virtual Reality Implementation
In the standard protocol of PCIT, parents are required to practice the skills taught throughout the sessions in 'special time' at home with their children. This means that the families participating in PCIT have homework to do at home, to improve weekly. The current study suggests for parents to also practice the skills they are taught in the sessions at home in the virtual environment that will be created for them. As such, the implementation of VR is an additional element to the standard protocol of PCIT as described above.
There are multiple different systems within VR; one of which is a head-mounted display (HMD) (25). The current study will be using 360-degrees videos that can be played on a smartphone device, which is placed in the HMD. As a result, the person with the headset on, can look around 360 degrees from a singular point in the virtual environment, meaning that the environment does not change if the person wearing the headset decides to move. In technical terms, this is referred to as having three degrees of freedom. This option of technology was chosen as opposed to other forms of VR, as it is inexpensive and easy to use through a simple (but protected) link, thus allowing people to access it from anywhere and on their own time, which increases accessibility and applicability.
The 360-degree video content for this study is recorded using child actors for different scenarios. The various scenarios depict the positive parenting skills that are taught in the CDI phase (Praise, Behavioural Description, Re ective Statement, Ignoring Unwanted Behaviour), thus allowing parents to repeatedly practice these skills in the comfort of their own home. The recorded video is edited into fragments that contain key events that call for a response from the parent by answering a multiple-choice question asked at the end of each fragment. The fragments are assembled into a therapy scenario using an interactive scenario editor in which a therapy is described by a non-linear sequence of fragments, the order of which is determined by a parent's responses. Coaching -in the way that the therapist would do in therapy sessions -, is implemented by providing textual feedback to each given answer. The technology to support PCIT-VR is based on a web-based design that (1) provides the interactive scenario editor, (2) functions as the server that delivers the content of a therapy scenario to the parent's headset and (3) records performance data for future analysis. The web-based design allows the therapy scenarios to be used on any device that runs a web browser and allows use from any location with internet access.

Assessment instruments
The primary endpoint of positive parenting skills will be measured through the Dyadic Parent-Child Interaction Coding System (DPICS; (43)) and the added value of VR will be measured through Virtual Reality assessments. The secondary endpoints that include psychological functioning of the child, family dynamics and competences, and general therapy evaluation, will be measured through questionnaires and qualitative interviews. This is further explained below.
Dyadic Parent-Child Interaction Coding System (DPICS; (43)): This is a reliable and valid behavioural observational coding system designed to measure the quality of the social interactions between parent and child. By coding open verbal and physical behaviour exhibited by both parent and child, the DPICS is able to assess the quality of the parent-child interaction as a construct. Behaviours that are coded are divided into categories displayed in Table 1 below: Coding during assessments is done through three standard 5-minute play situations (Child-Led Play (CLP), Parent-Led Play (PLP) and Clean Up (CU). Within the 5-minute observational videos, the open verbal and physical behaviour is scored on a score sheet. To progress within PCIT, parents must achieve 10-10-10 of positive behaviours (10 behavioural descriptions, 10 re ective statements and 10 labelled praises) and must display no negative leading behaviours (questions, negative talk, and commands in CDI) within the 5-minute observation time. At every assessment point during treatment, the therapist will score the parent during that 5-minute observation time with the DPICS. A video-recording will be made of this, which will be used to assess the parents' progression. In addition to this, an independent researcher will randomly select a few of the sessions' video-recordings and score the DPICS as well to assess the quality of the coding. Researchers are trained beforehand to score the video-recordings through DPICS and are required to achieve an agreement rate of 80% on the scored behaviours. All observations will be transcribed verbatim to monitor interrater reliability.
Further assessment instruments are divided into four subcategories and explained below: I. Virtual Reality Evaluation VR user analytics: Each parent has a personal sign-in, which registers their answers when using VR. Their progress is monitored anytime they log in and use a therapy scenario.
Virtual Reality completion: The parents are required to use a VR tracking sheet, where they ll out the number of days that they completed the VR-training. This is then divided by the total number of days in the week, yielding a percentage of VR completion per week. Parents are required to ll this out weekly (alongside their 'special time homework completion', which is incorporated in the standard protocol of PCIT) as soon as they receive VR.
Qualitative interview parents: Parents evaluate the learnability and usability of their VR experience.
Questions re ect on how their experience with VR was, what they believe is the added value of VR to the therapy and whether VR helped to strengthen and comprehend the parenting skills at a faster pace. They are asked qualitative questions about their VR experience at the post-treatment and follow-up assessments.

II. Psychological Functioning Child
Questionnaire background information: This questionnaire contains questions with regards to background information about the child and his or her family. Questions concerning sex, age, ethnicity, family composition, parental work situation and education level are included. This questionnaire will be lled out at the pre-treatment assessment. If necessary, their treatment le can be consulted for supplementary information.
Eyberg Childhood Behaviour Inventory (ECBI): The ECBI (37) is a questionnaire containing 36 items, lled out by parents and addressing child behavioural problems for children between the ages of 2 and 16 years. The ECBI is part of the standard protocol of PCIT. It has good psychometric properties.
The frequency of problem behaviour (Intensity Scale; 7-point scale from 1 (never) to 7 (always)) and the extent to which parents experience the behaviour as problematic at that moment in time Norwegian versions of the screening questionnaire, it is valid and reliable. Parents will complete this questionnaire at the pre-treatment assessment and at the post-treatment assessment.
Child Abuse Potential Inventory (CAPI): The CAPI (47) is a questionnaire lled out by parents that measures the chances that parents (physically) abuse their child (aged between 0-18 years old). All 34 items include a statement with which parents can either 'agree' or 'disagree'. The Dutch translation has shown to be reliable and valid. This study will use the short version. Parents ll out this questionnaire at the pre-treatment assessment and at the post-treatment assessment.

III. Family Dynamics and Competences
Opvoedingsbelasting Vragenlijst (OBVL): The OBVL (38) is a Dutch questionnaire with 34 questions that address ve aspects on the burden of parenting; being Problems in the parent-child relationship, Problems in parenting, Depressed Moods, Role Restriction and Health complaints. The total score on the parenting burden can be calculated. The OBVL has a high reliability and norms have been measured for different age groups. Parents will complete this questionnaire at pre-and posttreatment and the follow-up assessments.
Opvoedingsbelasting Vragenlijst -Kort (OBVL-K): The OBVL-K (48) is a shortened version of the OBVL. It consists of 10 questions that together provide a total score on the burden of parenting. This questionnaire has a high reliability and norms have been measured for different age groups. Parents will complete this questionnaire weekly to get a general overview of their parental burden throughout the process.
Adult Self Report (ASR): The ASR (49) is a self-report that measures competencies, and emotional and behavioural problems for adults and it is based on the DSM. The questionnaire is split up in two sections, one concerning competencies and work, and the second section concerning emotional and behavioural problems. The English version has shown to be reliable and have a good validity. The Dutch version has yet to be evaluated. The parents will complete this self-report at pre-and posttreatment assessments.
Attachment Insecurity Screening Inventory (AISI): The AISI (50) is a questionnaire that measures attachment problems. The questionnaire contains 20 items that together form total attachment, but are divided into three subcategories, namely avoidant, ambivalent/resistant and disorganised attachment. The AISI 2-5 was found to valid and reliable according to recent research (50). The AISI 6-12 is practically identical to the AISI 2-5 but has not been tested on validity and reliability. The AISI will be lled out by parents at the pre-and post-treatment assessments.

IV. Therapy Evaluation
Therapy Attitude Inventory (TAI): The TAI (51) is a 10-item scale that evaluates parental satisfaction of the treatment. This questionnaire is lled out once by parents at the end of the treatment.
Qualitative interview parents: A qualitative interview post-treatment and follow-up assessments will allow parents to share their experiences with regards to the new form of PCIT-VR and the extent to which this therapy strengthened their competences and made them see their own abilities.
Qualitative interview therapists: Therapists will also be asked about their experiences and vision on the effectivity of PCIT and their vision on the added value of VR to the therapy at the end of the data collection period. They will be asked what factors contribute to effectivity, including factors such as their relationship with parents or speci c characteristics. Through this interview, different bene ts and obstacles of PCIT-VR can be brought to light through the eye of the therapist.

Statistical Analysis
Data-analysis will take place in consultation with a statistics expert of the Clinical Research Unit (CRU) department of the Amsterdam University Medical Centre (Amsterdam UMC). Data analysis will include the following: Primarily the added value of VR to PCIT will be evaluated through analyses.
SCED analyses will be conducted with the Shiny App.
Repeated measures analyses will be conducted.
Graphical analyses will be conducted to render the change over time.
The qualitative information from the interviews will be summarized to a complete picture of experiences from both parents and therapists. This will include their experiences with PCIT-VR, the VR component in particular and how the therapy helped for them to gain insight into strengthening their competencies.

Con dentiality of Data
The data for this study will be collected by researchers from the Department of Child and Adolescent Psychiatry at the Amsterdam UMC. The CRU of the Amsterdam UMC provides support for the dataanalysis of this study. All data will be stored in coded form, by assigning personal numbers to each family and registering all information by that number. This ensures that only authorised people can trace which le belongs to which family. All study results will be published anonymously.

Discussion
This study is the rst study that implements a VR-element that functions as an additional 'skill training' to magnify the scope of the PCIT through practice. Similar to standard PCIT, the intervention is centred around parent-child play sessions, where they receive live feedback and support from the therapist. The additional VR-element provides parents with the opportunity to practice the positive parenting skills in the comfort of their own home, without their child present, whilst still receiving feedback. This allows for them to further engrain the skills, alongside the therapy sessions and special time. Therefore, we expect that the implementation of VR will lead to faster skill mastery. Furthermore, current study expects that parental stress, child disruptive behaviour, dropout rates and the amount of necessary therapy sessions will diminish as a secondary effect of the additional VR-element.
Currently, the technology behind the VR-element is being developed in parallel with the study, meaning that the development of VR is uid. This is an asset as it means that the application can be adapted if any technical problems arise. A potential problem that could be encountered and should be resolved as soon as possible, is the functioning of the application on different smartphones. The application, the videos and the technology that underlies this must be designed in such a way that it is supported by all types of smartphones and their software versions. In order to nd the right t for smartphones, testing must be done on multiple devices before offering it to parents. To minimise the potential obstacles and to rather use them as learning points, close contact is kept with both technical support and parents. Parental feedback is processed as soon as possible by the technical support, and adaptations are made where and when necessary. Fortunately, SCEDs lend themselves well for this exibility, as each individual trajectory has different phases (baseline and treatment phase), thus enabling comparison between phases and even between measurement points. This means that the VR obstacles that one family faces, may differ from another family's obstacles. Therefore, the current study offers an opportunity to exibly implement VR and register individual obstacles as they come; both qualitatively as quantitatively.
The strength of SCEDs designed with phases, randomisation and the opportunity to replicate, means that it allows for comparison of an individual in different phases of treatment. Thus, it allows to follow progression, whilst also being able to compare the treatment effects of multiple individuals at different times. As analyses are conducted both visually and statistically, a clear overview can be given of both individual and group treatment effects. However, SCEDs also come with some potential limitations. For example, there is a risk of Type I errors due to potential violation of distributional assumptions and the presence of serial dependencies. Additionally, it is possible that there is an unexpected data trend due to maturation effects (e.g., gradual bettering of VR analytics due to habituation to scenarios). With the right statistical support, these pitfalls can potentially be mitigated, which is something the research group intends to do.
The clinical practice in which the study is conducted lends itself well for the research of VRimplementation within this study. This means that every family that participates, already provides enough data to be able to say something meaningful about the treatment and about VR. Overall, this study can be seen as a steppingstone to implementing VR on a larger scale within PCIT, to ultimately magnify the scope of the treatment through the additional skill training.