Network Analyses of Oppositional Defiant Disorder (ODD) Symptoms in Children

Abstract

Based on parent and teacher ratings of their children, this study used regularized partial correlation network analysis (EBIC glasso) to examine the structure of DSM-5 Oppositional Defiant Disorder (ODD) symptoms. Parent and teachers (N = 934) from the general community in Malaysia completed questionnaires covering DSM-5 ODD symptoms. The most central ODD symptom for parent ratings was anger, followed by argue. For teacher ratings, it was anger, followed by defy. For both parent and teacher ratings, the networks revealed at least medium effect size connections for temper and argue, defy and argue, blames others and annoy, and spiteful and angry. Overall, the findings were highly comparable across parent and teacher ratings, and they showed a novel understanding of the structure of the ODD symptoms. The clinical implications of the findings for assessment and treatment of ODD are discussed.

introduction

Oppositional Defiant Disorder (ODD) is a common childhood disorder (American Psychiatric Association [APA], 2000, 2013), and refers to a recurrent pattern of negativistic, defiant, disobedient, and hostile behavior toward authority figures that persists for at least six months (APA, 2013). Recently, the network model was proposed for understanding psychological disorders (Borsboom & Cramer, 2013). In this framework, the symptoms of a disorder are understood as a causal system, interacting with each other in meaningful ways, resulting in the disorder (Borsboom & Cramer, 2013). Although there is some network data for ODD symptoms, it is argued that there are major gaps and limitations in this area of research. In view of this, the current study used network analysis (Epskamp et al., 2018) to examine the network structure of DSM-5 ODD symptoms, based on parent and teacher ratings. 

Conceptualization and Latent Variable Models of DSM-5 ODD Symptoms

For diagnosis, DSM-IV (APA, 1994) and DSM-IV-TR (APA, 2000) have eight ODD symptoms organized together under one group, thereby implying support for a unidimensional model for these symptoms. DSM-5 has the same eight symptoms, but they are placed into three symptom groups: anger/irritable (comprising symptoms of temper tantrums, anger, and touchiness), vindictiveness (comprising the symptom for spiteful/vindictiveness), and argumentative/defiant behavior (comprising symptoms of arguing with adults, purposefully annoying others, disobedience, and blaming others for one’s own mistakes). Nevertheless, as the symptoms are the same in DSM-IV/DSM-IV-TR and DSM-5, the findings based on DSM-IV/DSM-IV-TR ODD symptoms are directly applicable to DSM-5 ODD symptoms.

The three ODD groups in the DSM-5 hint at the possibility that ODD might be multidimensional. To date, the structure of DSM-5 ODD symptoms has been examined extensively using confirmatory factor analysis (CFA). Supplementary Table S1 shows the factors and the symptom compositions of the major ODD factors in CFA models proposed in the literature. As shown in the table, both two- (Burke et al., 2010, Rowe et al., 2010) and three-factor (Aebi et al., 2010, Burke et al., 2014, Stringaris & Goodman, 2009) models have been proposed. Reviewing these models, Gomez et al. (2018) concluded that while there is some agreement for distinct factors for irritable/negative affect and headstrong/spiteful (or oppositional), there is a lack of agreement on the best factor structure for ODD, and the composition of the symptoms in the comparable primary factors in the different models.

Novel Network Model for ODD

A feature of CFA is that it is a latent variable model. A latent variable model provides a reflective view of psychopathology. As applied to a psychological disorder, this means that there is a latent (unobservable) construct (which is the disorder in question) that causes a range of observable responses (that are the symptoms of the disorder). Seen in the context of ODD, the reflective view suggests that the ODD symptoms are responses arising from a latent ODD construct. This means that the ODD symptoms are interchangeable and equally reflective of ODD. Also, the ODD symptoms are considered to have nothing in common after controlling for the latent construct (an assumption referred to as local independence). 

Although the latent variable model (like that captured in a CFA) is currently the most dominant approach for understanding psychopathologies, a newly developed perspective, called the network approach, has a different view of psychopathologies. As noted by Armour et al. (2017), the idea that symptoms do not interact with each other causally (as assumed in latent variable models) is highly implausible. In the network framework, symptoms are understood as a causal system, interacting with each other in meaningful ways, resulting in the disorder (Borsboom & Cramer, 2013).

A network model can be tested empirically using ‘network analysis’ (Borsboom & Cramer, 2013, Boschloo et al., 2015). Network analysis is an exploratory approach that provides visual and quantitative information about symptoms that are “core” or “central” (important) to the overall network of symptoms, and the strength of connections between symptoms (Borsboom & Cramer, 2013, Fried et al., 2015). In network analysis, one creates a network, preferably based on partial correlations between variables. As noted by Epskamp and others (Epskamp & Fried, 2018, Epskamp et al., 2017), such a network can identify unique interactions between variables that cannot be identified using multiple regression analysis, and when the network analysis is exploratory it is advantageous over structural equation modelling (SEM), because there are no equivalent undirected models possible in SEM. More details of network analysis are provided in the methodology section. 

Empirical Research on ODD Networks

To date, there have been at least three studies that have examined the network structure of ODD symptoms (Hukkelberg & Ogden, 2018, Preszler & Burns, 2019, Smith et al., 2017). Smith et al. examined the structure of DSM-5 ODD symptoms in a mixed (those with and without ODD) group of preschool children, using zero order correlations for the network. The findings showed that the symptoms of angry, annoy, and argue were central in the network, with angry being most central. Preszler and Burns (2019) examined the network structure of DSM-5 ODD symptoms together with Attention Deficit/Hyperactivity Disorder (ADHD, APA, 2013) symptoms in a group of primary school-aged children from the general community, based on mother and father ratings, using a partial correlation procedure for the network.  The network found symptom clusters for ODD, and the ADHD dimensions of inattention (IA) and hyperactivity/impulsivity (HI). For the ODD symptoms, the symptom of refuse was most central, and the symptom of spiteful was least central. Also, the stronger connections were between argue and temper, and annoy and spiteful. Hukkelberg and Ogden (2018) examined the network structure for ten ODD symptoms (extracted from the Antisocial Behavior Scale of the Home and Community Social Behavior Scales, Merrell & Caldarella, 1999) for parent ratings of primary-school aged children. The study used a partial correlation procedure. Although the ten ODD items were selected to cover the ODD factors of irritable, headstrong, and hurtful, it did not have a spiteful symptom, and it included symptoms not listed as ODD symptoms in DSM-5 (i.e., ignores parents or supervisors, destroys or damages others’ property, and insults peers). Keeping this in mind, the stronger connections were between temper and easily provoked, easily provoked and easily irritated, disrupts on-going activities and bothers and annoys others, and bothers and annoys others and insults peers. Also, bothers and annoys others and blames others were more central symptoms, and defiant to parents showed the least centrality. 

Limitations and Omissions in Network Analysis of DSM-5 ODD Symptoms 

The findings from past network studies of the ODD symptoms do not allow for any clear interpretation of the network structure of the ODD symptoms. This is not surprising as the studies have included different symptoms in their network. To date, the study by Smith et al. (2017) is the only one that focused exclusively on DSM-5 ODD symptoms. Even so, the findings in this study are limited. First, the study used zero-order correlation for constructing the network, which has the potential to inflate correlations, thereby resulting in difficult to interpret and misleading results (Epskamp et al., 2018). Second, the study examined preschool children, and used parent ratings of the ODD symptoms. Thus we have no data for children and adolescents, and for teacher ratings. As ODD is highly relevant to children, and as the DSM-5 views severity of ODD in terms of the presence of ODD across settings, and as teachers are useful sources of information for clinical diagnosis, it will be also useful to know the network structure of ODD symptoms for children based on teacher reports. Third, the study did not examine the accuracy and stability of the findings for centrality and edge weights. This is a limitation, as network analysis experts have recommended that a network must also be evaluated for its accuracy and stability (Epskamp et al., 2018). Fourth, as all previous studies in this area have been in western countries, we do not have any network data for ODD in non-western countries. Such data could be useful to understand ODD cross-culturally. Given these limitations and omissions, there is clearly a need for more network analysis studies, applying partial correlation approaches, involving parent and teacher ratings of Malaysian primary school-aged children, and for the network findings to be examined for accuracy and stability. 

Clinical Importance of Network Analysis of the ODD Symptoms

Results from network analyses of the symptoms of a disorder can have important implications for theory, assessment and diagnosis, treatment, and prevention. Traditionally, the theoretical importance of a symptom is viewed in terms of its severity which is ascertained in terms of its mean score. However, in network models, centrality—which is different from the mean score—defines the importance of a symptom. Indeed, the mean levels of symptoms can change without changes in their centrality in the network (Yang et al., 2016). Thus, different conclusions about what are core symptoms in a disorder could be arrived at when looking at symptom centrality and symptom severity (Mullarkey et al., 2019). 

In relation to treatment, as symptoms for a disorder identified as central in a network are considered most influential in producing or maintaining the disorder, intervening on these symptoms can be expected to maximize the impact of intervention. In this respect, and given its network characteristic, focusing on the central symptoms could potentially have a downstream effect in improving other symptoms. 

Aims of the Present Study

Despite the novelty and noted advantages of the network approach, to the best of our knowledge, to date, no study has so far used network analysis to examine the structure of DSM-5 ODD symptoms for primary school-age children for parent and teacher ratings. Consequently, the major aim in the current study was to use network analysis, with regularized partial correlation, to examine the network structure of the eight DSM-5 ODD symptoms (temper, argue, defy, annoy, blames others, touchy/annoyed, angry/resentful, spiteful/vindictive), in a large community sample, based on parent and teacher ratings (separately). For each respondent, we produced a network graph, displaying the topology of the symptom network. We then evaluated statistically (edge width and centrality) the symptoms most influential in the network, and the robustness and stability of the network. 

Method

Participants, Measure, and Procedure 

The participants in this sample were the same as those involved in previous studies that examined the prevalence of ADHD in Malaysian primary school children (Gomez & Hafetz, 2011), the structure of the ODD symptoms (Gomez, 2017), and gender invariance (Burns et al., 2006). Since details of the participants were provided in those papers, and due to space limitation, only a brief description of participants, measures, and procedure is provided in this paper. In all, 934 parents and teachers from the State of Johor in Malaysia participated in this study. These respondents provided ratings for ODD symptoms (obtained using the ODD symptoms in the Disruptive Behavior Rating Scale (DBRS, Barkley & Murphy, 1998) for 436 boys and 496 girls, between 6 and 12 years of age. The children were from fourteen randomly selected schools. For the current study, the DBRS was translated into Malay (developed via forward and backward translation by experts in both languages), with some parents completing the English version, and others completing the Malay version. Respondents rated the occurrence of each symptom over the past 6-months on a 4-point scale (0 = never or rarely, 1 = sometimes, 2 = often, 3 = very often). The Cronbach’s alpha values for parent and teacher ratings for the ODD symptoms were 0.89 and 0.94 respectively. There was no significant difference for age between boys and girls, and their ethnic background and fathers’ occupational levels did not differ significantly from the general Malaysian population. Ethical approval for the study was obtained from the University of Ballarat Human Research Ethics Committee, the Federal Ministry of Education of Malaysia, and the Ministry of Education of the State of Johor, where the study was conducted. Prospective parent participants were provided a plain language statement providing the background of the study, the consent form, the DBRS, and a return envelope. Parents were also asked to provide the child's age, gender, and ethnic background, and their willingness to have the DBRS completed by their child’s class teacher. When h consent was available, the child’s teacher was requested to complete the DBRS for the child. There was a   participation rate of approximately 93%. 

Statistical Network Analyses                       

A network analysis graph comprises of nodes (the variables used for the network), and s edges (the relationships between the nodes). Edge weight refers to the strength of the relationship between two nodes. For the study, edge weights were estimated using a regularized partial correlation approach, such as g-lasso (Tibshirani, 1996) that shrinks small partial correlations to 0, resulting in a sparse network, and showing only the most important relationships in it. 

Jeffreys' Amazing Statistics Program (JASP) version 0.14.1.0 statistical software (Team, JASP, 2018) that uses the bootnet (Epskamp, Borsboom, & Fried, 2018) and the qgraph (Epskamp et al., 2012) packages from R, was used in the study to conduct the network analysis. As used in y others (Heeren, Bernstein, & McNally, 2018, Isvoranu et al., 2017, McNally, Heeren, & Robinaugh, 2017), we applied the extended bayesian information criterion (EBIC) glasso for the network analysis to produces the optimal degree of shrinkage, setting the hyperparameter at 0.5 (Foygel & Drton, 2010, Epskamp & Fried, 2018).  The network shows blue and read connections between nodes that are indicative of positive and negative relations, respectively. For both, stronger relationships are shown in terms of them being thicker and denser in colored, and nodes with stronger similarities are placed closer together. 

A network can also be descried statistically (Borgatti, 2005) in terms of edge weights (indicating the correlation or partial correlation between nodes) and centrality (indicating the relative importance of the individual nodes in the network). As a central symptom is one that is highly connected to other symptoms, and its activation can be expected to spread to other symptoms. Strength (also called degree), betweenness, closeness, degree, and expected influence are commonly reported indices of centrality (Opsahl et al. 2010). We used degree and expected influence as measures of centrality in the study. Degree is the sum of all direct associations a given symptom exhibits with all other nodes, and it reflects the direct influence a given node has on the network. The expected influence for a node is the absolute sum of edge weights associated with it, taking into consideration negative nodes. It is therefore easier to interpret than degree (Bringmann et al., 2019). Nodes with high expected influence centrality values indicate that they are more central. 

A network must also be evaluated for its accuracy and stability. The accuracy of edge weights can be evaluated using bootstrap 95% non-parametric confidence intervals (CIs) (Epskamp et al., 2018a), with narrower CIs suggest a more precise estimation of the edge (Epskamp et al., 2018), and therefore more accuracy.  The stability of the centrality indices can be examined using case-dropping bootstrapping (Epskamp et al. 2018). In brief, it examines if the order of centrality indices remains the same after re-estimating the network with less cases (or nodes), quantified in terms of correlation stability coefficient. This coefficient reflects the correlation between the original centrality indices (based on the full data) and the correlation obtained from the subset of data representing different percentages of the overall sample. Although a correlation stability coefficient of 0.7 or higher has been suggested as being the threshold, Epskamp et al. (2018) have suggested that the correlation stability coefficient should be at least 0.5. The stability of the centrality indices and edge accuracy of the network were examined using the procedures just described, with 1000 bootstraps. 

Results

Descriptive Information of Data 

There was no missing data. Initially we examined the mean and standard deviation (SD) scores. The findings are presented in Supplementary Table S2. As show, the mean score for the eight symptoms for parent ratings ranged from 0.44 to 1.08. For teacher ratings, the mean score for the eight symptoms ranged from 0.38 to 0.63. The two most severely rated symptoms were temper and touchy for parent ratings, and angry and temper for teacher rating. For both respondents, the symptom with the lowest severity rating was spiteful. Inspection of the distribution of the frequencies for each symptom indicated continuous but non-normal distribution for both parent and teacher ratings. Although the mean symptom scores for parent and teacher ratings indicate relatively low levels of the ODD symptoms, all four response options for all eight symptoms for both respondents were endorsed. Thus it can be assumed that the ratings provided by both parents and teacher captured the full spectrum of the ODD symptoms. 

Visualization of the ODD Network

With eight symptoms or nodes, the maximum number of edges in this network was 24. Although the EBIC glasso estimation used in the analysis did not reduce the number of edges for parent ratings, it reduced it to 22 for teacher ratings. 

Figure 1 shows a visualization of the network of the eight ODD symptoms. As shown, for parent ratings (Figure 1, left side), the symptoms were somewhat evenly disbursed. Also, all symptoms were associated positively (blue edges) with one another. Angry (7) was placed more to the center of the network, having connections with all symptoms except for argue (2). For teacher ratings (Figure 1, right side), all the symptoms were also relatively evenly disbursed. All symptoms were associated positively (blue edges) with one another, except for defy (3) with touchy (6), and defy (3) with temper (1). Annoy (4) was placed more to the center of this network, and it was linked to all other symptoms in the network. Also, visually, for both parent and teacher ratings, touchy (6), angry (7), and spiteful (8) were linked together relatively closely in one group in one section of the network, and temper (1), argue (2), defy (3), annoy (4) and blames others (5) were linked together relatively closely in another group in a different section of the network.  

Edge Weights of ODD Symptoms in the ODD Network

The accuracy of the edge weights, estimated using bootstrap 95% non-parametric Cis, are shown in Supplementary Figure S1. As shown, for both parent (left in the figure) and teacher (right in figure) ratings, the CI ranges around most of the estimated edge-weights were moderate, thereby indicating moderate precision for the edge weights. Thus, the interpretation of the edges in the networks for parent ratings and teacher ratings may be somewhat problematic.

Table 1 shows the weights matrix between the ODD nodes from the network analysis for parents (below the diagonal) and teachers (above the diagonal). For ease of interpretation, based on Cohen’s (1992) effect size guidelines for correlation (r) values (0.1 = low, 0.3 = medium, and 0.5 = large), we interpreted r ≥ 0.3 (medium effect size) as important. As shown in this table, the edge weights for parent ratings (left side) meeting this cut-off score were for temper (1) and argue (2), argue (2) and defy (3), annoy (4) and blames others (5), and angry (7) and spiteful (8).  Also as shown in Table 1, the edge weight findings for teacher ratings (right side) was the same as for parent ratings, with the addition that the teacher rating for annoy (4) with defy (3) also met this cut-off score of 0.30. Indeed, for both respondents, the edge weights between defy (3) and argue (2), and spiteful (8) and angry (7) were of large effect sizes. The edge weights for parent ratings showed zero values for argue (2) with touchy (6) and angry (7), and annoy (4) with touchy (6). For teacher ratings, the values were zero for temper (1) with spiteful (8), argue (2) with blames others (5) and spiteful (8), defy (3) with blames others (5), annoy (4) with angry (7), and blames others (5) with spiteful (8).

Centrality of the ODD Symptoms in the ODD Network 

The stability of the centrality indices (betweenness, closeness, and strength) examined using case-dropping bootstrapping are shown in Supplementary Figure S2 (left side for parent ratings, and right side for teacher ratings). For all centrality indices, the figure shows the correlation stability (CS) coefficients from the subsets of data representing different percentages of the overall sample. Supplementary Figure S2 shows that there was a slight drop in the correlations between the subsample estimates and the estimate from the original entire sample as the subset samples decreased from 95% of the original sample to 25% of the sample. For parent ratings, however, the correlations for the centrality indices for betweenness and strength remained above .7 for the decrease from 95% of to 25% of the sample, thereby indicating stability for the betweenness and strength centrality indices (Epskamp et al., 2018). For teacher ratings, the correlations for only the centrality index for strength remained above .7 for the decrease from 95% to 25% of the sample. 

The standardized estimates of the centrality indices for betweenness, closeness, strength, and expected influence are presented in Table 2. To ease interpretation, plots for the centrality measures in terms of z scores were created, and this is displayed in Figure 2. Both Table 2 and Figure 2 present the centrality indices for betweenness, closeness, and degree (strength). As shown in Table 2 and Figure 2, for the different nodes, there was notable variability in their relative values. Thus to ensure clear interpretation of centrality, we examined strength centrality, as this index was the only one that showed stability for both parent and teacher ratings, and also because it is known to reflect reasonably precise centrality estimates for psychology networks (Santos et al., 2018). In general, higher strength centrality values indicate more centrality. 

As shown in Figure 2 and Table 2, the two symptoms (in descending sequence) with the highest strength centrality values were angry (7) and argue (2) for parent ratings. For teacher ratings, they were angry (7) and defy (3). For both respondents, spiteful (8) had the lowest strength centrality value.

Discussion

The current study is the first to use network analysis to examine the structure of the eight DSM-5 ODD symptoms (APA, 2013) in a group of primary school-aged children, based on parent and teacher ratings. It examined the centrality of the ODD symptoms in the network, the edge weights for the ODD symptom pairs, and the stability and accuracy of indices for centrality and edges.

Topology of the ODD symptom in the Networks

Overall, there was reasonable level of comparability in the topology of the ODD symptom networks across parent and teacher ratings. For both parent and teacher ratings, touchy, angry, and spiteful were linked together relatively closely in one section of the network, and temper, argue, defy, annoy, and blames others were linked together relatively closely in a different section of the network. For parent ratings, angry was placed more to the center of the network, having connections with all, except the symptom for argue. For teacher ratings, annoy was placed more to the center of this network, and it was linked to all other symptoms in the network. 

ODD Network Findings

For parent ratings, the highest strength centrality values were angry, followed by argue. It therefore follows that for parent ratings, angry and argue may be most important ODD symptoms. For teacher ratings, the highest strength centrality values were angry, followed by defy, thereby indicating that for teacher ratings, angry and defy may be the most important ODD symptoms. 

For both parent and teacher ratings, there was high connectivity between the symptoms, thereby raising the possibility that many of the ODD symptoms are closely connected with each other. For both parent and teacher ratings the edge weights with at least medium effect sizes were for temper and argue, defy and argue, blames others and annoy, and spiteful and angry. For teacher ratings only, this was also so between annoy and defy. For both parent and teacher ratings, the edge weights between defy and argue, and spiteful and angry, were of large effect sizes. Overall, therefore, there was high degree of comparability in terms of the centrality and edge weights of the symptoms across parent and teacher ratings. 

Comparison of Current and Past ODD Network Findings

Although there have been three studies that have examined networks that included the OSS symptoms (Hukkelberg & Ogden, 2018, Preszler & Burns, 2019, Smith et al., 2017), only the Smith et al. study focused exclusively on DSM-5 ODD symptoms, based on parent ratings.  Preszler and Burns (2019) examined the network structure of DSM-5 ODD symptoms together with ADHD symptoms, and Hukkelberg and Ogden (2018) examined the network structure of ten relevant ODD symptoms and not the eight DSM-5 ODD symptoms. Given this, only the network for ODD in the Smith et al. study provides an appropriate comparison with the findings in the current study, and that with the findings derived from parent and not teacher ratings. In both studies, for parent ratings, all the symptoms were relatively evenly disbursed in the network. However, as already noted, in the current study, the two symptoms (in descending sequence) with the highest strength centrality values were angry, and argue parent ratings. In contrast, the findings in the Smith et al. study showed that the symptoms for angry, annoys, and argues were central in the network, with anger being most central. Thus, although there were some differences, there was reasonable comparability in the findings in the current study and that reported in the Smith et al. study. This is so, despite major differences across these studies. In contrast to the current study that examined primary-school age children from the general community in an Asian country (Malaysia), Smith et al. examined the network of the ODD symptoms in a group of preschool children with and without ODD in a US sample. Additionally, the Smith et al. study used zero order correlations in the network analysis, whereas the current study used regularized partial correlation. Taking all these into consideration, it can be argued that existing findings from network analysis of ODD symptoms are robust. 

Novel Clinical Implications

Our findings have novel implications for theory, classification, assessment and diagnosis, and treatment and prevention. We focus here on the major implications.  

Firstly, in a network, symptoms with high centrality values are considered most influential in producing or maintaining the disorder. For parent ratings, the highest strength centrality values were angry, followed by argue, and for teacher ratings, it was angry, followed by defy. Thus it can be argued that the angry, argue, and defy symptoms are especially important for understanding and diagnosis of ODD. Individuals with serious problems related to angry and/or argue are likely to demonstrate or to be at risk for more serious ODD presentations. Thus clinicians may wish to pay special attention to the presence of these symptoms during assessment and diagnosis of ODD. For both respondents, the spiteful symptom had the lowest centrality value, and therefore this symptom may not be critical for ODD. 

Secondly, the theoretical importance of a symptom is traditionally viewed in terms of its severity which is ascertained in terms of its mean score. The two most severely rated symptoms were temper and touchy for parent ratings, and angry and temper for teacher ratings. However given that in the network analysis for parent ratings, the highest two centrality symptoms were angry and argue, and for teacher ratings, it was angry and defy, different conclusions about what are core symptoms in ODD are found when looking at symptom centrality compared to symptom severity (Mullarkey et al., 2019). Thus it will be useful for clinicians to also consider symptom centrality when assessing and treating children with ODD. 

Thirdly, because the symptoms with high centrality values are considered most influential, intervening on these symptoms could maximize the impact of an intervention, including reducing the effects of other symptoms. This, therefore, could mean that focusing intervention efforts on angry, argue, and defy symptoms rather than the other symptoms could maximize treatment effects, and also likely cascade to reduce the effects of other symptoms. Where relevant, focusing on the symptoms with high centrality values (angry, argue, and defy in the case of ODD) may also prevent the on-set and development of ODD in the context of primary prevention protocols implemented in the community. 

Fourthly, the edge weights for parent ratings showed zero values for argue with touchy and angry, and annoy with touchy. For teacher ratings, the values were zero for temper with spiteful, argue with blames others and spiteful, defy with blames others, annoy with anger, and blames others with spiteful. The absence of a connection between two symptoms in a network implies that they are conditionally independent of each other given the other symptoms in the network. Thus the absence of connections for these pairs of symptoms could mean that the symptoms in these relations are conditionally independent of each other. This is a novel finding and indicates that there may be a need to review the relevance of some of the DSM-5 symptoms for ODD. 

Fifthly, for both parent and teacher ratings there was one group of closely linked symptoms comprising touchy, angry, and spiteful symptoms (group 1), and another group of closely linked symptoms comprising temper, argue, defy, annoy, and blames others symptoms (group 2). Viewed in terms of how the ODD symptoms are grouped in DSM-5, the first group for both respondents can be considered as an irritability/spiteful group, and second group can be considered as a defiant group. For parent ratings, the highest strength centrality values were angry, followed by argue. As angry and argue are symptoms in the irritability/spiteful and defiant groups, respectively, it follows that for parent ratings, angry may be the most important symptom for the irritability/spiteful group, and argue may be most important symptom for the defiant group. 

For teacher ratings, the highest strength centrality values were angry, followed by defy. As angry, and defy are symptoms in the irritability/spiteful and defiant groups, respectively, it follows that for teacher ratings, angry may be most important symptom for the irritability/spiteful group, and defy may be most important symptom for the defiant group. Thus, there was some degree of comparability in terms of the centrality of the symptoms across parent and teacher ratings, especially for the irritability/spiteful group. Also, across the two symptom groups, the dominant connections were temper and argues for parent ratings, and spiteful and angry for teacher ratings. 

Sixthly, as the irritability/spiteful group comprised the irritability and spiteful DSM-5 ODD symptoms, and the defiant group comprised the DSM-5 ODD defiant symptoms, the findings suggest the possibility that unlike how the ODD symptoms are grouped in DSM-5, the spiteful symptom could be placed in the angry/irritable symptom group, rather than on its own. Also, when ODD symptoms are viewed in terms the latent variable framework, a two-factor model (with latent factors for irritability/spiteful and defiant) may be most appropriate. To date ODD models with one symptom group comprising touchy, angry, and spiteful, and another symptom group comprising temper, argue, defy, annoy, and blames others, has not been proposed (see Supplementary Table S1). This model may be worthy of exploration in future studies.

Summary of Findings in the Study

The most central ODD symptom for parent ratings was angry, followed by argue.  For teacher ratings, it was angry, followed by defy. For both parent and teacher ratings, the networks revealed at least medium effect size connections for temper and argue, defy and argue, blames others and annoy, and spiteful and angry. Overall, the findings were highly comparable across parent and teacher ratings, thereby attesting to their robustness. Also worthy of note is that the stability and accuracy of indices for centrality and edges were supported.

Limitations and Directions for Further Studies

Despite the positive value of the findings in the current study, the results in the study have to be interpreted in the light of a number of limitations. Firstly, as the study showed only moderate stability and accuracy for edge weights, the edge weight findings need to be interpreted with some caution. Secondly, network analysis assumes that mental disorders (and therefore ODD) are causal systems. However as we used cross-sectional data in the current study, causality cannot be securely assumed.  At best, we were able to eliminate spurious candidates for causal relations. Causality assessment would require longitudinal data, collected repeatedly. Further studies may wish to examine such concerns, using longitudinal network analysis. Thirdly, as we conducted the network analysis using a normative-community sample, the findings cannot be directly generalized to other samples, like specific racial and clinical groups. Fourthly, as we used parent and teacher rating measures, the findings may not be applicable to data collected via clinical interviews, or from other sources. Fifthly, as our findings are based on group-level analyses, it may not be directly applicable at the individual level. It is possible that some of the associations found in the current study may not be applicable to some individuals. Clearly, we need more network studies of the ODD symptoms, using longitudinal data, collected using multiple sources and methods and different racial and clinical groups. Individualized networks would also be beneficial for a comprehensive understanding of the ODD network.  Despite these limitations, our findings do offer novel insights on the structure of ODD symptoms, their relative importance that can be used effectively for theorizing, assessing and treating ODD.

Declarations

Ethics approval and consent to participate

The research was approved by the Human Research Ethics Committee of Federation University, Australia (HREC Project No. 02/41). The research was conducted under their guidelines, and the ethical standards of the Australian National Medical Research Council, which adheres to the Declaration of Helsinki. All methods were carried out in accordance with these guidelines. All parents provided written consent for their child to participate, as well as their child’s teachers to participate in completing the required questionnaires. Following this, approvals were sought from the relevant authorities for distributing the relevant documents, including the dissemination of the DBRS to parents through schools. Approvals were obtained from the Federal Ministry of Education of Malaysia, and the Ministry of Education of the State of Johor. These consent procedures were approved by the ethics committee.

Consent for publication

Not applicable.

Availability of data and materials

All data generated or analyzed during this study are included in this published article and the supplementary information file (titled data). 

Competing interests

The authors declare that they have no competing interests.

Funding

No specific fund supported the study beyond the usual salaries paid by the authors’ institutions.

Authors’ contributions

RG were responsible for designing the study, ethic application, data collection, statistical analyses, and writing and revising all sections of original and subsequent drafts of the paper.  VS was responsible for designing the study, ethic application, data collection, and revising all sections of the drafts of the paper. AG was involved in statistical analyses and revising all sections of the drafts of the paper. SW was involved in revising all sections of the drafts of the paper, formatting and submission. 

Acknowledgements

Not applicable. 

References

1. Aebi, M., Plattner, B., Metzke, C.W., Bessler, C., Steinhausen, H.C. (2013). Parent-and self-reported dimensions of oppositionality in youth: construct validity, concurrent validity, and the prediction of criminal outcomes in adulthood. Journal of Child Psychology & Psychiatry, 54, 941–949. https://doi.org/10.1111/jcpp.12039
2. American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). American Psychiatric Publishing, Inc. 
3. American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed.) [Text Revision]. American Psychiatric Publishing, Inc. https://doi.org/10.1016/B978-1-4377-2242-0.00016-X 
4. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). American Psychiatric Publishing, Inc. https://doi.org/10.1176/appi.books.9780890425596 
5. Armour, C., Fried, E. I., Deserno, M. K., Tsai, J., & Pietrzak, R. H. (2017). A network analysis of DSM-5 posttraumatic stress disorder symptoms and correlates in US military veterans. Journal of Anxiety Disorders, 45, 49-59. https://doi.org/10.1016/j.janxdis.2016.11.008
6. Barkley, R.A., Murphy, K.R. (1998). A Clinical Workbook: Attention-deficit Hyperactivity Disorder. Guilford. https://doi.org/10.1007/978-1-4615-4881-2_13
7. Borgatti, S. P. (2005). Centrality and network flow. Social Networks, 27, 55-71. https://doi.org/10.1016/j.socnet.2004.11.008
8. Borsboom, D., & Cramer, A. O. (2013). Network analysis: an integrative approach to the structure of psychopathology. Annual Review of Clinical Psychology, 9, 91-121. https://doi.org/10.1146/annurev-clinpsy-050212-185608
9. Boschloo, L., van Borkulo, C. D., Rhemtulla, M., Keyes, K. M., Borsboom, D., & Schoevers, R. A. (2015). The network structure of symptoms of the diagnostic and statistical manual of mental disorders. Plos One, 10(9), e0137621. https://doi.org/10.1371/journal.pone.0137621" https://doi.org/10.1371/journal.pone.0137621
10. Burke, J. D. (2012). An affective dimension within oppositional defiant disorder symptoms among boys: personality and psychopathology outcomes into early adulthood. Journal of Child Psychology & Psychiatry, 53, 1176–1183. https://doi.org/10.1111/j.1469-7610.2012.02598.x
11. Burke, J. D., Boylan, K., Rowe, R., Duku, E., Stepp, S. D., Hipwell, A. E., & Waldman, I. D. (2014). Identifying the irritability dimension of ODD: Application of a modified bifactor model across five large community samples of children. Journal of Abnormal Psychology, 123(4), 841-851. https://doi.org/10.1037/a0037898 
12. Burke, J. D., Waldman, I., Lahey, B. B. (2010). Predictive validity of childhood oppositional defiant disorder and conduct disorder: Implications for the DSM-V. Journal of Abnormal Psychology, 119, 739–751. https://doi.org/10.1037/a0019708
13. Burns. G. L., Walsh, J. A., Gomez, R., Hafetz, N. (2006) Measurement and structural invariance of parent ratings of ADHD and ODD symptoms across gender for American and Malaysian children. Psychological Assessment, 18, 452–457. https://doi.org/10.1037/1040-3590.18.4.452
14. Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159. https://doi.org/10.1037/0033-2909.112.1.155 
15. Epskamp S., Rhemtulla M. T., Borsboom D. (2017). Generalized Network Psychometrics: Combining Network and Latent Variable Models. Psychometrika, 82, 904–927. https://doi.org/10.1007/s11336-017-9557-x
16. Epskamp, S., & Fried, E. I. (2018). A tutorial on regularized partial correlation networks. Psychological Methods, 23, 617-634. https://doi.org/10.1037/met0000167
17. Epskamp, S., Borsboom, D., & Fried, E. I. (2018). Estimating psychological networks and their accuracy: A tutorial paper. Behavior Research Methods, 50, 195-212. https://doi.org/10.3758/s13428-017-0862-1
18. Epskamp, S., Cramer, A. O., Waldorp, L. J., Schmittmann, V. D., & Borsboom, D. (2012). qgraph: Network visualizations of relationships in psychometric data. Journal of Statistical Software, 48, 1-18. https://doi.org/10.18637/jss.v048.i04
19. Foygel, R., & Drton, M. (2010). Extended Bayesian Information Criteria for Gaussian Graphical Models. Advances in Neural Information Processing Systems, 23, 2020-2028. 
20. Fried, E. I., Bockting, C., Arjadi, R., Borsboom, D., Amshoff, M., Cramer, A. O. J., Epskamp, S., Tuerlinckx, F., Carr, D., & Stroebe, M. (2015). From loss to loneliness: The relationship between bereavement and depressive symptoms. Journal of Abnormal Psychology, 124, 256-265. https://doi.org/10.1037/abn0000028
21. Fruchterman, T. M., & Reingold, E. M. (1991). Graph drawing by force‐directed placement. Software: Practice and Experience, 21, 1129-1164. https://doi.org/10.1002/spe.4380211102
22. Gomez, R. (2017). Factor structure of parent and teacher ratings of the ODD symptoms for Malaysian primary school children. Asian Journal of Psychiatry, 25, 22-26. https://doi.org/10.1016/j.ajp.2016.10.013
23. Gomez, R., & Hafetz, N. (2011). DSM-IV ADHD: Prevalence based on parent and teacher ratings of Malaysian primary school children. Asian Journal of Psychiatry, 4(1), 41-44. https://doi.org/https://doi.org/10.1016/j.ajp.2010.12.003
24. Gomez, R., Stavropoulos, V., & Van Doorn, G. (2018). Factor structure of teacher ratings of the ODD symptoms in children. Psychiatric Quarterly, 89, 219-234. https://doi.org/10.1007/s11126-017-9527-6
25. Heeren, A., Bernstein, E. E., & McNally, R. J. (2018). Deconstructing trait anxiety: a network perspective. Anxiety Stress Coping, 31(3), 262-276. https://doi.org/10.1080/10615806.2018.1439263 
26. Hukkelberg, S. S., & Ogden, T. (2018). Dimensionality of oppositional defiant disorder. Child and Adolescent Mental Health, 23, 121–129. https://doi.org/10.1111/camh.12248
27. Isvoranu, A. M., Boyette, L. L., Guloksuz, S., & Borsboom, D. (2017). Chapter G1. Symptom network models of psychosis. Dimensions of Psychosis. Oxford University Press. https://doi.org/10.31234/osf.io/nk8yv
28. JASP Team (2020). JASP (Version 0.14.1) [Computer software].
29. Krämer, N., Schäfer, J., & Boulesteix, A. L. (2009). Regularized estimation of large-scale gene association networks using graphical Gaussian models. BMC Bioinformatics, 10, 1-24. https://doi.org/10.1186/1471-2105-10-384
30. McNally, R. J., Heeren, A., & Robinaugh, D. J. (2017). A Bayesian network analysis of posttraumatic stress disorder symptoms in adults reporting childhood sexual abuse. European Journal of Psychotraumatology, 8 (sup3), 1341276. https://doi.org/10.1080/20008198.2017.1341276
31. Merrell, K. W., & Caldarella, P. (1999). Social-Behavioral Assessment of At-Risk Early Adolescent Students: Psychometric Characteristics and Validity of a Parent Report Form of the School Social Behavior Scales. Journal of Psychoeducational Assessment, 17(1), 36-49. https://doi.org/10.1177/073428299901700104 
32. Mullarkey, M. C., Marchetti, I., & Beevers, C. G. (2019). Using network analysis to identify central symptoms of adolescent depression. Journal of Clinical Child & Adolescent Psychology, 48, 656-668. https://doi.org/10.1080/15374416.2018.1437735
33. Opsahl, T., Agneessens, F., & Skvoretz, J. (2010). Node centrality in weighted networks: Generalizing degree and shortest paths. Social Networks, 32, 245-251. https://doi.org/10.1016/j.socnet.2010.03.006
34. Preszler, J., & Burns, G. L. (2019). Network analysis of ADHD and ODD symptoms: Novel insights or redundant findings with the latent variable model? Journal of Abnormal Child Psychology, 47, 1599–1610. https://doi.org/10.1007/s10802-019-00549-y
35. Rowe, R., Costello, E.J., Angold, A., Copeland, W.E., Maughan, B. (2010). Developmental pathways in oppositional defiant disorder and conduct disorder. Journal of Abnormal Psychology, 119, 726–738. https://doi.org/10.1037/a0020798
36. Santos Jr, H. P., Kossakowski, J. J., Schwartz, T. A., Beeber, L., & Fried, E. I. (2018). Longitudinal network structure of depression symptoms and self-efficacy in low-  income mothers. Plos one, 13(1), e0191675. https://doi.org/10.1371/journal.pone.0191675
37. Smith, T. E., Lee, C. A., Martel, M. M., & Axelrad, M. E. (2017). ODD symptom network during preschool. Journal of Abnormal Child Psychology, 45, 743–748. https://doi.org/10.1007/s10802-016-0196-y
38. Stringaris, A., Goodman, R., 2009. Three dimensions of oppositionality in youth. Journal of Child Psychology & Psychiatry, 50, 216–223. https://doi.org/10.1111/j.1469-7610.2008.01989.x
39. Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58, 267-288. https://doi.org/10.1111/j.2517-6161.1996.tb02080.x
40. Yang, Z., Algesheimer, R., & Tessone, C. J. (2016). A comparative analysis of community detection algorithms on artificial networks. Scientific Reports, 6, 30750. https://doi.org/10.1038/srep30750

Tables

Table 1 Weights Matrix Between the ODD Symptoms from the Network Analyses for Parent and Teacher Ratings 

Note. Values above the diagonal are those for teachers, and values below the diagonal are those for parents.

Table 2 Centrality Indices of ODD Symptoms from the Parent and Teacher Ratings Network Analyses

 Note. Higher numbers indicate that the variable is more central to the network, highest two values are underlined within each index.