Parental mental health and reporting of their child’s behaviour: measurement invariance of the French version of the parental Strengths and Difficulties Questionnaire

doi:10.21203/rs.3.rs-3404463/v1

Download PDF

Research Article

Parental mental health and reporting of their child’s behaviour: measurement invariance of the French version of the parental Strengths and Difficulties Questionnaire

https://doi.org/10.21203/rs.3.rs-3404463/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 25 Feb, 2024

Read the published version in European Child & Adolescent Psychiatry →

You are reading this latest preprint version

Symptomatic effects of mental disorders in parents could bias their reporting on their child’s mental health. This study aimed to investigate the measurement invariance of the French version of the parental Strengths and Difficulties Questionnaire (SDQ) across parental mental health in a sample (N=20,765) of parents of children aged 3 to 17 years in France. Confirmatory Factor Analysis (CFA) and Exploratory Structural Equation Modelling (ESEM) were used to evaluate the fit of three known alternative SDQ factor structures (five, three, or second-order factor structures). Invariance was tested across parental mental health (present anxiety and depressive symptoms, psychiatric history) and across socio-demographic characteristics (child's age, child's gender, parent's gender, parent's educational level). CFA models showed a poor fit, while all ESEM models achieved acceptable or good fit, with the five-factor model presenting the best fit. Invariance was observed for all characteristics tested, indicating that the SDQ can be used to study the links between parental mental health and their child's mental health without bias. However, ESEM showed that the Hyperactivity/Inattention and Conduct Problems dimensions were not well differentiated in the French version of the SDQ.

Measurement invariance

Parental mental health

Factor structure

Exploratory Structural Equation Modelling (ESEM)

France

With over 13% of young people suffering from a psychopathological disorder [1], mental health problems among children and adolescents are a public health issue arousing considerable concern. Apart from the immediate distress associated with these health problems, one of the main concerns is that they can persist or lead to other psychological and behavioural disorders later in life [2]. Furthermore, the COVID-19 pandemic and the various associated restrictions aggravated young people's mental health, particularly anxiety and depression [3]. Accurately identifying mental health difficulties in childhood and adolescence is essential to monitor changes in their prevalence over time and to study factors that could influence their occurrence or resolution.

On this topic, the Strengths and Difficulties Questionnaire (SDQ) is one of the most widely used questionnaires [4] for assessing children's mental health. Initially, the SDQ was designed to assess four dimensions of psychological difficulties among children: emotional symptoms (ES), conduct disorders (CD), hyperactivity/inattention (H/I), peer relationship problems (PRP) and a fifth dimension related to pro-social behaviours (PB) [5]. Available in 60 languages, fast to administer and designed for children aged 3 to 17, this screening tool can be filled in by parents, teachers or adolescents themselves from the age of 11.

In several studies that have used the SDQ to evaluate mental health difficulties among children, the informant was the parent. In these studies, parental mental health appears to be one of the strongest factors associated with psychological difficulties among children [6–9]. Therefore, the fact that the parent responds on behalf of the child raises a potential issue, since parental psychopathology is considered as a major risk factor for the child's mental health. As pointed out by Breaux et al. [10], there are many elements that could explain how parental psychopathology has an impact on a child’s functioning: shared genetic vulnerability; exposure to the parent's maladaptive cognitions, affects and behaviours; exposure to a more stressful living environment, etc. However, some studies have shown that parental reports of their children’s behaviours could also be influenced by their own functioning [11–13]. These studies, which have mainly focused on mothers, have shown that parents with psychopathology tend to report more disorders for their children. Indeed, several cognitive impairments (attention, memory recall, interpretation) associated with different psychopathologies [14] could bias a report. Thus, the association between parental and child's mental health could be at least partly due to a measurement bias.

This raises questions about measurement invariance, i.e. to what extent the construct being studied is measured in the same way across different groups [15, 16]. Invariance ensures that the differences observed between groups are due to differences in the level of the phenomenon, rather than a different understanding of the items of the questionnaire. Following this reasoning, a recent study by Olino et al. [17] investigated the invariance of the Child Behaviour Questionnaire (a scale measuring temperament) across mothers' mental health. Participants were asked to complete the questionnaire for their 3-year-old child. Despite the evidence suggesting that parental reporting could be biased, their analysis indicated that the questionnaire was invariant across respondent’s depression, anxiety, substance use and psychiatric history. To our knowledge no such study is available for the SDQ, nor on a broader age span of the children assessed, nor when the informant is the father. Alongside, the invariance of the parent version of the SDQ across socio-demographic characteristics has already been extensively studied, as studies have shown that it is invariant across the child's age [18–21], parental socioeconomic status [20, 22], parental educational level [23], and parental gender [24]. The literature presents contrasted results for the child's gender [18–23, 25] and ethnic origin [20, 23].

The objective of this study was to assess the measurement invariance of the French version of the parental SDQ across parental mental health (depressive and anxiety symptoms, psychiatric history). The secondary objectives were to assess the invariance of the French version of the parent SDQ across different socio-demographic characteristics (child’s age, gender of child and parent, parental educational level), its factor structure as well as its test-retest reliability and internal consistency, as these properties have never been investigated in France.

Study sample

The study sample was drawn from the “Epidémiologie et Conditions de vie sous le COVID-19 » (EpiCoV) cohort, a national random population-based survey which followed 134,391 participants selected from the national administrative and tax register through four rounds of data collection from Spring 2020 to Autumn 2022. Details of the sampling methods are available in [26]. Data collection was achieved through computer-assisted web interviews (CAWI) or computer-assisted telephone interviews (CATI) and covered a wide range of health characteristics, including mental health, healthcare and socio-demographic characteristics.

For this study, the sample was a subgroup of the original cohort composed of participants with at least one child aged 3 to 17 years at the time of the third data-collection round (N = 21,406 in Spring 2021), and who had filled in the SDQ questionnaire for one randomly selected child (along with the questionnaires on their own mental health). Since it was a complete data study (Supplementary Fig. 1), our final sample consisted of 20,765 participants. A subset of this sample (N = 14,339) participated in the fourth round (Autumn 2022), which made it possible to investigate the test-retest reliability of the questionnaire.

Measures

Strengths and Difficulties Questionnaire: The parental version was used to evaluate the mental health of the children and adolescents. This questionnaire consists of 25 items (Supplementary Table 1), to which the respondent answers on a three-point scale: "not true" (0), "somewhat true" (1) or "very true" (2). The factor structure of the SDQ has been the subject of several studies with contrasted results. Three dominant models have been identified [5, 27](Fig. 2): the original 5-factor model; the 3-factor model, which combines items from the ES and PRP dimensions to form an "Internalizing Problems" factor and items from the H/I and CD dimensions to form an "Externalizing Problems" factor; and the 2nd order model, which places the internalizing and externalizing factors on a second level. Factor scores can be calculated by summing the scores of the associated items. Confirmatory Factor Analysis (CFA) studies have shown contrasting factorial validity for the 3-factor models [18, 19, 22, 27–31] and the 2nd order model [18, 19, 22, 27–29]. The 5-factor structure generally presents the best fit [19, 20, 22, 27, 29, 31, 32], but there are also several large studies that have failed to obtain fit indices above acceptable thresholds [28, 30, 33]. Regarding the French version more specifically, it should be noted that the parent version has never undergone a factorial evaluation in France.

Parental Mental Health: The PHQ-9 scale [34] was used to assess various symptoms associated with a characterized depressive episode (loss of pleasure, sadness, sleep or appetite disturbances, low self-esteem, etc.). The total score (items sum) is used to measure the intensity of depressive symptoms: no depressive symptoms (0–4), mild depressive symptoms (5–9) and moderate/severe depressive symptoms (10–27). Similarly, the GAD-7 scale [35] was used to screen for anxiety symptoms, as it assesses various aspects of anxiety (nervousness, worry, irritability, difficulty relaxing, etc.). Participants are categorized as having no anxiety symptoms (0–4), mild anxiety symptoms (5–9) or moderate/severe anxiety symptoms (≥ 10) on the basis of the total score (items sum). Both questionnaires use a Likert frequency scale ranging from 0 ("not at all”) to 3 ("nearly every day”) and assess symptoms over the past two weeks. The factorial structure and internal consistency of the PHQ-9 and GAD-7 were checked in our sample (Supplementary Table 2). Psychiatric history was assessed using the following question: "During your life, has a doctor ever told you that you have a psychiatric, psychological or addiction disorder? Yes or No”.

Some of the socio-demographic characteristics (child's age, child's gender, parental gender, parental educational level) collected in the cohort were used to describe the survey sample and to conduct the invariance analysis. In this analysis, the characteristics were integrated in the form of categorical variables (with the categories presented in Table 1)

Analytical approach

The factor structure of the French version of the parental SDQ has classically been studied using CFA, and later using a more recent latent variable approach, Exploratory Structural Equation Modelling (ESEM), as several authors have argued that this new approach could be better suited to the assessment of psychological constructs [36, 37]. Indeed, ESEM [37–39] integrates the possibility of cross-loading into a CFA framework, in the same way as in exploratory factor analysis (EFA). Similar to EFA, ESEM offers several rotation options. The target rotation was used, as it enables an a-priori specification matrix to be constructed, which maximizes the loading of the targeted items for each factor, and minimizes all others. Marsh et al. [37] state that with this rotation, ESEM can provide a robust and flexible confirmatory approach to investigate the structural validity of a questionnaire. Some studies [21, 24] have already applied ESEM to the SDQ.

Because the SDQ items are ordinal variables, the models were estimated with the Weighted Least Squares with adjusted Mean and Variance estimator (WLSMV), using a polychoric correlation matrix with probit regression. The three different SDQ factor structures described above (Fig. 2) were fitted using CFA and ESEM. To note, for ESEM, the 2nd order model required an alternative approach, namely ESEM-within-CFA (EwC), which involves reintegrating the estimated coefficients from an ESEM as starting values within a CFA. The following fit thresholds were used: fit was considered acceptable if the Comparative Fit Index (CFI) and the Tucker-Lewis Index (TLI) were ≥ 0.90, the Root Mean Square Error of Approximation (RMSEA) and the Standardized Root Mean square Residual (SRMR) were ≤ 0.080, and good if CFI and TLI were ≥ 0.95, RMSEA and SRMR were ≤ 0.060 [40]. Factor loadings were also examined to ensure that each manifest variable was adequately explained by its expected latent factor. Target factor loadings were expected to have values greater than 0.50 with CFA [41] or 0.35 with ESEM [39].

Once the best-performing model was selected, invariance measurement was studied across parental mental health (depressive and anxiety symptoms, psychiatric history) and socio-demographic characteristics (child's age, child's and parental gender, parental educational level). In line with the literature [16], three levels of invariance were tested sequentially: configural invariance (a multi-group factor model is examined without constraints), metric invariance (factor loadings are constrained to be equal across groups), and scalar invariance (additional equality constraints on the item response category thresholds across groups). The invariance is established when no meaningful variation of the fit is observed when constraints are imposed at each level. Lack of invariance was thus indicated when CFI or TLI decreased by more than 0.010 or RMSEA increased by more than 0.015 or SRMR increased by 0.030 (for metric invariance) or 0.010 (for scalar invariance) [42]. A stratified invariance analyses across the three mental health characteristics was also performed separately according to the gender of the parent who filled in the SDQ.

Finally, internal consistency reliability was evaluated using McDonald's ω, computed using the factor loadings and item residuals of the models. To note, there is no specific method to incorporate cross-loadings in the calculation of ω for ESEM models [39]; the current recommendation is to calculate ω by considering only the parameters of the "target" items. For test-retest reliability, longitudinal ESEM and CFA models were run to investigate the correlation of a latent dimension between measurements in round 3 and round 4. The two types of reliability are both considered acceptable if they are greater than 0.70 [43, 44].

Data cleaning and descriptive analyses were performed using R Software (version 4.2.3), while all psychometric analyses were conducted using Mplus (version 8.8).

Sample characteristics

Among the 20,765 parents who filled in the SDQ in spring 2021, 12,069 (58.1%) were women, with a mean age of 43.9 years (standard deviation (SD) = 7.3) and 12,548 (60.4%) reported a higher educational level than high school diploma (Table 1). Regarding their mental health, 5121 (24.7%) had mild depressive symptoms, and 2098 (10.1%) moderate to severe depressive symptoms, 4171 (20.1%) had mild anxiety, and 1593 (7.7%) had moderate to severe anxiety. A psychiatric history was reported by 1720 individuals (8.3%). Regarding the children selected, 10,556 (50.8%) were boys, 3771 (18.2%) were aged 3–5 years, 6296 (30.3%) 6–10 years, 5768 (27.8%) 11–14 years and 4930 (23.7%) 15–17 years. The scores for each of the five SDQ dimensions are described in Table 1.

Table 1

Participant characteristics and descriptive statistics of respondents to the *Strengths and Difficulties* Questionnaire in the EpiCoV survey, 3rd round of data collection (N = 20,765, 2021, France).
	n (N = 20,765)	%
Parental age
Under 30 years	418	2.0%
31–40 years	6935	33.4%
41–50 years	10,151	48.9%
51–60 years	3007	14.5%
Over 60 years	254	1.2%
Parental gender
Man	8696	41.9%
Woman	12,069	58.1%
Parental educational Level
High school or below	8217	39.6%
Above High school	12,548	60.4%
Parental depressive symptoms
No depressive symptoms	13,546	65.2%
Mild depressive symptoms	5121	24.7%
Moderate or severe depressive symptoms	2098	10.1%
Parental anxiety symptoms
No anxiety	15,001	72.2%
Mild anxiety	4171	20.1%
Moderate or severe anxiety	1593	7.7%
Parental psychiatric history
Yes	1720	8.3%
No	19,045	91.7%
Child’s age
3–5 years	3771	18.2%
6–10 years	6296	30.3%
11–14 years	5768	27.8%
15–17 years	4930	23.7%
Child’s gender
Boy	10,556	50.8%
Girl	10,209	49.2%
	Mean	Standard Deviation
SDQ scores
Emotional symptoms (ES)	1.92	2.02
Peer relationship problems (PRP)	1.54	1.62
Conduct disorders (CD)	1.52	1.57
Hyperactivity/Inattention (H/I)	3.00	2.40
Pro-social behaviours (PB)	7.72	1.96

SQD factor structure

Using CFA, the fit of the three factor structures investigated were not found to be satisfactory. The best fit was observed for the 5-factor model whose RMSEA (0.065) and SRMR (0.078) values were acceptable, but whose CFI (0.874) and TLI (0.851) were not. All ESEM models showed satisfactory or good indices, with the 5-factor model presenting the best fit indices values (CFI = 0.973, TLI = 0.956, RMSEA = 0.035, SRMR = 0.029).

Table 2

Fit statistics for the six SDQ factor models tested (N = 20,765, 2021, France).
Model	χ²	df	CFI	TLI	RMSEA	SRMR
CFA five correlated first-order factors¹	22649.1	253	0.875	0.851	0.065 [0.065;0.066]	0.078
CFA three correlated first-order factors¹	30029.6	260	0.833	0.808	0.074 [0.074;0.075]	0.088
CFA three 2nd order factors¹	27393.4	256	0.848	0.822	0.071 [0.071;0.072]	0.084
ESEM five correlated first-order factors	5016.8	185	0.973	0.956	0.035 [0.035;0.036]	0.029
ESEM three correlated first-order factors	15791.0	228	0.913	0.885	0.057 [0.057;0.058]	0.048
EwC three 2nd order factors	5697.5	188	0.970	0.952	0.037 [0.036;0.038]	0.030
Notes: ¹ Integration of residual co-variances between semantically related items (fidget-restless; task-attention) and between the five reverse items (obedient, friend, liked, impulse, task)

Table 3 shows the factor loadings of the ESEM five-factor model. Except for latent factor CD, all the "targeted" items of latent factors had factor loadings above 0.35. Cross-loadings higher than 0.35 were observed for five items, all targeted in the Conduct Disorder and Hyperactivity/Inattention dimensions. Fidgeting ("Constantly fidgeting") and Restlessness ("Restless, overactive, cannot stay still for long") were the H/I items that had higher factor loadings on the CD dimension (λ = 0.549 and λ = 0.645) than those on the H/I dimension (λ = 0.447 and λ = 0.405). This is particularly noteworthy as these two loadings were higher than all targeted items for the CD dimension (λ ranging from 0.236 to 0.462). Similarly, the H/I factor loaded on the Lies ("Lies or cheats") item (λ = 0.395) more than the CD factor (λ = 0.236).

Table 3

Factor loadings of the SDQ ESEM 5F model (N = 20,765, 2021, France).
Items	ES	PRP	CD	H/I	PB
	λ	λ	λ	λ	λ
Complaints	0.412	0.011	0.094	0.010	0.023
Unhappiness	0.610	0.174	0.169	-0.024	-0.074
Worries	0.733	0.063	-0.039	-0.071	0.093
Anxiety	0.775	0.005	-0.046	0.075	-0.055
Fear	0.792	-0.005	-0.019	-0.004	-0.041
Solitude	0.201	0.455	-0.159	-0.038	0.186
Friends	-0.117	0.633	0.061	-0.012	0.221
Popularity	0.011	0.553	0.040	0.073	0.157
Bullying	0.252	0.477	0.079	0.137	-0.262
Adults	0.047	0.604	0.003	-0.032	-0.083
Anger	0.342	-0.129	0.462	0.148	-0.075
Obedience	-0.033	-0.010	0.317	0.285	-0.216
Aggressiveness	0.098	0.194	0.450	0.167	-0.306
Lies	0.080	0.031	0.236	0.395	0.186
Stealings	0.020	0.261	0.344	0.175	0.228
Restlessnesss	0.001	-0.010	0.645	0.405	-0.156
Fidgeting	0.064	0.007	0.549	0.447	-0.362
Distraction	0.113	0.061	-0.145	0.849	-0.090
Reflection	-0.055	-0.075	0.029	0.605	-0.100
Concentration	-0.064	0.076	-0.260	0.934	-0.053
Consideration	0.080	-0.098	-0.081	-0.150	0.547
Sharing	0.008	-0.297	0.040	0.113	0.350
Helpfulness	-0.006	-0.029	-0.107	0.082	0.783
Kindness	0.036	-0.225	-0.166	0.054	0.578
Volunteering	-0.106	0.129	-0.051	-0.054	0.841
Notes: λ = standardized factor loading; bold values indicate target loadings for the factor; shaded values represent cross-loadings <\|0.35\|. ES = Emotional symptoms; PRP = Peer relationship problems; CD = Conduct disorders; H/I = Hyperactivity/Inattention; PB = Pro-social behaviour.

Correlations between the latent factors of the ESEM five-factor model were low to moderate (range: 0.16 to 0.43) as shown in Table 4.

Table 4

Correlation matrix of the SDQ latent factors – ESEM 5F model (N = 20,765, 2021, France).
a.	ES	PRP	CD	H/I	PB
ES	1.00
PRP	0.38	1.00
CD	0.24	0.16	1.00
H/I	0.37	0.24	0.43	1.00
PB	0.05	-0.32	-0.05	-0.31	1.00

Invariance analysis

The results of the invariance analysis using the ESEM five-factor model are presented in Table 5. According to the reference thresholds, the SDQ was invariant across all the characteristics considered. The gender-stratified invariance analysis (Supplementary table 4) showed that the SDQ was invariant across mental health regardless of parental gender. To provide a sensitivity analysis, invariance was also studied across the same characteristics (Supplementary table 5) using the best-performing CFA model (five-factor model). Invariance was also found across all characteristics tested.

Table 5

Invariance analysis– ESEM 5F model (N = 20,765, 2021, France).
Variable	CFI	TLI	RMSEA	SRMR	Δ CFI	Δ TLI	Δ RMSEA	Δ SRMR
Parental gender
Configural	0.973	0.956	0.035	0.029	\(\cdots\)	\(\cdots\)	\(\cdots\)	\(\cdots\)
Metric	0.984	0.980	0.024	0.031	+ 0.011	+ 0.024	-0.011	+ 0.002
Scalar	0.983	0.980	0.024	0.031	-0.001	0.000	0.000	0.000
Parental diploma
Configural	0.973	0.957	0.035	0.029	\(\cdots\)	\(\cdots\)	\(\cdots\)	\(\cdots\)
Metric	0.981	0.977	0.026	0.033	+ 0.008	+ 0.020	-0.009	+ 0.004
Scalar	0.980	0.977	0.026	0.033	-0.001	0.000	0.000	0.000
Parental depressive symptoms
Configural	0.971	0.953	0.035	0.030	\(\cdots\)	\(\cdots\)	\(\cdots\)	\(\cdots\)
Metric	0.983	0.981	0.022	0.033	+ 0.012	+ 0.028	-0.013	+ 0.003
Scalar	0.982	0.981	0.022	0.033	-0.001	0.000	0.000	0.000
Parental anxiety symptoms
Configural	0.971	0.953	0.035	0.030	\(\cdots\)	\(\cdots\)	\(\cdots\)	\(\cdots\)
Metric	0.984	0.982	0.022	0.033	+ 0.013	+ 0.029	-0.013	+ 0.003
Scalar	0.983	0.982	0.022	0.033	-0.001	0.000	0.000	0.000
Parental psychiatric history
Configural	0.974	0.957	0.035	0.029	\(\cdots\)	\(\cdots\)	\(\cdots\)	\(\cdots\)
Metric	0.985	0.982	0.022	0.031	+ 0.011	+ 0.025	-0.013	+ 0.002
Scalar	0.985	0.983	0.022	0.031	0.000	+ 0.001	0.000	0.000
Child’s gender
Configural	0.973	0.956	0.035	0.029	\(\cdots\)	\(\cdots\)	\(\cdots\)	\(\cdots\)
Metric	0.981	0.978	0.025	0.032	+ 0.008	+ 0.022	-0.010	+ 0.003
Scalar	0.980	0.977	0.026	0.032	-0.001	-0.001	+ 0.001	0.000
Child’s age
Configural	0.974	0.957	0.035	0.030	\(\cdots\)	\(\cdots\)	\(\cdots\)	\(\cdots\)
Metric	0.971	0.968	0.030	0.041	-0.003	+ 0.011	-0.005	+ 0.011
Scalar	0.967	0.967	0.031	0.042	-0.004	-0.001	+ 0.001	+ 0.001
Reliability analysis

Internal consistency and test-retest reliability estimators are reported in Table 6. Internal consistency indicators were acceptable, with the exception of the CD dimension in the ESEM model (ω = 0.53). The questionnaire also demonstrated good test-retest reliability with all dimensions, except pro-social behaviour, which had a V3-V4 correlation of over 0.70.

Table 6

Internal consistency reliability (McDonald's ω) and test-retest reliability (round 3-round 4 correlation) of the two best models (CFA and ESEM) of the SDQ, (N = 20,765, 2021 ; N = 14,339, 2022 ; France).
	ES	PRP	CD	H/I	PB
MacDonald’s ω (N = 20.765)
CFA 5-Factor model	0.823	0.725	0.794	0.853	0.802
ESEM 5-Factor model	0.819	0.726	0.535	0.874	0.789
Correlation round 3-round 4 (N = 14.339)
CFA 5-Factor model	0.723	0.747	0.768	0.767	0.681
ESEM 5-Factor model	0.721	0.743	0.764	0.815	0.668

In a French random population-based cohort, our study did not evidence any measurement invariance issues for the SDQ across parental mental health (depressive and anxiety symptoms, psychiatric history), nor across other socio-demographic characteristics (child's age, child and parental gender, parental educational level). When the factor structure of the French version of the SDQ was classically investigated using CFA, it showed a less than acceptable fit, whereas this was not the case using ESEM, which showed that the five-factor model had the best fit. We also estimated good reliability indices for the SDQ.

Because cognitive and affective alterations resulting from mental disorders could bias the parents' reporting about their child [11–13], this study aimed to detect measurement invariance issues on the SDQ across parental mental health. It did not in fact detect any issues. In practice, this indicates that if an association is observed between parental mental health and their child’s behaviour assessed using the French version of the SDQ in a study [6–8], it is unlikely to be the result of an invariance bias. Our results are thus in line with the study by Olino et al. [17] who, using the Child Behaviour Questionnaire on 3 year-old children, showed invariance across maternal mental health, and extends it to fathers’ reports and to the assessment of the behaviour of children aged 3 to 17 years. Additionally, the invariance analysis carried out on other socio-demographic characteristics complements the literature: as in several other studies on the SDQ, we found no invariance issues linked to the parental gender [24], the child’s gender [18–22], the child's age [18–21] or parental educational level [23].

Regarding the SDQ factor structure, our study is one of the first conducted on the parent version of the SDQ in French language and the largest ever conducted in France. None of the three CFA models reached an acceptable fit in our sample. This is in line with a number of CFA studies on the SDQ in various languages and populations [18, 28–30, 33]. Conversely, the ESEM approach, recommended by several authors, showed acceptable or satisfactory fit for the three-factor models, with the ESEM 5-factor model being the best. These fit indicators are consistent with the results of the few studies that have applied ESEM to the SDQ [21, 24]. Examining the factor loadings of the best ESEM model enabled us to observe structural issues in the questionnaire in the Hyperactivity/Inattention and Conduct Problems dimensions. These factors were indeed not clearly distinct as evidenced by the items "Fidgeting" and “Restlessness” for which the cross-loadings were higher than their targeted loadings. This could be explained by a relative dissociation within the Hyperactivity/Inattention factor between items related to hyperactivity (fidget and restless) and items related to concentration difficulties (attention, impulse, and task). Furthermore, this would mean that the items related to hyperactivity are more strongly associated with Conduct Problems than with Inattention. In addition, the invariance analysis enabled us to conclude that these results were observable regardless of parent or child characteristics. Putting these results into perspective with other studies using ESEM is particularly relevant, as a similar pattern has been observed in the parental SDQ in English [24] and in self-administered questionnaire in Spanish [21] samples, which suggests that this might not be specific to the French version of the SDQ. These findings are consistent with the recent literature, which now considers hyperactivity and attention deficit as two related but separate constructs [45]. It also reinforces the idea that, from the parents' point of view, hyperactivity symptoms are a behavioural problem. Clinicians do not necessarily share this view, but it does help their clinical practice to take the parents’ perception on board.

This study has provided new information on the association between parent and child mental health, since it has shown that there is no invariance bias. On a national level, this is the largest (N = 20,765) psychometric study ever carried out on the SDQ in France, and we have provided some information regarding the factor structure in this context, notably through a comparison of the CFA and ESEM approaches. Typically, CFA is used because it helps to determine whether it is psychometrically viable to consider that an item refers to a single latent process. Satisfactory structural validity enables a score to be considered as a good representation of the latent factor to be measured, thus justifying the most widely used measurement model in clinical practice. However, in our study, to reach an acceptable fit, the absence of cross-loading constraints had to be relaxed, which meant that some items substantially contributed to the measurement of latent factors other than the one that was targeted. In this respect, the ESEM approach showed that scores related to the CD and H/I factors could lack validity. This therefore highlights difficulties in accurately identifying these two factors and it leads us to recommend the use of latent modelling, namely ESEM, rather than scores when mobilizing the SDQ in a research context.

However, several limitations to this study should be acknowledged. The evaluation of parental mental health did not include severe psychiatric problems such as schizophrenia or intellectual disability, which could have affected the invariance of the SDQ differently. However, given the low prevalence of this type of disorder in a general population sample, this is not likely to have biased our results. The choice of performing the analysis on complete data is also debatable, as it is difficult to assume that missing data is Missing Completely At Random. However, we did observe very low proportions of missing values for the SDQ (between 0.93% and 1.10% depending on items), and measures of depressive symptoms (0.12% − 0.24%) and anxiety symptoms (0.07% − 0.11%). In addition, the use of ESEM comes with relative uncertainty: it is a recent methodology and the performance and validity of which are not necessarily well known, depending on the study configuration. It is also essential to remember that the study focused on parental perceptions of their child's difficulties, so that the measure was not direct. Therefore, it is not certain that the difficulties reported by the parents were really those experienced by their children. Finally, other characteristics of the questionnaire should be investigated to provide a more exhaustive psychometric analysis of the SDQ: convergent validity; sensitivity and specificity; inter-parent consensus. This was not possible with our data, however readers can refer to the systematic reviews [4, 46] which provide some insights into these aspects.

Thanks to the EpiCoV cohort, we were able to show that the SDQ does not present problems of invariance across the parental mental health characteristics, nor to socio-demographic characteristics. Furthermore, the SDQ factor structure was studied using CFA and ESEM, which evidenced difficulties in distinguishing the Conduct Disorders and Hyperactivity/Inattention dimensions. Consequently, this leads us to recommend using ESEM rather than the scores when mobilizing the SDQ in research, so that the complexity of its structure can be incorporated into the study.

Funding and conflicts of interest: This study is part of the EpiCoV cohort, which was funded by the Institut National de la SantÉ et de la Recherche Médicale (INSERM), the Direction de la Recherche, des Études, de l'Évaluation et des Statistiques (DREES), the French ministry of research and the Île de France region . The authors have no competing interests to declare that are relevant to the content of this article.

Consent: Informed consent was obtained from all individual participants included in the study.

Author contributions: AS conducted the data management and the analysis, supervised by AR, AV and J-BH. AS wrote the main manuscript text. Redactional and methodological support was provided by AV, AD, GM-M, CV, CG, CD-P, J-BH and AR. BF, AR and JW provided overall supervision. All authors reviewed the manuscript.

Polanczyk GV, Salum GA, Sugaya LS, et al (2015) Annual Research Review: A meta-analysis of the worldwide prevalence of mental disorders in children and adolescents. J Child Psychol Psychiatry 56:345–365. https://doi.org/10.1111/jcpp.12381
Kessler RC, Amminger GP, Aguilar-Gaxiola S, et al (2007) Age of onset of mental disorders: a review of recent literature. Curr Opin Psychiatry 20:359. https://doi.org/10.1097/YCO.0b013e32816ebc8c
Nearchou F, Flinn C, Niland R, et al (2020) Exploring the Impact of COVID-19 on Mental Health Outcomes in Children and Adolescents: A Systematic Review. Int J Environ Res Public Health 17:8479. https://doi.org/10.3390/ijerph17228479
Bergström M, Baviskar S (2021) A Systematic Review of Some Reliability and Validity Issues regarding the Strengths and Difficulties Questionnaire Focusing on Its Use in Out-of-Home Care. J Evid-Based Soc Work 18:1–31. https://doi.org/10.1080/26408066.2020.1788477
Goodman R (2001) Psychometric properties of the strengths and difficulties questionnaire. J Am Acad Child Adolesc Psychiatry 40:1337–1345. https://doi.org/10.1097/00004583-200111000-00015
Kallas K-A, Marr K, Moirangthem S, et al (2023) Maternal Mental Health Care Matters: The Impact of Prenatal Depressive and Anxious Symptoms on Child Emotional and Behavioural Trajectories in the French EDEN Cohort. J Clin Med 12:1120. https://doi.org/10.3390/jcm12031120
Monnier M, Moulin F, Bailhache M, et al (2023) Parents’ depression and anxiety associated with hyperactivity-inattention and emotional symptoms in children during school closure due to COVID-19 in France. Sci Rep 13:4863. https://doi.org/10.1038/s41598-023-31985-y
van Der Waerden J, Galéra C, Larroque B, et al (2015) Maternal depression trajectories and children’s behavior at age five years. J Pediatr 22. https://doi.org/10.1016/j.jpeds.2015.03.002
O’connor TG, Heron J, Glover V (2002) Antenatal Anxiety Predicts Child Behavioral/Emotional Problems Independently of Postnatal Depression. J Am Acad Child Adolesc Psychiatry 41:1470–1477. https://doi.org/10.1097/00004583-200212000-00019
Breaux RP, Harvey EA, Lugo-Candelas CI (2014) The Role of Parent Psychopathology in the Development of Preschool Children with Behavior Problems. J Clin Child Adolesc Psychol Off J Soc Clin Child Adolesc Psychol Am Psychol Assoc Div 53 43:777–790. https://doi.org/10.1080/15374416.2013.836451
Seifer R, Sameroff A, Dickstein S, et al (2004) Your own children are special: clues to the sources of reporting bias in temperament assessments. Infant Behav Dev 27:323–341. https://doi.org/10.1016/j.infbeh.2003.12.005
Briggs-Gowan MJ, Carter AS, Schwab-Stone M (1996) Discrepancies among mother, child, and teacher reports: Examining the contributions of maternal depression and anxiety. J Abnorm Child Psychol 24:749–765. https://doi.org/10.1007/BF01664738
Chilcoat HD, Breslau N (1997) Does Psychiatric History Bias Mothers’ Reports? An Application of a New Analytic Approach. J Am Acad Child Adolesc Psychiatry 36:971–979. https://doi.org/10.1097/00004583-199707000-00020
Mathews A, MacLeod C (2005) Cognitive Vulnerability to Emotional Disorders. Annu Rev Clin Psychol 1:167–195. https://doi.org/10.1146/annurev.clinpsy.1.102803.143916
Meredith W (1993) Measurement invariance, factor analysis and factorial invariance. Psychometrika 58:525–543. https://doi.org/10.1007/BF02294825
Putnick DL, Bornstein MH (2016) Measurement Invariance Conventions and Reporting: The State of the Art and Future Directions for Psychological Research. Dev Rev DR 41:71–90. https://doi.org/10.1016/j.dr.2016.06.004
Olino TM, Guerra-Guzman K, Hayden EP, Klein DN (2020) Evaluating maternal psychopathology biases in reports of child temperament: An investigation of measurement invariance. Psychol Assess 32:1037–1046. https://doi.org/10.1037/pas0000945
Ortuño-Sierra J, Aritio-Solana R, Fonseca-Pedrero E (2018) Mental health difficulties in children and adolescents: The study of the SDQ in the Spanish National Health Survey 2011–2012. Psychiatry Res 259:236–242. https://doi.org/10.1016/j.psychres.2017.10.025
Hoffmann MD, Lang JJ, Guerrero MD, et al (2020) Evaluating the psychometric properties of the parent-rated Strengths and Difficulties Questionnaire in a nationally representative sample of Canadian children and adolescents aged 6 to 17 years. Stat Can. https://doi.org/10.25318/82-003-X202000800002-ENG
He J-P, Burstein M, Schmitz A, Merikangas KR (2013) The Strengths and Difficulties Questionnaire (SDQ): the Factor Structure and Scale Validation in U.S. Adolescents. J Abnorm Child Psychol 41:583–595. https://doi.org/10.1007/s10802-012-9696-6
Garrido LE, Barrada JR, Aguasvivas JA, et al (2020) Is Small Still Beautiful for the Strengths and Difficulties Questionnaire? Novel Findings Using Exploratory Structural Equation Modeling. Assessment 27:1349–1367. https://doi.org/10.1177/1073191118780461
Bridger Staatz C, Kelly Y, Lacey RE, Hardy R (2023) Investigating the factorial structure and measurement invariance of the parent-reported strengths and difficulties questionnaire at 11 years of age from the UK Millennium Cohort Study. Eur Child Adolesc Psychiatry. https://doi.org/10.1007/s00787-023-02156-1
Hill CR, Hughes JN (2007) An Examination of the Convergent and Discriminant Validity of the Strengths and Difficulties Questionnaire. Sch Psychol Q Off J Div Sch Psychol Am Psychol Assoc 22:380–406. https://doi.org/10.1037/1045-3830.22.3.380
Chiorri C, Hall J, Casely-Hayford J, Malmberg L-E (2016) Evaluating Measurement Invariance Between Parents Using the Strengths and Difficulties Questionnaire (SDQ). https://doi.org/10.1177/1073191114568301
Shibata Y, Okada K, Fukumoto R, Nomura K (2015) Psychometric properties of the parent and teacher forms of the Japanese version of the Strengths and Difficulties Questionnaire. Brain Dev 37:501–507. https://doi.org/10.1016/j.braindev.2014.08.001
Warszawski J, Bajos N, Barlet M, et al (2021) A national mixed-mode seroprevalence random population-based cohort on SARS-CoV-2 epidemic in France: the socio-epidemiological EpiCov study. 2021.02.24.21252316
Goodman A, Lamping DL, Ploubidis GB (2010) When to use broader internalising and externalising subscales instead of the hypothesised five subscales on the Strengths and Difficulties Questionnaire (SDQ): data from British parents, teachers and children. J Abnorm Child Psychol 38:1179–1191. https://doi.org/10.1007/s10802-010-9434-x
McCrory C, Layte R (2012) Testing competing models of the Strengths and Difficulties Questionnaire’s (SDQ’s) factor structure for the parent-informant instrument. Personal Individ Differ 52:882–887. https://doi.org/10.1016/j.paid.2012.02.011
Niclasen J, Skovgaard AM, Andersen A-MN, et al (2013) A Confirmatory Approach to Examining the Factor Structure of the Strengths and Difficulties Questionnaire (SDQ): A Large Scale Cohort Study. J Abnorm Child Psychol 41:355–365. https://doi.org/10.1007/s10802-012-9683-y
Klein AM, Otto Y, Fuchs S, et al (2013) Psychometric properties of the parent-rated SDQ in preschoolers. Eur J Psychol Assess 29:96–104. https://doi.org/10.1027/1015-5759/a000129
Sosu EM, Schmidt P (2017) Tracking Emotional and Behavioral Changes in Childhood: Does the Strength and Difficulties Questionnaire Measure the Same Constructs Across Time? J Psychoeduc Assess 35:643–656. https://doi.org/10.1177/0734282916655503
Janitza S, Klipker K, Hölling H (2020) Age-specific norms and validation of the German SDQ parent version based on a nationally representative sample (KiGGS). Eur Child Adolesc Psychiatry 29:123–136. https://doi.org/10.1007/s00787-019-01337-1
Moriwaki A, Kamio Y (2014) Normative data and psychometric properties of the strengths and difficulties questionnaire among Japanese school-aged children. Child Adolesc Psychiatry Ment Health 8:1. https://doi.org/10.1186/1753-2000-8-1
Kroenke K, Spitzer RL, Williams JBW (2001) The PHQ-9: Validity of a Brief Depression Severity Measure. J Gen Intern Med 16:606–613. https://doi.org/10.1046/j.1525-1497.2001.016009606.x
Spitzer RL, Kroenke K, Williams JBW, Löwe B (2006) A Brief Measure for Assessing Generalized Anxiety Disorder: The GAD-7. Arch Intern Med 166:1092–1097. https://doi.org/10.1001/archinte.166.10.1092
Morin AJS, Herbert MW, Nagengast B (2013) Exploratory Structural Equation Modeling. In: Hancock GR, Mueller RO (eds) Structural Equation Modeling: A Second Course, 2nd Revised ed. edition. Information Age Publishing, Charlotte, N.C
Marsh HW, Morin AJS, Parker PD, Kaur G (2014) Exploratory Structural Equation Modeling: An Integration of the Best Features of Exploratory and Confirmatory Factor Analysis. Annu Rev Clin Psychol 10:85–110. https://doi.org/10.1146/annurev-clinpsy-032813-153700
Asparouhov T, Muthén B (2009) Exploratory Structural Equation Modeling. Struct Equ Model Multidiscip J 16:397–438. https://doi.org/10.1080/10705510903008204
van Zyl LE, ten Klooster PM (2022) Exploratory Structural Equation Modeling: Practical Guidelines and Tutorial With a Convenient Online Tool for Mplus. Front Psychiatry 12:795672. https://doi.org/10.3389/fpsyt.2021.795672
Hu L, Bentler PM (1999) Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Model Multidiscip J 6:1–55. https://doi.org/10.1080/10705519909540118
Hair JF, Black W, Anderson R, Babin B (2019) Multivariate data analysis, Eighth edition. Cengage, Andover, Hampshire
Chen FF (2007) Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance. Struct Equ Model Multidiscip J 14:464–504. https://doi.org/10.1080/10705510701301834
Evers A, Hagemeister C, Høstmælingen A, et al (2013) EFPA review model for the description and evaluation of psychological and educational tests: Test review form and notes for reviewers. Version 4.2.6. European Federation of Psychologists’ Associations
Nunnally JC (1978) Psychometric theory, 2d ed. McGraw-Hill, New York
Willcutt EG, Nigg JT, Pennington BF, et al (2012) Validity of DSM-IV attention deficit/hyperactivity disorder symptom dimensions and subtypes. J Abnorm Psychol 121:991–1010. https://doi.org/10.1037/a0027347
Stone LL, Otten R, Engels RCME, et al (2010) Psychometric Properties of the Parent and Teacher Versions of the Strengths and Difficulties Questionnaire for 4- to 12-Year-Olds: A Review. Clin Child Fam Psychol Rev 13:254–274. https://doi.org/10.1007/s10567-010-0071-2

No competing interests reported.

Supplementarymaterial.docx

Download PDF

Journal Publication

published 25 Feb, 2024

Read the published version in European Child & Adolescent Psychiatry →

Editorial decision: Revision requested
01 Nov, 2023
Reviews received at journal
18 Oct, 2023
Reviewers agreed at journal
10 Oct, 2023
Reviewers invited by journal
10 Oct, 2023
Editor assigned by journal
02 Oct, 2023
Submission checks completed at journal
02 Oct, 2023
First submitted to journal
02 Oct, 2023

You are reading this latest preprint version

Parental mental health and reporting of their child’s behaviour: measurement invariance of the French version of the parental Strengths and Difficulties Questionnaire

Status:

Journal Publication

Version 1

Abstract

Figures

1. Background

2. Methods

3. Results

4. Discussion

Declarations

References

Additional Declarations

Supplementary Files

Status:

Journal Publication

Version 1