Validation of the parent version of the Strengths and Difficulties Questionnaire (SDQ) to screen child and adolescent mental health problems in Mongolia


 BackgroundChild and adolescent mental health problems are urgent health issues in low- and middle-income countries (LMICs). To promote mental health services for child and adolescent mental problems, simple validated screening tools are helpful. In Mongolia, the Strengths and Difficulties Questionnaire (SDQ), an internationally used child and adolescent mental health screening tool for children aged 4 to 17, was translated but not yet validated. To use the questionnaire for surveillance, research, and clinical purpose, validation is necessary.MethodsA community sample and a clinical sample were used. The discriminating ability of the parental version of the SDQ was examined using Receiver Operating Characteristics (ROC) analysis on the SDQ total difficulties score. The area under the ROC curve was used as a measure. Cut-off score was determined by normative banding that categorizes children with the highest 10% score range as abnormal and the second highest 10% score as borderline following the original method; this cut-off score was compared with the cut-off score candidates with good balance between sensitivity and specificity using ROC analysis.ResultsWe included 2301 children in the community sample, and 429 children in the clinical sample. Mean age was 9.7 years (SD 0.4) among the community sample and 10.4 years (SD 3.8) among the clinical sample. A total of 88.8% of the community sample and 98.8% of the clinical sample answered the SDQ. Using ROC analysis, the area under the curve was 0.82 (95% confident interval 0.80-0.85), which meant moderate discriminating ability. The mean total difficulties score was 12.9 (SD 4.8) among the community sample and 20.4 (SD 6.2) among the clinical sample. Using normative banding, the cut-off score between normal and borderline was 16/17 and between borderline and abnormal was 19/20. Sensitivity was 71.9% and 53.8% and specificity was 78.5% and 90.5% for cut-off scores of 16/17 and 19/20, respectively. The cut-off score candidates by ROC analysis was 16/17 and 17/18.ConclusionsThe parental version of the SDQ had moderate discriminating ability in Mongolia. For the screening of mental health problems among community children, cut-off score of 16/17 is recommended.

The parental version of the SDQ had moderate discriminating ability in Mongolia. For the screening of mental health problems among community children, cut-off score of 16/17 is recommended.

Importance of child and adolescent mental health in LMICs
Children and adolescents comprise a third of the world's population, and 10-20% of them are considered to suffer from mental health problems. (1,2) According to the Global Burden of Disease study, mental disorders and substance use disorders account for 15-30% of years lived with disability among young people.(3) Despite the growing burden of disease and its long-lasting consequences beyond childhood and adolescence, there are significant gaps between need and resource availability, particularly in LMIC where 90% of the world`s children and adolescents live. (1,4) Studies of mental health services suggest that a small proportion of those with mental health needs are receiving care. (5)(6)(7) Valid and simple screening instruments can contribute to the promotion of child and adolescent mental health services in LMICs in multiple ways. For example, it enables an identification of people with service needs at health care facilities as well as an epidemiological surveillance at the population level.
The Strengths and Difficulties Questionnaire (SDQ) is a mental health screening instrument for children and adolescents aged 4-17 years old used in over 100 countries including LIMCs. (8)(9)(10)(11)(12)(13)(14)(15)(16) The SDQ is extensively used in both research and clinical settings as it is quick and easy to complete and score has good psychometric properties. (17) Moreover, it has very broad acceptance by non-health professionals, children and their parents. However, normative scoring and psychometric proprieties of the SDQ have been extensively assessed predominantly in samples from high-income countries.
Several cross-cultural issues have been raised and the original cut-off values derived from UK data may not be appropriate in different settings. (18) Mongolia is a lower middle-income country. The epidemiological transition from communicable diseases to non-communicable diseases is occurring and mental health needs are assumed to be high. (19) The Mongolian version of the SDQ was developed through translation and back-translation and made available on the official website. Researchers are using the SDQ in Mongolia. (20) However, validation of the Mongolian version of the SDQ has not been conducted yet, and the international cutoff scores, which were originally the cut-off scores derived from the UK children, have been used. Therefore, the present study aimed to analyze the discriminative validity of the parent version of the Mongolian SDQ and to define the appropriate cut-off scores for the categories in Mongolia by banding normative data. Appropriate cut-off scores and proven validity of the SDQ are necessary for the effective use of the SDQ in Mongolia for various purposes such as epidemiological surveillance and clinical needs assessment.

Study settings
This study compared two samples to validate the SDQ. These are: (1) a community sample; and (2) a clinical sample. The community sample consisted of children recruited from the community; one district in the capital city. The clinical sample consisted of children who visited a psychiatry outpatient service in Mongolia. There was no gold standard questionnaire to compare with the SDQ in Mongolia.
In addition, there were no known groups which consist only of children with mental disorders and only of children without mental disorder. Thus, in the present study, we aimed to validate the SDQ through examining the discriminating ability of the SDQ between a community sample and clinical sample although there was a possibility that the community sample might include children with mental health problems and the clinical sample might include children without mental health problem. This method for SDQ validity has been applied in previous studies. (21,22) Community sample The community sample consisted of participants in a study that evaluated the effectiveness of physical activity on academic achievement and cognitive function among children in elementary schools. The details of this study are described elsewhere. (23) Participants were children in their 4 th year at 10 public primary schools in Sukhbaatar district, which is one of nine districts in the capital city, Ulaanbaatar. Inclusion criteria were: (1) attendance at a public school in the Sukhbaatar district; (2) written consent from parents or guardians; and (3) age-appropriate literacy in Mongolian. Exclusion criteria were: (1) comorbidities or contraindications prohibiting participation in an exercise program; and (2) enrollment in a special needs program. The population of the district is roughly 10% and 5% of the population of Ulaanbaatar and Mongolia, respectively. The district stretches from urban city center to non-urban area where the infrastructure is not enough developed so that the socioeconomical background of the participants are diverse. There is no apparent difference in demographic characteristics in this district compared with other districts in Ulaanbaatar. Thus, population representativeness of the community sample was considered high.

Clinical sample
The clinical sample was recruited at the child and adolescent mental health outpatient service at the Mongolian. There were no exclusion criteria. In this analysis, children aged between 4 to 17 years old were included in the analysis due to the target age range of the SDQ.

Measures
Socio-demographic characteristics and the Strengths and Difficulties Questionnaire were obtained from a parent or guardian in both community and clinical samples. Clinical diagnosis was obtained in the clinical sample.

The Strengths and Difficulties Questionnaire
The Strengths and Difficulties Questionnaire (SDQ) is a 25-item questionnaire for child and adolescent mental health problems. It is for 4-17-year-old children and adolescents. It is used for clinical assessment, epidemiological study and screening of psychiatric disorders. The 25-items are answered using a 3-point scale, "certainly true", "somewhat true" and "not true" and scored from 0-2 points.
The items yield 5 subscale scores that range from 0 to 10 including: (1) emotional symptoms; (2) conduct symptoms; (3) hyperactivity/inattention; (4) peer relationship problems; and (5) prosocial behavior. Summing emotional, conduct, hyperactivity/inattention and peer relationship subscale scores yields a total difficulties score that ranges from 0 to 40. The SDQ uses cut-off scores that are defined using normative data banding in three categories: normal (80 th percentile and less), borderline (80 th -90 th percentile) or abnormal (90 th percentile and more). The Mongolian version was obtained from the official website.(13) For the screening of psychiatric disorders, the SDQ has been shown to have moderate sensitivity and high specificity in other languages. (14,22,(24)(25)(26) In the clinical sample, clinical diagnosis was assessed by recording consultation or reviewing medical

records. The 10 th revision of International Statistical Classification of Diseases and Related Health
Problems is used conventionally in Mongolia and was used in the present study. This analysis had an assumption that the prevalence of psychiatric disorders among the clinical sample was substantially higher than that of the community sample. This analysis did not have an assumption that either none of the participants in the community sample had a psychiatric disorder or all the participants in the clinical sample had a psychiatric disorder.

Statistical analysis Receiver Operating Characteristic analysis
To assess the discriminating ability of subscale scores, AUC of each subscale score was calculated.
Although the clinical sample consisted of patients at a child and adolescent psychiatric outpatient service, some participants in the clinical sample might not have a psychiatric disorder. If many in the clinical sample did not have a psychiatric disorder, it might be difficult to examine the discriminating ability of the SDQ. To solve this problem, a sensitivity analysis was conducted using the community sample and a subsample of the clinical sample participants which only included those with definite psychiatric diagnoses.

Cut-off score by normative banding
Normative data for the SDQ total difficulties score of the entire community sample were described.
As the etiology of child and adolescent mental health problems has gender differences, normative data by gender were also described. (29) Normative data of the 5 subscale scores were described (Supplementary material).
To determine the original UK version SDQ cut-off scores, banding of the normative data of the SDQ total difficulties scores was done to divide percentiles into abnormal and borderline categories. (22) In the present study, the same banding method was applied to the normative data to determine the cutoff scores of the Mongolian version.

Comparison with the cut-off score candidates by ROC analysis
The cut-off score by normative banding was compared with the cut-off score candidates using ROC analysis which has a balance between sensitivity and specificity. For ROC analysis, the best cut-off score was analyzed by two methods: (1) determining the point closest to the top-left point of the plot which means perfect discriminating ability (100% sensitivity and 100% specificity); and (2) Youden's J statistics which uses the point that maximizes the distance to the line of no discriminating ability (connecting the point of 100% sensitivity and 0% specificity and the point of 0% sensitivity and 100% specificity). (27,30,31) The candidates from the ROC determined cut-off score were compared with the cut-off score by normative banding.

Sensitivity and specificity
Though we did not have an assumption that either the community sample did not include any participants with psychiatric disorders or that the clinical sample did not include any participants without psychiatric disorders, sensitivity, specificity, positive likelihood ration, and negative likelihood ratio to discriminate participant's group (clinical sample or community sample) were calculated for each cut-off score. Sensitivity meant the proportion of above threshold participants in the clinical sample. Specificity meant the proportion of below threshold children in the community sample The proportion of children above each cut-off score was calculated.

Age and gender
The mean age of the community sample was 9.7 years (SD 0.4). Of all participants, 51.3% were male.
Mean age of the clinical sample was 10.4 years (SD 3.8) and 60.1% were male. Mean age was higher in the clinical sample (t=-3.67, p < 0.001) and the proportion of males was also higher (χ 2 = 1066.7, p < 0.001).  Among males, the mean total difficulties score was 13.3 (SD 4.9) in the community sample and 20.4 (SD 6.1) in the clinical sample ( Supplementary Fig. 1). AUC was 0.81 (95% CI 0.78-0.84). Among females, the mean total difficulties score was 12.4 (SD 4.6) in community sample and 20.4 (SD 6.5) in the clinical sample ( Supplementary Fig. 1). AUC was 0.84 (95% CI 0.80-0.87). This meant that the higher mean total difficulties score in the clinical sample was not due to the higher proportion of males in the clinical sample. Cut-off score by normative banding Among the entire community sample, the cut-off score between normal and borderline and between borderline and abnormal was 16/17 and 19/20 respectively. Cut-off scores by normative banding were compared to that of the UK.(13) Cut-off scores of subscales are presented in Table 2. The cut-off score for the total difficulties score was 3 points higher than that of the UK. The cut-off scores of emotion, conduct and hyperactivity/ inattention and peer relationship subscales were 0-2 points higher than those of the UK. The cut-off score for the prosocial subscale was 2 points lower than that of UK.
Normative data for the SDQ total difficulties score were demonstrated in Supplementary Table 1.
Normative data of the five subscale scores were described in Supplementary Table 2.  Comparison with cut-off score candidates by ROC analysis For the first method, the best cut-off score was determined by the point closest to the top-left point and was 16/17. Sensitivity was 0.72 and specificity was 0.78. For the second method, the Youden method, the cut-off score was 17/18. Sensitivity was 0.67 and specificity was 0.84. According to the comparison between cut-off scores by normative banding and these cut-off score candidates using ROC analysis, the cut-off score of 16/17 was considered to have better balance between sensitivity and specificity than the cut-off score of 19/20. The cut-off score of 19/20 weighs more on specificity.
Thus, the cut-off score of 16/17 is considered to be a good cut-off score for the screening of mental health problem among community children in Mongolia.

Sensitivity and specificity
For each cut-off score, sensitivity, specificity, positive likelihood ratio, negative likelihood ratio and proportion of high risk among the community sample were calculated and displayed in Table 3.

Summary of results
The SDQ score of 2301 community representative children and 429 mental health service user representative children were compared. The AUC value of the total difficulties score was 0.82, which means moderate discriminating ability. As for cut-off scores, normative banding suggested 16/17 for a cut-off between normal and borderline and 19/20 for a cut-off between borderline and abnormal. Both cut-off scores were three points higher than the international cut-off scores. The cut-off score of 16/17 had good balance between sensitivity and specificity by ROC analysis. We recommend a cut-off score of 16/17 for the screening of mental health problems among community children. This analysis demonstrated that the use of international cut-off scores in Mongolia leads to an over estimation of high risk children.

Comparison with previous studies
Previous validation studies of the parental version of the SDQ has demonstrated AUC ranges between 0.66 to 0.87, which moderated discriminating ability. (14,22,(24)(25)(26) AUC of the parental version of the Mongolian SDQ was 0.82 and consistent with previous studies. This suggested that the parental version of the SDQ could be used in Mongolia.

Cut-off score
Internationally, normative banding has been used to determine the cut-off score of the SDQ. However, ROC analysis suggested that cut-off scores between borderline and abnormal disproportionately weighed on specificity rather than sensitivity and false negatives might be a problem. Although this study suggests a cut-off score of 16/17 for normal/borderline and 19/20 for borderline/abnormal following the methods of previous studies, other cut-off scores can also be considered according to the purpose and nature of the target population. For example, if a human resource to perform an assessment for screened children is depleted, minimizing false negative is an important strategy. In that case, higher cut-off score must be considered.

Difference from the results of previous survey in Mongolia
One previous study has used the SDQ parental version among Mongolian adolescents. (20) In the study, children aged 11 to 18 from both Ulaanbaatar and outside Ulaanbaatar were included. The mean total difficulties score was 16.59 (SD 4.36) by parental report. This mean score was higher than the mean total difficulties score among the current community sample. The difference in age range and residential area of the study sample might explain the difference.

Limitations
The community sample consisted of children attending the same year at primary school and did not include younger children or adolescents. However, the mean age of community and clinical samples was similar and the mean age of the community sample was similar to the median age of the target age range for the SDQ. In addition, the community sample consisted of children at around 10 years old, which is the age between childhood and adolescent. Thus, we assumed that problems which mainly occur during childhood and during adolescent could be seen in the community sample.
In our study, the community sample might have included children with mental health problems and the clinical sample might have included children without mental health problems. Regarding the community sample, if the community sample had exclusively consisted of children without mental disorders, the discriminating ability would have been higher and we did not overestimate the discriminating ability. Similarly, for the clinical sample, it did not exclusively consist of children with mental disorders. However, the sensitivity analysis using only children with definite psychiatric disorder yielded the same level of discriminating ability. Thus, the present study did not overestimate the discriminating ability due to the sampling methods.
In this study, we used children living in Ulaanbaatar as the community sample. However, the lifestyle of children is very diverse in urban and rural areas. For rural areas, it might be necessary to obtain normative data and set appropriate cut-off scores.

Conclusions
The parental version of the SDQ demonstrated moderate discriminating ability in Mongolia. The cutoff score between normal and borderline was 16/17 and between borderline and abnormal was 19/20.
For the screening of mental health problems among community children, the cut-off score 16/17 is recommended. The suggested cut-off score was considerably different from the cut-off score used internationally. If the internationally used cut-off score is used in Mongolia, specificity would be very low and false positives would be more likely.

Consent for publication
Not applicable

Availability of data and material
The datasets generated and/or analyzed during the current study are not publicly available but are available from the corresponding author on reasonable request.