Development and Initial Validation of The Russian Version of The RAADS-14: A Self-Report Questionnaire to Assess Autistic Traits

The aim of our research was to construct and validate a Russian version of the RAADS-14 (the RAADS-14 Rus), a brief self-report questionnaire designed for assessment of autistic traits, to provide a reference point in developing autism self-assessment tools in Russian-speaking population. The data were collected for 1724 participants, including a general sample of non-psychiatric young adults (n=849) and adults with ASD (n=49). Conrmatory factor analysis revealed that the RAADS-14 Rus has the same factor structure as the original Swedish version yielding acceptable t indices. The discriminating properties, however, were substantially worse than in the original study (AUC = .92 vs .99); it stems from higher mean RAADS-14 scores for the Russian control sample vs the original controls (mean = 13.9 vs 3.9) conrming prior anecdotal evidences of greater mean scores yielded by international ASD assessment tools on Russian-speaking populations. The correlations between the RAADS-14 Rus domain scores and the Big Five factors were similar to previously obtained ndings. The results of the study indicate that the RAADS-14 Rus can be used for assessment of autistic traits and, with some reservations, as an ASD screening tool for adult Russian-speaking population. Further research is necessary to develop more precise screening tools.


RAADS-14 as a screening tool for autism spectrum disorders in adult population
Autism spectrum disorder (ASD) is a range of neurodevelopmental conditions characterized by persisting de cits in social communication and interaction, repetitive and stereotyped behavior, and restricted interests 1 . The prevalence of ASD in adult population is steadily increasing, it greatly affects the quality of life of adults with ASD and impairs a signi cant burden on health and social care 2 . Autism in adults with no intellectual impairments is also quite frequent 3 , and often underdiagnosed 4 , particularly in women 5 . Late identi cation and general underdiagnosis and misdiagnosis of ASD leads to a problem of 'a lost generation' of autistic adults with no intellectual impairments 6 .
The comorbidity of ASD and other neuropsychiatric conditions is quite high 7 , differential diagnostics can also pose a problem, therefore proper assessment of autistic traits and related problems can be crucial for increasing the quality of life of people with ASD 8 . There is a range of self-report questionnaires available for screening of ASD in adults with no intellectual impairments, the most widely used being: the Autism-Spectrum Quotient (AQ, 50 items) 9 and its shorter versions, AQ-1. AQ-2. and AQ-J-21 10 ; the Ritvo Asperger and Autism Diagnostic Scale -Revised (RAADS-R, 80 items) 11 and its short version RAADS-14 8 ; the Social Responsiveness Scale, 2nd edition -Adult form (SRS-A or SRS2-A, 65 items) 12 ; and the Adult Social Behavior Questionnaire (ASBQ, 44 items) 13 . A systematic review found that stronger evidence of good or satisfactory measurement and diagnostic properties exist for AQ-5.
The RAADS-14 is a 14-item self-report questionnaire speci cally designed to be used as a screening tool for ASD and distributed under CC BY 2.0 Creative Commons License 8 . It was developed on the basis of the longer and more comprehensive RAADS-R, which has demonstrated good overall psychometric properties, test-retest reliability, discriminant and convergent validity, sensitivity and speci city 11,14,15 . The RAADS-14 was developed using the Swedish version of the RAADS-R 16  indicating duration of each symptom (3 = 'true now and when I was young',2= 'true only now',1= 'true only when I was younger than 16' and 0 = 'never true'). Item scores are summed to produce the total score and the subscale scores. The recommended cutoff of 14 for the original English version yielded excellent sensitivity of 0.97 and speci city of 0.95 over non-psychiatric population. The gender differences were moderate: neurotypical males scoring somewhat higher than females in the mentalizing de cits and social anxiety domains, while females scored higher than males in the sensory reactivity domain.
The RAADS-14 is very short and easy to administer and has already demonstrated the potential to distinguish e ciently between ASD and other neuropsychiatric conditions in adult population.
Assessing autistic traits in adult population in Russia: current problems and possible directions The situation with clinical diagnostics of ASD in adult population in Russia is very problematic, as several historical factors add to the above-mentioned diagnostic issues leading to even greater ASD underdiagnosis in Russian adult population. According to the o cial statistical data by Rosstat (Russian statistical agency), in 2016 there were only 96 adults o cially diagnosed as having ASD (less than .001% of adult population) 17 , which is at least 1000 times less than the estimates based on European cohort statistics. This huge disagreement reveals the fact that it was virtually impossible to get an o cial diagnosis of ASD for high functioning adolescents and adults for two major reasons. The rst reason is related to underdiagnosis of ASD in childhood during 1970-2010s which was less pronounced than for adults but still quite prominent: as of 1999, the o cially recognized ratio was 1 in 385 18 . For children without intellectual impairments the diagnosis was almost never o cially established, often in order to help the parents to avoid the stigma 19 . The second reason is an almost routine obligatory change of the o cial clinical diagnosis from ASD to some other neuropsychiatry condition performed when a person reaches adolescence or adulthood, most commonly, to schizoaffective disorder and/or mental retardation, this clinical practice was o cially reproved only on the June 30 th , 2014 20 . On the October 4 th , 2017, Russian Ministry of Health issued a letter of instructions (№ 17-1/10/1-6371) explicitly prohibiting an unfounded change of diagnosis from ASD to another mental condition for individuals reaching adulthood, as some psychiatrists tended to adhere to the old diagnosisswitching practice 21 . Due to the efforts of governmental and non-governmental institutions, autism professionals, parental community, and autism community it is now widely recognized that ASD in adults is currently underdiagnosed in Russia but implementing the diagnostic routines will have to take some time, and is hindered by the lack of diagnostic tools and trained professionals. The Russian version of ADOS-2 was published only in 2016 22,23 , and its adult module still remains under validation.
In this situation, the need for self-assessment tools in the autistic community has long been perceived as high.
Despite that, to our best knowledge, no self-assessment tools for evaluating autistic traits and for screening of ASD in adult population or adolescents has been o cially developed, adapted, and validated on Russian-speaking samples. Several attempts were made by members of autistic community to provide Russian versions of internationally recognized questionnaires, even if just for a rough reference; the largest collection of uno cial translations (including AQ, RAADS-R, Aspie Quiz, and The Broad Autism Phenotype Test) is presented at a web-site related to autism and to the problems of people with ASD 24 developed and maintained by members of Russian autism community. There are multiple anecdotal pieces of evidence that most of these questionnaires used with cutoffs established for European samples tend to overestimate the prominence of autistic traits but no data on this issue have been o cially reported.
The lack of ASD diagnostic tools and inventories assessing autistic traits validated on Russian-speaking samples signi cantly limits the options to assess criterion validity of any emerging ASD screening questionnaire. In order to get additional circumstantial evidence, special efforts could be made to evaluate the questionnaire's discriminant validity comparing it to other personality trait inventories. There are numerous studies exploring the links between the autism traits and other personality factors and constructs. The Big Five personality model is probably the most well-established general personality construct 25 which was found to strongly relate to autism traits in a large number of studies, consistently revealing moderate positive correlations of ASD scores to neuroticism, and weak to moderate negative correlations to all the other Big Five factors 26 . A study utilizing multivariate approaches to explore the relations between IPIP-NEO-120 and RAADS-R scores found that Big Five personality factors accounted for 70 % of variance in autism trait scores 27 . These ndings suggest that a Big Five personality questionnaire would be a good tool to assist in a rming convergent and discriminant validity of an ASD assessment tool.
Aim and general design of the study The general aim of our research was to construct and validate a Russian version of the RAADS-14 (the RAADS-14 Rus) to provide the initial reference point in developing the body of instruments for assessing autistic traits in Russian-speaking adult population. This measure was chosen due to its good psychometric characteristics, discriminative properties, and simplicity of use. The study aims to assess the reliability of the RAADS-14 Rus, its factor structure, and to provide the initial estimates of the questionnaire's validity, diagnostic properties, and potential for use as a screening tool for ASD in adults with no intellectual impairments. We have also formulated three speci c research hypotheses: 1. The RAADS-14 Rus factor structure would be equivalent or similar to the factor structure of the original Raads-14; 2. The RAADS-14 Rus total scores and subscale scores will have weak to moderate correlations to the scores of the Big Five personality factors, in particular, positive to neuroticism and negative to extraversion, consistent to the previous studies; 3. The distribution of the RAADS-14 scores could be different for general Russian-speaking population, producing greater mean total score values and yielding potentially higher cutoff scores to provide reasonable speci city of the questionnaire.

Results
Reliability and the factor structure Cronbach's α for the 14 items of the RAADS-14 Rus was 0.839 demonstrating good reliability of the questionnaire with an average inter-item correlation of 0.272.
CFA performed for a three-factor model re ecting subscale item composition of the original RAADS-14 (Fig 1) revealed acceptable t indices (RMSEA = .067, CFI = .954, df=91, χ 2 =12076.025, p <.0001). CFI demonstrating a good t 28 and RMSEA demonstrating a moderately good t 29 . All the factor loads were above .42. The data suggest that the factor structure of the RAADS-14 Rus is very similar to the factor structure of the original Swedish version, therefore the three domains (Mentalizing de cits, Social anxiety, and Sensory reactivity) of the RAADS-14 RUS were scored as in the original version.

Distribution of scores and discriminative properties
The distributions of total RAADS-14 Rus scores in non-psychiatric young adults (n = 794, mean = 13.9, SD = 7.5, median = 13), and ASD participants (n=49, mean = 29.3, SD = 8.7, median = 30.5) are shown at Figure 2. The score distribution in non-psychiatric young adults, as expected, was skewed to the left, while the score distributions for self-diagnosed and clinically diagnosed adults with ASD were very similar and were skewed to the right, with no prominent ceiling effects observed for any of the samples. An additional comparison revealed that the RAADS-14 Rus scores of ASD participants with ADOS-2 veri ed diagnosis were slightly higher (n=13, mean = 33.2, SD = 7.1, median = 35.0) than the scores of the rest of ASD participants but the difference was not statistically signi cant (U= 170.5, p= 0.150). The distribution of scores for the ASD participants was similar to the data obtained with the RAADS-14 for Swedish ASD population 8 . The distribution of the RAADS-14 Rus scores for the non-psychiatric adults was less extremely skewed to the left compared to the Swedish data, producing much greater mean and median (13.9 and 13 vs. 3.9 and 3), con rming our prediction that mean RAADS-14 Rus scores are expected to be higher than the original scores. The comparison of the domain scores between the non-psychiatric participants (Sample 1, n=794) and the participants with clinically established ASD diagnosis (n=49) revealed very robust effects (U MD =4773, p two-tailed <.0001; U SR =8404,5, p two-tailed <.0001; U SA =3685,5, p two-tailed <.0001) indicating that the ASD participants scored higher on each domain compared to the non-psychiatric participants. Age and Gender differences Considering large difference in mean age and in the RAADS-14 Rus scores between the samples, the assessment of possible relations between age and the RAADS-14 Rus total scores was performed separately for all the samples and, additionally, for the participants with ASD pooled across the samples. A signi cant negative correlation with age was observed for Sample 2 (n=509, r = -.32, p two-tailed <.001) but not for Sample 1 (n = 793, r = -.017, p twotailed =.629) and Sample 3 (n = 364, r = -.1, p two-tailed =.058). This correlation was not signi cant for the ASD subgroup (n = 49, r = -.13, p = .388).
The gender differences were also assessed separately for all the samples and for the participants with ASD. The participants who reported their gender as 'other' (Sample 2 n=20; Sample 3 n=16) were characterized by signi cantly higher RAADS-14 Rus total scores than the participants stating their gender as both male or female for Sample 2 (H (2, N=509) =3.92, p<.001) and for Sample 3 (H (2, N=364) =10.4. p= .006) (see Table 1). There were no signi cant differences between the males and the females in the RAADS-14 Rus total scores for any of the samples, nor for the ASD subgroups.
The RAADS-14 Rus domain score comparison for males and females in the non-psychiatric population revealed that, similarly to the RAADS-14 validation study, females scored higher than males for Sensory reactivity domain  The aim of this study was to develop a Russian version of the RAADS-14 (the RAADS-14 Rus) and to assess its reliability, factor structure, validity and discriminative properties. The RAADS-14 Rus has exactly the same item composition as the original version; con rmatory factor analysis (CFA) revealed that all the model t indices were reasonably good, suggesting that the factor structure of the Russian version is equivalent to the original version 8 .
There was a very robust difference in the scores of all the RAADS-14 Rus screening domains between the nonpsychiatric population and the ASD sample, further con rming construct validity of the RAADS-14 Rus.
The original RAADS-14 was composed of the items yielding the highest discriminative properties for ASD diagnosis; owing to this approach, the questionnaire was characterized by excellent discriminative properties at distinguishing between ASD participants and non-psychiatric controls (AUC = .99) and by good discrimination between ASD and other neuropsychiatric conditions. The total RAADS-14 Rus score has good discriminative properties for distinguishing between ASD participants and general population control sample (AUC of .92), that are still not nearly as good as in the original study. This difference can be best attributed to the fact that mean RAADS-14 scores were much higher for the Russian sample than for the original Swedish sample, particularly for non-psychiatric controls.
At the moment we would recommend using a provisional cutoff of 22 points and above. This cutoff yields a good sensitivity of 85.7% and a speci city of 85.4%. While, taken per se, these discriminative properties can be considered operational, it should be noted that using the RAADS-14 Rus as a screening tool with this cut-off would yield a rather low positive predictive value, given the estimated prevalence of high functioning forms of ASD. Using an o cial prevalence estimate of high functioning autism of 1/200 for Russian samples 18 , the positive predictive value would be as low as 2.9%. Still, at the moment, with no other validated ASD assessment tools available for Russianspeaking population, we suggest that one may use RAADS-14 Rus for screening if all the above-mentioned considerations are taken into account. Further research is sought to develop more sensitive autism assessment tools speci cally for Russian-speaking samples.
While this paper was in preparation, the results of another recent study assessing the RAADS-14 properties for a New Zealand national sample were published 30 . The two studies are similar in their purpose and design, and there are notable similarities in the results as well. In the both studies the mean RAADS-14 scores for non-ASD groups were much higher than in the original study (Russia 13.9, New Zealand 9.85 vs Sweden 3.9); this, combined with other possible factors, led to substantial worsening of the discriminative properties of the questionnaire. The mean RAADS-14 scores were the highest for the Russian study; it con rms our rst speci c prediction stemming from prior anecdotal evidences that pilot non-validated versions of autism assessment self-report questionnaires produce higher scores for Russian samples than for Western European and American samples. The underlying reasons for this effect are yet unclear; several factors may contribute but at this moment we would suggest that this difference should be largely attributed to culture-speci c biases in self-reporting and speaking about one's problems and social competences rather than to elevated autistic traits in Russian neurotypical population. Despite that, the RAADS-14 performed somewhat better in this Russian study than in the New Zealand study, producing acceptable model t indices and better speci city for the author-suggested cut-offs.

Age and Gender differences
The RAADS-14 Rus total score revealed that for one of the three samples used in this study a negative correlation of the total scores of the RAADS-14 was observed; it is consistent with the previous data indicating that different facets of autistic traits in adults tend to somewhat decrease with age 3132 . The female to male differences in the RAADS-14 Rus scores were quite minor and generally similar to data from the original RAADS-14 validation study collected for Swedish samples 8 . In our study a signi cant part of subjects in Samples 2 and 3 indicated their gender as 'other' or non-binary (total n =36). The non-binary participants scored signi cantly higher than the males and females; this also corresponds to the prior ndings in other national samples showing that non-binary individuals often report elevated levels of autistic traits [33][34][35] .
The RAADS-14 Rus and the Big Five personality model The correlations of the Big Five personality traits with the total RAADS-14 Rus score, taken as a general measure of autistic traits, were highly consistent with the earlier ndings 36 , revealing moderate positive correlations with Neuroticism, moderate negative correlation with Extraversion, and weaker negative correlations with Agreeableness, Openness to experience, and Conscientiousness, similarly to a study using RAADS-R and IPIP-NEO-120 37 . These results support good discriminative validity of the RAADS-14 Rus and con rm our prediction that the character of the relation between autistic traits, as measured by the RAADS-14 Rus, and the Big Five personality traits for the Russian sample is similar to such relation for other national populations. It can serve as a circumstantial evidence of criterion validity of the RAADS-14 Rus.

Limitations and future directions
The main limitations of the study stem from the lack of validated ASD assessment tools currently available in Russian and from very high estimates of underdiagnosis of ASD in Russian adults with no intellectual impairments. Due to our limited options to assess concurrent validity of the RAADS-14 Rus and to the limited size of the ASD sample in this study (particularly, to the small number of participants with ADOS-veri ed diagnosis) all the estimates of the discriminative properties of the RAADS-14 Rus should be interpreted strictly as preliminary. These ndings require further validation on larger clinical and non-clinical samples. However, considering that a. to our best knowledge, the RAADS-14 Rus is the rst self-assessment inventory addressing autistic traits in adults with no intellectual impairments, and b. the non-psychiatric sample of young adults is relatively large and represents several Russian regions, the data were collected with high consent rate and can be considered a satisfactory initial approximation of the age cohort, at least for larger cities 38 , the RAADS-14 Rus can become one of the rst stepping-stones in establishing a comprehensive system of ASD assessment and diagnostics for adult Russian-speaking population.
Further efforts should be made to ensure better screening sensitivity and may lead to development of new screening instruments based on larger initial pools of ASD-related items and using item selection centered around single-item discriminatory power, similarly to the process used for developing the original RAADS-14.

Participants and Procedure
A total of 1724 adults have agreed to voluntarily participate in the study. The data were collected for three different samples: Sample 1: Non-psychiatric control sample of young adults (college and vocational school students, n=849) who volunteered to participate for research credits. The recruitment was performed in 4 educational institutions in Moscow (n=522) and in 5 educational institutions in other Russians regions (n=327), the consent rate was above 0.7. A part of the participants (n= 652) used paper and pen version, other participants used web-based survey forms and completed them in class settings. 57 participants from Sample 1 failed to provide the answers to all the questions and were excluded from further analyses, reducing the number of the accepted participants for Sample to 794. Three participants failed to report their gender, therefore the total number of participants from the sample is greater than the sum of female (n=504) and male (n=287) participants.
Sample 2: A sample with a higher than normal ratio of adults with ASD (total n=509) was recruited using snowball sampling by targeting autism-related communities to ensure a higher than normal ratio of participants with ASD. To provide the initial estimates of the sensitivity of the RAADS-14 Rus in this study we relied on self-reported ASD diagnosis, reportedly established in clinical settings (n=49), similarly to another recent study assessing the RAADS-14 properties for a different national sample 30 ; for a smaller subset of the participants that gave their consent (n=13) it was further veri ed by ADOS-2. All the participants in this sample used web-based survey forms and completed them at their convenience.
Sample 3: An additional web-based sample was recruited using snowball sampling (total n=364) to provide the data for evaluating possible relations between the autism traits measured by the RAADS-14 Rus and the Big Five personality traits. All the participants in this sample used web-based survey forms and completed them at their convenience.
After all the exclusions the data from a total of 1667 participants used for further analyses. The data collection was anonymous for all the participants but the participants from samples 2 and 3 were suggested to provide their emails (this was optional and the emails were used for the sole purpose of providing the feedback with participant's personal RAADS-14 Rus scores and general follow-up for the study and were later removed from the resulting datasheet). For detailed age and gender statistics for all the samples see Table 2. 2= 'true only now', 1= 'true only when I was younger than 16' and 0 = 'never true'). The item scores were summed to produce the total score (for general total score statistics by samples see Table 1). The Russian version of the form is provided in Supplementary information (S1 Table).
The Big Five personality model. To assess the Big Five personality factors we used a Russian version of the NEO-FFI 41 . This questionnaire is reasonably brief (50 items) and its factor structure has been consistently replicated for different Russian-speaking samples 42,43 .

Data analysis
Reliability and the factor structure. To evaluate the reliability and the factor structure of the RAADS-14 Rus we pooled the participants from all the samples resulting in a total of 1667 participants. Cronbach's α was used to assess the reliability of the scale. In order to assess whether the factor structure of the RAADS-14 Rus was equivalent to the factor structure of the original version, con rmatory factor analysis (CFA) was performed using a single-order threefactor model reported in the original validation study. Weighted least-squares with mean and variance (WLSMV) procedure was used to estimate the model. The goodness of model t was tested by calculating CFI (comparative t index) and root-mean-square-error of approximation (RMSEA).
Demographics and correlation analyses.
Age and gender effects were assessed separately for all the samples due to prominent differences in the demographics and clinical characteristics of the samples. A Kolmogorov-Smirnov test was performed for total RAADS-14 Rus for the pooled sample revealing that the data do not have a normal distribution (n=1667, Mean=17,42, SD=9,96, D= .07, p<.001). Due to the non-normal nature of the data, all the group-related differences were assessed with Kruskal-Wallis ANOVAs and Mann-Whitney U-tests. Spearman's rank-order correlations were used for subscale cross-correlations and for correlations with the Big Five factor scores.
Screening properties. The discriminatory power was assessed using a receiver operating characteristic (ROC) curve, estimating sensitivity (the rate of true positives for the ASD sample), speci city (1 minus the rate of false positives for the non-psychiatric control sample), and the area under the curve (AUC) as a measure of the discriminatory power (an AUC greater than 0.7 is generally considered satisfactory) 44 .
Statistical analysis. All the statistical analyses were performed using Statistica 10, IBM SPSS Statistics 23, and Mplus 8.2 software 45 . Bonferroni corrections for multiple comparisons were used where appropriate.

Data availability
All data generated or analyzed during this study were in anonymized and are included in this published article (and its Supplementary Information les).