The Internet Addiction Test: Factor structure, measurement invariance, and psychometric properties of a six-item version among Spanish women with eating disorders

Background: Eating disorders and internet addiction are widespread conditions that have considerably increased because of the current lockdown. Their comorbidity is frequently undiscovered, which may increase patients’ distress and hamper recovery. Identifying the constructs of internet addiction may facilitate its detection and management. The Internet Addiction Test is widely used to evaluate internet addiction among students and healthy adults, with less agreement on its dimensional structure. Objectives: To examine the structure of the Internet Addiction Test and its invariance among patients with eating disorders. Methods: In this cross-sectional study, exploratory and conrmatory factor analyses evaluated the structure of the Internet Addiction Test among 123 women with eating disorders (59 women with anorexia nervosa (group 1) and 64 women with other eating disorders (group 2)) recruited from the General University Hospital of Ciudad Real between February and November 2018. Multigroup analysis examined invariance of the Internet Addiction Test across eating disorder groups, and correlation with the Bergen Facebook Addiction Scale examined its criterion validity. Results: One factor explained 83.4%, 74.4%, and 87.4% of the variances in the whole sample, group 1, and group 2, with excellent reliability (coecient alpha = 0.99, 0.98, and 0.99, respectively). In conrmatory factor analysis, two- and bifactor structures (a general factor with two specic factors: Emotional and cognitive preoccupation; Loss of control and interference with daily life) expressed some satisfactory t in all groups, but they displayed metric and scalar variance—less tendency of women with anorexia nervosa to endorse items 14, 15, and 16. A 12-item version expressed a better t. However, the six-item Internet Addiction Test expressed the best t, along with congural and metric invariance across groups, and excellent reliability (coecient alpha = 0.97, 0.95, and 0.98, respectively). The criterion validity of the Internet Addiction Test, the12-item version, and our six-item version was conrmed by strong positive correlations with the Bergen Facebook Addiction Scale (r = 0.906, 0.883, and 0.878, respectively). Conclusion: The Internet Addiction Test is a unidimensional or bidimensional measure. The six-item version expresses better t, less variance, and comparable reliability and criterion validity to the Internet Addiction Test and the 12-item version. Its brevity allows test batteries to include several measures. Scalar variance implies that differences in the properties of the Internet Addiction Test across eating disorder groups may cause statistical differences in group means, which should be considered when conducting interventional studies. Further investigations are intensely needed. The six-item Internet Addiction Test demonstrates good t, less variance, excellent reliability, and good criterion validity among women with eating disorders.


Background
The use of the internet for socialization and gaming has dramatically increased among children, adolescents, and younger adults during the last two decades as a result of the expansion of internet technology [1,2]. Excessive internet use has brought about several negative consequences such as decreased sleep duration, cyberbullying, nomophobia, and internet addiction, especially among individuals with special emotional needs [3][4][5]. The current coronavirus disease 2019 pandemic has been associated with an increase in the prevalence and severity of internet addiction among the general population, especially individuals with poor social support [4,6]. In particular, the duration of internet use has considerably increased secondary to social isolation, a key protective measure against coronavirus disease 2019, and anxiety associated with fear of contracting infection [4,7].
Internet addiction is a maladaptive pattern of excessive or problematic use of the internet for nonessential, personal internet activities (e.g., gaming, social networking, gambling, and online sex) that increase the time spent online and cause remarkable alterations in one's life [8,9]. Individuals with problematic internet use demonstrate the diagnostic criteria of addiction proposed in the DSM-5: salience indicated by preoccupation, repeated and uncontrolled use of the internet despite its negative psychosocial effects expressed as con icts in work and academic relationships, loss of interest in other recreational activities, using the internet to escape negative emotions or to improve mood, misleading others regarding the amount of time spent online, intolerance-a need for increased use to achieve the previous desired effect, withdrawal reactions (e.g., anxiety and depression) following deprivation of the internet, and relapsetendency to revert to earlier use following abstinence [3,8,10].
Problematic use of the internet predisposes adolescents to higher levels of depression, suicidal ideation [11,12], and suicidal attempts due to the development of brain dysfunction: increased activity in the gyrus frontalis inferior of the right pars triangularis and the right pars opercularis [13]. Despite its widespread and serious adverse effects, internet addiction may not be detected by psychologists, psychiatrists, and other health professionals due to lack of training [14].
The Internet Addiction Test, developed by Kimberly Young, is one of the most widely used measures to diagnose internet addiction [9,15]. Because the scale comprises numerous indicators of recurrent addiction behaviors, it detects attributes of obsessive use of the internet (e.g., escapism, compulsivity, and dependency) and consequences of addictive use (e.g., personal and social con icts, personal and occupational performance de ciency) [16]. It has been adapted to evaluate online sexual activities [17], and a modi ed version has been utilized to assess smartphone addiction [18].
The Internet Addiction Test has been translated and validated into more than 20 languages other than English [14], including French [17][18][19], Spanish [20], Finnish [21], German [22,23], Italian [15,24], Polish [25] [29], Indonesian [30], Malay [9], etc. However, there is less agreement on its construct structure. Several studies report a unidimensional factor structure of the Internet Addiction Test [15,18,19,21,26], including the smartphone Internet Addiction Test [18]. Meanwhile, in some studies the one-factor structure displays poor t compared with a two-factor structure [8,17,20,[22][23][24][25]. In few instances, the one-factor structure comparably ts data same as the alternative two-factor model [15,28]. A bifactor structure is reported to account for the high correlations between two extracted factors [5]. Some other studies report a three-factor structure [29,30] or even a ve-factor or six-factor structure [9,31]. Interestingly, one study reported better t for a one-factor solution based on the 20 items of the measure than a welltting three-factor solution based on 18 items (after excluding item 5 and item 7) [30]. In some studies the best tting models are produced by the removal of one or two items [20,28,30] while in some studies, tting models are indicated by t indices expressing values out of the acceptable range [8,24].
Variations in the reported structure of the Internet Addiction Test can be accounted, in part, by the method of extraction. Obviously, studies reporting more than two-factor structures used exploratory factor analysis or principle component analysis [9,30,32], which employ methods that overestimate the number of extracted factors mainly counting on the criteria of eigenvalues >1 [22,33,34]. In addition, the Internet Addiction Test is largely validated in high school and university students as well as in healthy young adults [14,30,32]. Internet addiction is reported to be high among individuals with comorbidities. Therefore, the diagnostic potential of the Internet Addiction Test should be explored in other diverse groups, including individuals with diseased conditions [14].
Very few studies evaluated invariance of the Internet Addiction Test across different groups such as gender [8]. Measurement invariance implies psychometric equivalence of a construct across groups i.e., it has the same meaning to those groups [33,35]. Several forms of invariance are assessed in psychometric studies: con gural invariance-examines global model t without imposing constrains across groups, metric invariance-examines the sensitivity of groups to each item on the measure by constraining factor loadings to equality across groups, scalar invariance-examines variations in true mean differences by constraining intercepts of the regression equations of the observed variables on the latent factors to equality across groups, and strict invariance-examines the uniqueness of each observed variable by constraining residuals to equality across groups [33,36]. Among all types, scalar variance/non-invariance is the most important because it may cause serious misinterpretation of true mean differences. One-third of psychometric tests exhibits partial invariance [35]. Lack of evaluation of invariance of the Internet Addiction Test may cast doubt on the statistically signi cant differences in internet addiction across different groups [35]. Therefore, establishing measurement invariance of the Internet Addiction Test is necessary for cross-group comparisons of mean differences and other structural parameters [36] of internet addiction.
Eating disorders commonly develop in young groups, especially among adolescent girls and young women, and they are associated with high levels of distress. Emotional distress may trigger emotional eating or discourage eating in some patients in effort to control body weight and shape [37]. The coronavirus disease 2019 outbreak has been associated with increased symptoms of dietary restriction, binge eating, purging, weight gain, and exercise behaviors in people with eating disorders and in the general population [38,39]. This is especially because individuals with eating disorders use de cient coping with the current lockdown. As a result, their psychopathology (depression and anxiety) heightens, which furthers eating pathology [38].
Some eating disorders display an addictive nature. Hippocampal and insular levels of dopamine can be manipulated by certain foods (e.g., sugary and processed) leading to loss of control over the intake of these foods along with symptoms of craving and withdrawal [40]. Indeed, people with eating disorders have a high genetic tendency toward addictive and impulse control disorders, which justi es the high comorbidity of these diseases [41][42][43]. Few studies associate problematic use of the internet with symptoms of bulimia and binge eating among university students [44]. Given the innate nature of distress in eating disorders, internet addiction may be an additional source of distress that may promote a prolonged course of the disease and threaten patients' wellbeing and quality of life [44,45]. Therefore, careful identi cation and proper management of internet addiction in patients with eating disorders may have implications for improving their recovery. The current study aimed to examine the construct structure and invariance of the Internet Addiction Test among patients with different eating disorders. Because the evaluation of criterion validity is an important aspect of psychometric evaluations [46], we included it in our analysis by examining the correlation between internet addiction and excessive use of the Facebook. We hypothesized that patients with higher scores on the Internet Addiction Test would express higher dependence on the Facebook.
2. Methods 2.1. Study design, participants, and procedure Participants of this cross-sectional study were Spanish females (>12 years of age) consecutively treated for eating disorders at the outpatient and inpatient departments of the General University Hospital of Ciudad Real between February and November 2018 who agreed to sign an informed consent. Patients with physical or mental disabilities were excluded. Although the Ethics and Clinical Research Committee of Ciudad Real (Spain) approved data collection (ref. 2017C/123) [45], we did not obtain ethical approval for the current study because it is based on the analysis of a publicly accessible dataset [47].

Data collection measures
Data were collected from eligible participants through a questionnaire that comprised a section inquiring about participants' sociodemographic and clinical characteristics such as age, weight, height, type of eating disorder, treatment history, comorbidities, and smoking status [45].
Problematic use of the internet was assessed by the Internet Addiction Test, a 20-item measure with responses rated on a ve-point Likert scale (1 = never to 5 = always). In general, items of the Internet Addiction Test re ect on aspects of internet dependence (e.g., become defensive or secretive (item 9), fear that life without the internet would be boring (item 12), and hide how long you've been online (item 18)) as well as drawbacks of internet addiction (neglect household chores (item 2), lose sleep (item 14), your job performance or productivity suffer (item 8)). The scale has a maximum score of 100. Scores below 36 indicate normal use, and higher scores indicate problematic internet use [22]. The scale has been validated among Spanish university students [20], and its reliability in the current study is excellent (Cronbach's alpha = 0.99).
The validated Spanish version of the Bergen Facebook Addiction Scale [48] was used to evaluate problematic use of the Facebook in the samples. The scale comprises six items, which measure different aspects of Facebook addiction. Items are rated on a scale that ranges between 1 = very rarely and 5 = very often.
Higher scores indicate higher levels of addiction [3]. The internal consistency of the Bergen Facebook Addiction Scale in the current study is excellent (Cronbach's alpha = 0.99). The scores of the Bergen Facebook Addiction Scale were used to test for criterion validity.

Statistical analysis
The dataset was checked for missing data in the key variables. Accordingly, one incomplete response was deleted. The normality of the Internet Addiction Test was examined by Shapiro-Wilk and Kolmogorov-Smirnov tests of skewness and kurtosis. Because anorexia nervosa was the most dominant diagnosis while other eating disorders were less presented, we created two major categories of eating disorders for subgroup analysis: patients with anorexia nervosa (group 1) and patients with other eating disorders (group 2). These two groups were established based on the fact that body mass index was signi cantly lower in group 1 than in group 2.
The factor structure of the Internet Addiction Test was evaluated in two stages. First, we carried out exploratory factor analysis using maximum likelihood method with Varimax rotation for the whole sample, group 1, and group 2; for simplicity, we refer to them herein as the samples. The Kaiser-Meyer-Olkin measure of sampling adequacy and Bartlett's test of sphericity were included in the analysis to ensure suitability of the data for factor analysis. In the second stage, we examined the one-factor, two-factor, three-factor, and bifactor structures of the Internet Addiction Test through con rmatory factor analysis ( Table 1). Maximum likelihood method with bootstrapping based on 2000 random samples and a 95% con dence interval (95% CI) was used for factor extraction. Based on our exploratory factor analysis, we examined data t to a one-factor structure of the Internet Addiction Test both without and with correlating some error terms revealed by modi cation indices (Model 1 and Model 2, respectively). To replicate the 19-item one-factor model tested previously among Spanish students, we removed item 7 and examined data t without and with error correlations (Model 3 and Model 4, respectively). Then, we examined the two-factor structure (Emotional investment; Performance and time management) [20], reported by Fernández-Villa et al (2015) both comprising all items of the Internet Addiction Test (Model 5 and Model 6, respectively) and after the removal of item 7 (Model 7 and Model 8, respectively). We also examined the two-factor structure reported by Barke et al (2012) (Emotional and cognitive preoccupation with the internet; Loss of control and interference with daily life) [23], both comprising all items of the Internet Addiction Test (Model 9 as a crude model and Model 10, which correlates residuals) and after the removal of item 7 (Model 11, which correlates residuals). In addition, we evaluated the three-factor structure reported by Widyanto  A non-signi cant chi square index was used to re ect global data-t to the model [34]. Model-data t was considered good and acceptable if values of the Comparative Fit Index and the Tucker-Lewis Index were equal to or above 0.95 and 0.90, respectively, along values of the root mean square error of approximation and the standardized root-mean-square residual less than 0.06 and 0.08, respectively [20,33]. This combination facilitates a sound judgement because some indices such as the root mean square error of approximation and the standardized root-mean-square residual may be sensitive to misspeci cation of the factor loadings or misspeci cation of the co-variances, respectively [20].
Multigroup analysis was conducted to examine invariance of the one-factor and two-factor structures (Model 2, Model 4, Model 6, Model 8, and Model 17) as well as the one-factor six-item Internet Addiction Test (Model 19) across eating disorder groups. To identify items that represent the source of invariance, we created several models in which the parameter estimates of each item were constrained to equality between the two groups while the rest of the items were left to load freely between the two groups. Nested model comparisons with signi cant chi square test were used to denote a signi cant decrease in model t when path coe cients were constrained to equality between groups [35]. Critical ratios for difference between parameters with values out of the range of - Allowing the residuals of some items to correlate improved the data-model t in the whole sample and group 2, albeit the root mean square error of approximation was always exceptionally high. Noticeably, items with correlating residuals in most models were (item 8 and 9), (item 14 and 15), (item 15 and 16), and (item 17 and 18). Meanwhile, the data t for all models tested in group 1 was unsatisfactory. However, values of the Comparative Fit Index in Model 8 and Model 11-which tested the two-factor structures proposed by Fernández-Villa and Barke (for 19 items after the removal of item 7)-were within the acceptable range (0.903 and 0.901, respectively), indicating some t of these two models. The t of a second order factor was the same as the two-factor structures. A bifactor structure based on Model 8 had a poor t (Supplementary material), but a bifactor structure based on Model 11 had the best t in all the samples, Model 15. Table 1, the unidimensional structure of the six-item Internet Addiction Test expressed the best t in the samples. For the rst time, the root mean square error of approximation considerably decreased when error residuals were correlated (Figure 1), and chi square index became insigni cant only in Model 19, denoting that this model best describes the psychometric properties of the Internet Addiction Test.

Invariance of the Internet Addiction Test across eating disorder groups
In Fernández-Villa's analysis, item 7 was removed because it originally did not load on any factor in exploratory factor analysis. However, in our study, item 7 loaded on the extracted factor with high communalities in the samples while its removal slightly improved model t in group 1. Therefore, we conducted multigroup analysis to compare data t to the one-factor, and two-factor structures both with and without item 7: Model 2, Model 4, Model 6, Model 8, and Model 11. All these models hold con gural invariance among eating disorder groups (Table 3). However, those models, except for Model 11, do not hold metric and scalar invariance across groups. Examinations of each individual path revealed that items 14, 15, and 16 were the source of invariance in the tested models (more details in Supplementary materials). In particular, women with anorexia nervosa seem to be less sensitive to these items than women with other

Reliability and criterion validity
The Internet Addiction Test, the 12-item Internet Addiction Test, and the six-item Internet Addiction Test demonstrate excellent reliability in all the samples as noted by high values of coe cient alpha (Table 4). All corrected item-total correlations and values of alpha if-item-deleted were remarkably high, indicating that all items considerably contribute to the overall latent construct under measurement, which re ects good convergent validity of these measures. The total scores of the three versions of the Internet Addiction Test strongly correlate with problematic use of the Facebook in all the samples (Table 4) denoting adequate criterion validity of these measures.

Discussion
Given that the structure of the Internet Addiction Test has always been tested among students (in 20 out of 25 studies) and healthy young adults [14], the current study enriches the existing knowledge by reporting on its structure and invariance among patients with eating disorders. In addition to examining the data t to several structures of this test via numerous validation techniques, the study represents the rst attempt to extensively revise this measure denoting usability and comparable or even better psychometric properties of the six-item Internet Addiction Test.
Our results suggest that the Internet Addiction Test may be a unidimensional measure. In a previous study, exploratory factor analysis indicated that the Spanish Internet Addiction Test had three factors, with a single item loading on the third factor and item 7 not loading on any factor. Therefore, Fernández-Villa et al. deleted item 7 and reported the ndings of exploratory factor analysis and con rmatory factor analysis based on a two-factor solution.
Nonetheless, both factors together explained 55% of the variance in total, and reliability of the second factor was relatively poor (Kappa = 0.65) [20]. Unlike Fernández-Villa' s results, all items of the Internet Addiction Test in our analysis had high loadings on a single factor, which explained the majority of the variances in the samples ( Table 1). The single factor structure gets further support from the sound internal consistency of the Internet Addiction Test, higher values of corrected item-total correlations, and lack of change in the internal consistency of the Internet Addiction Test if-item-deleted (Table 4). In our con rmatory factor analysis, the one-factor structure involving all items of the Internet Addiction Test had an acceptable t comparable with the two-factor structure (Emotional Investment; Performance and Time Management). Meanwhile, tests of a second-order factor revealed a similar or lower t than the twofactor structure (Supplementary material). However, a bifactor structure involving a general internet addiction factor with two speci c factors (Emotional and cognitive preoccupation; Loss of control and interference with daily life), indicated by Model 15, had the best t among all structures of the 20-item Internet Addiction Test in all the samples. Our ndings are consistent with those of previous studies [15,28] and a meta-analysis signifying that one-factor or twofactor structures represent the psychometric properties of the Internet Addiction Test in well-designed studies [14].
Although item 7 was not a source of noise in the structure of the Internet Addiction Test in our study, it caused mis t in a sample of Spanish university students [20]. The loading of item 7 was remarkably low in German [22], Romanian [8] and Indonesian students [30] as well as German young adults [22].
Therefore, we ran all the models with all the 20 items and without item 7. Remarkably, the 20-items one-factor structure with correlating errors (Model 2) exhibited a slightly better t than the 19-items one-factor structure with correlating errors ( It is not clear why the removal of item 7 improved the t in these structures and in women with anorexia nervosa, in particular. However, other studies reporting issues associated with item 7 denote that it may be an outdated item [8, 20,22]. This is because the e-mail represents a fast and useful tool of communication nowadays. Thus, checking the e-mail box frequently may be a normal behavior or a routine of everyday life (e.g., for professional reasons). Within this context, addiction may be better re ected by excessive time used for checking email rather than the frequency of email checking [8].
To our knowledge, only one study examined invariance of the Internet Addiction Test-the Romanian two-factor structure of the Internet Addiction Test was invariant across gender groups on the con gural, metric, and scalar levels, albeit the overall all t of the model was fairly low [8]. In our study, the Internet Addiction Test structures demonstrated con gural invariance denoting that women in different eating disorder groups conceptualize excessive use of the internet similarly. Nonetheless, the response of women in group 1 to items 14, 15, and 16 of the Internet Addiction Test is less than women in group 2 for all models, except for Model 11, which demonstrated scalar invariance-expressed by differences in the shared variance of both factors and inter-factor correlation between groups (Supplementary material). Of interest, correlating the residuals of item 15 with the two other items improved t indices in most models. Meanwhile, when Model 15 was compared across eating disorder groups, it failed to converge; most items with correlating residuals expressed standard errors >1. Increasing the iteration limit and removing these items failed to support model convergence and resulted in more dysfunctional items. In line with our results, item 15 caused noise in the Hebrew Internet Addiction Test, and its removal along with item 12 achieved an acceptable t of the twofactor structure [28]. Altogether, correlating and variant items on the Internet Addiction Test may threaten the validity of its measurement.
Pawlikowski and colleagues developed the 12-item Internet Addiction Test based on the removal of items with low loadings [22]. Although this 12-item version expressed a better t than the full scale in our analysis, the chi square index (a measure of global model t) and root mean square error of approximation (a measures of absolute model t) [35] denote that its t is unsatisfactory. In addition, the model exhibited between group variance involving item loading, item residuals, and the variance explained by factors underlying the scale (Table 3, Supplementary material). Accordingly, we have extensively revised the Internet Addiction Test by including a single item (the most relevant) on each addiction domain ending with a scale comprising six items that evaluate con ict, mood modi cation, salience, tolerance, withdrawal, and relapse (Supplementary material). This six-item version expressed the best t in all the samples, with less invariance compared with the 20-item and 12-item Internet Addiction Test. Its internal consistency and criterion validity are comparable to both measures. Thus, this six-item version may be a superior diagnostic measure of internet addiction. An exceptional merit of this version is that its brevity permits test batteries to include more other measures. It is worth mentioning that because neither the 20-item Internet Addiction Test nor the 12-item or six-item versions hold scalar invariance, we cannot substantiate multigroup comparisons of factor means (e.g., t-tests or ANOVA). In other words, statistically signi cant differences in group means may be attributed to differences in scale properties across different groups [35,36]. More investigations are necessary to examine invariance of these measures across various groups.
This study has some limitations. Because participants were females, it was not possible to examine invariance across gender groups. It is true that the prevalence of internet addiction does not vary by gender [12], and the Internet Addiction Test was invariant among men and women university students [8].
However, eating disorders are more frequent and more comorbid among women than men [49]. Women also express higher psychological distress than men [50] while research con rms differences in the brain activity between men and women with internet gaming disorder [51]. Thus, invariance of the Internet Addiction Test across gender groups may not hold in eating disorders patient population. Patients with other eating disorders (e.g., bulimia nervosa, binge eating, etc.) were not detailed in invariance analysis because they were less presented in the sample. In addition, the sample was not powered entailing a need for further testing in larger samples. The study is limited to Spanish patients with eating disorders, which limits the generalizability of the results. Future studies should overcome these limitations.

Conclusion
The Internet Addiction Test is best described by a bifactor structure or a two-factor structure among women with eating disorders. The six-item Internet Addiction Test expresses better psychometrics compared with the Internet Addiction Test and the 12-item Internet Addiction Test. The criterion validity of these measures is demonstrated by high correlations with Facebook addiction. Despite the fact that women with anorexia nervosa and with other eating disorders have the same global view of internet addiction, their endorsement of some aspects of internet addiction varies. Interventions addressing internet addiction among patients with eating disorders should take these differences into consideration. Further investigations of the six-item Internet Addiction Test in larger samples with more representation of different eating disorders will be valuable.