Recruitment and Procedure
As we intended to use the scale in a clinical setting , participants were German-speaking persons with MS or people with suspected MS who were aged ≥ 18 years, and who had access to the Internet. The necessary sample size for validating questionnaires is contentious . In accordance with our resources and the COSMIN Guidelines (Consensus-based Standards for the selection of health status Measurement Instruments) , we aimed to reach a sample size of 150.
Open recruitment took place from November 2019 to Mai 2020 through newsletters of the MS day hospital at the University Medical Center Hamburg-Eppendorf and as part of regular newsletters of four regional associations from the German Multiple Sclerosis Society (DMSG): Hamburg, Baden-Württemberg, Schleswig-Holstein, and Lower Saxony.
Persons with MS were invited to access an anonymous online survey by clicking on an electronic link. After reading the patient information, and giving informed consent online, patients were asked to fill out the following measures: the eHealth Literacy Questionnaire (eHLQ), eHIQ-part 1 and the General Self-Efficacy Scale (GSE). Afterwards, participants were directed to spend at least 10 minutes browsing either the section ‘living with MS’ of the website of the DMSG Baden-Württemberg, called AMSEL  or the whole website of the DMSG Hamburg . In addition to factual information, the AMSEL website also contained explanatory films about living with MS from patients and health professionals. The DMSG Hamburg website contained only factual information at the time of the study. The websites were chosen to test if there is variation in the items of the eHIQ-G part 2 when rating websites with different types of health information such as facts, figures and personal experiences. After browsing one of the websites, patients had to return to the online survey and answer the eHIQ-part 2, and demographic as well as MS-related questions.
The eHIQ is divided into two parts. The 11-item eHIQ-part 1 must be completed before accessing the website to be evaluated. It consists of two subscales 1) attitudes towards online health information and 2) attitudes towards sharing health experiences online. The 26-item eHIQ-part 2 measures the impact of using a specific health-related website on three subscales: 1) confidence and identification, 2) information and presentation, and 3) understanding and motivation. Response options range from 1 (‘strongly disagree’) to 5 (‘strongly agree’). The eHIQ part 2 must be administered after accessing the website to be evaluated. The scores were converted to a 0-100 metric. The total eHIQ score for part 1 and 2 was calculated as the sum of all subscale scores, divided by the number of subscales. Higher scores correspond with more positive responses [10, 12]. The translation of the eHIQ into German was performed in the context of a medical dissertation . The translation was carried out according to the TRAPD (Translation, Review, Adjudication, Pretesting, and Documentation) team translation model in accordance with the Cross-Cultural Survey Guidelines . Three staff members of the Institute of General Practice Göttingen produced three full translations of the eHIQ . In the review phase, a team of five staff members of the same institute agreed on a single common translation. The translation of the individual questions either corresponded to one of the available translation suggestions or represented a new variant. This new version was submitted for a backward translation to a translator who had not been involved in any of the previous steps and was not familiar with the original English version of the eHIQ. The comparison of the original questionnaire with the backward translation led to further changes in the German translation resulting in a preliminary version of the eHIQ-G. Afterwards, the eHIQ-G was pretested in a convenience sample of 25 participants. The German version of the eHIQ can be found in the dissertation .
The eHLQ is a validated measure of eHealth literacy in English and Danish language covering user interaction with a given eHealth system and the user’s experience of engaging with it . The eHLQ is valuable for the characterization and understanding of digital health literacy in a broad range of target groups. It contains 35 items in seven domains: 1) using technology to process health information, 2) understanding of health concepts and language, 3) ability to actively engage with digital services, 4) feel safe and in control, 5) motivated to engage with digital services, 6) access to digital services that work, and 7) digital services that suit individual needs . Response options for all items range from 1 (strongly disagree) to 4 (strongly agree). The subscale scores are calculated summing up the scores of each item and dividing it by the number of items . We used the eHLQ from the German eHLQ-validation study after back- and forward translation and a qualitative pre-test. The study has not yet been published.
The 10-item GSE scale was developed and validated to assess a general sense of perceived self-efficacy. Responses are made on a 4-point scale from 1 (not at all true) to 4 (exactly true). The total score is calculated by summing up all item scores. The total score ranges from 10 to 40, with a higher score indicating more self-efficacy .
Demographic data such as sex, age, educational level, and highest professional qualification were collected as well as MS-related information, e.g. the disease course, years since diagnosis, and the 9-item-‘Patient Determined Disease Steps’ (PDDS), which asks for the patient-reported walking ability and disability (from 0 = normal to 8 = bedridden) .
The analysis was performed in SPSS (version 25.0; IBM Corp.) and SPSS Amos (version 26.0; IBM Corp.) software. All analyses were carried out on complete cases. For sample description, continuous variables are described using mean and standard deviation (SD), and categorical items are presented as counts and percentages. To examine the internal consistency reliability of the five subscales, Cronbach's alpha (α) was estimated. A Cronbach’s alpha value of > 0.7 was considered adequate .
Confirmatory factor analysis (CFA) was applied to investigate construct validity. The Full Information Maximum Likelihood estimation was used to estimate model parameters and to examine goodness-of-fit of all the CFA models with: the Root Mean Square Error of Approximation [RMSEA] ≤ 0.06, Standardized Root Mean Square [SRMR] ≤ 0.08, Tucker-Lewis-Index [TLI] ≥ 0.95, and Comparative Fit Index [CFI] ≥ 0.95 judged as adequate. Additionally, the minimum discrepancy (chi-square) per degree of freedom [CMIN/DF] ≤ 3 rule was also used [28, 29]. An exploratory factor analysis (EFA) using Oblimin rotation and principal component analysis was run to investigate an alternative to the original structure.
Moreover, convergent validity was assessed by testing hypotheses about expected relationships with eHLQ and GSE by calculating Pearson correlation coefficients. Correlations with instruments measuring related, but dissimilar constructs (eHLQ, GSE) should be 0.30–0.50 . Convergent validity was considered adequate if at least 75% of the correlations were as expected. P values less than 0.05 are interpreted as statistically significant.
Hypothesis 1: Particular subscales of the eHIQ-G correlate with subscales of the eHLQ and with the GSE, which measure related, but dissimilar constructs such as the user's interaction and experience with a given eHealth tool  and the perceived self-efficacy.
The eHIQ-G 1 subscale 1) ‘attitudes towards online health information’ correlates positively with the eHLQ subscales 1), 3), 5), 6), and 7).
The eHIQ-G 1 subscale 2) ‘attitudes towards sharing health experiences online’ correlates positively with the eHLQ subscales 1), 3), 5), 6), and 7).
The eHIQ-G 2 subscale 1) ‘confidence and identification’ correlates positively with the eHLQ subscales 5) and 7).
The eHIQ-G 2 subscale 2) ‘information and presentation’ correlates positively with the eHLQ subscale 2) and 4).
The eHIQ-G 2 subscale 3) ‘understanding and motivation’ correlates positively with the eHLQ subscales 2) and 5) as well as with the GSE score.
The differences of the eHIQ-G according to characteristics of the participants were compared using t test, analysis of variance (ANOVA) and analysis of covariance (ANCOVA) to demonstrate convergent and discriminant validity.
Hypothesis 2: Higher educational levels predict higher scores on the eHIQ part 1 as persons with lower education seek health information online less likely .
Hypothesis 3: Younger persons are more likely to search for health-related information on the Internet . Therefore, younger age predicts higher scores on the eHIQ part 1.
Hypothesis 4: The website of AMSEL, which contains factual and experiential information gets a higher sum index score on eHIQ part 2 than the website of DMSG Hamburg, which shows only factual information.