A reliability and validity study of the electronic health literacy scale among stroke patients in China

ABSTRACT Background Patients with stroke usually use smartphones to obtain online information to maintain their health. But their ability to identify, evaluate and apply this information is still unknown. Aim This study was designed to examine the reliability and validity of the electronic Health Literacy Scale among patients with stroke in China. Design This is a cross-sectional survey. Methods A demographic questionnaire, the electronic Health Literacy Scale (e-HLS) and the eHealth Literacy Scale (eHEALS) were administered to a sample of 648 patients with ischemic stroke recruited from December 2020 to March 2021 in a tertiary hospital. Results The Cronbach’α coefficient on the e-HLS-CHI was 0.907. Kappa consistency coefficient of test-retest reliability was 0.691 (p < .05). Three factors were extracted by Exploratory Factor Analysis (EFA), accounting for 90.84% of the total variance. Confirmatory Factory Analysis (CFA) revealed that three factors of e-HLS-CHI fit well (NFI = 0.979, RFI = 0.955, IFI = 0.987, TLI = 0.972, CFI = 0.987, RMSEA = 0.070, CMIN/DF = 2.586). Good simultaneous validity was suggested by the positive correlation of 0.94 (p < .001) between the e-HLS-CHI and eHEALS. When using eHEALS as the standard, the area under the ROC curve of e-HLS-CHI was 0.896 (95% CI: 0.831–0.960, p < .001). The sensitivity and specificity were 97.8% and 70.4% respectively. Conclusions The e-HLS can be used to evaluate electronic health literacy of patients with stroke in China after translation and cultural adaption.


Introduction
Stroke is an urgent public health problem in China due to the large number of patients and the high possibility of recurrence. 1 According to the Global Burden of Disease (GBD), the age-standardized incidence of stroke in China is higher compared with western countries and other eastern countries (such as the US and Japan), rising from 335.63/ 100,000 in 1990 to 353.70/100,000 in 2017. 2 Patients with stroke are often hospitalized due to stress, insomnia, infection, hypotension, and other causes of recurrent attacks. 3 After being discharged from the hospital, patients with stroke usually prefer to complete post-stroke rehabilitation at home rather than seek professional treatment. 4 The Internet has brought a lot of health information, which is usually used to solve personal health problems. 5,6 Smartphones are the devices most commonly used by patients to access online health information. 7 Although the Internet has been used in stroke prevention and treatment, the related health information provided by the Internet is not completely reliable. Patients with stroke need to have the ability to distinguish online health information. 8 Electronic health literacy refers to the identification, understanding, and evaluation of health information presented on the Internet and using this information to solve health problems. [9][10][11] Comprehensive Model of eHealth Literacy believes that individuals who want to solve health problems must not only have the ability to analyze and apply health information from the Internet but also have the ability to communicate with health service providers. 12,13 Electronic health literacy is easily affected by factors such as sex, education level, selfevaluated health, social support and attitude of online information. [14][15][16] The current instrument used to estimate electronic health literacy is the eHealth Literacy Scale (eHEALS) compiled by Norman. 10 Although the scale is widely used, there are some problems, including unclear scoring parameters and inaccurate judgment of the users' actual level of electronic health literacy. In addition, eHEALS is not fully suitable for assessing the electronic health literacy of Internet users in interactive network. 12,17 Internet users can spread health information to each other. To know whether users can distinguish health information and whether they are willing to consult professionals, Seckin 18 developed the electronic Health Literacy Scale (the e-HLS). The scale is divided into dimensions of action, attitude, and communication, including 19 items. It has been proved to have good reliability and validity. 18 And it has not been used in China.

Aims and objective
The purpose of this study is to translate the e-HLS into a version suitable for patients with stroke in China and to verify the reliability and validity of the translated version.

Design and participants
A cross-sectional survey was conducted among patients with stroke in China. Inclusion criteria were: (1) Diagnosed as ischemic stroke by head CT or MRI 19 ; (2) 18 years old or above; (3) can communicate normally; (4) voluntary consent and to participate in the study. Exclusion criteria were: (1) presence of suffering from serious illness or accompanied with disturbance of consciousness or (2) multiple organ failure or other serious somatic diseases. And our manuscript conforms to the STROBE guidelines. 20

Instruments
Demographic variables: age, sex, marital status, the number of children, education level, per capita wage, place of residence, live with family, self-rated health, frequency of online health information search, attitude to online health information, and online time.
The e-HLS consists of three areas: behavioral literacy (dimension of action), cognitive literacy (dimension of trust), and interactive literacy (dimension of communication). Dimension of action has the most items (13 items), followed by the dimension of trust (4 items) and communication (2 items). Each item is rated as a 5-point Likert scale, from 1 = "never or strongly disagree" to 5 = "always or strongly agree." It is worth mentioning that The four items in the trust dimension are scored in reverse: "Trust the Internet to provide accurate information"; "Think information on the Internet as credible"; "Think information on the Internet as balanced and accurate"; "Think information on the Internet better than what most health providers supply." It means the lower the score on these four items, the higher the electronic health literacy. 710 participants were surveyed by this tool during the course of Seckin's research; 194 of those surveyed constituted a subsample of the elderly. These results showed that the reliability of e-HLS was good. 18 The eHEALS is the first electronic health literacy assessment that estimates the self-perceived skills of Internet users. 10 There are 8 items in the scale and each item is answered using 5-point Likert response alternatives: "very inconsistent," "not consistent," "not clear," "consistent" and "very consistent." The higher the score, the higher the self-perceived electronic health literacy. It has been proved to be used in China. 21 This scale was used as the gold standard in this study to judge the concurrent validity of the e-HLS.

Translation procedure
After obtaining consent from Seckin, we followed Brislin's guide for the translation. 22 Firstly, the English version was translated into Chinese by two graduate students of Nursing. Differences were discussed until the two students reached a consensus to form the first draft of the translation. Secondly, two bilingual teachers (Doctor, Professor) translated the first draft back into English and compared it with the original content. When the two versions were inconsistent, a Nursing Professor translated the divergent items. A Doctor of Nursing sorted out and formed an agreed-upon translation version of the scale. Thirdly, a panel of experts consisting of a nursing Professor, a nursing manager, and a clinical nurse reached a consensus on the wording, clarity, and cultural equivalence of the scale after a face-to-face discussion. Finally, 30 patients with stroke were selected from a tertiary hospital providing medical and health care services to approximately 100 million people in central China to check whether there were any ambiguous or incomprehensible items. The scale was revised according to their feedback results, resulting in the final Chinese version of the e-HLS (e-HLS-CHI).

Ethics
This study was approved by the ethics review committee of the First Affiliated Hospital of Zhengzhou University (Ethical review number:2020-KY-459). All participants were informed of the purpose of the study before data collection. 23

Sample size
According to the advocated standards, the sample size should be at least 5-10 times the number of items on the measuring instrument. And to ensure sufficient samples, 10% of the potential data loss (i.e. missing) can not be ignored. 24 The calculated sample size in this study is not less than 211 due to 19 items of the e-HLS-CHI. We distributed more than 211 questionnaires, of which 648 copies were analyzed and 19 were excluded due to insufficient information.

Data collection
From December 2020 to March 2021, a convenience sample of participants from patients with stroke was recruited in the First Affiliated Hospital of Zhengzhou University. Before questionnaires were distributed by two graduate students of Nursing, the purpose of the study was explained to participants who met the inclusion criteria. Afterward, they were told how to complete the questionnaires. Questionnaires that have been completed were taken back immediately. Two weeks after data collection, a small pilot study was undertaken to retest the questionnaires.

Data analysis
SPSS21.0 and AMOS24.0 were used to process and analyze the collected data. Descriptive analysis and frequency statistics were used to describe the characteristics of the sample and the items.
Validity analysis: the content validity of the e-HLS-CHI was evaluated by the content validity index (CVI), which included I-CVI and S-CVI. S-CVI included S-CVI/UA and S-CVI/Ave. Pearson correlation coefficient analysis was used to perform the correlation between e-HLS-CHI and e-HEALS. Using the e-HEALS as the gold standard, the predictive validity of e-HLS-CHI was tested by the ROC curve. All samples were numbered from 1 to 648 and then randomly divided into 2 groups by "metools" (a random number generator) for Exploratory Factor Analysis (EFA) and Confirmatory Factory Analysis (CFA) (generator).
Reliability analysis: The internal consistency of the e-HLS-CHI was tested by Cronbach's α coefficient and the split-half reliability coefficient. The stability of the scale was examined by the Kappa consistency coefficient.

The sample
Of the 648 patients with stroke, 364 (56.2%) were male and 284 (43.8%) were female. The ages ranged from 29 to 83 years, with a mean age of 58.54 ± 8.53 years. Participants were randomly divided into two groups, each with 324 samples, which are used for Confirmatory Factor Analysis (CFA) and Exploratory Factor Analysis (EFA) respectively. More detailed results appear in Table 1.

Descriptive analysis of e-HLS-CHI
Scores on individual items comprising the e-HLS-CHI ranged from 0 to 5, and the average score on each item was 1.89 (SD = 1.20). As shown in Table 2, item 17 (3.69 ± 0.93) had the highest mean score. Item 4 had the lowest mean score (1.29 ± 0.61).

Reliability
The Cronbach'α coefficient of the e-HLS-CHI was 0.907. The correlation coefficients between individual items and total scale ranged between r = −0.46 and r = 0.90, and the average correlation coefficient was r = 0.56 ( Table 2). The split half reliability coefficient was 0.765. In the small pilot study, 30 participants were selected by "metools" to complete the questionnaires again. At this time, these participants had been discharged and some of them were unwilling to cooperate with the investigation by telephone. So 6 out of 30 participants refused to participate again and the results from 24 participants were available for test-retest reliability analysis. And the Kappa consistency coefficient for test-retest reliability was 0.691 (p < .05).

Content validity
The S-CVI/UA and S-CVI/Ave were 0.630 and 0.940, respectively. Six experts evaluated the I-CVIS of each item, and the values ranged from 0.83 to 1.00.

Constructive validity
Data analysis showed that the Kaiser-Meyer-Olkin (KMO) was 0.850 and Bartlett Test of Sphericity was 15435.63, with a significant difference (p < .01).
After principal component analysis, three factors with eigenvalues greater than 1.00 were extracted. They explained 90.84% of the total variance. Table 3 lists  Figure 1.

Concurrent validity
There was a notable positive correlation between the total score of e-HLS-CHI and the total score of eHEALS (r = 0.94, p < .01). In comparison with the other items, item 17 had the strongest positive correlation with the eHEALS subscale application of electronic health information and service (r = 0.55, p < .01), judging (r = 0.56, p < .01), decision making (r = 0.55, p < .010), and the total score of eHEALS (r = 0.66, p < .01).

Predictive validity
Considering eHEALS as the gold standard, electronic health literacy was divided into higher-level (total score ≥ 20) and lower-level (total score < 20). The predictive validity of e-HLS-CHI was analyzed by the ROC curve. Results the critical point were 32 points, the sensitivity was 97.8%, the specificity was 70.4%, the Youden index was 0.682; the area under the curve was 0.896 (95% CI: 0.831-0.960, p < .01). The results for reliability and validity analysis of the e-HLS-CHI appear in Figure 2.

Discussion
Post-stroke rehabilitation exercises need to be implemented after being discharged from the hospital. Although exercise programs are provided by medical professionals, sometimes they don't seem to be completed. 25 Health literacy is very important for the rehabilitation of patients with stroke. 26 With the rapid development of information technology, the Internet has had a huge impact on stroke rehabilitation. 27 Patients need to judge whether the health information they are exposed to is credible. 28 Therefore, it is necessary to develop assessment measures and to understand the current state of patients' electronic health literacy. 29 In this study, with the agreement of Seckin, the e-HLS was translated to Chinese for the first time and tested in patients with stroke. Our research shows that the Chinese version of the e-HLS has good reliability and validity. The Cronbach'α coefficient of the e-HLS-CHI was 0.907, which exceeds the recommended value (0.70). 30,31 EFA and CFA were used to test the construct validity. The KMO sampling adequacy measure and Bartlett Test of Sphericity were used to determine our data were suitable for EFA. The results showed that the KMO) was 0.850 and the Bartlett Test of Sphericity was 15435.63 (p < .01). When the KMO value of 0.80 or above or the Bartlett Test of Sphericity has significant statistical significance (p < .05), it shows that there is sufficient correlation among the variables and EFA can be performed. 32 Three factors with an eigenvalue greater than 1 were obtained by the principal component method, which explained the cumulative variance was 90.84% of the total variance. The factor loadings of 19 items of e-HLS-CHI were above 0.40. 33 After analysis of the factor loadings, items 1-13, 14-17, and 18-19 aligned with the three dimensions. This finding is consistent with Seckin's findings when he compiled the scale. 18 The item score represents the electronic health literacy level of patients with stroke. The mean score of all items was 1.89. The highest mean score was 3.69 for item 17, followed by item 16 (3.15) or item 15 (3.15). These three items are reverse scored. In our study, the higher the score is, the more participants trusted online health information rather than medical professionals. These results showed that patients with stroke are more willing to believe the online health information. Actually, this result can reflect the current medical situation. In China, the relationship between doctors and patients is quite tense, and there is grave concern regarding trust between patients and medical professionals. 34 In Seckin's study, it showed the item with the highest mean score was item 15 (3.09). 18 We can see that item 15 and item 17 are all in the dimension of trust. This may indicate that even if the cultural background is different, it is common for patients to neglect to verify the authenticity and effectiveness of the website when using online health information. And online health information may easily affect patients. This study has some limitations. First, the sample is from a tertiary hospital in Henan Province, China. Although the hospital provides medical services to 100 million people, it may limit the application of the research results in other regions. Second, this study verified the reliability and validity of the e-HLS in Chinese patients with stroke, which may not apply to other participants, such as with other medical conditions. The method of including or excluding participants also needs to be improved. Some patients with stroke have unilateral dysfunction of eyes and limbs. With the help of a neurologist, we selected participants who were able to operate smartphones and fill out questionnaires despite visual impairment and motor dysfunction. It cannot be ignored that although communication ability was mentioned as one inclusion criterion, we did not evaluate the patient's cognitive ability. In future research, we should use assessment measures to judge the cognitive ability of patients with stroke and then decide whether to include them.

Conclusion
This study is the first to explore whether the e-HLS can be used to measure electronic health literacy and verify the reliability and validity of the scale in Chinese patients with stroke in the context of Web 2.0. These findings may be helpful for clinical nurses in assessing the current status of electronic health literacy in patients with stroke and understanding how they identify, judge and use online health resources, so as to provide a reference for future health promotion programs.