Reliability and Validity of the electronic Health Literacy Scale Among People at High Risk of Stroke in China: A Cross-sectional Study

doi:10.21203/rs.3.rs-512841/v1

Download PDF

Research Article

Reliability and Validity of the electronic Health Literacy Scale Among People at High Risk of Stroke in China: A Cross-sectional Study

https://doi.org/10.21203/rs.3.rs-512841/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 20 Dec, 2021

Read the published version in Topics in Stroke Rehabilitation →

Version 1

posted

You are reading this older preprint version

Read the latest preprint version →

Background: It is of great significance for brain and heart health managers to assess the electronic health literacy of people at high risk of stroke for improving the current situation of stroke in China. Although various measuring instruments have been developed, there is still a lack of suitable tools to match the development of network.

Aim: To examine the reliability and validity of the electronic Health Literacy Scale（e-HLS）among people at high risk of stroke in China, so as to provide appropriate measurement tools for brain heart health managers.

Methods: A demographic questionnaire, the electronic Health Literacy Scale（e-HLS）and the eHealth Literacy Scale (eHEALS) were administered to a sample of 648 people at high risk of stroke recruited from November to December 2020 in a tertiary hospital.

Results: The Cronbach’α coefficient on the e-HLS-CHI was 0.91. Three factors were extracted by Exploratory Factor Analysis (EFA), accounting for 90.84% of the total variance. Confirmatory Factory Analysis (CFA) revealed that three factors of e-HLS-CHI fit well ( NFI = 0.979, RFI = 0.955, IFI = 0.987, TLI = 0.972, CFI=0.987, RMSEA = 0.070, CMIN/DF= 2.586). Good simultaneous validity was suggested by the positive correlation of 0.94 between the e-HLS-CHI and eHEALS. And when using eHEALS as the standard, the area under the ROC curve of e-HLS-CHI was 0.896 (95% CI: 0.831-0.960, P = 0.000). After calculation, the sensitivity and specificity were 97.8% and 70.4% respectively, indicating that it has nice predictive validity.

Conclusions: The e-HLS can be used to evaluate electronic health literacy of people at high risk of stroke in China. This may be helpful for brain and heart health managers to assess the current situation of electronic health literacy in people at high risk of stroke and provide reference for future health promotion programs.

Medical Informatics

Stroke

Instrument development

electronic Health Literacy Scale

Internet

Stroke is the second leading cause of death in the world, characterized by high incidence rate, high recurrence rate, high disability rate and high mortality rate[1]. According to statistics, about one third of the world's new stroke patients are in China, with the incidence rate of stroke increasing rapidly at 8.7% per year [2]. In addition, due to the aging population, high incidence of risk factors and lack of reasonable management, the stroke burden is expected to increase further [3]. It has been shown that stroke can be prevented by controlling the risk factors and improving the health behaviour of people at high risk of stroke[4]. However, a national screening and intervention program found that the proportion of people at high risk of stroke in China increased from 13.05–18.51% within one year[5]. The large number of people at high risk makes stroke prevention a challenge. This program listed the following as characteristics of people at high risk of stroke: a) the presence of three or more of the eight major risk factors, including a history of hypertension/atrial fibrillation or valvular disease/dyslipidemia/diabetes mellitus, smoking, obesity (BMI > 26 kg/m²), low physical activity, and family history of stroke; b) history of Transient Ischemic Attack(TIA); c) history of stroke.

Often the health behaviour of people at high risk of stroke is not satisfactory. A community-based survey showed that people at high risk of stroke were more likely to suffer from malnutrition and cognitive impairment[6]. Although it is known that exercise is good for stroke prevention, many people with history of stroke do not exercise because of limb dysfunction, cost, weather and other reasons[7]. In order to reduce the burden of stroke in China, there is a tremendous need to improve the health behaviour of people at high risk of stroke. A recent study classifying health behaviour among people at high risk of stroke found that electronic health literacy is a predictor of health behaviour, and a positive correlation was found between the scores on electronic health literacy and health behaviour[8]. Therefore, in order to prevent stroke, it is vitally important to understand the electronic health literacy level of people at high risk of stroke in China.

Electronic health literacy refers to the identification, understanding and evaluation of health information presented on the Internet considering the influence of personal or social factors, and using this information to solve health problems[9–11]. With the rapid development of information technology, the Internet has had a huge impact on all aspects of social life, including the medical and health fields. Some government departments, medical and health institutions, and non-profit organizations communicate more and more health information on the Internet. The Internet has gradually become an important resource for the public to obtain health information and seek medical and health care activities [12].

The China Internet Network Information Center (CNNIC) released information that by June 2020, China's online medical users had reached 276 million, accounting for 29.4% of the total number of Internet users, according to the 46th Statistical Report on China's Internet Development[13]. More and more people are gradually beginning to search for health information through electronic resources, as well as to resolve health problems with the information they obtain[14, 15]. Using electronic health information correctly has shown to be good for both physical and mental health[16, 17]. However, most studies have also shown that people generally lack the ability to identify and evaluate this information and the level of electronic health literacy was relatively low[18, 19]. Moreover, due to security problems with network health information, poor health information can damage health. Thus, it is essential to understand and strengthen people's electronic health literacy.

Nursing is an important part of health service. Nurse-led management of risk factors in patients with history of stroke has been found advantageous[20]. “Brain and heart health manager” is a new profession in China and most of these managers are nurses. Nurses received multidisciplinary theoretical knowledge and management skills training, and become brain and heart health managers through examination[21]. One study has shown that health education led by brain and heart health managers can help patients with history of stroke gain more health knowledge, maintain stable blood pressure, and improve self care ability. They have made a huge contribution to the secondary prevention of stroke[22]. If the electronic health literacy of people at high risk of stroke can be evaluated, then brain and heart health managers can consider whether they can promote health behaviour through the intervention of electronic health literacy of people at high risk of stroke, so as to achieve the effect of health management; or consider whether health resources can be disseminated on the Internet to promote the health of people at high risk of stroke.

The instrument used to estimate Chinese electronic health literacy is the eHealth Literacy Scale (eHEALS) compiled by Norman[9]. Although the scale is widely used, there are some problems, including unclear scoring parameters and inaccurate judgment of the users' actual level of electronic health literacy. In addition, eHEALS is not fully adapted to Web 2.0[23, 24]. Considering the digitalization of health care and the wide use of Web 2.0 applications, Seckin[25] has developed the electronic Health Literacy Scale (the e-HLS), with 19 questions, including three dimensions of action, attitude and communication. It has been shown that the scale has good reliability and validity [25]. However, it has not been used in China. The purpose of this study is to translate the e-HLS into a version suitable for people at high risk of stroke in China and to verify the reliability and validity of the translated version.

2.1 DESIGN AND PARTICIPANTS

A cross-sectional survey was conducted among people at high risk of stroke in China. Inclusion criteria were: (1) three or more of the eight major risk factors or a history of TIA or stroke; (2) greater than or equal to 40 years old; (3) can communicate normally; (4) voluntary consent and to participate in the study. Exclusion criteria were: (1) presence of a suffering from serious illness or accompanied with disturbance of consciousness or (2) multiple organ failure or other serious somatic diseases. These persons were excluded because they may not have the ability or energy to complete the entire investigation.

2.2 INSTRUMENTS

Demographic variables: age, gender, marital status, level of education, household income/month, habitation, alcohol drinking status, smoking status, history of TIA /stroke, hypertension, atrial fibrillation or valvular disease, dyslipidemia, diabetes mellitus, obesity (BMI > 26 kg/m²), physical activities, and family history of stroke. Demographic characteristics of the sample are summarized in Table 1.

Table 1

Characteristics of the sample
Variables	Total sample N = 648	EFA N = 324	CFA N = 324
Age in years (mean ± SD)	58.54 ± 8.53	58.50 ± 8.70	58.58 ± 8.37
≥ 40 years old (N/%)	97 (15.0)	50(7.7)	47(7.3)
≥ 50 years old (N/%)	241 (37.2)	120(18.5)	121(18.7)
≥ 60 years old (N/%)	245 (37.8)	122(18.8)	123(19.0)
≥ 70 years old (N/%)	65 (10.0)	33(5.0)	32(5.0)
Gender (F/%)
Male	364(56.2)	188(29.0)	176(27.2)
Female	284(43.8)	136(21.0)	148(22.8)
Marriage (F/%)
Yes	582(89.8)	295(45.5)	287(44.3)
No	66(10.2)	29(4.5)	37(5.7)
Education (F/%)
Elementary school or less	183(28.2)	92(14.2)	91(14.0)
Middle school	268(41.4)	136(20.9)	132(20.5)
High school	139(21.5)	73(11.3)	66(10.2)
Professional education	55(8.5)	21(3.3)	34(5.2)
Undergraduate or more	3(0.4)	2(0.3)	1(0.1)
Habitarion (F/%)
Urban	168(25.9)	83(12.8)	85(13.1)
Rural	480(74.1)	241(37.2)	239(36.9)
Household income/month (F/%)
≤ 3000 RMB	55(8.5)	25(3.8)	31(4.7)
༞3000 and ≤ 5000 RMB	407(62.8)	208(32.0)	199((30.8)
༞5000 and ≤ 10000 RMB	182(28.1)	90((13.9)	92(14.2)
༞10000RMB	4(0.6)	2(0.3)	2(0.3)
History of TIA (F/%)
Yes	34(5.2)	17(2.9)	17(2.9)
No	614(94.8)	307(47.4)	307(47.4)
History of stroke (F/%)
Yes	396(61.1)	187(28.9)	209(32.2)
No	252(38.9)	137(21.1)	115(17.8)
Hypertension (F/%)
Yes	419(64.6)	219(33.8)	200(30.8)
No	229(35.4)	105(16.2)	124(19.2)
Atrial fibrillation or valvular disease (F/%)
Yes	21(3.2)	11(1.7)	10(1.5)
No	627(96.8)	313(48.3)	314(48.5)
Diabetes mellitus (F/%)
Yes	70(10.8)	39(6.0)	31(4.8)
No	578(89.2)	285(44.0)	293(45.2)
Dyslipidaemia (F/%)
Yes	228(35.2)	118(18.2)	110(17.0)
No	420(64.8)	206((31.8)	214(33.0)
Family history of stroke (F/%)
Yes	150(23.1)	82(12.7)	68(10.4)
No	498(76.9)	242(37.3)	256(39.6)
Few physical activities (F/%)
Yes	191(29.5)	99(15.3)	92(14.2)
No	457(70.5)	225(34.7)	232(35.8)
Obesity (BMI > 26 kg/m²) (F/%)
Yes	265(40.9)	140(21.6)	125(19.3)
No	383(59.1)	184(28.4)	199(30.7)
Smoking (F/%)
Yes	211(32.6)	101(15.6)	110(17.0)
No	437(67.4)	223(34.4)	214(33.0)
Alcohol drinking (F/%) (F/%)
Yes	98(15.1)	40(6.2)	58(8.9)
No	550(84.9)	284(43.8)	266(41.1)
Note. SD, standard deviation; F, frequency; %, percentage; EFA, exploratory factor analysis; CFA, confirmatory factor analysis; BMI, body mass index.

The e-HLS has a total of 19 items, including the three dimensions of action, trust and communication. Each item is rated as a 5-point Likert scale, from 1= "never or strongly disagree" to 5= "always or strongly agree". The four items in the trust dimension are scored in reverse: “Trust the Internet to provide accurate information”; “Think information on the Internet as credible”; “Think information on the Internet as balanced and accurate”; “Think information on the Internet better than what most health providers supply”. The lower the score on these four items, the higher the electronic health literacy. Although this scale has not been used by others since it was compiled by the author, 710 participants were surveyed using this measure during the course of the original author's research; 194 of those surveyed constituted a subsample of the elderly. Research results from the original author showed that this scale’s Cronbach's α was 0.93 and the reliability was good[25].

The eHEALS is the first electronic health literacy assessment that estimates the self-perceived skills of Internet users when seeking and applying online health knowledge[9]. There are 8 items in the scale and each item is answered using five Likert response alternatives: "very inconsistent", "not consistent", "not clear", "consistent" and "very consistent", respectively marked as 1, 2, 3, 4 and 5 points. The total score of each respondent is the sum of the scores of each question. The higher the score, the higher the self-perceived electronic health literacy. It has been translated into at least seven languages: Italian[26], Chinese[27], Japanese[28], Spanish[29], German[30], Dutch [31]and Korean[32], making it the most widely used electronic health literacy assessment. This scale was used as the gold standard in this study to judge the concurrent validity of the e-HLS.

2.3 TRANSLATION PROCEDURE

After obtaining the original version of e-HLS from the first author, we followed Brislin's translation guide for the next translation steps [33]. Firstly, the English scale was translated into Chinese by two brain and heart health managers. Then the translated Chinese scales from these two persons were compared with the original scale. The differences were discussed by two nursing postgraduates until they reached a consensus to form the first draft of the translation. Throughout this process, the two translators worked separately.

Secondly, a bilingual teacher (Doctor, Professor) translated the first draft back into English, and compared it with the original content. When the two versions were inconsistent, a nursing expert translated the divergent items. An English teacher sorted out and formed an agreed-upon translation version of the scale.

Thirdly, an expert group composed of two nursing professors, two middle-level professional title workers and two nursing lectures was invited to judge whether the items of the scale reflected the original item contents and whether the items were easy to understand and express clearly to form the Chinese version of e-HLS.

Finally, 8 people at high risk of stroke were selected from a tertiary hospital to investigate their understanding and suggestions on the items, and to check whether there were any ambiguous or incomprehensible items. The scale was revised according to their feedback results, resulting in the final the Chinese version of the e-HLS (e-HLS-CHI). A consensus was reached regarding diction, articulation and cultural equivalence of the measure.

2.4 ETHICS

This study was approved by the ethics review committee of the First Affiliated Hospital of Zhengzhou University. All participants were informed about the study and informed consent was obtained prior to data collection[34].

2.5 SAMPLE SIZE

This study included 27 demographic variables, 19 items of e-HLS and 8 items of eHEALS, with a total of 54 variables. According to the standard advocated[35], the sample size is at least 5–10 times the number of items on a measuring instrument, plus 20% of potential loss of data (i.e. missing) to ensure a sufficient number of people. Finally, the calculated sample size is 648 cases.

2.6 DATA COLLECTION

From November to December 2020, a convenience sample of participants from people at high risk of stroke was recruited in a tertiary hospital in China. Study participants(N = 648) came from a cerebrovascular disease prevention clinic(N = 144), a physical examination center(N = 139), a cardiology department(N = 121), a rehabilitation department(N = 136) and an endocrine department(N = 108). To prevent novel coronavirus pneumonia, researchers wore masks when collecting data, tested body temperature and provided hand washing liquid for participants.

Questionnaires were distributed by two brain and heart health managers. Before questionnaires were issued, the purpose and significance of the study were explained to participants who met the inclusion criteria and gave their consent. After that, they were informed of the methods to complete study the questionnaire and precautions, using unified guidelines. They were asked to complete the questionnaire on their own. In order to ensure participant’s understanding of the content of the e-HLS-CHL, a preliminary survey was conducted on 10 people at high risk of stroke (not included in the the study) before the formal survey, and the questionnaire was revised as needed. The data were collected in a face-to-face manner. All study participants were numbered from 1 to 648. Two weeks after data collection, 30 participants were selected by “metools”(a random number generator) to complete the questionnaire a second time[36]. However, because six of the 30 participants had no contact information, the results from only 24 participants were available for test-retest reliability analysis.

2.7 DATA ANALYSES

SPSS24.0 and AMOS24.0 were used to process and analyze the collected data. Descriptive analysis and frequency statistics were used to describe characteristics of the sample and the items.

Validity analysis: the content validity of the e-HLS-CHI was evaluated by the content validity index (CVI), which included I-CVI and S-CVI. S-CVI included S-CVI / UA and S-CVI / Ave. Construct validity of the scale was evaluated by Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA). Pearson correlation coefficient analysis was performed between e-HLS-CHI and e-HEALS to examine the correlation with the e-HEALS as the criterion. Based on the e-HEALS score, the study participants were divided into two groups: total score ≥ 20 and total score < 20. Using the e-HEALS as the gold standard, the predictive validity of e-HLS-CHI was tested by the ROC curve.

Reliability analysis: The internal consistency of the e-HLS-CHI was tested by Cronbach's α coefficient and the split-half reliability coefficient. The stability of the scale was examined by the Kappa consistency coefficient.

3.1 THE SAMPLE

Of the 648 people at high risk of stroke, 364 (56.2%) were male and 284(43.8%) were female. The ages ranged from 40 to 83 years, with a mean age of 58.54 ± 8.53 years. Participants who had a spouse numbered 582(89.8%) and 66 (10.2%) had no spouse. The number of people with middle school education was the largest (41.4%). More detailed results appear in Table 1.

3.2 DESCRIPTIVE ANALYSIS Of e-HLS-CHI

Scores on individual items comprising the e-HLS-CHI ranged from 0 to 5, and the average score on each item was 1.89 (SD = 1.20). As shown in Table 2, the item “Think information on the Internet better than what most health providers supply”(3.69 ± 0.93) had the highest mean score. The item “Check who sponsors the website” had the lowest mean score (1.29 ± 0.61).

Table 2

Mean score and reliability analysis of e-HLS-CHI (N = 648)
Items	M	SD	Item-to-total correlation coefficient (r)	Cronbach's α if item deleted
Q1-Read disclosure statements	1.29	0.61	0.84	0.74
Q2-Check credentials and affiliations of author	1.32	0.71	0.82	0.74
Q3- Check who owns the website	1.28	0.58	0.85	0.74
Q4- Check who sponsors the website	1.27	0.58	0.84	0.74
Q5- Check if there is a financial tie between information and sponsor	1.29	0.60	0.85	0.74
Q6- Appraise whether information provider’s credentials seem adequate	1.35	0.72	0.85	0.74
Q7- Check whether an address is listed on the website	1.40	0.79	0.85	0.74
Q8- Check whether goals and objectives of the website are clearly stated	1.47	0.88	0.88	0.73
Q9- Appraise whether there is a clear and comprehensive coverage of the topic	1.51	0.95	0.90	0.73
Q10- Check whether other print or Web resources confirm the information	1.54	0.98	0.90	0.73
Q11- Check whether information is current and updated recently	1.55	0.97	0.89	0.73
Q12- Check whether the last update of information is prominent on the website	1.55	0.97	0.89	0.73
Q13- Confident of being able to appraise information quality on the Internet	1.41	0.80	0.87	0.74
Q14- Trust the Internet to provide accurate information	3.15	1.80	-0.46	0.76
Q15- Think information on the Internet as credible	3.15	1.09	-0.47	0.76
Q16- Think information on the Internet as balanced and accurate	3.15	0.08	-0.51	0.76
Q17- Think information on the Internet better than what most health providers supply	3.69	0.93	-0.67	0.75
Q18- Discuss the information with a health provider	2.24	1.26	0.74	0.73
Q19- Ask a health provider where to find credible information on the Internet	2.23	1.26	0.74	0.73
Overall scale	1.89	1.20	0.56	0.91
Note. e-HLS-CHI: the Chinese version of the e-HLS ; Q1-Q19:Item1-Item19; Q14-Q17 are scored in reverse.

3.3 RELIABILITY

The Cronbach’α coefficient of the e-HLS-CHI was 0.907, and deleting any item would not improve the Cronbach's α of the scale. The correlation coefficients between individual items and total scale ranged between r = -0.46 and r = 0.90, and the average correlation coefficient was r = 0.56 (Table 2). The split half reliability coefficient was 0.765. As shown in Table 3, the Kappa consistency coefficient for test-retest reliability was 0.691(p < 0.05).

Table 3

**Symmetric measures**
	Value	Asymp. Sta. Error^a	Approx. T^b	Approx Sig.
Measure of Agreement Kappa	.691	.097	14.140	.000
N of Valid Cases	24
a: Not assuming the null hypothesis.
b: Using the asymptotic standard error assuming the null hypothesis.

3.4 VALIDITY

3.4.1 CONTENT VALIDITY

The S-CVI/UA and S-CVI/Ave were 0.630 and 0.940, respectively. Six experts (two doctor's degree, intermediate title; two master's degree, senior title; two masters, lecturers) evaluated the I-CVIS of each item, and the values ranged from 0.83 to 1.00, indicating that the items of e-HLS-CHI had good content validity.

3.4.2 CONSTRUCT VALIDITY

Data analysis showed that the Kaiser-Meyer-Olkin (KMO) was 0.850, and Bartlett Test of Sphericity was 15435.63, with significant difference (p < 0.01), indicating that the e-HLS was suitable for factor analysis. After principal component analysis, three factors with eigenvalues greater than 1.00 were extracted. They explained 90.84% of the total variance. Table 4 lists the factor loadings and the values of communality. As presented in Table 5, the CFA revealed the following measurements of the structural equation model of the e-HLS: CMIN/DF = 2.586, NFI = 0.979, RFI = 0.955, IFI = 0.987, TLI = 0.972, CFI = 0.987, and RMSEA = 0.070. The standardized regression coefficients and the structural equation model of three-factor of e-HLS appear in Fig. 1.

Table 4

Factor loading of e-HLS-CHI items from factor analysis (N = 324).
Items		Factor 2
Q1-Read disclosure statements	.858	.258	.290	.887
Q2-Check credentials and affiliations of author	.846	.256	.302	.873
Q3- Check who owns the website	.895	.256	.325	.971
Q4- Check who sponsors the website	.898	.226	.301	.948
Q5- Check if there is a financial tie between information and sponsor	.901	.223	.262	.930
Q6- Appraise whether information provider’s credentials seem adequate	.916	.167	.178	.898
Q7- Check whether an address is listed on the website	.930	.101	.001	.875
Q8- Check whether goals and objectives of the website are clearly stated	.931	.099	.131	.893
Q9- Appraise whether there is a clear and comprehensive coverage of the topic	.944	.084	.201	.939
Q10- Check whether other print or Web resources confirm the information	.933	.085	.241	.936
Q11- Check whether information is current and updated recently	.939	.067	.224	.936
Q12- Check whether the last update of information is prominent on the website	.931	.072	.220	.921
Q13- Confident of being able to appraise information quality on the Internet	.911	.102	.140	.860
Q14- Trust the Internet to provide accurate information	.480	.851	.117	.968
Q15- Think information on the Internet as credible	.475	.858	.113	.976
Q16- Think information on the Internet as balanced and accurate	.473	.857	.124	.974
Q17- Think information on the Internet better than what most health providers supply	.287	.829	.175	.800
Q18- Discuss the information with a health provider	.378	.205	.806	.835
Q19- Ask a health provider where to find credible information on the Internet	.368	.204	.814	.840
Note. e-HLS-CHI: the Chinese version of the e-HLS; Q1-Q19:Item1-item19.

Table 5

The e-HLS-CHI model fit measure (n = 324)
Structure factor model	χ2	df	CMIN/DF(χ2/df)	RMSEA	NFI	RFI	IFI	TLI	CFI
three-factor model	204.315	79	2.586	0.070	0.979	0.955	0.987	0.972	0.987
Note. e-HLS-CHI: the Chinese version of the e-HLS; χ2/df, chi-square/degree of freedom; RMSEA, root mean square error of approximation; NFI, normal fit index; RFI, relative fit index;IFI,incremental fit index; TLI, Tucker Lewis index; CFI, comparative fit index.

3.4.2 CONCURRENT VALIDITY

As shown in Table 6, there was a notable positive correlation between the total score of e-HLS-CHI and the total score of eHEALS (r = 0.94, p < 0.01). In comparison with the other items, the 17th item had the strongest positive correlation with the eHEALS subscale application of electronic health information and service(r = 0.55, p < 0.01), judging(r = 0.56, p < 0.01), decision making(r = 0.55, p < 0.010), and the total score of eHEALS(r = 0.66, p < 0.01).

Table 6

Correlation of e-HLS-CHI with eHEALS (n = 648).
Dimensionality	Application of ehealth information and service	Judging	Decision making	eHEALS
Q1-Read disclosure statements	0.35	0.39	0.37	0.41
Q2-Check credentials and affiliations of author	0.37	0.43	0.42	0.45
Q3- Check who owns the website	0.28	0.45	0.32	0.48
Q4- Check who sponsors the website	0.36	0.48	0.39	0.51
Q5- Check if there is a financial tie between information and sponsor	0.45	0.46	0.47	0.49
Q6- Appraise whether information provider’s credentials seem adequate	0.44	0.41	0.43	0.48
Q7- Check whether an address is listed on the website	0.51	0.38	0.42	0.54
Q8- Check whether goals and objectives of the website are clearly stated	0.46	0.42	0.43	0.51
Q9- Appraise whether there is a clear and comprehensive coverage of the topic	0.47	0.47	0.37	0.49
Q10- Check whether other print or Web resources confirm the information	0.39	0.46	0.43	0.48
Q11- Check whether information is current and updated recently	0.41	0.48	0.41	0.53
Q12- Check whether the last update of information is prominent on the website	0.39	0.37	0.38	0.45
Q13- Confident of being able to appraise information quality on the Internet	0.38	0.41	0.39	0.47
Q14- Trust the Internet to provide accurate information	0.52	0.58	0.55	0.61
Q15- Think information on the Internet as credible	0.48	0.54	0.49	0.58
Q16- Think information on the Internet as balanced and accurate	0.54	0.57	0.53	0.62
Q17- Think information on the Internet better than what most health providers supply	0.55	0.56	0.55	0.66
Q18- Discuss the information with a health provider	0.45	0.53	0.52	0.59
Q19- Ask a health provider where to find credible information on the Internet	0.47	0.54	0.53	0.57
e-HLS-CHI	0.48	0.68	0.53	0.94
eHEALS, application of ehealth information and service (item 1,2,3,4 and 5), judging (items 6 and 7), decision making (item 8).

3.4.3 PREDICTIVE VALIDITY

Considering eHEALS as the gold standard, electronic health literacy was divided into higher level (total score ≥ 20) and lower level(total score < 20). The predictive validity of e-HLS-CHI was analyzed by ROC curve. Results the critical point were 32 points, the sensitivity was 97.8%, the specificity was 70.4%, the Youden index was 0.682, as shown in Table 7; the area under the curve was 0.896 (95% CI: 0.831–0.960, p < 0.01). The ROC curve appears in Fig. 2.

Table 7

Sensitivity and specificity for e-HLS-CHI with eHEALS as a criterion (n = 648).
Cut-off points	Sensitivity (%)	Specificity (%)	Youden's index
22.00	1.000	0.000	0.000
24.00	1.000	0.019	0.019
25.50	0.978	0.037	0.015
26.50	0.978	0.093	0.071
27.5	0.978	0.241	0.219
28.5	0.978	0.259	0.238
29.5	0.978	0.500	0.478
30.5	0.978	0.574	0.552
31.5	0.978	0.704	0.682
32.5	0.913	0.741	0.654
33.5	0.891	0.778	0.669
34.5	0.848	0.778	0.626
35.5	0.739	0.870	0.610
36.5	0.696	0.870	0.566
37.5	0.652	0.889	0.541
39.0	0.630	0.889	0.519
40.5	0.609	0.907	0.516
41.5	0.609	0.926	0.535
43.0	0.565	0.944	0.510
45.0	0.500	0.944	0.444
48.0	0.478	0.944	0.423
49.5	0.413	0.944	0.357
50.5	0.413	0.981	0.395
51.5	0.391	0.981	0.373
52.5	0.326	0.981	0.308
53.5	0.304	0.981	0.286
54.5	0.261	0.981	0.242
56.5	0.217	0.981	0.199
59.0	0.196	0.981	0.177
60.5	0.174	0.981	0.155
61.5	0.174	1.000	0.174
62.5	0.152	1.000	0.152
63.5	0.130	1.000	0.130
64.5	0.109	1.000	0.109
65.5	0.043	1.000	0.043
68.5	0.022	1.000	0.022
72.0	0.000	1.000	0.000
Note. e-HLS-CHI, the Chinese version of the e-HLS

With the development of Internet technology, online health information is no longer limited to professional health websites, but accompanied by social media into the Internet consumers' daily information behaviour and practice. Thus consumers need to judge whether the health information they are exposed to is credible[37]. Therefore, it is necessary to develop assessment measures and to understand the current state of people's electronic health literacy [38].

With the agreement of Profession Seckin, the e-HLS was translated and adjusted in Chinese for the first time and its reliability and validity were tested in people at high risk of stroke. The results showed that the scale has good reliability and validity.

In this study, the mean score of all items was 1.89. The highest mean score was 3.69 for the item “think information on the Internet better than what most health providers supply”, followed by two items “think information on the Internet as balanced and accurate” (3.15) or “think information on the Internet as credible” (3.15). These three items are reverse scored. The higher the score is, the more participants trusted online health information rather than medical professionals, and the worse electronic health literacy. These results showed that people at high risk of stroke are more willing to believe the health information they see on the Internet. Actually, this result can be explained by the current medical situation. In China, the relationship between doctors and patients is quite tense, and there is grave concern regarding trust between patients and medical professionals. So many people choose to believe the health information they find on the Internet[39]. The lowest mean score was 1.27 for the item “check who sponsors the website”. In the results, Seckin showed the item with the highest mean score was “think information on the Internet as credible” (3.09), and the lowest was “check whether an address is listed on the website” (1.96). Interestingly, in comparing our results with those of the original study, we can see that the highest score are all in the dimension of trust and the lowest are in the dimension of action. This may indicate that even if the cultural background is different, it is common for people to neglect to verify the authenticity and effectiveness of the website when using Internet health information.

The item-to-total correlations ranged from − 0.46 to 0.90 (p < 0.01). Item 14 (“trust the Internet to provide accurate information” ) had the weakest correlation with the total scale, while items 9 (“appraise whether there is a clear and comprehensive coverage of the topic”) and 10 (“check whether other print or Web resources confirm the information”) had the strongest correlation. In Seckin’s research, item 14 also has the weakest correlation with the total scale, and item 6 (“appraise whether information provider’s credentials seem adequate”) has the strongest correlation.

The Cronbach’α coefficient of the e-HLS-CHI was 0.907, which exceeds the recommended value(0.70)[40, 41]. In addition the Cronbach's α’s if items deleted were all greater than 0.7. Although it is lower than the coefficient of the original scale(0.93), it still shows that e-HLS-CHI has high internal consistency. This phenomenon may be due to the differences across samples between this study and other studies. Previous studies have shown that the Cronbach's α of eHEALS ranged from 0.88 to 0.91[42, 43]. Thus, it is shown that the e-HLS-CHI had good reliability when applied to people at high risk of stroke.

In the current study, the I-CVIs were between 0.83 and 1.00. The S-CVI/UA was found to be 0.63 and the S-CVI/Ave was 0.94. The existing literature has shown that an I-CVI > 0.83, an S-CVI/UA > 0.40, and an S-CVI/Ave > 0.90 indicate good content validity[44]. As a result, the e-HLS-CHI demonstrated good content validity.

Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) were used to test the construct validity. Firstly, the Kaiser-Meyer-Olkin (KMO) sampling adequacy measure and Bartlett Test of Sphericity (BTS) were used to determine our data were suitable for EFA. The results showed that the Kaiser-Meyer-Olkin (KMO) was 0.850 and Bartlett Test of Sphericity was 15435.63(p < 0.01). When the Kaiser-Meyer-Olkin (KMO) value is above 0.50, EFA can be carried out, and the Kaiser-Meyer-Olkin (KMO) value of 0.80 or above is considered to be very suitable for EFA. When the Bartlett Test of Sphericity (BTS) has significant statistical significance (p < 0.05), it shows that there is sufficient correlation among the variables, and EFA can be performed[45]. Then the principal component method was used to extract three potential dimensions, three factors with eigenvalue greater than 1 were obtained, which explained the cumulative variance was 90.84% of the total variance. The factor loadings of 19 items of e-HLS-CHI were above 0.40[46]. After analysis of the factor loadings, items 1–13, 14–17 and 18–19 aligned with the three dimensions. This finding is consistent with Seckin’s findings when he compiled the scale. After determining the scale structure, we conducted a CFA with the items of the scale. Generally, the CFI and NFI values should be close to 1, and an RMSEA index less than 0.10 is considered to be a good fitting model [47]. Our examination of model fits showed that the e-HLS-CHI had good validity in people at high risk of stroke in China.

There were substantial correlations between the total e-HLS-CHI score with total eHEALS score, and individual e-HLS-CHI items with eHEALS subscales, which suggests good concurrent validity analysis in the expected direction. The area under the curve of e-HLS-CHI was 0.896. Some authors believe that the area under the ROC curve should be between 0.5 and 1.0, preferably close to 1.0 [48, 49]. The satisfactory result suggests our work is highly important.

In conclusion, we translated a new English version of the electronic health literacy scale into a Chinese version and modified some sentences according to the cultural background to be understood by the participants. Then we conducted a questionnaire survey among 648 people at high risk of stroke, and analyzed the reliability and validity of the scale. Finally, we obtained meaningful results.

4.1 LIMITATIONS

This study has some limitations. First, the sample is from a tertiary hospital in Henan Province, China. Although this hospital processes the largest area and the most patients in China, it may still limit the application of the research results in other regions. In future research, we need to expand the geographical scope and select samples from different places. Second, a face-to-face survey was used in this study. This may have affected the real response of people at high risk of stroke to certain items, and another method of data collection may be needed in the future[50]. Finally, this study only verified the reliability and validity of the e-HLS in Chinese people at high risk of stroke, which may not be applicable to other types of study participants, such as with other medical conditions.

A large number of nurses are trained to become qualified brain and heart health managers every year. Smart phones have gradually become a source of patient health information. This study explored whether the e-HLS scale can be used to measure electronic health literacy and verified the reliability and validity of the scale in Chinese people at high risk of stroke in the context of Web 2.0. These findings may be helpful for brain and heart health managers in assessing the current status of electronic health literacy in people at high risk of stroke, and to understand how they identify, judge, and use online health resources, so as to provide reference for future health promotion programs. This provides clinical nurses with a new measuring tool that keeps pace with the times. At the same time, this may also be a new starting point for primary stroke prevention.

BMI: Body Mass Index

BTS: the Bartlett Test of Sphericity

CFA: Confirmatory Factory Analysis

CFI: comparative Fit Index

CI: Confidence Interval

CMIN/DF: Chi-square/Degree of Freedom

CNNIC: the China Internet Network Information Center

CVI: Content Validity Index

EFA: Exploratory Factor Analysis

eHEALS: the eHealth Literacy Scale

e-HlS: the electronic Health Literacy Scale

e-HLS-CHI: Chinese Version of the electronic Health Literacy Scale

I-CVI: Item-level Content Validity Index

IFI: Incremental Fit Index

KMO: Kaiser-Meyer-Olkin

NFI: Normative Fit Index

RFI: Relative Fit Index

RMSEA: Root Mean Square Error of Approximation

ROC: Receiver Operating Characteristic

S-CVI: Scale-level Content Validity Index

S-CVI/Ave: S-CVI, Average

S-CVI/UA: S-CVI, Universal agreement

TIA: Transient Ischemic Attack

TLI: Tucker-Lewis Index

Ethics approval and consent to participate

This study was approved by the ethics review committee of the First Affiliated Hospital of Zhengzhou University (Approval NO. 2020-KY-459). Written informed consent was obtained from each participant.

Consent for publication

Not applicable.

Availability of data and material

Data supporting the research results can be obtained from Figshare without any administrative permissions (https://doi.org/10.6084/m9.figshare.14298845 ). If anyone wants to obtain other relevant data, please contact the first author YH ([email protected]).

Conflict of interest statement

None of the authors of this study had a conflict of interest.

Funding

This study is funded by Medical Science and Technology Project of Henan Province (Grant No. SBGJ2018003) and Chinese Nursing Association (Grant No. ZHKY202025). The funders only provides funds for the research and does not participate in the research process.

Authors' contributions

YH completed this research and manuscript. LNG helped YH design this study. JA. Z corrected errors and grammar for this manuscript. YJL helped YH calculate and select samples. MW, GGZ and XYL helped YH collect the data. All authors have reviewed the manuscript.

Acknowledgements

We are grateful for all the participants, especially Profession Seckin.

[1] B.C.V. Campbell, D.A. De Silva, M.R. Macleod, S.B. Coutts, L.H. Schwamm, S.M. Davis, G.A. Donnan, Ischaemic stroke, Nat Rev Dis Primers 5(1) (2019) 70.

[2] Q. Bi, L. Wang, X. Li, Z. Song, Risk factors and treatment of stroke in Chinese young adults, Neurol. Res. 32(4) (2010) 366-70.

[3] S. Wu, B. Wu, M. Liu, Z. Chen, W. Wang, C.S. Anderson, P. Sandercock, Y. Wang, Y. Huang, L. Cui, C. Pu, J. Jia, T. Zhang, X. Liu, S. Zhang, P. Xie, D. Fan, X. Ji, K.L. Wong, L. Wang, Stroke in China: advances and challenges in epidemiology, prevention, and management, Lancet Neurol. 18(4) (2019) 394-405.

[4] L.H. Wan, J. Zhao, X.P. Zhang, S.F. Deng, L. Li, S.Z. He, H.F. Ruan, Stroke prevention knowledge and prestroke health behaviors among hypertensive stroke patients in mainland China, J. Cardiovasc. Nurs. 29(2) (2014) E1-9.

[5] L. Wang, L.M. Liu, Y. Yang, B. Peng, Y.L. Wang, The Prevention and Treatment of Stroke Still Face Huge Challenges——Brief Report on Stroke Prevention and Treatment in China, Chinese Circulation Journal 34(02) (2019) 105-119.

[6] N.N. Zhao, K.X. Zeng, Y.L. Wang, P.J. Sheng, C.Z. Tang, P. Xiao, X.W. Liu, Research on the nutrition and cognition of high-risk stroke groups in community and the relevant factors, Eur. Rev. Med. Pharmacol. Sci. 21(23) (2017) 5408-5414.

[7] E. Preston, C.M. Dean, L. Ada, R. Stanton, S. Brauer, S. Kuys, G. Waddington, Promoting physical activity after stroke via self-management: a feasibility study, Top. Stroke Rehabil. 24(5) (2017) 353-360.

[8] L. Guo, Y. Liu, Y. Zhu, M. Wei, Identification of health behaviour clusters among people at high risk of stroke: A latent class profile analysis, J. Adv. Nurs. 76(11) (2020) 3039-3047.

[9] C.D. Norman, H.A. Skinner, eHEALS: The eHealth Literacy Scale, J. Med. Internet Res. 8(4) (2006) e27.

[10] S.R. Paige, M. Stellefson, J.L. Krieger, C. Anderson-Lewis, J. Cheong, C. Stopka, Proposing a Transactional Model of eHealth Literacy: Concept Analysis, J. Med. Internet Res. 20(10) (2018) e10175.

[11] L. Griebel, H. Enwald, H. Gilstad, A.L. Pohl, J. Moreland, M. Sedlmayr, eHealth literacy research-Quo vadis?, Inform Health Soc Care 43(4) (2018) 427-442.

[12] D. Becker, J. Grapendorf, H. Greving, K. Sassenbeerg, Perceived threat and internet use predict intentions to get bowel cancer screening（colonoscopy）：longitudinal questionnaire study, J. Med. Internet Res. 20(2) (2018) e46.

[13] CNNIC released the 46th《 Statistical Report on The Development of Internet in China》, China Internet (10) (2020) 6-7.

[14] T.H. Wang, X.F. Zhou, Y. Ni, Z.G. Pan, Health information needs regarding diabetes mellitus in China: an internet-based analysis, BMC Public Health 20(1) (2020) 990.

[15] H. Guo, J. Li, T. Dai, Consumer Health Information Needs and Question Classification: Analysis of Hypertension Related Questions Asked by Consumers on a Chinese Health Website, Stud. Health Technol. Inform. 216 (2015) 810-4.

[16] A. Yogarajah, R. Kenter, Y. Lamo, V. Kaldo, T. Nordgreen, Internet-delivered mental health treatment systems in Scandinavia - A usability evaluation, Internet Interv 20 (2020) 100314.

[17] N. Smith, S. Liu, A systematic review of the dose-response relationship between usage and outcomes of online physical activity weight-loss interventions, Internet Interv 22 (2020) 100344.

[18] Z.D. Xu, S. Zhang, J. Geng, J. Li, The association between eHealth literacy and health promoting lifestyle in high risk population of type 2 diabetes, Chinese Journal of Nursing Education 17(09) (2020) 849-853.

[19] Y. Zhao, H. Chen, C. Zou, H.L. Gong, Y.J. Wu, Analysis on status and influencing factor of e-health literacy among outpatients, Modern Preventive Medicine 46(06) (2019) 1070-1073+1078.

[20] A.M. Mendyk, A. Duhamel, Y. Bejot, D. Leys, L. Derex, O. Dereeper, O. Detante, P.Y. Garcia, O. Godefroy, F.M. Montoro, J.P. Neau, S. Richard, T. Rosolacci, I. Sibon, D. Sablot, S. Timsit, M. Zuber, C. Cordonnier, R. Bordet, Controlled Education of patients after Stroke (CEOPS)- nurse-led multimodal and long-term interventional program involving a patient's caregiver to optimize secondary prevention of stroke: study protocol for a randomized controlled trial, Trials 19(1) (2018) 137.

[21] C. Yang, Y. Liu, Y. Guo, X. Dong, A. Wang, K. Ma, S. Liu, S. Pang, Brain-heart health care manager-led whole-process health management for ischemic stroke patients, Journal of Nursing Science 35(21) (2020) 36-39.

[22] S. Wang, Y. Li, J. Tian, X. Peng, L. Yi, C. Du, C. Feng, C. Liu, R. Deng, X. Liang, A randomized controlled trial of brain and heart health manager-led mHealth secondary stroke prevention, Cardiovasc Diagn Ther 10(5) (2020) 1192-1199.

[23] S. Quinn, R. Bond, C. Nugent, Quantifying health literacy and ehealth literacy using existing instruments and browser-based software for tracking online health information seeking behavior, Comput Hum Behav 69 (2017) 256-267.

[24] C. Norman, eHealth literacy 2.0: problems and opportunities with an evolving concept, J. Med. Internet Res. 13(4) (2011) e125.

[25] G. Seçkin, D. Yeatts, S. Hughes, C. Hudson, V. Bell, Being an Informed Consumer of Health Information and Assessment of Electronic Health Literacy in a National Sample of Internet Users: Validity and Reliability of the e-HLS Instrument, J. Med. Internet Res. 18(7) (2016) e161.

[26] N. Diviani, A.L. Dima, P.J. Schulz, A Psychometric Analysis of the Italian Version of the eHealth Literacy Scale Using Item Response and Classical Test Theory Methods, J. Med. Internet Res. 19(4) (2017) e114.

[27] M. Koo, C. Norman, H. Chang, Psychometric evaluation of a Chinese version of the eHealth Literacy Scale （eHEALS）in school age children, Int Electron J Health Educ 15 (2012) 29-32.

[28] S. Mitsutake, A. Shibata, K. Ishii, K. Oka, Associations of eHealth Literacy With Health Behavior Among Adult Internet Users, J. Med. Internet Res. 18(7) (2016) e192.

[29] G. Paramio Pérez, B.J. Almagro, Á. Hernando Gómez, J.I. Aguaded Gómez, Validation of the eHealth Literacy Scale (eHEALS) in Spanish University Students, Rev. Esp. Salud Publica 89(3) (2015) 329-38.

[30] R. Soellner, S. Huber, M. Reder, The concept of ehealth literacy and its measurement：German translation of the eHEALS, J Media Psychol 26(1) (2014) 29-38.

[31] R. van der Vaart, A.J. van Deursen, C.H. Drossaert, E. Taal, J.A. van Dijk, M.A. van de Laar, Does the eHealth Literacy Scale (eHEALS) measure what it intends to measure? Validation of a Dutch version of the eHEALS in two adult populations, J. Med. Internet Res. 13(4) (2011) e86.

[32] S. Chung, B.K. Park, E.S. Nahm, The Korean eHealth Literacy Scale (K-eHEALS): Reliability and Validity Testing in Younger Adults Recruited Online, J. Med. Internet Res. 20(4) (2018) e138.

[33] R.W. Brislin, Back-translation for cross-cultural research. Journal of Cross-Cultural

Psychology, 1(3) (1970) 185-216.

[34] W.M. Association, Declaration of Helsinki: ethical principles for medical research involving human subjects, JAMA 14(1) (2013) 233-238.

[35] The Practice of Nursing Research: Appraisal, Synthesis, and Generation of Evidence - Seventh edition Grove Susan K The Practice of Nursing Research: Appraisal, Synthesis, and Generation of Evidence - Seventh edition 752pp Elsevier 9781455707362 1455707368 [Formula: see text], Nurs. Stand. 27(31) (2013) 30.

[36] O.r.n. generator, http://www.metools.info/other/o43.html.

[37] D. Westerman, P.R. Spence, B.V.D. Heide, social media as information source: recency of updates and credibility of information, Journal of Computer-Mediated Communication 19 (2014).

[38] H. Kim, B. Xie, Health literacy in the eHealth era: A systematic review of the literature, Patient Educ. Couns. 100(6) (2017) 1073-1082.

[39] J.B. Nie, L. Li, G. Gillett, J.D. Tucker, A. Kleinman, The crisis of patient-physician trust and bioethics: lessons and inspirations from China, Dev. World Bioeth. 18(1) (2018) 56-64.

[40] H. Gao, U. Söderhamn, L. Zhang, H.X. Cui, K. Liu, Reliability and validity of the Chinese version of the Nutritional Form For the Elderly, Public Health Nutr. 18(14) (2015) 2559-64.

[41] L.N. Guo, J.A. Zauszniewski, X.F. Ding, Y.C. Liu, L.J. Huang, Y.J. Liu, Psychometric Assessment of the Depressive Cognition Scale Among Older Chinese People, Arch. Psychiatr. Nurs. 31(5) (2017) 477-482.

[42] M. Stellefson, S.R. Paige, B. Tennant, J.M. Alber, B.H. Chaney, D. Chaney, S. Grossman, Reliability and Validity of the Telephone-Based eHealth Literacy Scale Among Older Adults: Cross-Sectional Survey, J. Med. Internet Res. 19(10) (2017) e362.

[43] L.G. Fernandes, B.T. Saragiotto, Clinimetrics: eHealth Literacy Scale, J. Physiother. 67(1) (2021) 67.

[44] D.F. Polit, C.T. Beck, The content validity index: are you sure you know what's being reported? Critique and recommendations, Res. Nurs. Health 29(5) (2006) 489-97.

[45] A.F. Nunes, P.L. Monteiro, A.S. Nunes, Factor structure of the convergence insufficiency symptom survey questionnaire, PLoS One 15(2) (2020) e0229511.

[46] T. Konishi, Principal component analysis for designed experiments, BMC Bioinformatics 16 Suppl 18(Suppl 18) (2015) S7.

[47] S.G. Ravyts, J.M. Dzierzewski, E. Perez, E.K. Donovan, N.D. Dautovich, Sleep Health as Measured by RU SATED: A Psychometric Evaluation, Behav. Sleep Med. 19(1) (2021) 48-56.

[48] Z.H. Hoo, J. Candlish, D. Teare, What is an ROC curve?, Emerg. Med. J. 34(6) (2017) 357-359.

[49] L. Guo, M.C. Jones, Y. Liu, S. Yv, Y. Zhu, Y. Guo, Cross-cultural validation of the Student Nurse Stress Index Scale: A descriptive survey targeting student nurses in China, J. Affect. Disord. 251 (2019) 31-38.

[50] A.R. Greenleaf, A. Gadiaga, G. Guiella, S. Turke, N. Battle, S. Ahmed, C. Moreau, Comparability of modern contraceptive use estimates between a face-to-face survey and a cellphone survey among women in Burkina Faso, PLoS One 15(5) (2020) e0231819.

No competing interests reported.

Download PDF

Journal Publication

published 20 Dec, 2021

Read the published version in Topics in Stroke Rehabilitation →

Version 1

posted

You are reading this older preprint version

Read the latest preprint version →

Reliability and Validity of the electronic Health Literacy Scale Among People at High Risk of Stroke in China: A Cross-sectional Study

Status:

Journal Publication

Version 1

Abstract

Figures

1 Background

2 Methods

3 Results

4 Discussion

5 Conclusions

Abbreviations

Declarations

References

Additional Declarations

Status:

Journal Publication

Version 1