DOI: https://doi.org/10.21203/rs.3.rs-125631/v1
Background: Patient-centred healthcare is important, and patient experiences are considered a central pillar of quality in healthcare. The Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS®) is an instrument used to quantitatively evaluate patient perspectives. This study's objective was to conduct the translation and adaptation of the HCAHPS® into Japanese and to determine the validity and reliability of the translated version.
Methods: We translated the original English HCAPHS® into Japanese based on standardized guidelines. The content validity was assessed by an expert panel of eight members and 12 discharged patients. The discharged patients assessed the face validity. We assessed the average of all raters’ evaluations at the item level (I-CVI and I-FVI) and the scale level (S-CVI and S-FVI). We evaluated each item and the entire questionnaire, with cut-off points of 0.78 for the I-CVI and I-FVI and 0.9 for the S-CVI and S-FVI.
Results: Regarding the content validity, the S-CVI was 0.99, and the I-CVIs for the individual items ranged from 0.95 to 1.0. Regarding the face validity, the I-FVIs of all items except for Q26 were 0.78 or higher. The S-FVIs were 0.96 for clarity and 0.98 for comprehension. The expert panel reviewed the translated Japanese version of the HCAHPS and modified it based on the patient raters' feedback.
Conclusions: The translated HCAHPS® content is well adapted to the local context. Until now, there has been no standard instrument to measure the perspectives of hospitalized patients in Japan. The translated HCAHPS® is expected to have utility for measuring the quality of health indicators.
Trial registration:
No applicable.
Patient-centred healthcare policies and efforts to evaluate the healthcare system have increased.[1, 2] Patient experience is positively associated with clinical effectiveness and patient safety.[3] Recently, patient experience has been considered one of the central pillars of quality in healthcare and is used worldwide as a tool for assessing the quality of healthcare professionals.[4]
One of the instruments used to quantitatively evaluate patient perspectives is the Adult Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS®). The HCAHPS questionnaire was developed in the United States in 2006 by the Agency for Healthcare Research and Quality (AHRQ) in response to a request by the Centers for Medicare and Medicaid Services (CMS).[5] The HCAHPS® consists of three rating scales, including “HCAHPS Composites,” “Individual Items,” and “HCAHPS Global Items,” and seven items related to individual characteristics (Q23-Q29). It contains 19 core questions about critical aspects of patients' hospital experiences. HCAHPS Composites comprises six composite items, including 1) “Communication with Nurses” (Q1, Q2, Q3), 2) “Communication with Doctors” (Q5, Q6, Q7), 3) “Responsiveness of Hospital Staff” (Q4, Q11), 4) “Communication about Medicines” (Q13, Q14), 5) “Discharge Information” (Q16, Q17), and 6) “Care Transition” (Q20, Q21, Q22). HCAHPS Individual Items includes 1) “Cleanliness of Hospital Environment” (Q8), and 2) “Quietness of Hospital Environment” (Q9). HCAHPS Global Items includes 1) “Hospital Rating” (Q18), and 2) “Recommend the Hospital” (Q19). The HCAHPS® questionnaire has been translated into various languages, including Spanish, Chinese, Russian, Vietnamese, Portuguese, and German, and is used all over the world.[6] In Japan, although the Ministry of Health, Labor and Welfare has been promoting the measurement and publication of quality indicators for healthcare, there is no standardized measurement scale to assess hospitalized patients’ experiences. Thus, the objective of this study was to conduct the translation and adaptation of the HCAHPS® into Japanese and to determine the validity and reliability of the translated Japanese version.
Before starting this validation study, we obtained documents and permission to translate the HCAHPS® into Japanese from the U.S. AHRQ and its CAHPS® Program. We launched a panel of nine experts, including clinicians and faculty in the fields of epidemiology and medicine, to review the translated HCAHPS®. First, we followed the standardized translation procedure developed by the CAHPS® Cultural Comparability Team.[7] Two of the expert panel members separately translated the original HCAHPS® in English. After the forward translation, a reviewer compared the original English HCAHPS® and the two forward translations to ensure that the meaning of the content had been preserved and adopted the more faithful representation of the original text. Additionally, the reviewer confirmed the conceptual equivalence to ensure that both versions were identical in the context and meaning of the theoretical construct. Then, the expert panel, including the translators and translation reviewers, reconciled the translated and original English HCAHPS® versions and determined the accuracy of the translation by identifying the differences between the two versions.
We assessed the translated HCAHPS® following the guide for the cross-cultural adaptation process provided by Squires et al.[8] to avoid methodological problems.[9, 10]
In addition to the eight expert panel members, 12 discharged patients were included as experts who had experienced healthcare professionals’ health services and system operations to evaluate the content validity of the translated HCAHPS®.[11, 12] We established the inclusion and exclusion criteria for patients based on the criteria from the original HCAHPS® survey: 1) patients above 20 years of age; 2) patients who had stayed at the hospital for at least one night; and 3) patients who had been discharged 48 hours to 6 weeks ago were included. We excluded 1) patients diagnosed with psychiatric disease based on the Medicare severity diagnosis-related groups; 2) patients who did not allow the disclosure of patient information; 3) court/law enforcement patients (prisoners); 4) patients whose address was in a foreign country; 5) patient discharged to hospices, special nursing homes for the elderly, or geriatric health services facilities; and 7) patients with decreased cognitive function. We recruited the participants at the St. Luke’s International Hospital.
We assessed content validity using four-point Likert-scale questionnaires. Content validity refers to the extent to which the items on a test represent all aspects of the measured construct. We asked about the relationship between the domain and each item of the HCAHPS questionnaire on a scale from 1 (not at all relevant) to 4 (very relevant). The content validity index (CVI) was calculated using Microsoft Excel. For the CVI analysis, we recategorized the scores from the four scales as categorical data: 0 for not relevant (scores 1 and 2) and 1 for relevant (scores 3 and 4). We calculated the average of all the raters’ evaluations at the item level (I-CVI) and scale level (S-CVI).[13] The validity of each item and the entire survey was excellent, with the I-CVI being 0.78 or higher and the S-CVI being 0.9 or higher.[14]
We included the same 12 discharged patients who were recruited to assess content validity and asked them to evaluate face validity.
Face validity refers to the extent to which a test appears to measure what it is intended to measure. The participants evaluated the clarity and understandability of the Japanese version of the HCAHPS on a scale ranging from 1 (not at all clear) to 4 (very clear) and rated its comprehensibility from 1 (incomprehensible at all) to 4 (very understandable). For the face validity index (FVI) analysis, we recalculated the scores from the four rating scales as 0 for unclear and incomprehensible (scores 1 and 2) and 1 for clear and understandable (scores 3 and 4). The I-FVI was calculated by averaging the values for clarity and comprehension. The S-FVI was computed by calculating the scale average of the universal value.[15] Some studies have used 0.8 for the FVI cut-off score, but no consensus has been obtained.[15] Therefore, we applied the same criteria used to assess the content validity. The I-FVI was 0.78 or higher and the S-FVI was 0.9 or higher, and each item and the entire survey were evaluated as excellent.[14]
During the translation process, we removed Q27 (Are you of Spanish, Hispanic or Latino origin or descent?), Q28 (What is your race? Please choose one or more.), and Q29 (What language do you mainly speak at home?) because these questions were related to the cultural context of the United States of America (USA). Due to differences between the education systems in Japan and the USA, we revised the answer frame of Q29 (What is the highest grade or level of school that you have completed?) to reflect the Japanese educational system.
To ensure the diversity of the subjects who met the selection criteria regarding the patient survey, during the patient survey, we recruited four persons each from the three areas of obstetrics, internal medicine, and surgery.
We removed the items related to personal characteristics from the statistical analysis to calculate the CVIs. The screening questions did not provide any information on patient perceptions and could be removed from the questionnaire analysis.[16] We also removed screening questions, including Q10 (During this hospital stay, did you need help from nurses or other hospital staff to get to the bathroom or use a bedpan?), Q12 (During this hospital stay, were you given any medicine that you had not taken before?), and Q15 (After you left the hospital, did you go directly to your own home, to someone else’s home, or to another health facility?), to calculate the CVIs. The S-CVI was calculated to be 0.99, and the I-CVIs of the individual items ranged from 0.95 to 1.0 (Table 1).
|
Questions |
I-CVI |
I-FVI |
||
|---|---|---|---|---|
|
Clarity |
Comprehension |
|||
|
HCAHPS Composites |
||||
|
Communication with Nurses |
||||
|
1 |
During this hospital stay, how often did nurses treat you with courtesy and respect? (1 Never, 2 Sometimes, 3 Usually, 4 Always) |
0.95 |
0.92 |
1 |
|
2 |
During this hospital stay, how often did nurses listen carefully to you? (1 Never, 2 Sometimes, 3 Usually, 4 Always) |
1 |
1 |
1 |
|
3 |
During this hospital stay, how often did nurses explain things in a way you could understand? (1 Never, 2 Sometimes, 3 Usually, 4 Always) |
1 |
1 |
1 |
|
Communication with Doctors |
||||
|
5 |
During this hospital stay, how often did doctors treat you with courtesy and respect? (1 Never, 2 Sometimes, 3 Usually, 4 Always) |
1 |
1 |
1 |
|
6 |
During this hospital stay, how often did doctors listen carefully to you? (1 Never, 2 Sometimes, 3 Usually, 4 Always) |
1 |
1 |
1 |
|
7 |
During this hospital stay, how often did doctors explain things in a way you could understand? (1 Never, 2 Sometimes, 3 Usually, 4 Always) |
1 |
1 |
1 |
|
Responsiveness of Hospital Staff |
||||
|
4 |
During this hospital stay, after you pressed the call button, how often did you get help as soon as you wanted it? (1 Never, 2 Sometimes, 3 Usually, 4 Always, 9 I never pressed the call button) |
0.95 |
1 |
1 |
|
10 |
During this hospital stay, did you need help from nurses or other hospital staff in getting to the bathroom or in using a bedpan? (1 Yes, 2 No →If No, Go to Question 12) |
- |
1 |
1 |
|
11 |
How often did you get help in getting to the bathroom or in using a bedpan as soon as you wanted? (1 Never, 2 Sometimes, 3 Usually, 4 Always) |
0.95 |
0.92 |
0.92 |
|
Communication about Medicines |
||||
|
12 |
During this hospital stay, were you given any medicine that you had not taken before? (1 Yes, 2 No →If No, Go to Question 15) |
- |
0.83 |
0.92 |
|
13 |
Before giving you any new medicine, how often did hospital staff tell you what the medicine was for? (1 Never, 2 Sometimes, 3 Usually, 4 Always) |
1 |
0.92 |
0.92 |
|
14 |
Before giving you any new medicine, how often did hospital staff describe possible side effects in a way you could understand? (1 Never, 2 Sometimes, 3 Usually, 4 Always) |
1 |
0.92 |
0.92 |
|
Discharge Information |
||||
|
15 |
After you left the hospital, did you go directly to your own home, to someone else’s home, or to another health facility? (1 Own home, 2 Someone else’s home, 3 Another health facility → If Another, Go to Question 18) |
- |
1 |
1 |
|
16 |
During this hospital stay, did doctors, nurses or other hospital staff talk with you about whether you would have the help you needed when you left the hospital? (1 Yes, 2 No) |
1 |
1 |
1 |
|
17 |
During this hospital stay, did you get information in writing about what symptoms or health problems to look out for after you left the hospital? (1 Yes, 2 No) |
1 |
1 |
1 |
|
Care Transition |
||||
|
20 |
During this hospital stay, staff took my preferences and those of my family or caregiver into account in deciding what my health care needs would be when I left. (1 Strongly disagree, 2 Disagree, 3 Agree, 4 Strongly agree) |
1 |
0.92 |
1 |
|
21 |
When I left the hospital, I had a good understanding of the things I was responsible for in managing my health. (1 Strongly disagree, 2 Disagree, 3 Agree, 4 Strongly agree) |
1 |
0.92 |
1 |
|
22 |
When I left the hospital, I clearly understood the purpose for taking each of my medications. (1 Strongly disagree, 2 Disagree, 3 Agree, 4 Strongly agree, 5 I was not given any medication when I left the hospital) |
1 |
1 |
1 |
|
HAHPS Individual Items |
||||
|
Cleanliness of Hospital Environment |
||||
|
8 |
During this hospital stay, how often were your room and bathroom kept clean? (1 Never, 2 Sometimes, 3 Usually, 4 Always) |
1 |
0.92 |
1 |
|
Quietness of Hospital Environment |
||||
|
9 |
During this hospital stay, how often was the area around your room quiet at night? (1 Never, 2 Sometimes, 3 Usually, 4 Always) |
0.95 |
0.92 |
0.92 |
|
HCAHPS Global Items |
||||
|
Hospital Rating |
||||
|
18 |
Using any number from 0 to 10, where 0 is the worst hospital possible and 10 is the best hospital possible, what number would you use to rate this hospital during your stay? (0 0 Worst hospital possible, 1 1, 2 2, 3 3, 4 4, 5 5, 6 6, 7 7, 8 8, 9 9, 10 10 Best hospital possible) |
1 |
0.92 |
1 |
|
Recommend the Hospital |
||||
|
19 |
Would you recommend this hospital to your friends and family? (1 Definitely no, 2 Probably no, 3 Probably yes, 4 Definitely yes) |
1 |
1 |
1 |
|
Personal characteristics |
||||
|
23 |
During this hospital stay, were you admitted to this hospital through the Emergency Room? (1 Yes, 2 No) |
- |
0.82 |
0.92 |
|
24 |
In general, how would you rate your overall health? (1 Excellent, 2 Very good, 3 Good, 4 Fair, 5 Poor) |
- |
1 |
1 |
|
25 |
In general, how would you rate your overall mental or emotional health? (1 Excellent, 2 Very good, 3 Good, 4 Fair, 5 Poor) |
- |
1 |
1 |
|
26 |
What is the highest grade or level of school that you have completed? (1 Until compulsory education 2 High school graduate or a high school diploma 3 Some college or 2-year degree 4 vocational college graduate 5 More than 4-year college degree 6 Graduate degree |
- |
0.75 |
0.75 |
|
S-CVI/Ave or S-FVI/Ave |
0.99 |
0.95 |
0.97 |
|
| Note: I-CVI: item-level content validity index, S-CVI: scale-level content validity index, I-FVI: item-level face validity index, S-FVI: scale-level face validity index | ||||
Table 1 shows the FVI results of the expert and patient rater evaluations. The S-FVI for clarity was calculated to be 0.96, and the I-FVI ranged from 0.75 to 1.0. The S-FVI for comprehension was calculated to be 0.98, and the I-FVI ranged from 0.75 to 1.0 (Table 1). Some patient raters commented that they were unsure of how education level was associated with patient satisfaction.
The expert panel reviewed the translated HCAHPS® and modified it based on the survey results and patients’ comments. For Q26, we changed the answer option “degree beyond 4-year college” to “graduate degree.” The final version of the translated HCAHPS® is shown in Appendix Table 1.
There is growing interest in assessing patients' experiences with healthcare and instruments to measure and assess the quality and safety of care.[4, 17, 18] Previous studies suggested that the HCAHPS® questionnaire can be used to assess patient care experiences in other countries.[12, 19, 20] This study aims to translate the HCAHPS® into Japanese and evaluate its validity and reliability. The translated HCHAPS® can be a useful resource for advocating patient-centred health care policy in Japan, as it allows the patient's views to be heard directly.[2] In addition, the translated HCAHPS® could be used as an evaluation instrument for domestic and international hospital comparisons as well as international collaborative research.
The original HCAHPS® was rigorously developed based on a multistep process that included a review of submitted items from organizations, item identification, consumer testing, and additional field testing,[21] and translation guidelines were provided.[7] We systematically translated the original HCAHPS® into Japanese following the guidelines. The original HCAHPS® has been translated and adapted for comprehensibility in the Japanese cultural context, taking into account its meaning and intent. In addition, a panel of experts reviewed the meaning and intent, and after discussion, a forward translated Japanese version was created.
A cross-cultural translation is not sufficient to translate a survey literally. The problem here relates to translation quality and the comparability of results across cultural and ethnic groups.[22] Polit et al.[14] also suggest strong concept and development work, robust items, exceptional experts, and clear instructions to experts on the underlying composition and evaluation tasks. We considered these challenges and gathered a panel of experts from various medical fields. In addition, the adequacy of the translated HCAHPS® was confirmed by the members of the expert panel and patients who had been discharged. Regarding the content validity, the translated HCAHPS® had I-CVIs of 0.78 or higher and S-CVIs of 0.9 or higher. The translated content was well adapted to the local context. The translated HCAHPS® was assessed as having excellent content validity.[14] Regarding the face validity, the I-FVI of all items except for Q26 was 0.78 or higher. The S-FVI was 0.96 for clarity and 0.98 for comprehension. A previous study in which the HCAHPS® was adapted for five European countries reported that many patient raters commented that they did not understand the association between education level and patient satisfaction scores.[12] This study received similar comments from the patient evaluators. Face validity refers to the clarity and comprehensibility of the HCAHPS® questionnaire; however, it is believed that the patients’ psychological perception affected the results. Q26, whose I-FVI was less than 0.78, was modified according to the patient raters' feedback, and the translated HCAHPS® was well adapted to Japanese culture.
As the instrument is translated and used, it has been pointed out that its structure could change due to subtle differences in the conceptual meaning of words.[9] This study evaluated content validity and face validity but not construct validity. Further studies need to investigate the actual use of the translated HCAHPS® for discharged patients to assess its construct validity.
We translated the HCAHPS® into Japanese following the guidelines and confirmed its validity and reliability. The translated HCAHPS® was assessed as having excellent content validity and face validity and was well adapted to Japanese culture. It is expected to have utility for assessing the quality of health indicators, as until now, there has been no standard instrument to measure the perspectives of hospitalized patients in Japan.
HCAHPS®: Hospital Consumer Assessment of Healthcare Providers and Systems
CVI: Content validity index
I-CVI: Item-level content validity index
S-CVI: Scale-level content validity index
I-FVI: Item-level face validity index
S-FVI: Scale-level face validity index
Ethics approval and consent to participate
Ethics approval for this survey was granted by St. Luke's International University (19-R157). We carried out this survey in accordance with Ethical Guidelines for Medical and Health Research Involving Human Subjects [23]. Researchers used the informed consent form to explain the research objectives and methods and the principles of voluntary participation, data management, and privacy protection to those who met the inclusion criteria. We surveyed the participants who consented to participate in this research. Informed consent was obtained from all participants.
Consent for publication
Not applicable.
Availability of data and materials
The datasets generated and analysed during the current study are not publicly available as analysis is ongoing for additional publications but are available from Ohde S. ([email protected]) on reasonable request.
Competing interests
The authors declare that they have no competing interests.
Funding
This work was supported by the Health Labor Sciences Research Grant (19IA2013).
Authors’ contributions
N.Y., S.O., O.T., and T.F. conceptualized and designed the study. N.Y. and S.M. analyzed the data. All authors contributed to drafting and revising intellectual content. All authors read and approved the final manuscript.
Acknowledgements
We would like to express our gratitude to our participants for sharing their opinions on the assessment of translated HCAHPS®.