Reliability and validity of ICU nurses' knowledge-attitude-practice scale for post-ICU syndrome in critically ill patients

Abstract

Background: Post-ICU syndrome has not been paid enough attention in Chinese medical institutions. Due to the large population base in my country, the number of people with post-ICU syndrome is also large, which has a great impact on the quality of life of patients in the later period.

Objective To develop and verify ICU nurses' knowledge, attitude and practice questionnaire on post-ICU syndrome in critically ill patients, to provide a basis for the investigation of post-ICU syndrome status.

Methods Using the principle of Knowledge, Attitude and Practice (KAP), a questionnaire item database was created through literature review and semi-structured interviews. The questionnaire was modified by Delphi expert correspondence method and pre-investigation. 451 ICU nurses were selected for formal investigation, and the reliability of the questionnaire was evaluated degree test.

Results A questionnaire of knowledge, attitude, and practice of ICU nurses on post-ICU syndrome in critically ill patients was formed, in which the number of items in the three dimensions of knowledge, attitude, and practice were 11, 7, and 13, respectively. The Pearson correlation coefficient between dimensions and the total score of the questionnaire ranged from 0.526 to 0.860, and the correlation coefficients between items and dimensions ranged from 0.517 to 0.921 (P＜0.001). Item content validity (I-CVI) ranged from 0.867 to 1, and questionnaire content validity (S-CVI) =0.989. Four common factors were extracted by exploratory factor analysis, and the cumulative variance explanation rate was 76.214%. Confirmatory factor analysis: Chi-square ratio of degrees of freedom (χ2/df) =2.789, RMSEA=0.077, RMR=0.048, GFI=0.947, CFI=0.922, NFI=0.943, IFI=0.922, TLI=0.915. The Cronbach's alpha coefficients of the questionnaire and dimension were 0.948, 0.976, 0.966, 0.910, split-half reliability = 0.951, and test-retest reliability = 0.927.

Conclusion The reliability and validity of the questionnaire meets the requirements of questionnaire preparation, and can reflect the knowledge, belief, and behavior of ICU nurses in post-ICU syndrome.

Relevance To Clinical Practice

In this study, we developed a post-ICU syndrome knowledge, belief, and behavior survey to understand the current status of post-ICU syndrome knowledge, belief, and behavior among ICU nurses in my country.

Introduction

Post-intensive care syndrome (PICS) is a new or persistent extensive damage in ICU patients after discharge, mainly including cognitive function, psychological function, and physiological dysfunction. Post-intensive care syndrome family (PICS-F) manifests as restlessness, irritability, difficulty falling asleep, etc. and the two are collectively referred to as post-ICU syndrome^[1]. PICS was first proposed in 2010^[2]. Several studies have shown that the average incidence of PICS in foreign countries is about 52.4%^[3]. Duration ranges from 1 month to 8 years^[4]. Studies have shown that cognitive dysfunction is mainly manifested as slow thinking, delirium, memory loss, etc.; psychological dysfunction is anxiety, depression and post-traumatic stress disorder (PTSD); physical dysfunction is mainly fatigue, sleep disturbance, bathing, shopping, decline in daily living skills such as going up and down stairs^[5]. Only 50% of patients discharged from the ICU return to school or work after 1 year, and the social integration and quality of life are low. The readmission rate of patients with post-ICU syndrome is 23%. bring many adverse effects^[6]. Preventing the occurrence of PICS in ICU patients is the key to solving this problem, and the medical staff who have the longest contact with patients are ICU nurses, whose knowledge, attitudes and behaviors about PICS play an important role in the prevention of PICS. Since there is still a lack of survey tools on the status quo of ICU nurses' knowledge, belief, and behavior of PICS in my country.The study hopes to understand the current situation of PICS knowledge, belief, and behavior of ICU nurses in my country by compiling a questionnaire on the current situation of PICS knowledge, belief, and behavior. The measures provide theoretical basis to reduce the incidence of PICS and improve the quality of life of patients after discharge.

1. Objects And Methods

1.1 Research objects

Registered nurses who worked in the intensive care department of a tertiary hospital in Nanchang for ≥ 2 years and volunteered to participate in this interview were selected as the interview subjects, and the interview location was the ICU medical and nursing lounge. A total of 10 subjects were included in this interview, numbered 1–10. Among them, there are 2 males and 8 females; the numbers of junior, intermediate and senior professional titles are 2, 6 and 2 respectively. The working time is between 3 and 22 years, and the age is between 25 and 42 years old. After the interview, the interview data were coded and organized.

1.1.2 Objects of expert correspondence

This letter inquiries select experts who have a bachelor's degree or above, intermediate professional titles or above, and have worked for more than 10 years and voluntarily participate in this letter inquiries. The specific results are shown in Table 1.

1.1.3 Survey Objects

In June 2022, registered nurses who worked independently in the ICU of a tertiary hospital in Nanchang were randomly selected; the working time in the ICU was ≥ 1 year; they volunteered to participate in this survey. According to the sample size of 5 to 10 times the number of items in the questionnaire^[7], since there were 41 initial items in this questionnaire, and considering the 10% sample loss rate, 451 subjects were finally considered for inclusion.

1.2 Research methods

Based on the theory of knowledge, belief, and behavior, we systematically searched domestic and foreign literatures with the definition, incidence, assessment tools, high-risk factors, high-risk groups, intervention measures, and follow-up of post-ICU syndrome as search keywords to build a pool of questionnaire entries.

1.2.2 Semi-structured interview method

By establishing an interview outline, interviews were conducted with the included interviewees. The interview outline is: ① Do you think it is necessary to investigate the current status of ICU nurses' knowledge, belief, and practice of post-ICU syndrome? ②What do you think ICU nurses need to know about post-ICU syndrome? ③What do you think the ICU nurses will think about the department's work on the prevention of post-ICU syndrome? ④In your daily work, what kind of work do you think can effectively prevent post-ICU syndrome? ⑤What is your opinion on the knowledge of post-ICU syndrome?

Through the literature review method and semi-structured interview method, a questionnaire on the status quo of ICU nurses' knowledge, belief, and behavior of post-ICU syndrome was initially formed, including a total of 41 items, including 17, 11, and 13 items in the three dimensions of knowledge, attitude, and behavior.

1.2.3 Delphi expert correspondence method

The content of the letter inquiries includes three parts: the general information of experts, the "Questionnaire on the Status of Knowledge, Belief and Practice of ICU Nurses on Post-ICU Syndrome", the expert's familiarity with the content of the questionnaire (Cs) and the basis for judging the importance of the items (Ca). The inquiry form will be returned within 2 weeks after the letter is issued in the form of mail. Experts evaluate the importance of each item, and explain the items that need to be added, deleted or modified. The importance of each item is in order of very unimportant, unimportant, important, very important, and very important, with a score ranging from 1 to 5. Items with an average score < 3.5 and a coefficient of variation (CV) > 2.5 were deleted^[8]. The specific amendments, deletions, and mergers are as follows: "I think it is necessary for ICU nurses to master the use of post-ICU syndrome assessment tools" is changed to "I think it is necessary for ICU nurses to master the assessment methods for post-ICU syndrome"; "In clinical work I will pay attention to the patient's psychological condition" to "I will pay attention to whether the patient has anxiety, depression, delirium and other problems in clinical work"; delete "I think my knowledge of post-ICU syndrome can meet the clinical needs"; Interested in knowledge about post-ICU syndrome", "I think it is necessary for ICU nurses to master knowledge about post-ICU syndrome"; "I would like to receive training on post-ICU syndrome" is merged into "I think it is necessary for ICU nurses to have knowledge about post-ICU syndrome" Knowledge of Post-ICU Syndrome". After 2 weeks, the second round of expert letter consultation will be carried out, and "Whether you will pay attention to the patient's physical activity in clinical work" is revised to "I will pay attention to the patient's muscle strength in clinical work", other items are retained, and the letter Inquiry ends. After two rounds of expert correspondence, the number of modified, deleted and merged entries was 8, 4 and 3 respectively.

1.3 Pre-investigation

Randomly selected 15 ICU nurses from tertiary hospitals in Nanchang City for pre-investigation to evaluate whether the content of the questionnaire is reasonable and the description is easy to understand. The filling time is about 6–9 minutes. According to the results of the survey feedback and amend the two items with ambiguity in description: change "the harm of post-ICU syndrome" to "the harm of post-ICU syndrome to patients"; "I will make reasonable adjustments according to the patient's mental and conscious state. "Use of sedatives" was changed to "I will adjust the use of sedatives reasonably according to the patient's mental and conscious state after communicating with the doctor and following the doctor's advice", without deleting and adding items.

1.4 Formal Investigation

Formal questionnaires were distributed to 451 research subjects through Questionnaire Star, and 428 were recovered, with a recovery rate of 94.90%. Excluding duplicate answers and answering less than 1 minute of questionnaires, a total of 28 questionnaires were obtained, and 400 questionnaires were finally obtained. Items 1–11 of the knowledge dimension of the questionnaire are graded likert 5, ranging from "completely ignorant" to "completely understood", with a score of 1 to 5; items 12–15 are multiple-choice questions, select " 3 points for "correct", 2 points for choosing "unclear", and 1 point for choosing "wrong", with a total score of 15 to 67 points. The attitude dimension ranges from "very unnecessary" to "very necessary", with scores ranging from 1 to 5, with a total score of 7 to 35. The behavior dimension ranges from "almost never" to "always", with a score of 1 to 5 and a total score of 13 to 65. The total score of the questionnaire is 35 to 167. The higher the level of syndrome knowledge, attitude and behavior.

1.5 Statistical methods

Using Excel2016, SPSS25.0, Amos24.0 to conduct data sorting and statistical analysis. General survey data are presented as frequencies and percentages. The results of this letter inquiries were evaluated by experts' enthusiasm, expert authority coefficient (Cr), and Kendall's W coefficient; extreme value method (CR) and Pearson's correlation coefficient were used for item analysis of the questionnaire; content validity was based on item content Validity (I-CVI) and scale content validity (S-CVI) were analyzed, and construct validity was evaluated by exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). The reliability and test-retest reliability were tested, and P < 0.05 was considered statistically significant^[9].

2. Results

2.1 Result of expert correspondence

The positive degrees of the two rounds of expert evaluation were 83.3% (15/18) and 100% (15/15) respectively; the expert authority coefficient (Cr) was the average of the expert familiarity (Cs) and the judgment basis (Ca), respectively 0.87 and 0.89, both greater than 0.7, indicating that the experts in this research have a high degree of authority^[10]. The average score of the first round of items was 4.07–4.93 points, and the coefficient of variation (CV) was 0.05–0.21; the average score of the second round of items was 4.20–4.93, and the coefficient of variation (CV) was 0.05–0.17, all in line with the item average score ≥ 3.5 Or the requirement of coefficient of variation ≤ 0.25^[11]. The Kendall harmony coefficients of the two rounds were 0.350 and 0.209, respectively, P < 0.001^[12].

2.2 General information of the respondents. The results are shown in Table 2.

2.3 Project Analysis

The difference of the questionnaire was tested by the extreme value method (CR), the total score of the 400 research subjects was calculated and sorted from high to low, and the 108 research subjects whose total score was higher than 138 points (top 27%) were classified as high. The 108 subjects whose total score was lower than 113 points (the latter 27%) were classified as low group, and two independent samples t-test was used for statistical analysis. so keep all entries. The Pearson correlation coefficient (r) was used to evaluate the correlation between each dimension, the total score of the questionnaire, and the items. The r of the dimension score and the total score of the questionnaire were 0.852, 0.526, and 0.860, respectively (P<0.001). In the correlation analysis, the r of A12, A13, A14 and the knowledge dimension were 0.257, 0.370, and 0.345 (r<0.4), respectively. The r of the other items and each dimension ranged from 0.517 to 0.921 (P<0.001)^[13]. According to the project analysis, the entries A12, A13, and A14 in the knowledge dimension are finally deleted.

2.4 Validity Analysis

2.4.1 Content Validity

Item content validity (I-CVI) and scale content validity (S-CVI) were used to evaluate the content validity of the questionnaire. After calculation, the I-CVI of the 15 experts who consulted by correspondence was 0.867-1, and the S-CVI was 0.989.

2.4.2 Construct validity

The 400 questionnaires were numbered by the random number table method, 400 random numbers were randomly generated and sorted by size, and the first 200 questionnaires were selected for the first factor analysis, KMO = 0.944, Bartlett sphericity test: χ2 = 9942.118, df = 496, (P < 0.001), where the factor loading of A15 was < 0.4, and the entry was finally deleted. After deletion, the second factor analysis was performed, KMO = 0.945, which was suitable for factor analysis. Bartlett test of sphericity: χ2 = 9923.017, df = 465, (P < 0.001), using principal component method and maximum variance method, a total of 4 common factors with eigenvalue > 1 were obtained after rotation, and the cumulative variance explained rate was 76.214%, 31 The factor loading of each item is 0.510–0.881, all of which are greater than 0.4. Finally, the official version of the questionnaire with 31 items is obtained. The analysis results are shown in Table 3. The gravel diagram is shown in Fig. 1.

2.4.2.2 Confirmatory factor analysis

After the selection of 200 questionnaires, Amos24.0 was used for confirmatory factor analysis, and the results are shown in Table 4.

2.5 Reliability test

Through reliability analysis to test the internal consistency and stability of the questionnaire, it is believed that Cronbach's α coefficient, split-half reliability > 0.7–0.9, and test-retest reliability > 0.8, which indicates that the consistency and stability of the questionnaire are good^[14]. The Cronbach's alpha coefficient test results of the questionnaire and each dimension were 0.910–0.976, the split-half reliability was 0.920–0.961, and the test-retest reliability was 0.916–0.978. The reliability test results are shown in Table 5.

3 Discussion

3.1 Scientific nature of questionnaire preparation

This questionnaire is based on Knowledge, Attitude and Action (KAP) theory, searches relevant databases and semi-structured interviews to construct an initial 41 items and 3 dimensions. The questionnaire was supplemented and revised through pre-experiment and expert letter inquiries, and a questionnaire consisting of 35 items was formed after the opinions tended to be unanimous.

The item analysis was evaluated by the critical ratio rate and the Pearson correlation coefficient. The decision value (CR) of the questionnaire items were all greater than 3, and the Pearson correlation coefficient with the item in the knowledge dimension was divided by K12, K13, and K14 < 0.4. After discussion by the members of the research group the above 3 items were deleted, and the correlation coefficients of the remaining items were all > 0.4. Finally, 32 items of the questionnaire were retained.

Validity analysis indicates that the questionnaire reflects the accuracy and reliability of the research results, and is mainly evaluated through content validity and construct validity^[15]. Content validity mainly includes item content-level validity (I-CVI) and scale-level validity (S-CVI). Studies have shown that I-CVI > 0.78 and S-CVI > 0.8 indicate that the questionnaire has good content validity^[16]. The I-CVI of this questionnaire was 0.867-1, and the S-CVI was 0.989, indicating good content validity. Construct validity is the degree of a certain structure and framework measured by measurement tools, reflecting the degree ofagreement between the measurement results and the theoretical framework^[17]. This study passed two exploratory factor analyses. Since the factor loading of the entry K15 of the knowledge dimension in the first factor analysis was less than 0.4, this entry was deleted. In the second factor analysis, the factor loadings of 31 items ranged from 0.663 to 0.909, there were no multiple loadings, and the cumulative variance explanation rate was 76.214%. After orthogonal rotation by the maximum variance method, a total of 4 common factors were obtained and named respectively. 3 dimensions of knowledge, attitude and behavior. Confirmatory factor analysis results showed that RMSEA, RMR, GFI, CFI, NFI, IFI, and TLI all met the model fitting criteria, and the fitting effect was good. The factor analysis indicated that the questionnaire had good structural validity.

Reliability means that the measurement tool reflects the consistency and stability of the measurement results, and the Cronbach's alpha coefficient and the split-half reliability (0.7–0.9) reflect the internal consistency of the questionnaire^[18]. Test-retest reliability (> 0.8) reflects the stability of the questionnaire^[19]. The Cronbach's alpha coefficients of this questionnaire and each dimension were 0.948, 0.976, 0.966, 0.910, and the split-half reliability was 0.951, indicating that the questionnaire had a good degree of consistency. The test-retest reliability was 0.927, indicating that the questionnaire was stable.

3.2 The importance of questionnaire preparation

Due to the accelerated aging of the population, the number of people admitted to the ICU in my country is as high as 10 million each year^[20]. With the improvement of my country's medical level, the cure rate of ICU has also increased significantly, but research shows that about 50% of patients discharged from ICU have psychological, physiological, cognitive and other functional disorders, mainly manifested as anxiety and depression, stress disorder, ICU acquired weakness, decreased memory and language skills, decreased daily activities, unable to complete basic life skills such as bathing, going shopping, and going up and down stairs^[21]. It can be seen that post-ICU syndrome has a greater impact on the quality of life of patients after discharge^[22]. The research on the post-ICU syndrome in my country is not yet in-depth, and the ICU nurses' understanding of the syndrome is not clear. Therefore, this study hopes to investigate the knowledge, attitudes and behaviors of the ICU nurses in my country on the post-ICU syndrome by compiling this questionnaire. To help hospital managers improve the ICU nurses’ mastery of the syndrome through relevant training, policy implementation, etc., so as to improve the ICU nurses’ compliance in preventing ICU patients from developing post-ICU syndrome, and reduce the occurrence of post-ICU syndrome in ICU patients. to help patients better reintegrate into society and reduce the burden on families and medical care.

4. Conclusion And Limitation

The questionnaire on the status of knowledge, belief and behavior of post-ICU syndrome constructed in this study has good reliability and validity. Policy provides a theoretical basis. Because the research subjects included in this study were all tertiary hospitals in Nanchang City, subjects from other provinces were not included, resulting in a small sample size, which could not well reflect the situation in other provinces. In the later stage, ICU nurses from other provinces should be included to further verify the questionnaire, so as to better understand the current status of knowledge, belief, and behavior of ICU nurses in my country on post-ICU syndrome, so as to better promote the questionnaire.

Declarations

Conflicts of interest 

All authors declare no conflict of interest

Acknowledgements：We would like to thank all researchers who generously shared thei time to participate in this survey.

Authors’ contributions  zhiqiang cheng ,baozhen zhang analyzed this data and wrote manuscript; jiaoyun xia revised this manuscript;  xiaoxing wang edited the language. All authors read and approved the final manuscript.

Ethics statement

This article does not address ethical issues

Consent for publication

No applicable.

Author details

Zhiqiang Cheng,：School of Nursing, Nanchang University, Jiangxi Province, China E-mail:[email protected]

Baozhen Zhang： The First Affiliated Hospital of Nanchang University, Jiangxi, China E-mail:[email protected] 

Jiaoyun xia：The First Affiliated Hospital of Nanchang University, Jiangxi, China E-mail:[email protected]

Xiaoxing Wang：School of Nursing, Nanchang University, Jiangxi Province, China E-mail:[email protected]

References

1. Needham D M, Davidson J, Cohen H, et al. Improving long-term outcomes after discharge from intensive care unit: report from a stakeholders' conference[J]. Crit Care Med, 2012, 40(2): 502-9.
2. Elliott D, Davidson J E, Harvey M A, et al. Exploring the scope of post-intensive care syndrome therapy and care: engagement of non-critical care providers and survivors in a second stakeholders meeting[J]. Crit Care Med, 2014, 42(12): 2518–26.
3. Wang S, Allen D, Perkins A, et al. Validation of a New Clinical Tool for Post-Intensive Care Syndrome[J]. Am J Crit Care, 2019, 28(1): 10–18.
4. Sakusic A, Gajic O, Singh T D, et al. Risk Factors for Persistent Cognitive Impairment After Critical Illness, Nested Case-Control Study[J]. Crit Care Med, 2018, 46(12): 1977–1984.
5. Kosilek R P, Schmidt K, Baumeister S E, et al. Frequency and risk factors of post-intensive care syndrome components in a multicenter randomized controlled trial of German sepsis survivors[J]. J Crit Care, 2021, 65: 268–273.
6. Myhren H, Ekeberg Ø, Stokland O. Health-related quality of life and return to work after critical illness in general intensive care unit patients: a 1-year follow-up study[J]. Crit Care Med, 2010, 38(7): 1554–61.
7. Moitra P, Verma P, Madan J. Development and validation of a questionnaire measuring knowledge, attitudes, and practices (KAP) to healthy eating and activity patterns in school children (HEAPS)[J]. Nutr Health, 2021, 27(2): 199–209.
8. Jiang H, Zhang S, Ding Y, et al. Development and validation of college students' tuberculosis knowledge, attitudes and practices questionnaire (CS-TBKAPQ)[J]. BMC Public Health, 2017, 17(1): 949.
9. Sun Y, Sun J, Zhao Y, et al. A new comprehensive oral health literacy scale: development and psychometric evaluation[J]. BMC Oral Health, 2021, 21(1): 429.
10. Degnan A, Berry K, James S, et al. Development, validation and cultural-adaptation of the knowledge about psychosis questionnaire for African-Caribbean people in the UK[J]. Psychiatry Res, 2018, 263: 199–206.
11. Albanesi B, Marchetti A, Facchinetti G, et al. Cross-cultural adaptation and validation of questionnaire for knowledge, attitudes, and behavioral intentions in Italian nurses with regard to provision of artificial nutrition and hydration to patients terminally ill with cancer[J]. Nutrition, 2020, 72: 110655.
12. Vance M E, Keele B. Development and validation of the cervical cancer knowledge and beliefs of Appalachian women questionnaire[J]. J Nurs Meas, 2013, 21(3): 477–501.
13. Reeve E, Low L F, Shakib S, et al. Development and Validation of the Revised Patients' Attitudes Towards Deprescribing (rPATD) Questionnaire: Versions for Older Adults and Caregivers[J]. Drugs Aging, 2016, 33(12): 913–928.
14. Tahmasebi R, Noroozi A, Dashdebi K G. Psychometric Evaluation of the Colorectal Cancer Screening Belief Scale Based on Health Belief Model's Constructs for the Fecal Occult Blood Test[J]. Asian Pac J Cancer Prev, 2016, 17(1): 225–9.
15. King R B, Champion V L, Chen D, et al. Development of a measure of skin care belief scales for persons with spinal cord injury[J]. Arch Phys Med Rehabil, 2012, 93(10): 1814–21.
16. Staggs V S. Pervasive errors in hypothesis testing: Toward better statistical practice in nursing research[J]. Int J Nurs Stud, 2019, 98: 87–93.
17. Schroderus-Salo T, Hirvonen L, Henner A, et al. Development and validation of a psychometric scale for assessing healthcare professionals' knowledge in radiation protection[J]. Radiography (Lond), 2019, 25(2): 136–142.
18. Lacasse A, Connelly J A, Choinière M. The Chronic Pain Myth Scale: Development and Validation of a French-Canadian Instrument Measuring Knowledge, Beliefs, and Attitudes of People in the Community towards Chronic Pain[J]. Pain Res Manag, 2016, 2016: 5940206.
19. Afolabi M O, Rennie S, Hallfors D D, et al. An adapted instrument to assess informed consent comprehension among youth and parents in rural western Kenya: a validation study[J]. BMJ Open, 2018, 8(7): e021613.
20. Tu W J, Zeng X, Liu Q. Aging tsunami coming: the main finding from China's seventh national population census[J]. Aging Clin Exp Res, 2022, 34(5): 1159–1163.
21. Lee M, Kang J, Jeong Y J. Risk factors for post-intensive care syndrome: A systematic review and meta-analysis[J]. Aust Crit Care, 2020, 33(3): 287–294.
22. Griffiths J, Hatch R A, Bishop J, et al. An exploration of social and economic outcome and associated health-related quality of life after critical illness in general intensive care unit survivors: a 12-month follow-up study[J]. Crit Care, 2013, 17(3): R100.