Validity and Reliability of the Korean Anesthesia Surrendering Instrument

Abstract

Background : In Korea, to reduce patients’ anxiety regarding anesthesia induction and to increase compliance, the nurses need a measurement for understanding of anesthesia surrendering. This study examined the reliability and validity of the Korean version of the anesthesia surrendering instrument (ASI), which was originally developed to measure anesthesia surrendering in Swedish adults.

Methods: The study population consisted of 306 patients who received general anesthesia for abdominal, breast, knee, hip, lower back, or shoulder surgery in ten hospitals across five regions of Korea, from June to September 2019. We assessed the validity of the content, construct, and criterion used, as well as the reliability of the ASI.

Results: The results showed that the instrument had appropriate content validity with the item-level content validity index ranging between 0.80 and 1.00 and a scale-level content validity index of 0.90. The construct validity test results showed that it comprised four sub-categories with a total of 26 items, and the internal consistency reliability tests showed Cronbach’s alpha values ranging between 0.71 and 0.88. In summary, the results confirmed that the Korean ASI had adequate reliability and validity.

Conclusion: The study findings confirmed the applicability of this instrument for measuring anesthesia surrendering in Korean adults. Moreover, these results may be used in future studies on anesthesia surrendering in Korean adult patients.

Background

As a result of the recent increase in the elderly population and advancement in medical technology, there has been a significant increase in the number of individuals visiting medical institutions to maintain and improve their health [1]. Consequently, the number of cases involving surgical treatment is also increasing [1]. According to the National Health Insurance Corporation, the total medical expenditure in Korea in 2016 was 6.76 trillion won, wherein surgical costs accounted for 68.0% at 4.65 trillion won [2]. Moreover, the number of surgical cases also increased 1.3-fold from 1.371 million in 2006 to 1.765 million in 2017 [3]. Additionally, surgery-related complications and mortality rates are increasing with the number of surgery cases. Patients who develop hypotension during surgery under general anesthesia have a four-fold higher risk of myocardial infarction than do healthy individuals [4]. Furthermore, complications during surgery could lower the survival rate of surgical patients by approximately 69.0% [5].

Anesthesia, a factor associated with surgical complications, can be broadly classified as general and regional anesthesia [6]. General anesthesia refers to the overall loss of consciousness accompanied by suppression of sensory, motor, and reflex stimulation; patients consider surgery requiring general anesthesia as a major invasive procedure [7]. Patients requiring general anesthesia for surgery express apprehension regarding surgical outcomes and recovery of consciousness during surgical procedures [8]. Moreover, patients who exhibit high levels of anxiety regarding anesthesia prior to surgery tend to distrust the medical personnel who administer the anesthesia [9] and show low levels of compliance during anesthesia induction [5]. Additionally, preoperative stress associated with anesthesia could stimulate the sympathetic nervous system to upregulate secretion of norepinephrine and epinephrine, as well as increase other physiological responses such as cardiac output, blood sugar level, blood pressure, bronchial dilation, contraction of the peripheral vessels, and muscle tension, leading to complications during surgery or postoperative recovery [10].

The efficacy of nursing interventions during administration of anesthesia depends on the dynamic, personal, and therapeutic relationship formed between the patient and the nurse during a brief meeting at the time of induction of anesthesia [8, 11]. It is imperative that nurses not only safely administer anesthesia and maintain a deep sleep state in patients until the completion of surgery, but also ensure that the patients fully understand the implications of anesthesia and safely surrender to anesthesia induction [10, 11]. In general, surrender can be defined as a psychological state that allows an individual to entrust their safety to another person without insecurity. Anesthesia surrendering in patients is the surrendering of one’s physical safety and health to the care of medical personnel [8]. Surrendering implies that an individual is assured of their safety, and it enables genuine psychological and it enables psychological relaxation necessary for mental health. The ease of the anesthesia surrendering experience depends on the patients’ interactions with medical personnel [8, 11, 12]. Therefore, medical personnel must facilitate anesthesia induction by addressing the concerns of patients so that they can voluntarily surrender to the process [8].

To alleviate patients’ anxiety regarding administration of anesthesia and to increase compliance, the nurses need a detailed understanding of anesthesia surrendering as well as a means of quantifying the experience. Therefore, there is a need for appropriate measurement tools that can aid medical personnel in providing necessary interventions during anesthesia induction. The anesthesia surrendering instrument (ASI) is one such tool that can be used to measure anesthesia surrendering in patients before surgery [13]. The ASI comprises 27 items related to the anesthesia surrendering experience, as studied in Swedish adults. These items are classified under four dimensions of the surrendering experience: preparation by avoidance, control, preparation by understanding, and acceptance. The instrument has been previously tested and found to be suitable for measuring the experience of anesthesia surrendering in patients. Since there is a need to reduce surgical complications and measure the performance of interventions for anesthesia surrendering, the present study aimed to translate the ASI, originally developed by Liebenhagen and colleagues [13], into a tool that could be utilized for Korean patients and to test the validity and reliability of this Korean version.

Methods

Research design

The present study tested the reliability and validity of the Korean version of an ASI, which was originally developed by Liebenhagen and colleagues [13], for measuring anesthesia surrendering in Korean adults.

Participants

The study population consisted of patients who received general anesthesia for abdominal, breast, knee, hip, lower back, or shoulder surgery. All participants were above 18 years of age and provided informed consent after thoroughly understanding the study protocols. They had no difficulty in reading and understanding the materials provided in Korean. Patients who had undergone surgery related to cancer in qualifying areas were excluded from the study since their level of anxiety could be different from that of other surgery patients owing to their cancer diagnosis. In the study sample design, the number of hospital beds was used as a stratification variable. The class intervals used for the number of hospital beds were based on a previous study [14] that used Dalenius & Hodges’s [15] cumulative-square root method to divide bed numbers into four strata: 100–299, 300–499, 500–999, and more than 1000 beds. In particular, hospitals with more than 1000 beds of anesthesia had high severity and were excluded from the sample size. In addition, based on the previous study [14] in which the characteristics of hospitals were clearly distinguished based on 500 beds, hospitals between 200 and 300 beds were selected. Second, five regions from the top ten were extracted from the list of 162 demographic cities, using the Republic of Korea National Statistical Portal [16]. Finally, the participants were administered anesthetic agents by specialists in medical institutions with capacities of 200 beds or more. Patients with similar durations of anesthesia induction were selected as participants based on both the discussions with medical personnel at each medical institution and the previous study [13]. To perform exploratory factor analysis (EFA), a ratio of items-to-sample-size of 1:5 or at least 100 respondents was required [17]. A sample size of at least 150–200 is appropriate for confirmatory factor analysis (CFA) [18]. The minimum sample needed for this study was estimated to be 300 subjects. A total of 306 participants were selected (retrieval rate, 95.9%), which satisfied the aforementioned sample size requirement. The questionnaires completed by the selected participants were used for preliminary factor structure testing (EFA, n = 100) and secondary factor structure testing (CFA, n = 206).

Data collection

We collected data for the period between June 1, 2019 and September 30, 2019. Data were collected from five regions in Korea (“S” city, “G” do, and “I,” “B,” and “C” metropolitan cities), which included major and small-to-medium sized cities. Ten hospitals within these regions were convenience-sampled, and we interacted with the nurses and various department personnel prior to data collection to explain the study protocols. Informed written consent was obtained from patients with the cooperation of relevant department heads. For patients who were scheduled to be discharged after surgery, the researchers made an effort to sufficiently explain the purpose and methods of the study on the day of or the day before discharge and obtained informed written consent from those who wanted to participate.

Measurement

Anesthesia Surrendering Instrument

The ASIis a measurement instrument developed by Liebenhagen and colleagues [13] that comprises 27 items classified under four dimensions, namely preparation by avoidance, control, preparation by understanding, and acceptance, with each containing 7, 7, 7, and 6 items, respectively. Each item was scored on a 4-point scale (1 point for “highly disagree” to 4 points for “highly agree”), with higher scores indicating a higher level of anesthesia surrendering. At the time of development, the reliability of the instrument, as indicated by Cronbach’s alpha was 0.76.

State-Trait Anxiety Inventory

We used the State-Trait Anxiety Inventory (STAI) form-Korean YZ (STAI-KYZ), which is a self-reported STAI developed by Spielberger [19] and adapted for Korean adults by Han et al. [20]. In the present study, only 20 items related to state anxiety were used. Each item was scored on a 4-point Likert scale, with higher scores indicating a higher level of anxiety. The reliability of the instrument, as measured by Cronbach’s alpha was 0.92 in the study by Han et al. [20] and 0.87 in our study.

Study procedure

The present study followed the guidelines for the translation and adaptation of English language instruments for use in other languages, as recommended by the World Health Organization [21]. The finalization of the items used in the instrument involved a process of preliminary translation, expert panel review, back translation, and cognitive assessment. Prior to the preliminary translation step, approval for translation into Korean and subsequent use was obtained from the original developer of the instrument. For the preliminary translation, a healthcare professional that was familiar with the technical terminology of the relevant field, and was well-versed in English-speaking culture but whose native language was Korean, was selected to perform the translation. In the present study, two Doctors of Nursing Practice (DNPs) who were fluent in English and had sufficient experience with technical terminology in the nursing field were assigned the preliminary translation. The preliminary translation was independently performed by the two DNPs, and a consensus was reached on the translated content such that the intended meaning of each sentence, not individual words, could be accurately conveyed while avoiding literal translation. Subsequently, an expert panel with a professional bilingual translator, two nursing professors fluent in both languages, and the two DNPs who performed the preliminary translation was established, and the translated sentences were reviewed by comparing with the original English version. The consistency and accuracy of the translated sentences were verified, and considering the readability and differences in cultural meaning, some parts were revised to make the tool suitable for use in Korea. Next, an American who was not familiar with the instrument and whose native language was English but was also fluent in Korean was assigned the back translation. Although the back-translated content did not perfectly match the content in the original instrument, it was confirmed that there were no changes in the intended meaning. The aforementioned rigorous translation process ensured content validity [21]. Moreover, content validity was verified by an anesthesiologist, a DNP with over 10 years of clinical nursing experience, two nursing professors, and a student in a nursing PhD program. The appropriateness, sufficiency, and representativeness of the items in each dimension were examined using the item-level content validity index (I-CVI) and the scale-level content validity index/Ave (S-CVI/Ave). In this study, the S-CVI was calculated as the average of the I-CVI by experts for the content validity analysis. The CVI is a 4-point scale reported by Polit et al. [22]. The I-CVI estimates that appropriate content validity is obtained at 0.80 or higher and when the S-CVI is 0.90 [22]. This study also confirmed content validity through cognitive validity testing with three people who have had similar experiences with surgery as the patients for whom the instrument would actually be used. Participants were asked to read the items of the tool aloud one by one, and then the readability and comprehension of the items were expressed in words.

Data analysis

Data were analyzed using the AMOS (version 22.0; IBM, Armonk, NY) and the Statistical Package for the Social Sciences (version 20; IBM). The frequency, percentage, mean, and standard deviation (SD) were calculated for the general characteristics of participants. We estimated the skewness and kurtosis to confirm normality of the data. This was done through item analysis and verified by estimating the multivariate kurtosis for a multivariate normality test using AMOS. In this study, both EFA and CFA were conducted to verify construct validity. In this case, construct validity should be conducted with different subjects for the EFA and CFA [23]. Therefore, for EFA, data for 100 subjects were randomly extracted from the collected data using Microsoft Excel (Microsoft; Redmond, WA), and the remaining data was used for CFA. For EFA, the principle axis factor analysis was used for the extraction in the common factor model and the varimax rotation, considering the correlations between factors. In particular, varimax rotation was applied to minimize the number of factors and information loss, and the Kaiser-Meyer-Olkin (KMO) test and Bartlett’s test of sphericity were performed to confirm whether the data were suitable for factor analysis. If the KMO test result was greater than 0.50, the data were found to be suitable for analysis [23]. Several methods are suggested for determining the number of factors, such as classification of more than one eigenvalue (commonly used), parallel analysis, and Velicer’s minimum average partial analysis [24, 25]. In this study, we used all three methods to determine the number of factors. The number of factors was extracted based on eigenvalues of 1.0 or higher, and items were selected based on factor loading of 0.40 or higher. The WLSMV (weighted least square mean and variance), calculated using Mplus (Los Angeles, CA), was applied to estimates that did not satisfy multivariate normality and we considered the Likert scales for the CFA [26]. The goodness-of-fit index was verified by χ2 values (degree of freedom or df, p value), weighted root mean square residuals (WRMR; reference value: ≤ 1.0), the Tucker-Lewis index (TLI; reference value: ≥ 0.9), the comparative fit index (CFI; reference value: ≥ 0.9), and the root mean square error of approximation (RMSEA; reference value: ≤ 0.05). Moreover, the convergent validity of the factor construct was checked based on the following criteria: standardized regression coefficients of observed variables ≥ 0.50, construct reliability (CR) ≥ 0.7, and average variance extracted (AVE) ≥ 0.5. The discriminant validity of the factors estimated by CFA was verified based on the criteria that inter-factor correlation coefficients should be ≤ 0.80 and the AVE of latent variables should be greater than the square of correlation coefficients of the latent variables. To test the criterion validity of the Korean ASI, a gold standard scale with proven validity and reliability should be used. However, because there are very few instruments designed to measure anesthesia surrendering and almost none have proven validity and reliability, we compared our findings to a previous study [7] on anxiety in general anesthesia patients. Hence, the Pearson’s correlation coefficient between the Korean ASI and STAI-KYZ was obtained for criterion validity. To measure internal consistency, the Cronbach’s alpha correlation coefficient was obtained using reliability testing.

Ethical considerations

This work was approved by the Institutional Review Board of Changwon University (1040271–201902-HR–002).

Results

General patient characteristics

The study population included 88 males (28.7%) and 218 females (71.3%), with a mean age of 45.91 ± 16.60 years. Among the participants, 202 individuals (66.0%) were married, 92 (30.1%) were unmarried, and 12 (3.9%) were widowed or divorced. Additionally, 260 individuals (84.9%) were employed and 46 (15.1%) were unemployed. The sites of surgery included the abdomen (n = 126, 41.2%), breast or ovary (n = 60, 19.6%), knee (n = 32, 10.5%), hip (n = 32, 10.5%), lower back (n = 36, 11.7%), and shoulder (n = 20, 6.5%).

Validity analysis

Content validity

Almost all items (26/27) showed an I-CVI ranging between 0.80 and 1.00 and S-CVI/Ave of 0.90, indicating adequate content validity. The item “I looked around the operating theater” had an I-CVI of 0.80 and S-CVI/Ave of ≤ 0.90. However, it was determined that this was a coping behavior in patients during anesthesia surrendering, and consequently, we decided to retain all 27 items.

Item analysis

We analyzed the skewness and kurtosis to check the normality of the data used in this study. The skewness of all measured variables was between 0.01–1.15 (absolute value) and did not exceed 3; kurtosis did not exceed 10 and had absolute values between 0.04–1.41. The data showed normal distribution. However, the multivariate normality test showed that the multivariate kurtosis was 43.5 and the critical ratio was 14.2 (Table 1). It is not practical to confirm the combined frequency distributions for all variables, and few data satisfy the assumption of multivariate normality. Therefore, if there is no major problem with a univariate normality test, it can be assumed that multivariate normality is satisfied [27]. In summary, the measured variables in this study satisfied univariate normality, and thus, the validity of the ASI was satisfactory.

Construct validity

This study determined the number of factors through classification of more than one eigenvalue, parallel analysis, and Velicer's minimum average partial analysis. We found four factors using eigenvalues, two factors using parallel analysis, and two factors using Velicer's minimum average partial analysis. Based on the number of factors used for the original ASI, we finalized four factors for the factor analysis. EFA was performed for all 27 items, applying the principle component analysis and varimax rotation. The KMO value for determining the suitability of samples for factor analysis was 0.85, which was higher than the cutoff value of 0.80. The approximate χ2 value for the Bartlett’s test of sphericity was 2942.88 (df = 231, p < 0.001), which indicated that the samples were suitable for factor analysis. Factor analysis of all 27 items showed four factors with an initial eigenvalue ≥ 1.0. Among all times, the item “I looked around the operating theater” was excluded due to a commonality of 0.10, whereas all other items showed commonality ≥ 0.50, satisfying the factor loading criterion of ≥ 0.40. Based on the EFA results, 26 out of the 27 initial items were retained (Table 2).

Confirmatory factor analysis

CFA was performed for a total of 26 items under the four factors identified by preliminary construct validity testing. Goodness-of-fit assessment showed χ2 = 1022.33 (df = 293, p < 0.001), WRMR = 1.05, TLI = 0.92, CFI = 0.93, and RMSEA = 0.056 (95% CI = 0.053–0.059). These results indicated that the goodness-of-fit was satisfactory according to our criteria. The final model was used for convergent validity and discriminant validity testing. The standardized regression coefficients of all items were statistically significant, ranging between 0.50 and 0.97, and the ranges of CR and AVE were 0.85–0.92 and 0.50–0.61, respectively (Table 3). Moreover, when the inter-factor correlation coefficients were examined to test the discriminant validity of factor construct, they were ≤ 0.80, and the square inter-factor correlation coefficients were lower than the AVE values, thereby confirming the discriminant validity of the factors (Table 3).

Criterion validity

For criterion validity, the correlation between the Korean ASI and STAI-KYZ was verified; the results showed a negative correlation (r = -0.50 to -0.62, p < 0.001; Table 4).

Reliability analysis

The values of Cronbach’s alpha ranged between 0.71 and 0.88 for the dimensions used in the instrument, as shown by the reliability analysis for the internal consistency of the Korean ASI (Table 4).

Discussion

Our study aimed to test the validity and reliability of the Korean ASI, which is an instrument used to measure anesthesia surrendering. The tool provides essential data related to a patient’s anesthesia surrendering experience to medical personnel, based on which appropriate intervention can be administered. The validity of an instrument cannot be adequately verified using a single type of validity analysis. Instrument validation studies need to present significant results from multiple analyses to recommend the use of the instrument in clinical settings [23]. Accordingly, the present study used various methods to analyze three types of validity, namely content, construct, and criterion.

We performed the EFA, item analysis, CFA, and criterion validity analysis to verify the construct validity of the Korean ASI. Importantly, the EFA and CFA were conducted using random samples from the collected data, thereby confirming the measurement equivalence of the instrument. This was necessary for providing robust evidence for the validity of the Korean ASI [13]. In contrast to the validity of the original ASI being proven by EFA alone, the present study also tested the discriminant and convergent validity of the Korean ASI by CFA and item analysis to present additional results. Based on the EFA results, four dimensions and a total of 26 items were extracted for the Korean ASI. Since the categorization of the dimensions in the Korean ASI was similar to the ASI developed by Liebenhagen and colleagues[13], our results confirmed that anesthesia surrendering could be measured using the four dimensions of preparation by avoidance, control, preparation by understanding, and acceptance in Korean patients too. However, the item “I looked around the operating theater” in the third dimension had a commonality of 0.10 in the EFA results, and thus, this item was excluded from the Korean ASI. This item also showed the lowest validity in content validity testing by an expert panel, and it is believed that this may be due to the layout of the operating rooms in Korea. The waiting time inside the operating room until anesthesia is insufficient for patients to examine the operating theater. We performed the CFA to ensure measurement equivalence of the Korean ASI. The results showed good fitness for the factor structural model of the Korean ASI, which confirms its construct validity. We conducted an item analysis of the factor structure confirmed by EFA to improve content validity. In this way, the preliminary convergent validity and discriminant validity were confirmed. Subsequently, factor structure, convergent validity, and discriminant validity were reconfirmed by CFA. In the two rounds of content validity testing, all items were found to meet the criteria presented in the previous studies, whereas only the item “I looked around the operating theater” was found to be unsuitable for Korean patients.

This study also tested the criterion validity of the Korean ASI. Criterion validity should be tested against a gold standard with the same construct as the translated instrument, but no such instrument was available. Previous studies [5, 9] have reported that higher anxiety and fear regarding anesthesia result in lower anesthesia outcomes. Therefore, we confirmed the correlation between the Korean ASI and STAI-KYZ. A previous validity assessment using correlation coefficients [28] determined that correlation coefficients ranging between 0.60 and 0.80 indicate “high validity” and coefficients ranging between 0.80 and 1.0 indicate “very high validity”. Accordingly, the Korean ASI was found to be a valid instrument for measuring anesthesia surrendering in Korean patients.

Finally, the internal consistency of the Korean ASI, as measured by the Cronbach’s alpha, ranged between 0.71 and 0.88 in our study. Since the reliability of the ASI at the time of development had a Cronbach’s alpha of 0.76, the Korean ASI was also confirmed to be an instrument with high reliability.

However, our study had certain limitations. A validation test of the ASI requires the use of a gold standard scale that has been validated. For this study, we could not use an anesthesia surrendering scale for a concurrent validity test. Thus, the STAI scale was used as an alternative. Nevertheless, our analysis is meaningful because this is the first time a study has verified the reliability and validity of a Korean ASI, and the tool is expected to be used for anesthetic nursing care in Korea.

Conclusion

We translated the ASI developed by Liebenhagen and colleagues for use in Korean patients and tested the validity and reliability of the translated ASI so that it may be used to determine appropriate interventions for anesthesia surrendering. The study findings confirmed adequate reliability and validity for the four dimensions and 26 items contained in the Korean ASI. This evidence supports its use as an anesthesia surrendering measurement tool in Korean healthcare settings. However, sociocultural factors such as attitudes toward healthcare providers, healthcare under a single payer, and expectations from surgery are expected to affect anesthesia surrender. Additionally, the experience of anesthesia surrendering may vary among adults, children, and adolescents. Thus, development of instruments for measuring anesthesia surrendering in these demographics is also required.

Abbreviations

ASI: anesthesia surrendering instrument, EFA: exploratory factor analysis, CFA: confirmatory factor analysis, STAI: state-trait anxiety inventory DNP =: doctors of nursing practice, I-CVI: item-level content validity index, S-CVI: scale-level content validity index, SD: standard deviation, KMO: Kaiser-Meyer-Olkin, WLSMV: weighted least square mean and variance, TLI: Tucker-Lewis index, CFI: comparative fit index, RMSEA: root mean square error of approximation, CR: construct reliability, AVE: average variance extracted.

Declarations

ACKNOWLEDGEMENTS

The authors would like to thank study subjects.

ETHICS DECLARATIONS

Ethics approval and consent to participate

We ensured the ethics of research by having the study approved by the IRB of the Changwon University; obtaining written informed consent from the respondents after briefing them about the purpose of the study.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

References

1. Kwon WK, Kim TY. Special considerations for general anesthesia in elderly patients. J Korean Med Assoc. 2017;60(5):371.376. https://doi.org/10.5124/jkma.2017.60.5.371 Korean.
2. National Health Insurance Service. Medical Care Statistics: Medical treatment result and main indicator [Internet]. Korea: Korean Statistical Information Service; 2019 [cited 2019 Sep 11]. Available from: http://kosis.kr/statHtml/statHtml.do?orgId=350&tblId=DT_920015_N001&vw_cd=MT_ZTITLE&list_id=D1_920015&seqNo=&lang_mode=ko&language=kor&obj_var_id=&itm_id=&conn_path=MT_ZTITLE Korean.
3. National Health Insurance Service. Health insurance major surgery statistics: Surgery status [Internet]. Korea: Korean Statistical Information Service; 2019 [cited 2019 Sep 11]. Available from: http://kosis.kr/statHtml/statHtml.do?orgId=350&tblId=TX_35004_A011&vw_cd=MT_ZTITLE&list_id=350_35004_3&seqNo=&lang_mode=ko&language=kor&obj_var_id=&itm_id=&conn_path=MT_ZTITLE Korean.
4. Van Waes JA, Van Klei WA, Wijeysundera DN, Van Wolfswinkel L, Lindsay TF, Beattie WS. Association between intraoperative hypotension and myocardial injury after vascular surgery. Anesthesiology. 2016;124(1):35.44. https://doi.org/10.1097/ALN.0000000000000922.
5. O’Connor ME, Kirwan CJ, Pearse RM, Prowle JR. Incidence and associations of acute kidney injury after major abdominal surgery. Intensive Care Med. 2016;42(4):521.530. https://doi.org/10.1007/s00134-015-4157-7.
6. Ulrica N, Maria J, Karuna D, Karin H. Postoperative recovery after general and regional anesthesia in patients undergoing day surgery: A mixed methods study. J Perianesth Nurs. 2019;34(3):517.528. https://doi.org/10.1016/j.jopan.2018.08.003.
7. Cardinale JP, Gillespie N, Germond L. Complications of general anesthesia. In: Fox, III C, Cornett E, Ghali G. Catastrophic Perioperative Complications and Management. Cham, Germany: Springer; 2019. P.61-131. https://doi.org/10.1007/978-3-319-96125-5_7.
8. Liebenhagen A, Forsberg A. The courage to surrender--placing one’s life in the hands of the other. J Perianesth Nurs. 2013;28(5):271.282. https://doi.org/10.1016/j.jopan.2012.12.003.
9. Rosiek A, Kornatowski T, Rosiek-Kryszewska A, Leksowski L, Leksowski K. Evaluation of stress intensity and anxiety level in preoperative period of cardiac patients. Biomed Res. 2016;2016:1-8. http://dx.doi.org/10.1155/2016/1248396.
10. Gürsoy A, Candaş B, Güner Ş, Yılmaz S. Preoperative stress: An operating room nurse intervention assessment. J Perianesth Nurs. 2016;31(6):495.503. https://doi.org/10.1016/j.jopan.2015.08.011.
11. Vermisli S, Cukurova I, Baydur H, Yılmaz E. Relationship between preoperative patient learning need and anxiety of patients hospitalized at ear nose throat and head neck surgery clinic for surgical treatment. Kulak Burun Bogaz. 2016;26(2):79.91. https://doi.org/10.5606/kbbihtisas.2016.42385.
12. Sundqvist AS, Holmefur M, Nilsson U, Anderzen-Carlsson A. Perioperative patient advocacy: An integrative review. J Perianesth Nurs. 2016;31:422.433. https://doi.org/10.1111/jocn.14181.
13. Liebenhagen A, Forsberg A, Kristensson J. Development and psychometric exploration of the Anesthesia Surrendering Instrument. J Perianesth Nurs. 2019;34(1):169.179. https://doi.org/10.1016/j.jopan.2017.12.008.
14. Dalenius T, Hodges JL. The choice of stratification points. Scand Actuar J. 1957:3(4):198.203. http:// 10.1080/03461238.1957.10405970
15. Centers for Disease Control. A Study on Sampling Design and Weighting for Korea Hospital Discharge Survey [Internet]. CDC; 2007 [cited 2019 DEC 11]. Available from: http://www.cdc.go.kr/CDC/cms/cmsFileDownload.jsp?fid=28&cid=1322&fieldName=attach1&index=1
16. Wikipedia. List of Demographic Cities in South Korea. 2019. [cited 2019 Sep 11]. Available from: https://ko.wikipedia.org/wiki/%EB%8C%80%ED%95%9C%EB%AF%BC%EA%B5%AD%EC%9D%98_%EC%9D%B8%EA%B5%AC%EC%88%9C_%EB%8F%84%EC%8B%9C_%EB%AA%A9%EB%A1%9D
17. Suhr D. Exploratory of Confirmatory Factor Analysis. Proceedings of SAS Users Group International Conference [Internet]. Cary:1-17: SAS Institute Inc.; 2006 [cited 2019 Sep 11]. Available from: http://www2.sas.com/proceedings/sugi31/200-31.
18. Streiner DL, Norman GR, Cairney J. Health measurement scales: A practical guide to their development and use. New York: Oxford University Press; 2014. p. 30.34, 83.84, 149.156.
19. Spielberger CD. Manual for the State-Trait Anxiety Inventory (STAI). Palo Alto, CA: Consulting Psychologists Press; 1983.
20. Hahn DW, Lee CH, Chon KK. Korean adaptation of Spielberger’s STAI (K-STAI). Hanguk Simni Hakhoe Chi Kongang. 1996;1(1):1.14. Korean.
21. World Health Organization (WHO). Process of translation and adaptation of instruments [Internet]. Geneva: WHO; c2015 [cited 2019 Sep 1]. Available from: http://www.who.int/substance_abuse/research_tools/translation/en/.
22. Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 2007;30(4):459.467. https://doi.org/10.1002/nur.20199.
23. Hinkin TR. A brief tutorial on the development of measures for use in survey questionnaires. Organ Res Methods. 1998;1(1):104.121. https://doi.org/10.1177/109442819800100106.
24. Velicer W, Eaton C, Fava J. Construct explication through factor or component analysis: a review and evaluation of alternative procedures for determining the number of factors or components. In: Goffin R, Helmes E, eds. Problems and Solutions in Human Assessment. New York: Springer US; 2000. p.41.71.
25. Schmitt TA. Current methodological considerations in exploratory and confirmatory factor analysis. J Psychoeduc Assess. 2011;29(4):304.321. https://doi.org/10.1177/0734282911406653

26. Muthén B, Kaplan DA. Comparison of some of the methodologies for the factor analysis of non-normal Likert variables. Br J Math Stat Psychol. 1985;38:171-189. https://doi.org/10.1111/j.2044-8317.1985.tb00832.x
27. Bae BR. Structural equation modeling with Amos 19: Principles and practice. Seoul: Chungram Books; 2014. p. 1.673. Korean.
28. Seong TJ. Validity and reliability. Seoul: Hakjisa Corp.; 2010. p. 35.82. Korean.

Tables