Study design and Participants
An open-label, parallel (1:1), comparative study design was conducted among undergraduate nursing students at Hiroshima University, Japan. Participants were third-year nursing students enrolled in the Practicum in Adult Nursing in 2023.
In the third year, students study each nursing science area after completing basic nursing subjects; they study theory and skills in parallel, and after completion, they go on to clinical practice.
Data collection procedure
Since this study was implemented in the regular class, the data for this analysis was obtained after the class completion as an opt-in basis. Prior to commencement of the specified class explained below, students were informed of the purpose of this experiment, procedure, voluntarily of participation, no disadvantages of withdrawal and/or no participation, and the secondary use of the data. Then, students submitted the written consent form for providing their data submitted in the class to the researcher who is not involved in the course.
After a 3-hour theoretical class of patient management on ventilator and closed-suction principles of mechanically ventilated patients in an ICU focused class, a seven-question knowledge pretest was conducted for all the available nursing students enrolled in the course; a total of 62 students completed the pretest for randomization.
Randomization and allocation
To assure equal distribution in terms of academic achievement or intelligence, the pretest score was used as a factor to block randomize the students into the 360 VR video training group (360 VR group: an intervention group) and the face-to-face traditional training group (face-to-face group: a control group). A block size of 2 resulted in 31 blocks, and students assigned from each block into the face-to-face and 360 VR groups. Figure. 1 shows the study procedure during the course.
Development of the 360 VR Video
A video of a certified nurse in critical care performing closed-suction procedure in a high-fidelity mannequin-based simulation was recorded with Insta360 (ONE X2). The procedure of suctioning was conducted in a step-by-step manner following a checklist developed by the research team. The video involved a voice over of the instructor explaining the procedure, and the nurse performing the procedure. This was edited using Adobe Premiere Pro ver (23.2.0); the final product of the video was 18 minutes divided into three phases for better understanding of the procedure: Preparation and assessment phase; instrument identification and oral suction phase; tracheal suction and patient report phase (Figure. 2).
360 VR group
After the theoretical class, the 360 VR group watched (see Figure. 2a) the 360 VR video using the HMD of Meta Quest 2 individually; after then, the students answered the VR safety questionnaire developed by the researcher. For hands-on practice, the students engaged in self-directed practice with group feedback in 5 subgroups for 90 minutes; each subgroup containing 6 students except one with 7 students.
Face-to-Face group
The face-to-face group attended the face-to-face demonstration of the same nurse that carried out the procedure in the video using the researcher developed checklist. After then, the students engaged in hands-on practice in 5 subgroups for 90 minutes; each subgroup containing 6 students except one having 7 students; clinical instructors (experienced registered nurses) were present in each subgroup during the practice.
For both groups, the critical care nurse captured in the video was available between the intervention and control groups to address the students’ concerns and questions. After the hands-on practice, the control group was provided the usual supplemental procedural video for closed suctioning attached to their study material, and the intervention group could either rewatch the 360 VR video with VideoLAN Client (VLC) player or request for use of the HMD at their convenience.
A week after the skill demonstration and hand-on practice, a total of 9 instructors (nursing faculties and registered nurses) assessed and evaluated the students’ closed suction skills (including oral suction) using the procedure checklist. To ensure consistent evaluation, a session was held to communicate the grading criteria. The students were requested to perform the procedure in an Objective Structured Clinical Exam (OSCE) and evaluated by clinical instructors. At the end of the skills evaluation, both groups answered the google forms on knowledge, satisfaction, and confidence. In addition, the 360 VR group answered the VR perception questionnaire. According to the study procedure, it was planned to explore the perception of the face-to-face group on VR by watching the 360 VR video after skills evaluation, but none of the students watched the 360 VR video.
Evaluation outcomes and Instrument
Evaluation was conducted under the framework of psychomotor skill, knowledge, confidence, and satisfaction of the closed tracheal suction technique. For the 360 VR group, video perception and VR sickness symptoms were also explored.
Closed tracheal suction checklist (including oral suction)
A Closed tracheal suction (including oral suction) checklist was developed from available literature review of evidence-based practice (32–38) to evaluate the skills of the nursing students. To ensure the validity of the checklist, the developed checklist was submitted to certified critical care nurses of Hiroshima University Hospital, and a version of procedure checklist available at the unit was received by the researchers as a guide. The checklist was further modified resulting in 38-steps procedure (items) checklist. To assign grades to the steps, each step was dichotomized to critical and non-critical. For a critical step, a score of 4, 2, 0 was assigned to satisfactory, unsatisfactory, and not performed, respectively; a score of 2, 1, 0 was assigned to a non-critical item as aforementioned for the level of performance. The criterion for the three level of performance was outlined for each item for consistent rating. To establish the content validity, using Lynn’s (1986) technique (39), the checklist was submitted to four certified critical care nurses; the relevance, accuracy of terminology, and grading of the steps were evaluated. The checklist was reviewed and modified based on the experts’ opinion, and the final checklist score ranges from 0 to 64. The item-level content validity index (I-CVI) was computed for each item; the scale-level content validity index of universal agreement (S-CVI/UA) was 0.97.
Knowledge test scores for suction in ventilated patients
The researchers developed practical knowledge questions on tracheal suctioning. A total of 24 questions were outlined, and after the researchers’ group discussion, it was reduced to 17 questions. This was pretested with two certified critical care nurses for an expert-driven pretest to assess the face and construct validity of the questionnaire. The nurses answered the questionnaire, and suggested modifications or discard of some questions were addressed accordingly. After then, two questions were added, and 19 questions were pretested with two different certified critical care nurses. In order to achieve a 20-questions questionnaire, one question was included to the final expert-driven pretest. From the 20 questions developed, 7 questions, which were identified to address the basics and overview of tracheal suctioning, were used for pretest. For the post test, the total of 20 questions was administered; the correct answer is given 1 point, and the incorrect answer is given 0 points.
Degree of satisfaction and confidence in learning
To assess the students’ satisfaction and confidence, the Japanese version (40) of the Students Satisfaction and Self-Confidence in Learning by the National League for Nursing (NLN) was adopted to assess the students’ satisfaction and confidence. It consists of 13 questions in two different questionnaire; five questions for satisfaction and eight questions for self-confidence. The questionnaire is on a 5-point Likert scale form 1 = strongly disagree, 2 = disagree, 3 = undecided, 4 = agree, and 5 = strongly agree; the higher the score, the higher the satisfaction and confidence. The satisfaction score ranges from 5 points to 25 points and confidence from 8 points to 40 points. As reported by NLN, the Cronbach alpha for the satisfaction and self-confidence are 0.94 and 0.87, respectively. In this study, the Cronbach alpha for satisfaction and self-confidence is 0.93 and 0.92, respectively.
Perception of VR use
The perception of students was assessed with an adapted tool from Peart et al(41) study. The tool was developed based on the Technology Acceptance Model (TAM) and includes 6 items on a 7-point Likert scale (strongly disagree, disagree, somewhat disagree, cannot decide, somewhat agree, agree, and strongly agree) and 2 open ended questions. However, to fit in our study, only one of the open-ended questions was retained (is there a way that the use of X could be improved). The tool was forward and back translated by the researchers, and an additional two questions (1. How was the comfort and ease of understanding of VR; 2. If you notice anything else or have any impressions, please write it down) were added to the open-ended questions. In Peart et al(41), the Cronbach alpha was > 0.7. As the tool was translated and adapted, the Cronbach alpha in this study is 0.61
The safety questionnaire was developed to explore the side effects of using the VR. The VR sickness symptoms explored in the questionnaire were based on the Meta Quest 2 health and safety manual and other VR studies(20, 26, 42, 43). It consisted of two questions (1) did you have any symptoms (2) please, pick all that applies. Ten symptoms were provided as option with an “other” option to allow for free answer.
Ethical consideration
The study was conducted according to the Declaration of Helsinki and the Ethical Guidelines on Clinical Studies of the Ministry of Health, Labor and Welfare of Japan. This study was reviewed and approved by the Hiroshima University Epidemiological Ethics Review Committee (E2023-0054). One of the researchers who is not part of the adult health nursing course explained the study purpose and data collection procedure, and consent was received from students agreeing to secondary use of the data. It was explained that not consenting to the provision of data obtained in class would not affect the class grade in any way, and there would be no disadvantage on the part of the students; a written informed consent was obtained from all the students. Therefore, to ensure the class instructors would not be able to know which students had consented, consent procedure and data extraction were done by the research coordinator.
Data analysis
Data analysis was performed with JMP, Pro 17 (SAS Institute Inc., Cary, NC, 1989–2023). Due to the non-normal distribution, descriptive data was presented in median, quartile, frequency, and percentage. Wilcoxon rank-sum test was used to compare the two groups in skill, knowledge, satisfaction, and confidence. The perception was presented as frequency, and percentages based on the level of agreement on the Likert scale; the open-ended questions were analyzed following the conceptual content analysis method to describe the attitudinal and behavioral responses of the students toward the 360 VR video. The VR side effects was presented as frequency and percentage. The level of statistical significance was considered at 0.05.