Results-Stage 1. Problem Identification
We derived these perspectives by interviewing four chiefs of nursing education.
Requirement of Education in Difficult-to-use Devices
Using difficult-to-use medical devices in healthcare can pose a significant challenge for medical staff due to the increased risk of errors, negatively impacting patient outcomes. Proper education and training on these devices are essential to ensure that medical staff can use them safely and effectively.
Lack of Resources; Space, Instructor, Time, and Cost
The lack of training resources in healthcare can be a significant challenge for healthcare organizations and medical staff. Training is essential to ensure that healthcare professionals possess the necessary knowledge, skills, and abilities to provide safe and effective care to patients. However, many healthcare organizations face barriers in providing adequate training resources to their staff, which can negatively impact patient outcomes.
One of the main challenges is a shortage of funding for training and education. Training budgets may be reduced or eliminated, making it difficult for healthcare organizations to provide the necessary resources to their staff.
Another challenge is the lack of adequate training time. Healthcare professionals are often required to work long hours, and finding time to attend training sessions and complete the necessary coursework can be difficult. This lack of time can make it difficult for healthcare organizations to provide training tailored to the specific needs of their staff.
Results-Stage 2. Needs Assessment
A series of interviews was conducted with two educators and two operators, with each interviewee holding a distinct role. The interviews were initially planned as semi-structured, allowing for flexibility in question selection based on the specific circumstances encountered.
Following the interviews, we identified crucial factors to consider when selecting educational topics. Educators highlighted the lack of training devices and instructors as significant challenges. Additionally, they emphasized the need for training in technically demanding skills. Trainers expressed a preference for educational topics that required hands-on practice. Their rationale stemmed from the recognition that devices with higher complexity are frequently utilized in the care of critically ill patients. The nurses' proficiency in operating such devices directly impacts the patients' prognoses.
Table 1
Trainer’s and operator 's Interview Responses (Table 1)
The details of the work
|
Identify training needs as well
Thinking about the training contents that fit the characteristics of AR devices
The production of educational proposals
Establishing a class operation plan
Buying Equipment
Rental of educational medical devices
Purchasing and subscribing to the operation of a class
|
Reasons for choosing a topic
|
Conventional training was difficult due to the lack of ECMO devices
Educators who can train ECMO have limitations
Not many devices or opportunities to practice
I believe that training on how to use the device will be appropriate for self-learning methods
|
Considerations when creating a program
|
Consideration is given to reducing the burden of education, even if you have less experience
Meet the level of nurses with the right experience for the program
Standardize the training phase
Apply it to the training subjects in the development stage and check their understanding
Try to refine the confusing order and educate them in detail
|
Issues encountered at the production
|
Not enough objects are available for creation
Insufficient features in the software
Slower work due to heat generation and developer fatigue
The recognition rate of AR devices is not high, so the working time is increased
|
Issues encountered at the training
|
Internet experience and connection are not smooth
The program has limitations in controlling difficulty
Trainees wearing glasses or difficulties in wearing masks exist
Difficult to see or focus clearly
Difficulty in recruiting educational devices and places
HoloLens 2 has different continuous use time and actual use time
It is not considered appropriate for nurse education in high age groups
High fatigue during training
|
Results-Stage 3. Goals And Objectives
Selected objectives
Following the results of interview objectives selected of ECMO and ventilator as difficult-to-use devices. Ventilators were chosen as important but difficult-to-use devices. The device’s complexity and many connections make it difficult for new nurses. ECMO is a high-risk medical device that COVID-19 patients must have. In case of ECMO problems, nurses must immediately find a solution. Owing to its rarity, even experienced nurses have not encountered this problem. Rare but appropriate training was needed as it could be a risk to the patient, so it was chosen as the ECMO problem situation.
Selected goals
We aimed to develop a sustainable AR learning platform. AR learning platforms can provide many benefits for education, such as increasing engagement and motivation for learners, providing interactive and immersive experiences, and allowing more personalized learning. Key considerations for developing a sustainable AR learning platform include designing for scalability and accessibility, incorporating user feedback to improve the platform, and ensuring that the platform is cost-effective to maintain and operate over time. We focused on self-directed learning and hands-on practice.
Results-Stage 4. Educational Strategies
Step 1. Development AR-based instructions
The ECMO training program consists of 45 slides with 4 parts, each explaining how to operate ECMO devices, handle device disruptions, respond to “Low Bat” alarms, and address “SIG” alarms. Each part consists of about 8–10 steps to train how to handle each situation.
The ventilator education program comprised 3 parts and 46 slides. Each part consisted of 26 steps of ventilator setting and pre-use inspection, 7 steps of application, alarm setting, and training content evaluation. The training was conducted by checking supplies, power and Wall O2, medical gas connection, exhalation cassette connection, power on, pre-use check, test tube connection, target and application method selection, humidifier power on, mode setting, parameter setting, patient connection, monitoring after patient application, and alarm setting.
Step 2. Adoption of new technology
A nurse from the nursing education team developing a nursing education program. The training will utilize a servo-I mechanical ventilator from Maquet and the PLS ECMO device was Rotaflow II System from getinge. An attempt was made to incorporate a 3D guide for hands-on practice and to utilize video for enhanced understanding. The AR education program is being developed using the Microsoft Dynamic 365 Guide program. The program’s content will be delivered to users through a Microsoft HoloLens 2 device.
Step 3. Operating plan
N ventilator devices and N ECMO devices were located in the simulation laboratory in the hospital, and a place was prepared to provide the minimum required space for AR training, at least 3 x 3 meters of space to provide enough physical room for the trainee to move and perform the necessary tasks[23]
Results-Stage 5. Implementation
The AR education platform was operational for a period of two months. Training sessions were scheduled from 9 a.m. to 5 p.m., allowing nurses to choose their preferred date and time within this timeframe. For the implementation of the program, five Hololens2 devices were utilized, along with two laptops for supervisor screen connection and two large screens for the research environment.
Furthermore, it is important to note that a total of 24 individuals enrolled in our survey and interview study. However, during the actual duration of the program, there were 28 participants who actively engaged in the education sessions.
Results-Stage 6. Evaluation
Participants
From January 1 to February 3, two educators and two education directors trained 24 intensive care unit nurses.
Demographics
During the study, a total of 24 trainees were instructed, and surveys and interviews were conducted. The participating trainees had varying levels of experience with a maximum of 13 years and a minimum of less than 1 year. The average level of experience was found to be 3.75(SD3.9) years. Of the participants, twelve were trained in ventilator usage, while the remaining twelve educated on extracorporeal membrane oxygenation. In terms of gender distribution, 17 participants (70.83%) were women, and 7 participants (29.17%) were men.All participants belonged to the general nursing field. Among them, 15 nurses (62.5%) were employed in the Medical Intensive Care Unit, while 9 nurses (37.5%) worked in the Surgical Intensive Care Unit. Additionally, the majority of the participants (24, 100%) possessed smartphones, tablet PCs, and laptops (23, 95.83%). Prior to the instruction, 13 nurses (54.17%) already experienced a head-mounted display.
Table 2
Demographics of the study participants (N = 24) (Table 2)
Variable
|
N
|
Percent
|
Method
|
|
|
ECMO
|
12
|
50%
|
Ventilator
|
12
|
50%
|
Sex
|
|
|
Male
|
7
|
29.17%
|
Female
|
17
|
70.83%
|
Classification of Internal/surgical Medicine
|
|
|
Internal Medicine
|
15
|
62.5%
|
Surgical Department
|
9
|
37.5%
|
Family group
|
|
|
Nurse
|
24
|
100%
|
Department
|
|
|
Internal Medicine Intensive Care Unit
|
5
|
20.83%
|
Neurology Intensive Care Unit
|
2
|
8.33%
|
Cardiology Intensive Care Unit
|
2
|
8.33%
|
Oncology Center Internal Medicine Intensive Care Unit
|
6
|
25%
|
Pediatric Cardiovascular Surgery Intensive Care Unit
|
1
|
4.17%
|
Neurosurgery Intensive Care Unit
|
1
|
4.17%
|
Cardiac Surgery
|
1
|
4.17%
|
Cardiac Surgery Intensive Care Unit
|
3
|
12.50%
|
Oncology Center Thoracic Surgery Intensive Care Unit
|
1
|
4.17%
|
Oncology Center Surgery Intensive Care Unit
|
1
|
4.17%
|
Pediatric Intensive Care Unit
|
1
|
4.17%
|
Variable
|
mean (sd)
|
median (Q1, Q3)
|
min, max
|
Work experience (Years)
|
3.75 (3.9)
|
3 (0, 6.25)
|
0, 13
|
Result of System Usability Scale(SUS) and Technology Acceptance Model(TAM)
In the usability test, the item "I think that I would like to use this system frequently" and "I don't think the system unnecessarily complex" received the highest rating of 4.38 out of a possible 5, while the lowest rated item "I thought there was too much inconsistency in this system" received an average score of 1.83. The responses regarding technology acceptance were classified into four categories: perceived usefulness, perceived ease of use, perceived enjoyment, and intention to use. The survey consisted of 15 questions on a 7-point scale. The item with the highest score was "It is fun to use" with a score of 6.71, while the lowest rated item was "It is easy to use" with a score of 5.17. In other survey results, age, sex, department of work, and years of work did not affect satisfaction with education or usability. All response results for the survey have been attached as Supplementary 1.
The evaluation of the AR learning
A total of 24 nurses utilized the AR platform, out of which 15 voluntarily agreed to participate in the study. These participants completed surveys and took part in interviews related to their experiences with the AR education platform.
The platform was located outside the hospitals. During the operation period, 24 nurses used the platform, 11 trainee and 4 trainers agreed to participate in this study by conducting surveys and interviews. The responses of the 11 trainees who participated in the interviews have been summarized in Table 3.
Table 3
Trainee's Interview Responses, an internal factor
Factor
|
Phase
|
Contents
|
1–1. Intrinsic Motivation
|
1) Goals for this education
|
I wanted to experience education through Augmented Reality technology (3)
I want to get an education that will help me with the job of a nurse (4)
Expect better results than traditional education (4)
Need to learn about devices that have no practical experience (3)
|
|
2) Reasons for participation
|
Recommendation from other nurses (4)
I have relevant educational experience (1)
Curiosity about new methods of education (4)
Solving questions about the contents of the training (1)
|
1–2. Self-management
|
3) thoughts on existing nursing skills education
|
Conventional training alone can confuse nursing techniques (1)
Takes a long time to understand concepts (1)
The content of training varies depending on the educator’s propensity and level (1)
|
1–3. Desire for learning
|
4) Preference among conventional education/self-directed learning
|
Conventional learning methods 6/11 (55.5%)
Self-directed learning 5/11 (45.5%)
|
|
5) Reasons for your preference
|
Conventional learning methods (6/11)
Feedback and questions can be solved (3/6)
Test or tutor-integrated verification (1/6)
Self-directed learning is difficult in the educational system (1/6)
Nursing techniques are always applied to the patient and require confirmation from the tutor (1/6)
Self-directed learning (5/11)
It’s good that I can learn as much as I need (1/5)
Existing education may not match the speed at which the instructor delivers information (2/5)
I feel many things while doing it (1/5)
Self-directed learning is necessary for self-development (1/5)
The conventional method has little left after the evaluation is over (1/5)
|
1–4. Awareness
|
6) Face-to-face/non-face education preferences
|
Face-to-face (6/11,54.5%)
Books are more comfortable than videos (1)
Unaccustomed to non-face-to-face training (2)
I think the educational effect is better (1)
Feedback is available (2)
Non-face-to-face (5/11 45.5%)
Focus on lecture content without worrying about surroundings (1)
Can study at speed (1)
Features such as pauses are available (1)
No space-time constraints (2)
Repeatable (1)
|
|
7) Whether the preference for learning methods has changed before and after the outbreak of COVID-19
|
Changed 4/10 (40%)
- Getting used to non-face-to-face after COVID-19
- Recognizing that non-face-to-face contact can also be concentrated
No change 6/10 (60%)
|
Table 4
Trainee's Interview Responses, an External factor
2 − 1.Extrinsic Motivation
|
8) whether this education will improve technical skills and help you evaluate your job
|
Will improve (9/11)
It’s a lot more helpful than just a theory (1)
It will be helpful to improve your technical skills by watching videos (1)
I can definitely learn the confusing parts (1)
It helps me think about how to deal with it in the clinic (1)
It’s easy to understand (1)
I will be able to perform with confidence, which will help my job (1)
Will not improve (2/11)
It is helpful to acquire concepts but not to improve technical skills (1)
It will not be a direct help (1)
|
2–2. Concrete experiences
|
9) Experience wearing an HMD
|
Yes (8/10) No (2/10)
|
|
10) Experienced AR/VR
|
VR game (6/8)
Skill training (2/8)
|
2–3. Reflective observation
|
11) Advantages of education through AR/VR
|
More realistic than virtual reality
Education at one’s own pace
Provide specific information
To be able to see the video and the description at the same time
|
|
12) Disadvantages of education through AR/VR
|
The AR machine itself is unfamiliar
Curiosity and attention tend to focus on the device’s operation rather than the training contents
Difficulty in asking questions
Difficulty moving content freely
Unaccustomed to using the instrument
The handwriting is small
Difficult to focus on content
|
|
13) Whether the contents of the training were appropriate
|
Suitable (9/11)
This is a high-risk device, and it is difficult to use daily. So it's less burdensome because I can practice it without using it in real patients.
Usually 2/11
AR education is good, but there is no big difference from education using existing videos
|
|
14) Supervisor help required while education
|
None (2/11)
Technical Assistance Required (9/11)
|
|
15) Advantages over conventional education methods
|
It was nice to repeat learning according to my own educational pace
Easily find my errors through comparison between real and video
Repeatable multiple times
|
|
16) Disadvantages over conventional education methods
|
Difficult to operate the instrument (8)
Limited time due to lack of practical equipment (1)
Lack of content for in-depth learning (1)
Since it is self-learning, it is difficult to solve once there is a blockage (1)
|
2–4. Abstract Conceptualization
|
17) Evaluation of Hands-on Education
|
Effective by touching and learning the actual device (8/11)
Effective by allowing you to learn situations that you won’t actually experience (1/11)
Effective only when learning new concepts (1/11)
Conducting hands-on education for many nurses will require significant resources, including manpower. (1/11)
|
|
18) whether the existing training replaceable
|
Conventional training can be replaced (4/11)
Only some of the training content can be replaced (7/11)
|
|
19) Will AR education be able to improve nursing skills
|
It is thought that it will be difficult to solve the limitations of existing education only with AR devices (1)
I can practice what you want to practice repeatedly(1)
Similar to self-practice watching a video (1)
I think it’s an educational method that helps you understand (1)
I think it would be more helpful to learn on your own than the conventional learning method (1)
Help to implement situations that are unpredictable or difficult to experience (2)
Iterative learning is possible (1)
|
2–5. Active experimentation
|
20) Most appropriate target for this training method
|
New nurse (4)
Nurse with experience but no specific device experience (2)
Department transfer, reinstatement, other department helper (4)
Nurse with machine experience (3)
Nurse student (1)
Target Restrictions (1)
|
|
21) Most helpful thing about self-learning using AR
|
Less pressure (2)
Self-learnable (1)
Save time (2)
Review as much as you want (3)
Ability to learn about unusual situations (1)
Repeatable Learning(1)
Selectable for learning time(1)
Educator time saved(1)
less burden on human relationships(1)
|
|
22) Most worried thing about self-learning using AR
|
Limitations of clinical application(1)
No sharing of personal experiences or tips(1)
It will take a long time(1)
Even if you don’t like this approach, you may have to participate in it(1)
Worries about system errors(1)
Limitations due to device quantity limitations(1)
|
Overall, the evaluation of AR-based education was positive with participants indicating that AR could improve their actual clinical performance. AR is a technology suitable for self-directed or hands-on learning theory those who interested in experience and self-learning rated the program particularly well. Nurses were found to be receptive to education using new technology.
When asked if they needed assistance with the curriculum, no participant answered negatively regarding the content. However, some participants did express a need for help in adapting to new devices and technologies.