Advances in VR technology enabling the production of high-quality simulation environments that train the management of patient cases are dramatically affecting educational concepts, teaching activities, and the outcome of student learning. In the present study, we developed and implemented a novel VR-based training environment for medical emergencies, evaluated its acceptance by students, and explored selected psychological and didactic aspects that were associated with the benefits of the VR method and teaching intervention.
A primary objective of the study was to demonstrate that implementation of a VR-based training session as a compulsory course in the medical degree curriculum is feasible and well accepted by students. With the chosen format, we did not experience any administrative or technical difficulties and thus were able to enroll 252 participants, of whom 227 took part in our study. Compared to other recent work examining the feasibility and outcomes of VR-based training (Birrenbach et al., 2021; Lerner et al., 2020), the sample size of our study was considerably large. Investigating sociodemographic parameters, we experienced that the willingness to act as AP was greater among male participants, of whom a higher percentage had prior experience in FPG. This is in line with recent publications demonstrating that twice as many men compared to women play computer games several times a week (Pawlik, 2022). Possibly as a consequence, female students, especially those with less computer affinity, exhibit greater skepticism towards VR technology for teaching purposes (Walter et al., 2021). For any broad application in the future, the reservations of these individuals in particular should therefore be identified and addressed specifically. However, considering the whole teaching approach, evaluation of the VR-based training session revealed a wide acceptance among both male and female participants. Of note, debriefing and the discussion of open questions with a tutor following the exposure to VR were seen as an integral part of the course. This speaks in favor of embedding VR into a sound didactic concept rather than having students experimenting on their own (e.g., in a “VR lab” or at home). Beyond any didactic principles, the guided application of VR-based teaching is mandatory to promote the necessary media competence among medical students (Walter et al., 2021).
Our second aim was to investigate the technical aspects of the simulation, namely simulation sickness and presence. In our pilot study, VR exposure caused simulation sickness. However, reported values in the three elements nausea, oculomotor, and disorientation did not indicate high levels of discomfort in our sample. Referring to a comprehensive literature review including 55 studies recording simulator sickness scores among > 3000 VR participants (Saredakis et al., 2020), values obtained in our study even appeared to be relatively low in comparison. In light of the fact that only few participants had any active experience with VR prior to the study, STEP-VR as a simulation-based training environment can clearly be offered to a wider audience without any concern relating to adverse events such as vegetative reactions. Completion of actions only possible in one ground plane, which was congruent with the actual bodily position of the participant, might be one reason for the good tolerance. Additionally, the restrictive use of content-rich text boxes and a high average frame rate per second attained by the equipment we employed reduced the perception and perhaps effects of simulation sickness. It is worth noting that research in healthy participants and vestibular patients revealed a decline in motion sickness susceptibility with increasing age (Paillard et al., 2013). In our study, the majority of participants were 22–25 years old and therefore fairly young.
We observed a high degree of presence among AP, who perceived the VR simulation as very realistic. This was confirmed by OBS, but to a significantly lower degree. However, when exploring the interaction with patients on item level, authenticity was only rated slightly above average by AP and average by OBS. One explanation for this finding may be attributed to the fact that the virtual patients were portrayed in a rather motionless manner and communication could be performed only via a dialogue menu. Owing to the focus lying on training procedural skills and acceptance of the lack of haptic handling in the VR environment, many procedures were only depicted in a simplified manner, e.g. the insertion of an IV cannula as “drag & drop” to the patient’s arm. Therefore, future development of the software should aim to include more realistic patient modeling and characteristics including responsive and life-like gestures. More research is also needed to clarify how real-time communication affects the perception of high-level patient interaction in VR. However, this requires the integration of sophisticated applications to enable speech recognition. Recent developments in natural language processing in virtual standardized patients only demonstrated a response accuracy of 74–92% in dialogues (Campillos-Llanos et al., 2021; Maicher et al., 2017). Especially in the management of emergencies, even higher levels of precision might prove mandatory, in order to prevent the misconception of information under time pressure.
Thirdly, an aim was to evaluate the extent and quality of psychological distress among the participants of the VR simulation. On average, AP reported being moderately challenged during the management of virtual patients. More specifically, EFA identified three distinct factors corresponding to subtypes of perceived stress. The first two, “sense of control” and “challenge to perform” represent more classic concepts of stress and have been reported as appearing in various contexts of medical education (Alzahem et al., 2011; Sanders and Lushington, 2002). Both parameters correlated negatively with the use of FPG, which is in line with enhanced spatial navigation and visuomotor coordination in this subgroup (Momi et al., 2018; Wu and Spence, 2013), presumably facilitating the use of VR-based simulations and decreasing perceived stress. A third identified factor of perceived stress entitled “social interaction to learn” is based on the assumption that the possibility of publicly unmasking knowledge deficits, as well as the feeling of being observed, also exert discomfort on AP. This might be a particular concern, as it is not possible to notice the audience around when wearing an HMD. Interestingly, by comparing age groups, we revealed that young participants experienced this factor significantly to a greater extent than their older counterparts. This suggests that age might have a moderating / mediating effect on the social experience in virtual reality, a correlation that has not yet been investigated in more detail. Future studies are needed to elucidate the role of (social) stress in VR-based simulations by comparing different settings (with and without observers).
Finally, we were also interested in the perceived learning benefits and the relation to putative associated factors. Estimated learning success was certainly high for both AP and OBS at the end of the training session. In our comparative analysis, AP rated learning benefit significantly higher. This might be attributable to the fact that a strong feeling of presence has been shown to enhance skill acquisition and factual memory (Lerner et al., 2020; Makowski et al., 2017; Piccione et al., 2019). Higher motivation scores among the AP compared to OBS, possibly owing to higher vigilance and increased activity whilst training, might serve as another explanation for the better learning success for AP. The promotion of motivation and self-efficacy are key concepts of simulation-based learning and have already been reproduced in the setting of VR-based approaches (Bracq et al., 2019; McGrath et al., 2018).
Performing EFA on the “estimated learning success” measure revealed that items contributing to the “didactic value” of the used VR simulation were rated very highly among AP, while the items comprising translation into an “individual learning benefit” were rated somewhat lower. A possible explanation for this difference is the limited time in the training session, which was approximately 30 minutes per AP. Furthermore, the version of the VR software in use did not provide any immediate and specific feedback interface (such as a text-based performance report) to the students currently active. Instead, feedback was only provided during debriefing by the tutor, which might lead to a lower rating for the learning benefit acquired through the training scenario.
Investigating the potential influence of perceived stress on estimated learning success among AP, we were able to demonstrate that the factor “sense of control” revealed a typical inverse U-shaped correlation to both subtypes of “estimated learning success”, resembling the relationship that Yerkes and Dodson described for arousal and performance in challenging tasks (Yerkes and Dodson, 1908). According to one study we found in the literature, stress responsiveness was a predictor of good performance (Keitel et al., 2011), while most work highlights the impairment of cognitive skills under excessive stress in high-fidelity surgical or medical emergency simulations (Mauriz et al., 2021; McGraw et al., 2013; Wetzel et al., 2010). The results of the aforementioned study are also concordant with the negative correlation that we discovered for the second factor “challenge to perform” and “estimated learning success”. These findings indicate that support should be provided, e.g. by means of a student assistant or a digital tutor to mitigate excessive stress caused by the demands of this teaching method. Interestingly, on the other side of the inverse U-shape, students with a high “sense of control” also reported poor learning. These participants may have had previous experience in dealing with medical emergencies, and/or the theoretical content presented was already familiar to them such that they found no gain in competence. However, owing to the lack of objective performance data, we cannot conclude whether this self-estimation was justified or misleading. In contrast, “social interaction to learn” was the only factor of “perceived stress”, which exhibited no correlation to the two factors of “estimated learning success”. Presence of an audience is described as a stressor in conventional healthcare simulation approaches (Shearer, 2016; Taylor et al., 2010) and correlated with signs of increased stress and a trend towards worse learning outcomes in a physical intubation model (Bensouda et al., 2018). However, the audience in our training session was generally perceived as supportive, as indicated by the results on item level, which might have somehow balanced positive and negative effects, thus making it difficult to establish any meaningful correlation.
Strengths of the Study
In our study, we used state-of-the-art technology to enable the best possible learning success by high-presence VR training and mitigate major side effects to a large extent. The number of participants was comparably large, coming from two unbiased semesters, and was thus a statistically meaningful dataset. In comparison with voluntary or incentive-driven studies to recruit students, this compulsory training session is also likely to include participants with more critical attitudes towards VR technology. In this way, the participants of our study may well prove to be representative of the heterogeneity of medical students in general.
Limitations of the Study
One limitation of the study is that the sample was restricted to undergraduate students in one academic year and at one institution. Performed at a single medical school in Germany, further studies at other universities and with different student cohorts will be required to assess the validity of the results. In addition, as the course was performed exclusively in small groups under guidance, the additional influence of both the student assistant and the observing peers could not be clarified. As participation in the role of AP was voluntary, we are clearly unable to exclude selection bias in this group.
A second limitation is the use of adapted or newly created questionnaires. As the combination of the teaching method (VR-based simulation) and setting (small group teaching) is a relatively new approach, the currently available questionnaires, especially to evaluate perceived stress, were not considered applicable and had to be adapted. Of course, these strongly modified / newly created questionnaires await re-validation and currently restrict our ability to compare with other work.
Most importantly, all the results presented are subjective data recorded using surveys. While, at least to our knowledge, no concept exists for the objective measurement of simulation sickness or feeling of presence (based on physiologic parameters, for example), the reported “estimated learning success” certainly needs confirmation through a randomized-controlled trial and test performance data derived from a pre- and post-intervention study design. Additionally, future work might place emphasis on objective measurements of stress, such as heart-rate variation or electrodermal activity (Bhoja et al., 2020; Kyriakou et al., 2019), and investigate any possible habituation as a result of the repeated use of VR-based simulation.