The aim of this paper was to assess the usability and user expectations of an augmented reality application that allowed health science students and health professionals to visualize internal anatomical structures during training of the clinical technical skills for the performance of invasive techniques.
ARSim2care application was developed as part of a European project entitled “ARSim2Care: Augmented Applications in Clinical Simulation”, in collaboration with three European higher education institutions. The aim of this project was to offer an innovative method for teaching and training clinical technical skills. Traditionally, training on these skills comprises a theoretical explanation and the corresponding practical demonstration of the procedure by the teacher, combined with the supervised practical training on the procedure by the students (Baillie and Curzio 2009; Weller et al. 2012; Kemery and Morrell 2020). This practical training uses manikins or task trainers that rarely include new technology beyond the materials used to make them or some mechanical or electronic interfaces (Lioce et al. 2020). The use of AR by the designed application offers a new way of learning to perform techniques such as intramuscular injection, nasogastric tube insertion, suctioning via tracheostomy tube or endotracheal intubation. This may enhance the internalisation of the technique by facilitating its correct performance, as it is possible to visualize the internal structures. Implementing the ARSim2care application in the Microsoft HoloLens® offers the possibility of using this technology while keeping hands free, interacting with them through voice commands or hand gestures (Klinker et al. 2020). In addition, the use of this type of Head-mounted displays (HMDs) devices allows, if necessary, sterile conditions to be maintained in healthcare environments. This is one of the most advantageous aspects of the application.
The results for the evaluation of the usability show that the user experience with the ARSim2care application was considered as good or excellent by 62.4% of the participants, with the mean score of 73.15 (SD 15.40). Only 7 students reported a problematic level of usability, difficulties in handling that could be solved by increasing the time to become familiar with the device26. This result is higher than the SUS scores obtained in similar studies. For example, in the usability evaluation of an AR triage scenario visualized in a HMD the mean SUS score was 57 (Anderson et al. 2021). Similar result (58.96) was obtained for the usability assessment of an AR prebrief activity using a head-mounted display which was designed to offer orientation information before a clinical simulation activity (Anderson et al. 2022).
As suggested by Frost et al, augmented and mixed reality technologies, in their various forms, provide the user with a greater ability to be exposed to ideas that would otherwise be difficult to visualize and contextualize (Frost et al. 2020). ARSim2care application is an example of this, as the use of AR for learning invasive procedures that require a deep understanding of internal anatomical structures has proven to be advantageous. Thus, the results confirm that users perceived that ARSim2care application enhanced their three-dimensional understanding of the anatomy related to the
procedure.
Moreover, the positive findings about user expectations in relation to learning with the use of augmented reality demonstrate that the application has the potential of improving motivation, attention and learning. The use of AR was assessed as effective for learning clinical procedures and for strengthening the interest on the subject by a vast majority of participants. These results are in line with the findings of other studies evaluating the use of AR smart glasses for teaching laboratory skills or nursing skill training (Kim et al. 2021; Kapp et al. 2022).
However, it is worth mentioning that the use of the ARSim2care application with Microsoft HoloLens® requires solving certain technical issues to avoid difficulties during teaching activities. For example, the room needs to be well-lit to support object recognition and not have too much background noise to avoid speech recognition problems for some people. The importance of environmental conditions is an issue also identified by other studies (Marschollek et al. 2016; Ingrassia et al. 2020).
Another potential problem is the time it takes to learn how the AR device works, in this case the Microsoft HoloLens®. To prevent this, in this study, the students received a preceding session on augmented reality technology and the smart glasses Microsoft HoloLens® to facilitate its use during the practical teaching session. Additionally, in line with what is suggested in the literature (Mendez et al. 2020), during use, each student was guided by an instructor, who was a member of the research team with experience using the app and Microsoft HoloLens® to resolve any difficulties or problems that arose.
Our results present evidence for a preliminary validation of the ARSim2care application as a learning tool in clinical technical skills training for the performance of invasive techniques. However, to further validate this application, it is envisaged to develop additional research with experimental designs to evaluate the students’ competence to perform the different procedures, comparing the training using the application against traditional training.
Finally, we should mention the possibilities offered by the application to popularize its use. As the ARSim2care design is the result of a project financed by the Erasmus + European project, the software is available for free download (https://erasmus-plus.ec.europa.eu/projects/search/details/2017-1-ES01-KA203-038514).
Limitations
As a potential limitation of this study, it is worth noting the limited sample size, which should be taken into account when interpreting the results of the study. However, for the usability evaluation, the number of participants involved can be considered adequate (Alroobaea and Mayhew 2014) and the inclusion of students from three different universities could have enhanced the representativeness of the sample. Another issue that could be within the potential limitations of the study is the use of a questionnaire adapted from another study (Ferrer Torregrosa 2014) to measure expectations in relation to learning with the use of AR. However, the adaptations were minimal and a pilot test was carried out beforehand to ensure adequate understanding and relevance of the items.