Learning outcomes
At baseline, students reported their knowledge of the clinical microbiology topics (Gram staining, media, biochemical tests and susceptibility). Students were expected to have some knowledge of these topics, as they had likely completed a microbiology subject as a prerequisite to them undertaking the pharmacology and therapeutics course in their Bachelor of Pharmacy degree, from which this study draws its data. Both interventions produced statistically significant differences (p < 0.05) in mean scores compared to baseline across the domains of knowledge, skills and confidence. As seen in Tables 1, 2 and 3, VUMIE™ produced higher post-intervention mean scores for knowledge, skills and confidence compared to post-intervention mean scores for the wetlab, however there was no statistical significance between the mean score for the two interventions. This suggests that the VUMIE™ activity produces learning outcomes that are comparable to the wetlab activity. Statistically significant differences (p < 0.05) were also recorded for endpoint compared to baseline for knowledge, skills and confidence, which suggests that completing the VUMIE™ activity in addition to the wetlab made a positive impact on student learning outcomes. Of the individual items assessed in the surveys, the largest mean score was reported for Gram staining interpretation, again across all three learning domains. These findings are consistent with a number of other studies which demonstrate that virtual simulation can produce comparable learning outcomes compared to traditional teaching methods [15, 17, 22, 23]. The current pandemic has meant that some students’ progression in their degree has been stalled, due to the inability to complete practical laboratory components. A simulation, such as VUMIE™, which produces comparable results to traditional education modes, may allow some health programs to deliver teaching which previously required a traditional laboratory, thereby allowing students to complete pre-requisite modules that may otherwise have been postponed.
Use of a virtual simulation also provides benefits for students who are able to repeat processes and skills that in a traditional wetlab they may only be able to practice once, due to time, cost, supervision and consumables availability. According to this study, the wetlab did not produce statistically significant improvement from baseline for overall Gram stain skills (p = 0.09), where the VUMIE™ did (p < 0.01). This may indicate that students did not feel that they had mastered Gram stain skills during the wetlab, because skills are often only able to be completed once due to time and consumables limitations. The virtual simulation, however, allows for deliberate practice (where a learner undertakes a specifically designed activity to improve performance in that given area or skill) and can be used in the domain of mastery learning, where learners participate in an iterative cycle, repeating the learning process until a certain outcome is met [24, 25]. These concepts are particularly applicable to simulations when used to teach or assess a procedure or technique (process-oriented or procedural simulation), like VUMIE™ which teaches the aseptic procedure for various microbiological testing processes [26]. The repeatability of the VUMIE™, also means that it could potentially be used as an assessment tool. Computer simulations as assessment have the advantage of removing bias and enhancing reliability in contrast to assessments involving humans (eg Objective Structured Clinical Examinations) [27]. The exact case study can be repeated for all students and will perform in the same manner each time. The results of this study also suggest that VUMIE™ could be beneficial as an orientation tool prior to wetlab activities being undertaken, which may improve both the performances and the safety of students during the live laboratory exercises. Similar findings have been reported for other virtual laboratory experiences, particularly for promotion of confidence and more efficient completion of laboratory activities [28, 29].
The delivery of the traditional wetlab allowed for feedback from a demonstrator during the lab session, though students were required to wait until the following week’s session before seeing whether their aseptic technique had been adequate, and their plates had recorded growth. VUMIE™ however, provided instant results and allowed the generation of a lab report where students could see any errors made during the activity. Timely feedback on simulation performance is a critical component of effective learning, encouraging reflective thinking and analysis of learning, so that improvements can be made based on feedback acquired during prior attempts [14, 26].
The ‘anywhere, anytime’ access to virtual learning tools for students has been referred to as ‘simulation on-demand’ and also ‘distributed simulation’, though for the latter term it traditionally referred to a high-fidelity physical unit [26]. Provided the VUMIE™ program is downloaded onto a user’s computer, it can be used anywhere with an internet connection. In addition to the convenience of off-campus use, the VUMIE™ software provided a suitable alternative for a number of students who were unable to physically take part in the wetlabs. Results measuring learning outcomes from this study indicate that VUMIE™ was non-inferior to the wetlab, indicating that it could be used again in future where students have contraindications to participating in traditional wetlab activities. In addition to physical safety, the software allows the learner to feel safe in their actions, without fear of negative consequences (such as those that come from making an error in the wetlab). Feeling psychologically safe is associated with better learning outcomes, as students are more likely to treat mistakes as learning opportunities, rather than perceiving them as failures [30, 31].
Not every simulation or virtual laboratory activity will produce successful learning outcomes. Technology-mediated laboratory activities should be used in accordance with preferred instructional design methods and based on sound teaching theories, as well as aligned to curriculum [32, 33]. When virtual activities are used as mere ‘add-ons’ to existing course content, and not directly related to the learning objectives, their usefulness is limited [34, 35]. We have demonstrated that the VUMIE™ software is a useful tool for teaching clinical microbiology to second-year Bachelor of Pharmacy students, however, as is the case with many commercial simulation products, there are components that users might wish to alter. Whilst the program provides an excellent opportunity to practice the interpretation of Gram stains, it does not demonstrate the staining process or the agar plate streaking process. Commercially available products will often deliver the majority of the requisite educational objectives, however, may not address all of these. As long as the employer of the simulation is aware of the limitations, learning outcomes can usually still be met using supplementary teaching. Another consideration for future use might be a program that allows modification by the educator or institution.
Another consideration for use of simulations is technology acceptance. For this study, self-reported technology acceptance was overwhelmingly positive. The majority of respondents were either willing or very willing to use technology ordinarily and reported that they felt having a virtual microbiology training tool available to them would be somewhat or very useful. The technology-acceptance model explains that perceived usefulness and ease of use are predictors of intention to use a simulation or computer-based activity [36]. Incorporating a simulation into a curriculum requires educators to consider the learner and their willingness to use technology, in order to design a learning activity that will suit the students.
Several factors should be noted when considering future implications and considerations of this research. The VUMIE™ program was accessible by students from the time they attended their first workshop and could be accessed from anywhere provided the student had internet access. Due to privacy reasons there was no way to track how frequently students logged in and used the simulation, including duration of use, or how often simulations were repeated, though this information may assist in understanding and explaining the impact on learning outcomes. Due to ethical guidelines at this institution, surveys must be completely voluntary, which contributed to uneven group numbers (due to attrition). However, a response rate of approximately 50%, which was observed for the post-activity survey compared to the baseline response, is a typical rate of response for data collected from individuals [37]. Additionally, this study examined short-term learning outcomes (approximately 8 weeks), whereas long-term retention of learning using the program in comparison to the live wetlab should be investigated for a more rigorous assessment of student learning. Further studies could also examine the integration of the simulation at a chosen time (as is the case with just-in-time simulation), to examine the effects on learning outcomes.