The present study demonstrated that students exposed to a single simulation practice performed better on OSCE compared to students that only had access to a remote video class, particularly with respect to behavioral aspects. Additionally, students’ perception about the interventions was extremely satisfactory, and even greater after the simulation training.
When measuring effectiveness of an educational intervention, the Kirkpatrick model is often used, which classifies training outcomes in four levels; level 1 – reaction, level 2 – learning, level 3 – behaviors and level 4 – results [15, 16]. When dealing with undergraduate students in an emergency department, is very unlikely to see level 4 outcomes, since students’ decisions are influenced by supervisors [17]. Reaction to learning interventions was very positive in both arms of the study, with a better reaction following the simulation training. Knowledge acquisition assessed in the cognitive items of OSCE checklist was similar in both groups. However, simulation was superior in changing behavioral aspects of the simulated cases. This is consistent with literature, where simulation-based training for health professions is consistently associated with large effects on behaviors [11]. Core skills recommended by the CW initiative include empathy and clear communication which were adequately addressed through simulation-based training.
The CW initiative has been presented to medical students in previous studies and results were similar to our findings. Fifth-year medical students also in the pediatric clerkship in Brazil were exposed to a 2-hour workshop about the CW campaign. Students’ performance was evaluated through an OSCE consisting of 3 clinical cases based on the recommendations of the CW initiative. The level of successes in the OSCE evaluation was higher in the group of students exposed to the CW campaign when compared with the group that was not exposed [9]. Similarly to the present study, medical students and residents that were taught about the CW initiative through different formats evaluated the CW campaign content positively [8, 9]. These results indicate that students are receptive to learning about the CW initiative regardless of the teaching strategy. However, these studies did not evaluate Kirkpatrick levels 2 and 3 outcomes, like knowledge and behavior. German medical students were also exposed to some CW recommendations in a randomized cross-over study that compared “text cases” with “video cases” through repeated testing of key feature questions. They observed a high prevalence of erroneous beliefs in medical students irrespective of item format (video or text) [10]. This finding reinforces the need to bring up the discussion about high-value, cost-conscious care among undergraduate students.
Our study has some limitations. First, our participants were selected in a convenience sample, in a non-randomized fashion. However, since the baseline demographics were similar between the groups, we do not feel that significant bias was introduced by groups allocation. Secondly, since simulation was given in addition to the recorded lecture it is not possible to infer if other learning methods would have the same effect on behavior during OSCE, although changing behaviors of healthcare professionals seems to be more likely when lectures are associated with practical elements [18]. Finally, since only two simulation scenarios were performed, and only 50 students were assessed it is possible the strength of the intervention was not enough to detect differences that a more robust simulation program would produce. Nonetheless, the observed differences in behavior aspects are a sign of the efficacy of this intervention.