Teaching medical students to choose wisely through simulation

The Choosing Wisely (CW) campaign aims to encourage dialog among physicians and patients about the costs and benefits of medical care. The purpose of the present study was to describe the implementation of the CW campaign among medical students in the pediatrics clerkship using different teaching strategies and to evaluate the students’ perception and performance. A prospective, interventionist, open study with a control group was conducted. All sixth-year undergraduate medical students that were on their pediatric clerkship at the Emergency Department during the study period were invited to participate. The study consisted of two strategies: a remote video class about the CW initiative and in situ simulation training. By the end of the rotation, all participants were evaluated through an Objective Structured Clinical Examination (OSCE). A total of 50 students were included, of which 24 watched only the online video (control group) and 26 were exposed to both the online video and the simulation scenarios (intervention group). Students in the intervention group had a significantly higher total score in the OSCE compared to students in the control group (median 90 vs 90; range 78–100 vs 50–100; p: 0.047). Median scores of behavioral items of the OSCE grouped together were significantly higher in the intervention group compared to the control group (median 60 vs 50; range 40–60 vs 20–60; p: 0.002). Conclusion: Simulation training about principles of the CW campaign had a greater impact on behavioral aspects of undergraduate students. This learning strategy was well accepted by participants. What is Known: • The Choosing Wisely (CW) campaign aims to encourage dialog among physicians and patients about the costs and benefits of medical care. • Teaching high-value and cost-conscious care to medical students is highly desired.. What is New: • Simulation training about principles of the CW campaign had a greater impact on behavioral aspects of undergraduate students. What is Known: • The Choosing Wisely (CW) campaign aims to encourage dialog among physicians and patients about the costs and benefits of medical care. • Teaching high-value and cost-conscious care to medical students is highly desired.. What is New: • Simulation training about principles of the CW campaign had a greater impact on behavioral aspects of undergraduate students.


Introduction
The unsustainable increase in health care costs elicited the need to discuss the balance between harms and costs of tests and treatments against the potential benefits [1]. In this context, the Choosing Wisely (CW) campaign was launched by the American Board of Internal Medicine in 2012 aiming to encourage dialog among physicians and patients about the costs and benefits of medical care [2]. The campaign's key guiding principle is to improve the quality of care, always based on evidence, increasing the likelihood of benefit and reducing the risk of harm to individuals' health [3,4].
It is critical to engage physicians to provide high-value, patient-centered care, and it is notorious that the practice habits developed during medical school, residency, and fellowship training often persist throughout a career. After one finishes training, the opportunities to be influenced by respected role models decrease considerably, thus making the habits of practicing clinicians particularly difficult to break [5]. Thus, teaching high-value and cost-conscious care to medical students is highly desired [6].
Few studies aimed to teach the CW principles to medical students using different strategies [7][8][9][10]. Students on their pediatrics clerkship were exposed to workshops about the CW campaign and later assessed through an Objective Structured Clinical Examination (OSCE). They achieved a higher level of success in the OSCE evaluation when compared to the non-exposed group [9]. A randomized controlled cross-over study of video-versus text-based case scenarios about the CW initiative observed that repeated video-based key feature testing produces superior short-term learning outcome compared to text-based testing [10].
However, the efficacy of the CW initiative lies not only on medical professionals being familiar with the principles, but also being able to build an empathic relationship with patient and family and being able to communicate effectively the risks and benefits of each treatment option, considering the patient's concerns and preferences. Simulation is considered an effective training method, with large effects on skills and behavioral outcomes [11].
Thus, this study hypothesized that the teaching strategy of simulation training could enhance the learning of CW principles by medical students. The aims of the present study were to describe the implementation of the CW campaign among medical students in the pediatrics clerkship using different teaching strategies (remote video class and in situ simulation training) and to evaluate the students' perception and performance.

Study population and instructor
All sixth-year undergraduate medical students at FMUSP that were on their pediatric clerkship at the Emergency Department during the study period were invited to participate. The pediatric clerkship lasts 6 weeks, during which students rotate in the Pediatric wards and in the Emergency Department. The instructor was a pediatric faculty member with great knowledge of the CW initiative and experience on simulation training.

Intervention overview
The educational materials were developed by three pediatric faculty members and approved for implementation by the pediatric clerkship leadership. The study consisted of two educational strategies: a remote video class about the CW initiative with pre-and post-test questionnaires (Appendix A) and in situ simulation training (Appendix B). All participating students were asked to watch the video class and answer the two questionnaires at home. The students were free to watch the video class at any time during the 6 weeks of pediatric clerkship. After this first step, some students had the opportunity to participate on the simulation training that occurred throughout the 6 weeks of pediatric clerkship if they were in the Emergency Department on Mondays and Thursdays mornings, when the instructor was available to deliver the in situ sessions. During the pediatric clerkship at the Emergency Department, each student is expected to have three shifts per week (day or night) of 12 h each, which are pre-established by the pediatric clerkship leadership before the rotation begins. Therefore, neither researchers nor students interfered in the allocation of participants in the simulation training. In the last day of the rotation, all students were evaluated through an OSCE with two scenarios addressing topics of the regular theoretical and practical program of the pediatric clerkship and two scenarios about the CW initiative (Appendix C). The study was approved by FMUSP's Research Ethics Committee (number 45672121.4.0000.0068).

Remote video class with pre-and post-test questionnaires
The students received a link named "Choosing Wisely" through Google Classroom® in which they were asked to follow 4 steps in the sequence below (they could not move to the next step without fully completing the previous one): The questionnaires were developed on Google Forms®. The pre-test consisted of five questions about clinical cases/ situations in which CW principles were crucial in order to mark the correct answer. The post-test had the same five questions previously answered and seven new questions about students' perception of the CW initiative.
The video class lasted 30 min and consisted of explanations about history and principles of the CW campaign and how to communicate with clarity and empathy, prioritizing a cost-conscious and patient-centered approach. In the video, two specific pediatric recommendations were detailed: "Antibiotics should not be used for viral respiratory illnesses (sinusitis, pharyngitis, bronchitis, and bronchiolitis)" and "Computed tomography (CT) scans are not necessary in the immediate evaluation of minor head injuries; clinical observation/Pediatric Emergency Care Applied Research Network (PECARN) criteria should be used to determine whether imaging is indicated."

Simulation training
Simulation training was carried out at the Emergency Department, in medical consultation rooms, through role playing. The instructor played the role of the mother and the students played the role of the physician. Students participated in pairs in two scenarios: in one turn, they acted as the physician, and in the other turn, they observed. Case 1 discussed the need for antibiotics in a child with upper respiratory tract infection symptoms, and case 2 discussed the need for a head CT scan in a child with minor head injury.
Before starting each case, the students received a brief description of the scenario and what objectives they were expected to accomplish (Appendix B). By the end of each scenario, the group came together to debrief, and, guided by the instructor, both students were expected to talk about strengths and difficulties as well as suggestions for future encounters [12]. Each scenario lasted 10 min and debriefing 20 min approximately, with a total duration of the entire simulation of 60 min. After completing all the above steps, the students were asked to answer a questionnaire consisting of five questions related to their perception of the simulation training (Appendix D).

OSCE
At the end of 6 weeks of pediatric clerkship, all students are evaluated through an OSCE that addresses topics of the regular theoretical and practical program of the pediatric clerkship. For study purposes, two scenarios about the CW initiative (Appendix C) were added, and students were aware they were not part of the clerkship's summative evaluation. The OSCE scenario building process consisted on the following steps: 1. The scenario was created by an author specialized in pediatric emergency and was based on an assessment scenario model. 2. In order to guarantee face and content validity, the scenario was reviewed by the graduation committee, composed of pediatricians and specialists in medical education. 3. A pilot was then conducted; the scenario was tested with a first-year pediatric resident and was edited as needed. This pilot also served to train the standardized patient. The two scenarios went through the same process.
The students were randomly assigned to one of the CW scenarios. The OSCE scenario regarding principles of the CW campaign had one pediatric faculty member observing and marking the students' performance in a checklist and one pediatric resident playing the parents' role. Both received training and discussed the script and the checklist prior the encounter. There were two possible scenarios which were randomly selected for each student: case 1 addressed the need for antibiotics in a child with upper respiratory tract infection symptoms, and case 2 addressed the need for a head CT scan in a child with minor head injury.
Upon entering the room where the scenario took place, the student received a written instruction about the OSCE with a brief description of the scenario and the objectives. The student was given 10 min to perform the OSCE. The pediatric faculty member evaluated the student's performance through a structured checklist consisting of 10 topics with a maximum score of 10 each, adding up to a total of 100 points.

Data recording and statistical analysis
The data was uploaded to a database built in the Microsoft Excel® 2013 program. The variables were obtained from the information recorded in the Google Forms®, in the questionnaire after the simulation and in the structured checklist collected during the OSCE. The following variables were analyzed: age, sex, answers to all above mentioned questionnaires, and OSCE score.
Continuous variables are shown as mean ± standard deviation (SD) in cases of normally distributed data or median values with range if not normally distributed. Categorical data are expressed as numbers and percentages. Responses to the pre-and post-test questionnaires were analyzed through Wilcoxon test. Students were divided into two groups: intervention (exposed to both the online video and the simulation training) and control (exposed only to the onlinde video). OSCE scores were compared between the groups. Normally distributed data were analyzed by unpaired t-tests. OSCE scores were not normally distributed according to Shapiro-Wilk's normality test. Therefore, Mann-Whitney U-test was used to compare groups. Some topics of the OSCE checklist did not have a continuous grading and therefore were analyzed as categorical data through Fisher exact test. For data analysis, GraphPad Prism software version 8 was used.
To enhance the transparency and reproducibility of our study, we used the checklist recommended by the Guideline for Reporting Evidence-based Practice Educational interventions and Teaching (GREET) [13] as well as recommendations of extensions for the CONSORT Statement (Consolidated Standards of Reporting Trials) that can help improve the quality of reporting for simulation-based research [14].

Results
Of the 60 students that were eligible for study participation, nine did not fully complete the video and/or pre-and post-test questionnaires, and one was not evaluated in the OSCE. Following exclusion of these students, complete data were available for 50 participants. Mean age was 24.98 years (SD ± 1.51 years), and 26 were women (52%). Of those, 24 watched only the online video (control group), and 26 were exposed to both the online video and the simulation scenarios (intervention group).
Students in the intervention group had a significantly higher total score in the OSCE compared to students in the control group (median 90 vs 90; range 78-100 vs 50-100; p: 0.047) (Fig. 1). Out of the 11 items included in the OSCE checklist (Appendix D), students in the intervention group had a significantly higher score compared to the control group in the following three items: "Asked about the father's concerns" (median 10 vs 0; range 0-10 vs 0-10; p < 0.001); "Acknowledged and legitimized the father's emotions" (median 10 vs 10; range 0-10 vs 0-10; p: 0.045); and "Involved the father in decision-making and/or questioned whether he agrees/is satisfied with the treatment plan" (median 10 vs 10; range 10-10 vs 0-10; p: 0.046) ( Table 1).
Additionally, we divided the 11 items of the OSCE checklist into two groups: those regarding behavioral aspects and those regarding cognitive aspects. Median scores of the behavioral items grouped together were significantly higher in the intervention group compared to the control group (median 60 vs 50; range 40-60 vs 20-60; p: 0.002), whereas scores of the cognitive items were similar between the groups (median 37.75 vs 40; range 28-40 vs 20-40; p: 0.839) (Fig. 1).
Median scores of the post-test were statistically significant higher than scores of the pre-test, regarding students' Perception of students regarding online activity and simulation training can be seen in Fig. 2. Only one question was equal in both surveys of student's perception: "Having a video class on the 'Choosing Wisely' initiative is important for medical training and changes the perception of medical care" and "Participating in a simulation exercise about the 'Choosing Wisely' initiative is important for medical training and changes the perception of medical care." Students' responses to this question were significantly different, with higher agreement to the statement after simulation training when compared to responses following online video (median 5 vs 5; range 4-5 vs 3-5; p: 0.001).

Discussion
The present study demonstrated that students exposed to a single simulation practice performed better on OSCE compared to students that only had access to a remote video class, particularly with respect to behavioral aspects. Additionally, students' perception about the interventions was extremely satisfactory and even greater after the simulation training.
When measuring effectiveness of an educational intervention, the Kirkpatrick model is often used, which classifies training outcomes in four levels; level 1, reaction; level 2, learning; level 3, behaviors; and level 4, results [15,16]. When dealing with undergraduate students in an Emergency Department, it is very unlikely to see level 4 outcomes, since students' decisions are influenced by supervisors [17]. Reaction to learning interventions was very positive in both arms of the study, with a better reaction following the simulation training. Knowledge acquisition assessed in the cognitive items of OSCE checklist was similar in both groups.
However, simulation was superior in changing behavioral aspects of the simulated cases. This is consistent with literature, where simulation-based training for health professions is consistently associated with large effects on behaviors [11]. Core skills recommended by the CW initiative include empathy and clear communication which were adequately addressed through simulation-based training.
The CW initiative was presented to medical students in previous studies, and results were similar to our findings. Fifth-year medical students also in the pediatric clerkship in Brazil were exposed to a 2-h workshop about the CW campaign. Students' performance was evaluated through an OSCE consisting of 3 clinical cases based on the recommendations of the CW initiative. The level of successes in the OSCE evaluation was higher in the group of students exposed to the CW campaign when compared with the group that was not exposed [9]. Similarly to the present study, medical students and residents that were taught about the CW initiative through different formats evaluated the CW campaign content positively [8,9]. These results indicate that students are receptive to learning about the CW initiative regardless of the teaching strategy. However, these studies did not evaluate Kirkpatrick levels 2 and 3 outcomes, like knowledge and behavior. German medical students were also exposed to some CW recommendations in a randomized cross-over study that compared "text cases" with "video cases" through repeated testing of key feature questions. They observed a high prevalence of erroneous beliefs in medical students irrespective of item format (video or text) [10]. This finding reinforces the need to bring up the discussion about high-value, cost-conscious care among undergraduate students.
Our study has some limitations. Firstly, our participants were selected in a convenience sample, in a non-randomized fashion. However, since the baseline demographics were similar between the groups, we do not feel that significant bias was Listed "red flags" that indicate when to return for further evaluation 8.75 (0-10) 10 (0-10) 0.289 introduced by groups' allocation. Secondly, since simulation was given in addition to the recorded lecture, it is not possible to infer if other learning methods would have the same effect on behavior during OSCE, although changing behaviors of health care professionals seems to be more likely when lectures are associated with practical elements [18]. Thirdly, as students were free to watch the video class at any moment and simulation occurred throughout the period of 6 weeks, it was not possible to evaluate the influence on OSCE scores of different time spams between delivery of the educational strategy and the OSCE exam. Additionally, as there was only one rater per scenario, it was not possible to calculate inter-rater reliability. Finally, since only two simulation scenarios were performed and only 50 students were assessed, it is possible that the strength of the intervention was not enough to detect differences that a more robust simulation program would produce. Nonetheless, the observed differences in behavior aspects are a sign of the efficacy of this intervention.

Conclusion
Simulation training about principles of the CW campaign had a greater impact on behavioral aspects of undergraduate students. This learning strategy was well accepted by participants. An appropriate future direction would be to study a larger randomized sample of students participating in either multiple simulation scenarios or other education intervention and assess effects on behavior not only in OSCE, but also clinically.
Authors' contributions All contributing authors had full access to the study data. All authors have read and approved the final version of the manuscript. Thayza Morato, Pedro Mendes, Danielle Bou Ghosn, Thomaz Couto, Paulo Mai, and Nara Cavalcanti were responsible for the study design, data acquisition, data analysis, and writing of the manuscript. Sylvia Farhat and Cláudio Schvartsman were involved in study design and writing of the manuscript.

Availability of data and material
This manuscript has data included as electronic supplementary information.