DOI: https://doi.org/10.21203/rs.3.rs-2167283/v1
COVID-19 had a significant impact on medical education. Aim of this work is to assess if the pandemic has influenced medical student performance on the surgical Objective Structured Clinical Examination (OSCE).
Student performance at a German University Hospital during surgical OSCE (before, during, and after pandemic) were compared. 3 OSCE stations were considered: visceral surgical cases with clinical abdominal examination, suturing and description of visceral surgical operation techniques. Student`s marks were compared among the three groups above using the Student t-test for independent samples.
423 students took part in one of the three OSCE. Performance in solving clinical cases and performing an abdominal examination was better before pandemic than during pandemic (p = 0,04) as well as after pandemic than during pandemic (p = 0.02). Suturing was better during pandemic than before (p < 0.001) as well as after pandemic (p < 0.001). Performance in surgical technique was not different among the groups.
Cessation of the practical internship reduced student performance on clinical case and abdominal examination section but not on suturing or surgical technique sections. These data suggest that alternative approaches to instruction including online education may be sufficient for some but not all aspects of the surgical internship.
The Objective Structured Clinical Examination (OSCE) was introduced by Harden et al. (1975) (1) as an instrument to assess clinical, theoretical, and practical competencies. The OSCE allows the assessment of a candidates` clinical skills, attitudes, problem-solving abilities and their application of knowledge in one examination (1) and is a tool for the assessment of performance within simulated environments (2). The two major underlying principles of the OSCE are „objectivity“ and „structure“. Objectivity predominantly depends on standardised scoring rubrics and the same, trained, examiner asking the same questions to every candidate. A well-structured OSCE station has a standardised station design assessing a specific clinical task which is blueprinted against the curriculum (2). During OSCE, students rotate through different examination stations, where practical skills including physical examination, blood draw, and establishing an intravenous line are examined in addition to theoretical skills like interpretation of an x ray. To make the OSCE objective and structured, the examination stations have a strict defined task and the rating system of the students is based on clear defined criteria within a check box formular (1). In order to perform a representative OSCE, the stations have to be designed carefully and be evaluated many times by a professional team. Examiners also have to be trained in great detail and retrained periodically. OSCEs are believed to be a superior test system for medical students within the practical-clinical context and should be implemented in the curriculum (2, 3). In addition to an objective theoretical-practical assessment of the students skills, OSCEs are also believed to be more representative than a multiple choice written test where educated guesses may not indicate clinical proficiency. Within the OSCE, students can present their knowledge and skills in an interactive environment that has been reported to be less stressful than traditional written examinations (4, 5). OSCE is regularly performed twice a year at the University Hospital of Ulm for more than 15 years in order to assess medical students in their 8th or 9th semester. The student's knowledge and practical skills are assessed in twelve surgical stations. Six minutes of time are allocated to complete each station. Some stations focus on basic surgical skills (e.g. skin closure and suture techniques, fixing of drainages such as easy flow or thoracic tube, or fixing of a central line catheters on a skin model) while others focus on theoretical knowledge combined with clinical examination of simulation-patients or on description of the main steps of surgical procedures. All disciplines of surgery are represented within these mini-exams (general and visceral surgery, trauma and orthopedic surgery, cardiac surgery, vascular and thoracic surgery). Practical skills and theoretical knowledge were gained before the COVID-19 pandemic within a three week practical clerkship on surgical wards with face to face seminars. During the first wave of COVID-19, students were not allowed to assess their clerkship at the hospitals thus this internship was pushed to the end of the according semester and also reduced to only two weeks with online-seminars. Later in the pandemic, the duration of the internship was restored to three weeks with online and face to face seminars.
Students usually had the opportunity to improve their practical skills under supervision of experienced clinical tutors during the semester at the surgical skills lab on an open invitation. Students were able to show up in the skills lab before COVID without making an appointment. Within the first wave of COVID, the skills lab was closed totally and started to open just for a few students with prior appointment. In the week before the OSCE, students were usually able to attend on special preparation days without prior appointment and for as long as they wanted. During COVID, students also had to make an appointment to attend these preparation days just during certain time slots (2–3 hours). Success on the OSCE was marked at > 60% of the maximal achievable total points (182–184) in all twelve stations.
On the 17th of March 2020, the Association of American medical colleges strongly suggested that medical students should not to be involved with patient contact (6). Consequently, the COVID-19 pandemic had a significant impact on medical education, particularly with respect to practical internships of medical students where in person learning was replaced with online seminars. In this regard, the pandemic affected medical education worldwide (7), since the opportunities of patient care and bedside teaching for medical students were restricted in attempts to lower viral transmission through isolation and social distancing (8).
The replacement of in-person classes with online equivalents creates a loss of collaborative experiences that has the potential to be a significant detriment to education. Likewise, the impact of cancellation of clerkships, which are necessary for both skill acquisition as well as for relationship building, should be investigated (9).
The aim of this work is to assess if the pandemic has had an influence on medical students` performance on surgical OSCE, specifically in the field of visceral surgery (suturing, solving of a visceral surgical case with clinical examination and description of the main steps of abdominal operations) before (2018/2019), during (2020/2021) and after pandemic (2021/2022), as well as to investigate if and if yes where the new form of instruction can replace satisfactory not only the frontal lessons but also the clerkships on surgical wards.
The results of three surgical OSCE examinations (winter semesters 2018/2019, 2020/2021 and 2021/2022) which took place at a German University Hospital before the beginning, during, and after the pandemic, respectively, were compared. In particular, the students` performance in three of the twelve OSCE stations were considered: suturing (0 to 20 points could be achieved), solving of visceral surgical cases with clinical abdominal examination (0 to 10 points) and visceral description of surgical operation techniques (0 to 8 points). The marks achieved by each student were compared before, during and after the pandemic using the Student t-test for independent samples. The study adhered to the ethical principles of the Helsinki declaration.
Description of the three evaluated stations in the field of visceral surgery.
Station 1
Students were required to close a wound on a skin model either with simple stiches or Donati`s.
Station 2
Students were required to solve a clinical case and perform a complete abdominal examination. The surgical cases were acute appendicitis, acute cholecystitis, acute sigma diverticulitis, lower gastrointestinal bleeding, incarcerated inguinal hernia in a newborn and incarcerated umbilical hernia in a cirrhotic patient.
Station 3
Students were required to describe the main steps of a surgical abdominal procedure, i.e. open and laparoscopic inguinal hernia repair, laparoscopic cholecystectomy, oncologic right and left colectomy, transverse colon resection, non oncologic resection of sigma because of diverticulitis.
Values are presented as mean [± standard deviation] and median [range] for continuous variables. Dichotomic variables are presented as absolute number as well as percent. Groups were compared with the Student t-test for independent samples. A two-sided p-value < 0.05 was considered statistically significant. There ware no missing values in the data set, and no imputation strategies were needed. All calculations were conducted using R Project for Statistical Computing (The R Foundation, Version 3.1.0, Vienna, Austria).
A total of 423 students took part to one of the three OSCE examinations, 125 students divided in 11 groups were examined in winter semester 2018/2019, 155 divided in 14 groups in winter semester 2020/2021 and 143 students divided in 13 groups in winter semester 2021/2022. All students, except for one in winter semester 2018/2019 passed the exam (99.2%). The pandemic-impacted group had a higher average score than the group before the pandemic (170.52 vs. 157.43) as well as than the group after pandemic (170.52 vs. 168.60). Students performance in suturing was better during pandemic than before pandemic (19.59 (± 0.95) vs. 17.00 (± 2.04), p < 0.001) as well as during pandemic than after pandemic (19.59 (± 0.95) vs. 18.25 (± 1.87), p < 0.001). Students performance in theoretical solving of a clinical case and performance of a complete abdominal examination was better before COVID-19 pandemic than during the pandemic (8.32 (± 1.37) vs. 7.94 (± 1.61), p = 0,04) as well as after pandemic than during pandemic (8.36 (± 1.37) vs 7.94 (± 1.61), p = 0.02). No difference was found in the comparison of the performance before and after pandemic (8.32 (± 1.37) vs. 8.36 (± 1.37), p = 0.83). Students` performance in surgical technique was not different by comparing the marks before pandemic and during pandemic (6.34 (± 1.66) vs 6.52 (± 1.64), p = 0,37) as well as during pandemic and after pandemic (6.52 (± 1.64) vs. 6.79 (± 1.47), p = 0,14). A significantly better score was achieved in this station by comparing the performance after and before COVID-19 (6.79 (± 1.47) vs. 6.34 (± 1.66), p = 0.02). The surgical techniques which had to be illustrated by the students could be divided into technically simple and frequent procedures (inguinal hernia repair and cholecystectomy) and more difficult and rare procedures (right and left hemicolectomy, transverse resection and sigmoid resection). Students receiving a simple procedure performed significantly better than students receiving a difficult procedure both before the pandemic and after. In particular, in the OSCE before COVID-19 71 students (56.8%) had to explain a simple procedure and 54 (43.2%) a difficult one, the latter group achieving a worse score (6.66 (± 1.46) vs. 5.93 (± 1.82), p = 0.01) and, in the OSCE after COVID-19, 71 students received a simple procedure (49.7%) and 72 (50.3%) a complicated one. Again this last group performed worse (7.11 (± 1.47) vs. 6.47 (± 1.41), p = 0.01). On the contrary, no difference in the performance was observed for the 75 students receiving a simple procedure (48.4%) and the 80 (51.6) receiving a complicated one during pandemic (6.69 (± 1.41) vs. 6.36 (± 1.82), p = 0,21).
Analogously, the randomly assigned clinical cases had different grades of difficulties. They could be divided into simple and frequent cases (acute appendicitis, acute cholezystitis and incarcerated inguinal hernia) and difficult and less frequent cases (incarcerated umbilical hernia in a cirrhotic patient, sigmadiverticulitis and lower gastrointestinal bleeding). In the OSCE before COVID-19 71 out of 125 students received a simple case (57%) and 54 a difficult one (43%). Students receiving a simple case performed significantly worse than students receiving a difficult one (8.08 (± 1.33) vs 8.63 (± 1.38), p = 0.03). During and after COVID-19, no difference was observed. During COVID-19 75 (48%) students received a simple case and 80 students (52%) a complicated one (7.69 (± 1.54) vs 8.17 (± 1.65), p = 0.06). After COVID-19 69 students received a simple case (48%) and 74 (52%) a complicated one (8.19 (± 1.22) vs 8.51 (± 1.19), p = 0.16).
Among the difficult cases, the lowest score was achieved by students receiving the case of an incarcerated umbilical hernia in a cirrhotic patient with ascites compared with students receiving the case of a sigmadiverticulitis and lower gastrointestinal bleeding both in the group who underwent OSCE during COVID-19 (7.97 (± 2.01) vs. 8.32 (± 1.33), p = 0.35) and in the group who underwent OSCE after COVID-19 (8.06 (± 1.74) vs. 8.9 (± 1.13), p = 0.01), been this last statistically significant. This most difficult case was not assigned to any students taking OSCE before COVID-19.
By reporting all scores to a minimal common multiple of 40 points, the lowest scores among the three OSCE analysed stations was achieved by the students at the surgical case stations during pandemic (31.76/40) and the surgical technique station before pandemic (31.7/40). The highest score was achieved within the suturing exercise during the pandemic (39.2/40) (Table 1).
The peculiarity of medical education is the integration of learning with practical application in the hospital setting (10). In clerkships, students are assigned to various departments, including internal medicine, surgical disciplines, pediatrics and emergency medicine. They acquire clinical knowledge by shadowing and assisting residential physicians in daily practice, primary patient care, bedside procedures, daily record writing and case presentations (11). The pandemic challenged the educational and healthcare system worldwide. During COVID-19, hospital clerkships for medical students were cancelled, making it difficult to collect practical experience.
Teaching hospitals developed strategies in response to the pandemic, first of all the establishment of online learning platforms (9). Virtual education played a central role in mitigating the impact of this pandemic on the educational process by providing an interactive communication platform (12).
The aim of this retrospective analysis was to investigate how students performed in surgical OSCE before, during and after pandemic. The focus was put on general and visceral surgery. The case station with complete abdominal examination was the unique station which was significantly better performed by the students before and after COVID-19 in comparison to the pandemic. Most students had difficulties completing the abdominal examination in the correct sequence, starting from inspection, through auscultation, percussion and palpation. Often, the students failed to initiate the examination a distance away from the point of maximal pain. This is something completely obvious if someone is practiced in hands-on abdominal examination, but not if this is learned only theoretically without having the possibility to practice on a real patient with real pain. Also, simulation is, in this case, not as effective as a real patient. Initiating the examination at the point of highest pain is associated with patient initiated cessation of examination and thus incomplete assessment and diagnosis, knowledge that would be gained during hands-on training in the hospital, either directly or indirectly through observation of others. Modifications to standardized online learning modalities which include these errors in video format could help reduce these issues during the exam and more importantly, early in the physicians practice.
In the suturing station, student performance was higher during the COVID-19 pandemic. A possible explanation for this could be that students had additional time to self practice with suturing on skin model or even at home on self made models. It is also possible that intrinsic bias is present given that this station was monitored / graded by medical students who changed with every exam cycle, thus limiting assessment reproducibility and consistency.
Students performed better before and after pandemic when explaining a simple surgical procedure (open/laparoscopic inguinal hernia repair and laparoscopic cholecystectomy) than students receiving a complicated one (colon surgery). Before and after COVID-19, medical students had the opportunity to be present within the operating room during a variety of procedures. Senior surgeons working in concert with surgical residents allows for greater detail in description of the procedure, regardless of complexity, increasing the learning opportunities for all trainees present, residents and students alike. During the pandemic, students were not afforded these opportunities for in person learning. Likewise, complex cases are often intricate, with many steps, requiring repetition of observation and in different contexts and patients. Online learning may have failed to provide the variety and context necessary to prepare students for the exam.
The opposite results were observed with the surgical case, where students performed better before and after the pandemic when receiving the complicated case than the simple one. Presence within the visceral surgical ward would have provided students with multiple opportunities to practice analysis of these cases. For example, lower gastrointestinal bleeding and acute diverticulitis are often treated somewhat conservatively, knowledge which would have again been obtained through routine rounds with physicians.
By reporting all score to a minimal common multiple of 40, the lowest score of all stations in all OSCE was achieved by the students at the surgical case stations during pandemic, marking again that bedside teaching is an important part of the medical university curriculum, which even in a pandemic, should not be stopped. Limitations of this study include missing information about the semester in which the students took the examination (6th or 7th). Later in the program of study could have affected their performance on the exam where students could have better integrated the knowledge received. Likewise, the ward in which clerkships were completed were not consistent (visceral vs. vascular vs. traumatology wards). Rotations within the visceral ward no doubt would improve their performance on such an exam, where their exposure to these types of procedures would have been more frequent and in depth. As noted earlier, within the suturing station, evaluators could have impacted the overall results with senior students rather than practicing physicians conducting the examinations and providing their graded assessments. Consistency in grading could be difficult given the larger number of evaluators, their limited level of experience, and the constant changing of these evaluators with each cycle of examinations (13). A last limitation noted by the authors is in the consistency of the examination itself, in that the same stations provided cases of varying degrees of difficulty to students during the same OSCE exam.
Future studies should be focused on assessing if our findings using the three OSCE visceral stations, i.e. that students performed worse in stations where a previous bed-side teaching was required, can be generalized to the other nine OSCE stations, with some of them related to clinical skills (e.g. repairing a drainage) and practical knowledge which assumes the student has already had practice with real patients.
The cessation of the practical internship in surgical wards during the pandemic led to a significant decrease in student performance on the OSCE clinical case and abdominal examination section. Suturing was better during pandemic likely due to more free time for practice due to the lack of practical internships and frontal lessons. The average students` performance in surgical technique did not differ over time. Therefore, it is concluded that clerkships play a central role in medical formation and, as far as applicable, should not be stopped.
objective structured clinical examination
Disclosure statement: The authors report there are no competing interests to declare.
Table 1 is not available with this version.