Assessing the Impact of the COVID-19 Pandemic on Medical Student Education

Background: The full impact of the COVID-19 global pandemic has yet to be seen, yet medical education has already been critically disrupted. As U.S. hospitals were forced to aggressively limit non-essential care to preserve personal protective equipment and minimize COVID-19 exposure, in-person education and hands-on training was nearly eliminated for students. The objective of this study was to immediately and comprehensively investigate the impact of the COVID-19 pandemic on medical student education. Medical students in the U.S. were invited to complete an online survey about the impacts of the COVID-19 pandemic on their medical education experience. Students provided basic demographic information and answered questions about the impact of COVID-19 on their training/education, nances, and mental health. Results: Medical students reported nearly 18 fewer hours of patient care per week, an immediate switch to virtual learning (74%) along with grading changes (62%), and widespread cancellation of national exams. Additionally, 55% of fourth year students graduated early to provide direct clinical care. Students across years felt that changes from the pandemic would negatively affect their residency applications and that upcoming rotations would be impacted (p<0.001). Students reported that the pandemic had negatively affected their nances, increased their anxiety/stress, increased their feelings of burnout, and negatively impacted their work-life balance (p<0.001). Conclusion: Medical education has been critically impacted by COVID-19. Student perceptions and evaluation of experiences to date should be considered as educators prepare to ready students for academic and professional transitions in the context of continued COVID-19 disruptions and distanced learning.


Introduction
The novel coronavirus pandemic (COVID-19) has critically disrupted medical education. 1,2 Beginning in March of 2020 and continuing into the present, federal and local governments in the United States (U.S.) addressed the COVID-19 pandemic by issuing stay-at-home orders, closure of non-essential businesses, and restrictions on gatheringsin an effort to promote social distancing and decrease viral transmission.
This new legislation, along with public health concerns, forced medical educators to rapidly adjust medical curricula and training. 1,3 Many in-person educational experiences were canceled or delivered virtually during Spring and Summer 2020 in an effort to reduce person-to-person contact and ensure trainee safety. 3 Although this was primarily handled on an institution-by-institution basis, guidelines for restricting away rotations, for example, were issued nationally. 4 Thus, many medical students were unable to obtain traditional experiences, such as participation in classroom-based lectures or labs, in-person simulation and skills training, or onsite hospital and clinical experiences. 3 This decrease in trainee exposure to the clinical experiences mirrored a decrease in in-person patient encounters (anywhere from 40-80%) 5 reported domestically.
International descriptions of suspended or adapted learning immediately following the COVID-19 pandemic outbreak have begun to emerge. [6][7] Another brief survey study (United Kingdom) asked lastyear medical students about the effect of the pandemic on their exams, assistantships, and objective structured clinical examinations. Medical students felt less prepared due to curricular changes and postponement of exams and in-person learning. 7 Limited reports detailing medical students' personal knowledge of COVID-19 have also been published internationally, 2,8,9 including preferences for returning to direct clinical patient care.
To date, there is no comprehensive description of the immediate and holistic impact of the COVID-19 pandemic on medical student education and training domestically. Given that closures and delays happened relatively quickly, immediate data on the impact of COVID-19 restrictions on medical training is necessary if guidelines to ensure continued education are to be made. Thererfore, we sought to evaluate the effects of COVID-19 on medical student education and training in the United States via an immediate, cross-sectional survey study.

Measures
We conducted a single prospective survey developed by Education and Project Leads. The survey consisted of ve parts: (1) demographic information, (2) impact of COVID-19 on training and education (including: disruption of in-person education, changes in on-site clinical hours, replacement with virtual learning, satisfaction with how changes were being handled, whether fourth-year students were given the option to graduate early, and impact on internal and national exams), (3) impact of COVID-19 on nances, (4) knowledge about COVID-19, and (5) mental health and well-being as a result of COVID-19.
The survey was created and distributed in May 2020 via known medical student listservs and Twitter postings. The survey remained open for participants to accrue from May 18 through June 18.

Statistical Analysis
Study data were collected and managed using REDCap electronic data capture tools hosted at our institution 10  (percentages). Pre-de ned comparisons were performed between groups (pre-clinical vs. clinical students) via chi-square tests for categorical variables and independent-samples t-tests for continuous variables. Paired-samples t-tests were used to compare two variables from the same group (e.g. pre vs. post). Single sample t-tests were used to compare single group means to a pre-determined standard (neutral). We explored the impact of demographic/descriptive variables, sources and amount of COVID-19 knowledge, mental health variables (work-life balance, anxiety-general, anxiety-education, burnout), and changes in clinical hours in a series of linear regressions models examining outcome variables of interest. Signi cant effects are positive unless otherwise noted.

Study Sample
Four hundred medical students (19.6% RR) were included in the nal study sample and were categorized as pre-clinical students (Med 1-2, n = 171, 42.9%) or clinical students (Med 3-4, n = 228, 57.1%). At the time this survey was distributed, 70.2% of students had a stay-at-home order issued by their city or state. Descriptive characteristics are presented in Table 1.

Impact of COVID-19 on Training and Education
Medical students reported large disruptions to in-person education in response to the COVID-19 pandemic: 50.4% reported disruptions to classroom-based lectures, 27.3% to research or clinical labs, 19.3% to 1-or 2-week rotations, 29.8% to 1-month rotations, and 25.3% to dedicated study time for a board exam. Since the COVID-19 outbreak, a greater proportion of clinical students worked directly with patients, with a general drop in patient care across the board (p < 0.001, Table 2). A higher proportion of clinical (16.2% vs. 3.5%) students had been asked to actively treat patients as part of their mandatory curriculum or had actively volunteered to assist in patient care (p < 0.05). Post outbreak, pre-clinical students did not differ in their average number of clinic hours (Mean difference in clinical hours per week: -0.88, p = 0.267), but did report fewer hours spent in the hospital (Mean difference in hospital hours per week: -3.9, p = 0.02). Clinical students reported decreases across both of these (Mean difference in clinic hours per week: -10.5; Mean difference in hospital hours per week: 29.1, p < 0.01).
In lieu of in-person classroom or clinical learning, students reported an increase in the use of online modules, videos, and lectures (pre-clinical: 67.5%, clinical: 77.1%, p = 0.041). Students felt COVID-19 changes affected the grading system at their institution (p < 0.001), and felt that that COVID-19 had negatively affected (p < 0.001) their education and grades (Fig. 1). Students also felt that COVID-19 changes would also affect future semesters: 36.0% of pre-clinical and 14.5% of clinical students (p < 0.001) said their upcoming semester of medical school had been either delayed or canceled. Clinical students, moreso than pre-clinical students, felt that COVID-19 changes had also negatively impacted their upcoming residency applications (p = 0.021, Fig. 1). Despite all of these changes, 79.2% thought their program administration adequately handled scheduling changes, grade changes (60.6%), exam changes (76.7%), updates regarding the pandemic and effects on education (70.2%), and responding to questions and concerns (72.5%; all p < 0.001).
Students reported that fourth year medical students at their institution had been given the opportunity to graduate early in order to provide needed clinical care (56.5%) and 64.8% of fourth-year medical students said they took that opportunity.
Respondents were also asked about the cancellation or delay of national exams: 65.9% of pre-clinical and 24.0% of clinical students reported their o cial NBME exams has been cancelled or delayed due to COVID (p < 0.001). USMLE Step 1 had been cancelled or delayed for 32.5% of students, Step 2 Clinical Skills for 46.2%, and Step 2 CK for 33.3%. Additionally, 49.3% of students had Shelf or Subject Exams scheduled, 10.8% reported they were cancelled completely, 17.5% reported they were delayed until a later date, and 19.9% reported they were taking taken virtually. Institutional nal exams were also heavily impacted, with 65.6% of pre-clinical students reporting their exams were being taken virtually (p < 0.001). Only 13.1% of pre-clinical students and 14.7% of clinical students said their typical nal exams were not affected at all.

Impact of COVID-19 on Finances
Among respondents, 43.2% of students felt COVID-19 had affected their nancial situation, and this did not differ by year (p = 0.621). Some students (18.7%) reported that their institution had offered a nancial return for tuition, while 46.9% offered additional nancial assistance (e.g., scholarships.) While 86.1% of pre-clinical students thought that their programs should refund them tuition money for missed time, 72.0% of clinical students thought this (p < 0.002). A higher proportion of public institutions offered nancial returns (p = 0.036), and a higher proportion of students from public institutions reported receiving nancial assistance (p < 0.001). Students across institution types wanted to receive tuition refunds (p = 0.897). Modeling indicated that how accurate students found COVID-19 information to be (p = 0.016), prior levels of burnout (p < 0.001), and trust in the local government (p = 0.006) affected how much their personal nances were impacted (Model: R 2 = 0.166, p < 0.001), while race (p = 0.021) and anxiety about one's medical education (p = 0.047) were associated with whether they thought institutions should refund their tuition money (Model: R 2 = 0.134, p = 0.05).

Knowledge of COVID-19
In general, students reported feeling knowledgeable about the COVID-19 pandemic (p < 0.001), with clinical students feeling more knowledgeable than pre-clinical students (p = 0.028). Students reported their primary sources of information regarding COVID-19 to be the internet (55.5%), television (28.3%), and social media (20.3%; Table 3), and felt the information they were receiving regarding COVID-19 was accurate (p < 0.001). Modeling indicated that how accurate students felt the information they received was impacted by trust in the federal government (p = 0.044) and their own COVID-19 knowledge (p < 0.001; Model: R 2 = 0.233, p < 0.001).

Mental Health and Well-Being
Students felt that COVID-19 had negatively impacted their work-life balance (p = 0.015), felt greater general anxiety or stress since the start of the outbreak (p < 0.001), and felt greater anxiety or stress about their medical education since the start of the outbreak (p < 0.001). When asked to report their feelings of burnout (on a scale of 1-10) before vs. after the COVID-19 outbreak, pre-clinical students reported a signi cant increase (p < 0.001), but clinical students did not (p = 0.233). Modeling indicated that those who suffered the greatest increases in burnout were those who had higher feelings of burnout prepandemic (p < 0.001), those with worse work-life balance (p < 0.001), and those who had greater reductions in clinical (p = 0.012) and hospital (p = 0.033) hours. A summary of activities that students partook in to take care of themselves in response to the COVID-19 outbreak appear in Table 4, with most students engaged in two or more wellness activities (89.1%).

Discussion
This immediate, comprehensive, post-COVID-19 outbreak study describes the long-term impact of the pandemic on medical student education and training. Although international descriptions of changes in medical education due to the pandemic have emerged, little to no work had described the impact of COVID-19 on the medical student experience in the U.S. Several major ndings emerged: (1a) Anywhere from 20-50% of in-person learning was cancelled immediately following the outbreak and clinical students reported an average loss of 11 hours/week in clinic and 29 hours/week in the hospital; (1b) To counteract that, online and virtual learning increased substantially; (2) A non-negligible percentage of students (up to 16%) had been required to provide needed clinical care immediately following the outbreak and more than half of fourth-year students were allowed to graduate early to join this direct care; (3) Students were also personally affected by the pandemic, with increases in nancial stress and burnout; (4) COVID-19 affected institutional and national policies, such as scheduling, grading, clerkships, and national exams, and students perceived the effects of COVID-19 changes to be long-lasting, with clinical students particularly concerned with their residency applications. These ndings are discussed in greater detail below.

Transitioning from In-Person to Virtual Learning
The majority of students (> 50%) reported an interruption in either classroom on patient-based clinical experiences immediately following the COVID-19 outbreak and subsequent reductions in in-person patient visits. With more than a quarter of students reporting immediate cancelation of third or fourth year rotations, this is particularly worrisome. In the third year of medical school, students rotate through required clerkships, 11 while fourth year medical students typically complete elective rotations and away rotations in their discipline of interest. 13 It is unknown whether these virtual opportunities provide the same learning experience as traditional opportunities, particularly in preparation for residency (e.g. core competencies, requisite skills, measures of readiness for the next level of training). 12,13 Challenges of virtual medical education have been a topic of discussion prior to the COVID-19 pandemic. 14-15 and common challenges that educators face when teaching virtually have been described, 16 including educators' lack of familiarity with new technology and additional time needed to prepare material for a new platform or interface. 17 Addressing these concerns (e.g. providing additional staff or technology support for educators) can improve the virtual educational experience both students and educators.

Students providing clinical care:
Students reported being asked to provide patient care despite health and safety concerns, with fourth year medical students given the option to graduate early (almost 65%). Besides for the clear health concerns faced by working directly with patients in a hospital setting, it is also unclear whether 3.5 years of training is su cient for entering residency. 11 Recruitment of nal-year medical students for early graduation and immersion in patient care has been seen both internationally and domestically. 1,18−20 Some posit that work on the medical front lines may aid in advanced medical students' education, particularly in an outpatient or non-COVID setting. 1,21 A thorough description of the bene ts of early graduation and joining of the workforce is currently unknown, given the recency of the pandemic.

Impact of COVID-19 on Financial and Mental Health
Respondents reported signi cant impacts of the COVID-19 pandemic on their nancial and mental health.
Some institutions offered some sort of nancial remittance or an additional kind of nancial assistance; interestingly, this was more likely seen in public than private institutions. Although this has been discussed at the undergraduate level nationally, this should also be considered for medical students who may have decreased clinical training during this time.
Students also reported higher anxiety both in general and about their education. Limited international work 22 has suggested this as well. Some work has discussed resident and healthcare personnel concerns regarding lack of personal protective equipment, 1, 23-24 but no empirical data has speci cally measured this in medical students. This is particularly important given pre-COVID literature indicating that medical students are more likely to show signs of depression and burnout compared to the general population. [25][26] Together, our ndings indicate that both students and educators should be aware of and directly address the challenges of COVID-19 not just in the classroom, but as it relates to nancial and mental health.

Long-Term Impact of COVID-19 at the Institutional and National Level
On May 11, the Coalition for Physician Accountability issued its recommendations for the upcoming (current) academic year, which limited or restricted medical student away rotations. 4 At the time this survey was distributed, many institutions had cancelled third-year clerkships, with some continuing to be impacted at present. More empirical work is needed to see if supplemental education (such as virtual "away" rotations, streaming-based open houses or town halls, etc.) [27][28][29] may provide an adequate alternative to typical in-person networking and training, given that away rotations provide students opportunities to network, obtain letters of recommendation, 12 and demonstrate an interest in speci c programs. On the other hand, this shift may be bene cial to students who will save money on housing and other rotation expenses (for example, in 2016, it was estimated that a single away rotation cost the average student $958.) 30 Although it is too early to tell how COVID-19 will affect the upcoming residency match cycle, students expressed worry that changes to their education would impact their applications. Recent articles responding to this "match panic" 31 by students have expressed concern that the important interview process cannot be replicated virtually. 32 Limited recent research has emphasized the importance of utilizing mentoring 33 during this process, getting to know programs via virtual means (such as open houses or town halls), and familiarizing oneself with relevant technology (e.g. video conferencing) for optimal participation in interviews and remote networking. 34 Students may also feel underprepared if applying to more competitive specialties, or for vulnerable applicants from smaller medical schools that may not have a wide breadth of a liated programs or home specialty experience.
Finally, a substantial portion of respondents also indicated that their national exams had been either postponed or delayed (up to 66%). The USMLE examinations administered at Prometric testing centers were cancelled from mid-March through May of 2020. 35 Examinations continue to be limited due to the small capacity of testing centers following social distancing protocols. Some virtual changes have been made, with the traditionally in-person Step 2 CS being conducted virtually until at least June 1, 2021. 36 This is of particular concern given that national examinations have a large impact on the careers of medical students. According to the NRMP 2018 Program Director Survey, 94% of program directors valued USMLE Step 1 score or COMLEX Level 1 score and 80% of PDs valued Step 2 CK score or COMLEX Level 2 score. 37 Rescheduling important national examinations adds to uncertainty for both medical students and medical educators, and a lack of credentialing at appropriate milestones presents predicaments for institutions wondering if they should pass students to the next level or not.

Limitations
Survey studies present the inherent bias of utilizing a convenience sample. Participants were recruited via email and social media, which may have some self-selection bias. However, this survey accrued a sample of participants on par with those published previously in this topic and via these recruitment methods. In addition, the demographic characteristics of the participants are similar to those reported across medical schools nationally.

Conclusion
We have yet to see the effects of COVID-19 on medical education in its entirety. This cross-sectional study found a tremendous impact of the pandemic on medical students' education and training. This impact includes a transition from in-person to virtual and remote learning, having to serve on the "front lines" during a global pandemic, feelings of anxiety and burnout related to nancial and mental health, and long-term impacts of these changes on grades, credentialing, and residency applications. Our ndings complement emerging international studies indicating that these changes will have long-last impact on trainees and we suggest concrete directions for future empirical research supporting this educational development.

Declarations
Ethics approval and consent to participate: All methods were performed in accordance with the relevant guidelines and regulations. This study was approved by The Ohio State University Institutional Review Board Consent for publication: All authors consent to the publication of this manuscript. All information is anonymized with no identi able participant information.
Availability of data and materials: The datasets used and/or analyzed during the current study are available in an anonymised format from the corresponding author on reasonable request.
Funding: Although funding was not provided to the authors, REDCap was provided via institutional support: The Ohio State University Center for Clinical and Translational Science grant support (National Center for Advancing Translational Sciences, Grant UL1TR002733).
Authors' contributions: We assert that all authors played a substantial role in the preparation of this manuscript. Figure 1 Proportion of students reporting the impact of COVID-19 on their medical education, grades, and future residency applications. Note that pre-clinical and clinical students did not differ in their perceptions of this impact on medical education or grades (p>0.1), but clinical students were more worried about the impact of COVID-19 on their residency applications (p=0.021) than pre-clinical students.