Simulation-based education in neurology: lessons from the COVID-19 pandemic.

doi:10.21203/rs.3.rs-2231838/v1

Download PDF

Short Report

Simulation-based education in neurology: lessons from the COVID-19 pandemic.

https://doi.org/10.21203/rs.3.rs-2231838/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 02 Nov, 2023

Read the published version in BMC Research Notes →

You are reading this latest preprint version

Objectives

The COVID-19 pandemic has led to the disruption of all sectors of the economy including education. According to UNESCO over 1.37 million young people including medical students, were affected by the lockdowns in response to COVID-19 and the subsequent closure of the education system. The primary challenge for medical education was to provide clerkships in a biosafety environment. This study aimed to determine the impact of a simulated hospital in a neurology clerkship of 5-year medical students during the coronavirus pandemic and compare their results with a non-pandemic group in Bogotá, Colombia.

Results

The students in the pandemic group answered a Likert scale survey regarding their satisfaction with the simulated hospital. Both groups were required to perform an oral, mid-term and final examination. From the results, it is clear that students perceived that exposure to a simulated hospital facilitated their learning process (93.1%) and allowed greater interaction with the teacher compared to a face-to-face environment (77.3%). There were no clinically significant differences in test results. This experience indicates that a simulated hospital is a valuable method to acquire clinical skills in trainees, that could be integrated into the curricular milestones of medical education programs regardless of the pandemic.

Medical Education

Neurology

Simulated Hospital

COVID-19 Pandemic

COVID-19 the etiologic agent of SARS-COV2 (Severe Acute Respiratory Syndrome-Coronavirus type 2) was declared as a pandemic disease by the WHO (World Health Organization) on March 11th, 2020 (1). As of November 2021 there were more than 253 million confirmed cases of COVID-19 (1). Colombia was the fourth country in the region with the highest incidence per 100,000 inhabitants (2, 3). According to the United Nations Educational, Scientific and Cultural Organization (UNESCO), over 138 countries completely closed their educational systems representing around 1.37 million students (4).

In response to COVID-19, medical education has been seeking alternative teaching methods focused on virtual lectures and online simulation practice. Despite these initiatives, students are at a disadvantage when it comes to acquiring practical clinical skills (5–7). Notwithstanding, efforts and research have been focused on postgraduate students compared to undergraduate trainees (8–12).

This study aims to describe the response to the emerging educational challenges implemented by the school of medicine of Universidad de los Andes (Bogotá, Colombia). A simulated hospital (SH) was developed with educational tools, including trained actors who could replicate symptoms and signs, allowing students to learn clinical skills such as anamnesis and physical examination. This hospital applies the principles that simulation could reduce the probability of error and improve the training of medical practitioners in a safe and controlled environment that replicates a real-world experience (13–17). The findings of this study may constitute a good starting point for discussion and potential implementation of this educational model, even beyond this overwhelming scenario.

Study Design and Sample

A quasi-experimental design was conducted on a population of 5th-year medical students during their neurology clerkship at the Universidad de los Andes. Our sample comprised two similar groups across a different time period. Participants in the first group, which was composed of 57 students, received a traditional face-to-face format during the second semester of 2019, whereas the second group, composed of 46 students, received a mixed virtual and simulation-based clerkship in the second semester of 2020.

The courses were essentially identical except for the delivery method. To reduce heterogeneity between both groups, theoretical and practical components were included and the same supplementary resources were provided. In addition, classes were taught by the same attendants and the entire clerkship lasted the same number of weeks.

Neurology Clerkship

The neurology clerkship included synchronous and asynchronous learning. The face-to-face cohort's synchronous learning component was a neurology clinical rotation. Students were exposed to neurology ward patients, daily rounds, seminars and journal clubs. Professional actors recreated pathological situations in a SH for the pandemic cohort. Additionally, students were required to attend different virtual activities based on the neurology competencies addressed in the face-to-face cohort.

The asynchronous component encompasses the learning resources provided at the beginning of the course to both groups. Participants had access to textbooks, articles and neurological exam videos suggested by the attendants.

Simulated Hospital

A SH recreated different environments that resembled emergency rooms and inpatient or outpatient care settings. During this clerkship, professional actors depicted several neurological disorders using a realistic screenplay developed by experts. Before each instance, the actors met with a neurologist who validated the standardized patient's characteristics and provided feedback before, during and after the performance.

The students were able to interview the standardized patient and perform a clinical history and neurological examination. All of the details of the anamnesis and neurologic exam were done in a controlled environment, observed and guided by an attendant. After a thorough debriefing, the attendant provided objective feedback. The assessment strategy for the student performance was the ability to give a reliable history, perform a complete neurologic examination, localize lesions and elaborate on a differential diagnosis and treatment. Finally, empathy, communication abilities and interpersonal skills were also explored.

Measurement of the acquired knowledge

To evaluate theoretical knowledge acquisition, students in the pandemic and the non-pandemic groups were required to take a mid-term and a final examination through a virtual platform. To minimize heterogeneity between both sets of data, all the assessments comprised questions of the same complexity. Additionally, an oral exam was conducted to measure the practical skills of both cohorts. Each exam was measured using a scale from 0.0 to 5.0.

Satisfaction evaluation

Students in the pandemic group answered two 11-question Likert scale surveys regarding their satisfaction with the virtual clerkship and the SH. This assessment explored learning abilities and other features such as effective feedback, difficulties treating actors as real patients, interaction with teachers and the overall quality of the information received. Likewise a similar 10-question Likert-type scale survey was given to the neurologists who attended the SH. The questionnaire evaluated the satisfaction of teachers regarding their perception of the SH and its effectiveness as a teaching tool. Equivalent questions about the learning environment, interaction with the students and the ability of the actors to represent a real patient were asked.

Statistical Analysis

To check the normality of the distribution of quantitative variables, a Shapiro–Wilk test was employed. Central tendency statistics (means) and dispersion (standard deviation) were employed to describe normal variables, otherwise median and interquartile ranges were used. Due to the non-normal distribution of continuous variables and to compare medians between pandemic and non-pandemic groups, a Wilcoxon rank sum test was performed. In the case of qualitative dichotomous variables, the difference in proportions Z-test was used.

Furthermore, scores on oral, mid-term and final exams were modeled using a classical linear regression. Two models were constructed, the first one used the subject's group (pandemic or non-pandemic) as the only independent variable (bivariate model), while the second model used covariates to evaluate the incremental value of the academic performance based on exposure levels adjusted by the other covariates selected by stepwise technique (multivariate model). Both models were evaluated through the coefficient of determination and the significance of each was evaluated using a student test. A p-value ≤ 0.05 was considered statistically significant and all analyses were performed using Stata V.17.1.

Sociodemographic characteristics of subjects in both groups are shown in Table 1. The pandemic group was statistically younger than the non-pandemic participants (− 1.49 years apart, p < 0.0001). Both groups were similar with respect to sex.

Table 1

Sociodemographic Characteristics
Characteristics	Group		p-Value
Characteristics	Pandemic (n = 46)	Non-pandemic (n = 57)	p-Value
Age (years)
Median (IQR)	23 (22–24)	24.49 (23.40–25.30)	< 0.0001*
Sex (n -%)
Female	22 - 47.83	34–59.65	0.2311**
Abbreviations: IQR: interquartile range Difference calculated using the Wilcoxon Rank-Sum Test *Difference calculated using Z-test for proportions

The satisfaction of both groups regarding the teaching method was one of our main outcomes. For this purpose, we employed survey data concerning virtual clerkship satisfaction. According to our results, most of the students had good internet access and an electronic device at home. In addition, most of them considered the seminars and clinical cases valuable for their learning process. However, our results demonstrate that the main obstacle faced by the students was their ability to stay focused on their virtual classes, which affected their emotional state of mind and utmost performance. Virtual clerkship Satisfaction Likert Scale results are shown in Table 2.

Table 2

Clerkship Satisfaction Likert Scale results
Questions	Strongly disagree	Disagree	Neutral	Agree	Strongly agree
I have good quality internet access available when I need it.	1 (2.17%)	2 (4.34%)	7 (15.21%)	26 (56.52%)	10 (21.73%)
It is easy for me to focus on virtual classes.	1 (2.17%)	12 (26.08%)	12 (26.08%)	15 (32.6%)	6 (13.04%)
It is possible to have an interaction with the teachers, and that allows me to ask questions in virtual activities.	0	2 (4.34%)	5 (10.86%)	24 (52.17%)	15 (32.6%)
The technological and audiovisual tools used during the virtual classes worked adequately.	0	1 (2.17%)	6 (13.04%)	26 (56.52%)	13 (28.26%)
I perceived that most of the teachers who taught me were comfortable with virtuality.	0	3 (6.52%)	6 (13.04%)	21 (46.65%)	16 (34.78%)
Emotionally, I feel good about the virtual teaching model.	0	10 (21.7%)	14 (30.43%)	17 (36.95%)	5 (10.86%)
The virtual seminars improved my skills as a general practitioner in the area of neurology.	0	1 (2.17%)	3 (6.52%)	29 (63.04%)	13 (28.26%)
The journal club fostered my interest in the topic seen.	0	3 (6.52%)	4 (4.34%)	31 (67.39%)	8 (17.39%)
The clinical cases fostered my interest in the topic seen.	0	0	1 (2.17%)	35 (76.08%)	10 (21.73%)
The essay topics were relevant to my medical training.	0	1 (2.17%)	4 (4.34%)	31 (67.39%)	10 (21.73%)

Most of the students considered that the SH is a useful addition that should be incorporated into their curriculum regardless of the pandemic. Students considered debriefings provided adequate feedback and increased teacher-student interaction (Table 3). A similar pattern of results was obtained in the questionnaire of the neurology attendants (n = 7). All of them agreed that this is an effective tool for transmitting knowledge to students as well as developing neurology skills needed as general practitioners (Table 3). Additionally, they felt comfortable teaching and evaluating the students on the oral exam. Despite the general satisfaction with the SH, a similar conclusion was reached by the neurologists and students regarding the actors’ ability to perform the clinical characteristics required by the script.

Table 3

Simulated hospital satisfaction Likert Scale for students and teachers
Students Satisfaction Survey (n = 44)
Questions	Strongly disagree	Disagree	Neutral	Agree	Strongly Agree
The interaction with the simulated patients was satisfactory and adequately recreated a real clinical scenario.	0	2 (4.5%)	11(25%)	20(45.5%)	11 (25%)
Feedback after simulation activities was adequate.	0	0	4 (9.1%)	21(47.7%)	19 (43.2%)
The simulated hospital allowed me to develop clinical decision skills that are applicable to a real clinical scenario.	0	0	6(13.6%)	20(45.5%)	18 (40.9%)
I believe that the simulated hospital tool should be incorporated into the medicine program regardless of the pandemic.	0	1(2.3%)	4 (9.1%)	16(36.4%)	23 (52.3%)
I think that the simulated hospital facilitated my learning.	0	0	3(6.8%)	24(54.5%)	17(38.6%)
I consider that the simulated hospital allows me to have a greater interaction with the teachers, compared to a face-to-face environment	0	3 (6.8%)	7 (15.9%)	16 (36.4%)	18 (40.9%)
Teachers Satisfaction Survey (n = 7)
Questions	Strongly disagree	Disagree	Neutral	Agree	Strongly Agree
I consider that a simulation-based hospital is an effective tool for transmitting knowledge to students	0	0	0	3 (42.9%)	4 (57.1%)
I believe that the simulated hospital model should be incorporated into the medical school curriculum regardless of the pandemic.	0	2(28.6%)	0	3 (42.8%)	2 (28.6%)
The simulated hospital benefits the students by developing neurological competencies/skills as a general practitioner.	0	0	0	6 (85.7%)	1 (14.3%)
I consider that the actors adequately represented the clinical characteristics required by the script	0	0	3 (42.9%)	2 (28.6%)	2 (28.6%)
I consider the time spent with students under this simulation-based education model to be adequate.	0	0	1(14.3%)	4(57.1%)	2 (28.6%)

Finally, statistically significant differences were found in the oral and midterm exam scores between the groups (Additional file 1). The results were higher in the non-pandemic group (+ 0.3 points in the oral (p = 0.0002) and midterm exam (p = 0.0167)) compared to the pandemic group. Non-statistically significant differences were observed in the results of the final exam.

Bivariate and multivariate models are consistent with the results reported in the additional file 1. The results showed that changing from the non-pandemic to pandemic group represented a decrease of -0.468 points in the oral exam (p < 0.001) and − 0.53 points in the mid-term exam (p = 0.028) after adjusting by confounding variables (Additional file 2). Despite being statistically significant, these results are not particularly different in a real-world situation, making the scores of both groups comparable. In all the cases, the models demonstrated they were a good fit for the data.

During the COVID-19 pandemic the main challenge for medical education was to offer high-quality clerkships within a biosafe environment. As a solution to this challenge, most countries worldwide opted for virtual education models (2, 5, 19–21). Electronic-learning research has shown numerous positive effects in education, such as the availability of a vast array of resources and accessibility from a variety of locations without compromising students' security, making it suitable for the pandemic scenario. However, with this model undergraduate and postgraduate medical students identified a variety of concerns regarding their learning process, including technological challenges, diminished emotional well-being, decreased interaction between students and instructors and most importantly, difficulty with the practice of clinical skills (7, 18, 20, 22, 23).

The SH emerges as an alternative that allows the exposure of students to clinical cases within a biosafety environment. It also provides neurology trainees with an alternative to improve their clinical skills when hospital access is difficult. Participants felt that the SH allowed the development of skills for clinical decision-making, communication abilities, neuroanatomical localization and the development of competencies as general practitioners.

The emotional wellbeing of students was a major concern in the literature regarding virtual education during the pandemic and was thus a secondary outcome to be measured in this study. Our results showed that 52% of the students did not feel emotionally well with a virtual-only learning modality, a result consistent with what has been reported (7). The SH might help with a number of issues affecting the emotional state of students, such as reducing the distractions inherent to virtual education, enhancing the learning experience and promoting peer interaction.

Another advantage of the SH is that students perceived that this experience permitted better interaction with their instructors. In this environment other frequent actors in clinical practice, such as medical residents, did not divert the attention of teachers allowing direct interaction between the student, neurology attendants and the standardized patient. Along these lines, teachers can solve students’ questions and provide formative feedback, one of the pillars of education that strengthens the knowledge acquired (24).

This experience ensures exposure to neurologic diseases despite the difficulties and limitations of not being in a face-to-face environment. Our clerkship demonstrates that simulation-based practice in neurology, with the addition of virtual tools, can improve medical knowledge and develop confidence in the management of a variety of neurological pathologies needed as general practitioners. This model is not only an alternative for pandemic education but also an excellent teaching complement for medical curricula. A similar conclusion is supported by the fact that we did not find a clinically significant difference between the performance in the different exams that were conducted to evaluate theoretical knowledge. Our results indicate that the SH appears to have an influence on the students' acquired knowledge and may be an excellent complement to conventional medical education

The present study shows that a SH is a valuable method to acquire clinical skills in trainees with improvement in medical knowledge, satisfaction and comparable results in academic evaluations. This experience suggests that exposure to a SH could be integrated into medical education program's milestones regardless of the pandemic.

Due to the quasi-experimental nature of this study, the absence of randomization was one of its most significant limitations. However, we attempted to control this by adjusting for confounding variables. Because of the unpredictability of the pandemic's onset, each group could be exposed to unknown circumstances that may influence the effect of intervention.

COVID-19: coronavirus disease 2019

UNESCO: United Nations Educational, Scientific and Cultural Organization

SARS-COV2: Severe Acute Respiratory Syndrome-Coronavirus type 2

WHO: World Health Organization

SH: Simulated Hospital

Ethics approval and consent to participate: The study was reviewed by the Research Ethics Committee of the School of Medicine of Universidad de los Andes, who granted an exemption from requiring ethics approval in accordance with relevant guidelines and regulations. The information collected was secondary data already in the domain of Universidad de los Andes. The school of medicine approved the re-use of these datasets, which were originally collected for feedback and satisfaction with the overall program. Participation in the survey was entirely voluntary, and all participants gave their informed consent electronically. The School of medicine of Universidad de los Andes rigorously ensured anonymity and verified that data cannot be traced.

Consent for publication: Not applicable

Availability of data and materials: The datasets generated and/or analysed during the current study are available in PAPYRUS, the repository associated to Universidad de los Andes, http://papyrus-datos.co/dataset.xhtml?persistentId=doi:10.57924/YQKBE2

Competing interests: J. Toro is an associate editor of NEJM Journal Watch Neurology and is an editorial board member for Multiple Sclerosis and Related Disorders. Manuel Yepes, J.S. Rivera, F. Cortés-Muñoz, D.S Rodríguez-Silva, T.F Medina, J. Gaitán, L.A Serna report no disclosures.

Funding: There was no funding for this article

Authors' contributions:

-FCM, did the statistical analysis.

-LASC, DR and TM participated in the recollection of participants and methodology application.

-JG, JSR, JT and MY did the analysis of the data and the subsequent manuscript redaction.

Acknowledgements: Not applicable

Health Organization W. WHO Coronavirus (COVID-19) Dashboard [Internet]. Covid19.who.int. 2021 [cited 19 November 2021]. Available from: https://covid19.who.int.
Organización Panamericana de la Salud PAHO. 2021. “COVID-19 Region of the Americas Update.” Pan American Health Organization: 1–6. https://www.paho.org/en/documents/paho-covid-19-daily-update-25-august-2021.
World Health Organization. 2021. “COVID-19 Weekly Epidemiological Update.” World Health Organization (December): 1–3. https://www.who.int/docs/default-source/coronaviruse/situation-reports/weekly_epidemiological_update_22.pdf
United Nations Educational, Scientific and Cultural Organization. 1.37 billion students now home as COVID-19 school closures expand, ministers scale up multimedia approaches to ensure learning continuity [Internet]. 2020. Available from: https://en.unesco.org/news/137-billion-students-now-home-covid-19-school-closures-expand-ministers-scale-multimedia
Rose S. Medical Student Education in the Time of COVID-19. JAMA - J Am Med Assoc. 2020;323(21):2131–2.
Li, Heidi Oi Yee, and Adrian M.J. Bailey. 2020. “Medical Education Amid the COVID-19 Pandemic: New Perspectives for the Future.” Academic Medicine: E11–12.
Wilcha R. J. 2020. Effectiveness of Virtual Medical Teaching During the COVID-19 Crisis: Systematic Review. JMIR medical education, 6(2), e20963. https://doi-org.ezproxyegre.uniandes.edu.co:8843/10.2196/20963
Braksick, Sherri A., Kianoush Kashani, and Sara Hocker. 2017. “Neurology Education for Critical Care Fellows Using High-Fidelity Simulation.” Neurocritical Care 26(1): 96–102.
Cuffaro L, Carvalho V, Di Liberto G, Klinglehoefer L, Sauerbier A, Garcia-Azorin D, et al. Neurology training and research in the COVID-19 pandemic: a survey of the Resident and Research Fellow Section of the European Academy of Neurology. Eur J Neurol. 2020;(November 2020):1–6.
Agarwal S, Sabadia S, Abou-Fayssal N, Kurzweil A, Balcer LJ, Galetta SL. Training in neurology: Flexibility and adaptability of a neurology training program at the epicenter of COVID-19. Neurology. 2020;94(24):e2608–14.
Zeinali M, Almasi-Doghaee M, Haghi-Ashtiani B. Facing COVID-19, jumping from in- person training to virtual learning: A reviewoneducational and clinical activities in a neurology department. Basic Clin Neurosci. 2020;11(2):151–4.
Yadala S, Nalleballe K, Sharma R, Lotia M, Kapoor N, Veerapaneni KD, et al. Resident Education During COVID-19 Pandemic: Effectiveness of Virtual Electroencephalogram Learning. Cureus. 2020;12(10).
Kononowicz AA, Woodham LA, Edelbring S, Stathakarou N, Davies D, Saxena N, et al. Virtual patient simulations in health professions education: Systematic review and meta-analysis by the digital health education collaboration. J Med Internet Res. 2019;21(7):1–20.
Akaike M, Fukutomi M, Nagamune M, Fujimoto A, Tsuji A, Ishida K et al. Simulation-based medical education in clinical skills laboratory. 2021.
Berman N, Durning S, Fischer M, Huwendiek S, Triola M. The Role for Virtual Patients in the Future of Medical Education. 2021.
Ermak, David Matthew et al. 2013. “Incorporating Simulation Technology into a Neurology Clerkship.” Journal of the American Osteopathic Association 113(8): 628–35.
Ermak DM, Bower DW, Wood J, Sinz EH, Kothari MJ. Incorporating simulation technology into a neurology clerkship. J Am Osteopath Assoc. 2013;113(8):628–35.
Ferrel MN, Ryan JJ. The Impact of COVID-19 on Medical Education. Cureus. 2020;12(3):10–3.
Ahmed H, Allaf M, Elghazaly H. COVID-19 and medical education. Lancet Infect Dis. 2020 Jul;20(7):777-778. doi: 10.1016/S1473-3099(20)30226-7. Epub 2020 Mar 23. Erratum in: Lancet Infect Dis. 2020 May;20(5):e79. PMID: 32213335; PMCID: PMC7270510.
Alsoufi, Ahmed et al. 2020. “Impact of the COVID-19 Pandemic on Medical Education: Medical Students’ Knowledge, Attitudes, and Practices Regarding Electronic Learning.” PLoS ONE 15(11 November): 1–20. http://dx.doi.org/10.1371/journal.pone.0242905.
Khasawneh, A. I., Humeidan, A. A., Alsulaiman, J. W., Bloukh, S., Ramadan, M., Al-Shatanawi, T. N., Awad, H. H., Hijazi, W. Y., Al-Kammash, K. R., Obeidat, N., Saleh, T., & Kheirallah, K. A. (2020). Medical Students and COVID-19: Knowledge, Attitudes, and Precautionary Measures. A Descriptive Study From Jordan. Frontiers in public health, 8, 253. https://doi.org/10.3389/fpubh.2020.00253
Kim, So Mi, Seok Gun Park, Young Koo Jee, and Il Han Song. 2020. “Perception and Attitudes of Medical Students on Clinical Clerkship in the Era of the Coronavirus Disease 2019 Pandemic.” Medical Education Online 25(1). https://doi.org/10.1080/10872981.2020.1809929.
Seifman, Marc Adam, Sibon K. Fuzzard, Henry To, and Debra Nestel. 2021. “COVID-19 Impact on Junior Doctor Education and Training: A Scoping Review.” Postgraduate Medical Journal: 1–11.
Sandrone S, Berthaud J V., Carlson C, Cios J, Dixit N, Farheen A, et al. Education Research: Flipped classroom in neurology: Principles, practices, and perspectives. Neurology. 2019;93(1):E106–11.

Competing interest reported. J. Toro is an associate editor of NEJM Journal Watch Neurology and is an editorial board member for Multiple Sclerosis and Related Disorders. Manuel Yepes, J.S. Rivera, F. Cortés-Muñoz, D.S Rodríguez-Silva, T.F Medina, J. Gaitán, L.A Serna report no disclosures.

Additionalfile1.pdf
Title of data: Table S1. Comparison of test scores by exam type and study group.
Additionalfile2.pdf
Title of data: Table S2. Bivariate and Multivariate Linear Regression Models by Exam Type.

Download PDF

Journal Publication

published 02 Nov, 2023

Read the published version in BMC Research Notes →

Editorial decision: Major revision
15 Feb, 2023
Editor invited by journal
30 Nov, 2022
Editor assigned by journal
10 Nov, 2022
Submission checks completed at journal
09 Nov, 2022
First submitted to journal
02 Nov, 2022

You are reading this latest preprint version

Simulation-based education in neurology: lessons from the COVID-19 pandemic.

Status:

Journal Publication

Version 1

Abstract

Objectives

Results

Introduction

Methods

Study Design and Sample

Neurology Clerkship

Simulated Hospital

Measurement of the acquired knowledge

Satisfaction evaluation

Statistical Analysis

Results

Discussion

Conclusions

Limitations

Abbreviations

Declarations

References

Additional Declarations

Supplementary Files

Status:

Journal Publication

Version 1