Impact of COVID-19 pandemic on students’ performance in a problem-based learning system: comparative study between face-to-face-and virtual learning


 Background

Amongst all other educational institutions, medical schools suffered the consequences of the COVID-19 pandemic. Medical education requires a great deal of interaction between instructors and students, and in the final years, patients as well. In response to the pandemic, the College of Medicine and Medical Sciences at the Arabian Gulf University has applied virtual teaching/learning since March 2020 as an alternative to face-to-face teaching. The college used Moodle and Zoom as online methods for education. The aim of the present study was to evaluate the effectiveness of virtual medical education by comparing students’ performance in final exams in face-to-face and virtual settings.
Methods

Following the college's ethical approval, this longitudinal study was performed on 183 medical students. Those students experienced 2 different successive methods of teaching/learning; Unit V as face-to-face followed by Unit VI as virtual settings. Students' performance in the final exams of both units was analyzed and compared.
Results

There was no significant difference in student performance between Units V and VI. Students' performance in the physiology part was equally effective in both units, while the difficulty index of both exams was insignificantly different.
Conclusion

Our results demonstrate that students’ performance in final exams could serve as an objective parameter when comparing different educational settings. Our results also support the idea that, in certain aspects, virtual is equal to face-to-face medical education strategies.


Introduction
In January 2020, the World Health Organization (WHO) announced worldwide the outbreak of COVID-19, which is a new strain of coronavirus. Two months later, in March 2020, the disease was declared as a pandemic [1]. COVID-19 has thrown its immense impact on all areas of our life. It affected our communication with each other, our social life, our rituals, and ways of worships and even our way of dressing and clothing, with wearing a mask becoming a must and a necessary piece of attire. The eld of learning and teaching is of no exception. Schools and universities found themselves compelled to adopt new ways of teaching, as students were unable to come to classes and share the space and place with other colleagues. Almost all educational institutions applied distant or virtual learning, i.e., online learning as a substitute to face-to-face learning. The current situation is unique and hopefully unrepeatable, at least in the near future, so it is important to evaluate the impact of this phenomenon on different aspects of higher education industry including students' performance.
In response to this unprecedented event, the government of the Kingdom of Bahrain imposed strict and draconian precautious measures to hinder the spread of the coronavirus disease. All walks of life were suspended or at least were put to a bottom-low-level. All educational institutions all over the Kingdom of Bahrain were put on suspension including the regional Arabian Gulf University-College of Medicine and Medical Sciences (AGU-CMMS). AGU is a regional university under the nancial support of 6 Arabian Gulf countries, i.e., United Arab Emirates, Kingdom of Bahrain, State of Kuwait, Sultanate of Oman, State of Qatar, and Kingdom of Saudi Arabia. CMMS at AGU adopted the Problem Based Learning (PBL) method for its medical program [2,3]. The AGU students are mainly from this region of the world. After the suspension of classroom teaching, AGU students went back to their countries in the hope of returning soon, but the news broke out that that they should stay home and resume their classes by e-learning or what is recently known as virtual or online learning. Hence, virtual classes for medical undergraduate students at AGU were commenced on 21st March 2020, as almost all AGU students have no access to AGU campus or face-to-face learning.
Virtual or online learning made the process of communication between students and their lecturers easier and more convenient. Students started receiving their medical education while staying at home [4][5][6][7]. For the last twenty years distant learning has played a signi cant role and created a severe competition among universities to expand their scope of teaching and their nancial income [8][9][10][11]. Previous studies on the e ciency of online learning stated that this method of teaching should be consistently evaluated to potentially enhance e-learning [12]. As this experience of teaching is a novel method of teaching at AGU, it was necessary to assess and examine the outcome of the last six months of online learning at AGU. Recent studies were carried out on the impact of COVID-19 on clinical practice [13,14], and on how e-learning is a conducive way and an interactive method during the COVID-19 pandemic [15].
The aim of the present study was to evaluate the effectiveness of virtual teaching-learning strategies on the students' performance in nal exams at AGU-CMMS amid COVID-19 pandemic, when compared to their performance in conventional teaching-learning environment.

Methods
The Arabian Gulf University -College of Medicine and Medical Sciences (AGU-CMMS) has applied the virtual teaching since March 2020 as an alternative to face-to-face teaching. The university used Moodle and Zoom as an online way of teaching.
AGU-CMMS medical curriculum consists of three phases, phase I, as year one with basic and fundamental courses in science and English. Phase II, which consists of 9 units: unit I, II and III are taught in year two, units IV, V and VI in year three, and units VII, VIII and IX are given in year four. Each unit contains several cases or disease-scenarios that tackle a speci c system, i.e. cardiovascular, respiratory, gastrointestinal, renal...etc. From the weekly cases, students can come up with learning needs or objectives that they are obliged to nd the answers for and discuss them with their tutors.
In this study during the academic year 2019-2020, year 3 students took unit V as face-to-face learning. However, after the COVID-19 pandemic, the same group of students were forced to take their resources and lectures for Unit VI online, which provoked our team to carry out a study to compare the students' performance under these two ways of learning. This is a longitudinal study that attempts to shed some light on the e ciency of AGU experience in online learning for year 3 students. The study is an evaluation of the outcome of online method of teaching as compared to face-to-face traditional learning. The group of students (192 students, 20-22 years old) experienced face-to face teaching from the end of December 2019 to mid-March 2020 studying gastrointestinal and renal concepts and cases (unit V). The same group of students after the outbreak of the pandemic were compelled to take unit VI, concepts and cases of hematology, in an online method of learning.
For each student out of a cohort of 183 students, and for both Unit exams, the number of A-Type multiple choice questions (MCQs) that each student answered correctly was calculated, and then transformed to a percentage of the total number of questions. Data is expressed as mean ± SD. Kolmogorov-Smirnov test was used to test for normality of data. Statistical tests were selected according to the normality of data.

Results
Unit V (Gastrointestinal and Renal System) was delivered with face-to-face educational methods, while Unit VI (Hematopoietic and Immune System) was conducted remotely online due to the restrictions of the CoVid-19 pandemic, For each student out of a cohort of 183 students, and for both Unit exams, the number of A-Type multiple choice questions (MCQs) that each student answered correctly was calculated, and then transformed to a percentage of the total number of questions. Data is expressed as mean ± SD.
For both Units, the data on percentage of MCQs answered correctly (scores) for each student passed the normality test (Kolmogorov-Smirnov test), so parametric analysis was utilized. The score data ranged between 41.3% to 95.8% for Unit V and 45.8-95.8% for Unit VI. The mean MCQ score was 76.0 ± 10.4% and 75.6 ± 10.9% for Units V and VI, respectively (Fig. 1). A paired t test revealed that the mean MCQ scores were not different between the two Units (t (182) = 0.66, P = 0.51). Additionally, Chi-square test of independence revealed no association between Unit and percentage of MCQs answered correctly: X2 (1, 4) = 5.88, P = 0.21.
The Physiology component in the end of Unit exam comprised of 21 MCQs in Unit V and 9 MCQs in Unit VI. One Physiology MCQ in Unit V had a negative discrimination index and was discarded from the analysis. The data of scores for correctly answered Physiology MCQs by each student failed the normality test in Unit V. The mean score was 64.9 ± 13.5% (Range: 28.6-95.2%) and 77.4 ± 18.4% (Range: 22.2-100.0%) for Units V and VI, respectively. A Wilcoxon Matched-pairs Signed-Ranks Test showed that the median Unit VI score ranks were signi cantly higher than the median Unit V score ranks (P < 0.0001). Chi-square test of independence revealed a strong association between Unit and percentage of MCQs answered correctly: X2 (1, 4) = 85.45, P < 0.0001. This is obvious in Fig. 2, which reveals that the number of students answering 85-100% of the Physiology MCQs correctly was high (n = 79/183 or 43% of the total student cohort).
We examined other factors that may have culminated in such a result. We compared mean MCQ di culty index for Physiology MCQs between the two Units. Di culty index data passed the normality test for both Units. The mean di culty index for Unit VI Physiology MCQs was 0.67 ± 0.24 (Range: 0.30-0.95) and 0.77 ± 0.14 (Range: 0.58-0.97), however unpaired t test analysis revealed no signi cant difference between the means (t(27) = 1.09; P = 0.28). However, Fig. 3 shows that for Unit V, there were 7 MCQs (35%) with di culty index well below the minimum value for Unit VI (value of 0.58).

Discussion
The rapid transition of COVID-19 into a worldwide pandemic has had serious consequences on the higher education institutions with special concerns for the medical schools. The necessity of commitment, discipline, and acquisition of clinical skills underlie the challenging nature of medical professionalism.
The traditional face-to-face learning format have stood the test of time providing direct interaction with the instructor, commitment to participate, and attention in small group teaching [16]. The responsibility of the medical schools to follow the advices of the local health authorities, as well as the accountability of the institutions to execute students' assessment and graduation timely, have seriously impacted decisions to promptly adopt online medical education strategies.
In the present study, the same cohort of the 3rd year medical students was followed up during units V (gastrointestinal and renal systems) and VI (hematopoietic and immune systems), with the earlier being delivered face-to-face before the COVID-19-induced shut down of university premises. This provided an almost optimal scenario to compare students' achievement independent of the method of teaching. Our results demonstrated that student's overall performance in both units was not statistically different implying no negative effect of shifting to online teaching strategies even within the same academic year. Our results were in agreement with previously published data which demonstrated that the effectiveness of e-learning in different medical education settings appeared to be as e cient as [17] or even superior to [18] traditional face-to-face learning approaches.
We focused then on the physiology component in both units for the interest of the physiology department. The overall students' performance in the physiology questions followed the pattern seen in the whole exam; no signi cant difference was observed between both groups. No doubt that there was fear of the shift from face-to-face to on-line methods, specially that physiology typically deals with mechanisms and interpretations which need proper student-instructor interaction. Previous reports demonstrated the effectiveness of online-physiology teaching methods in mandatory physiology modules when compared to the traditional instructor-dependent methods [13]. We notice substantial increase in the number of students answering 85-100% of the physiology questions correctly in unit VI when compared to the corresponding value in unit V. To eliminate the possibility of relative easiness of unit VI exam, we compared the di culty indices of both physiology exam questions which were insigni cantly different. We concluded that although there was no statistical difference in student's performance in the physiology component between both units, yet student's performance in unit VI tended to be superior to their performance in unit V, and this was not due to easiness of the unit VI exam.
Our results highlighted an objective parameter denoting the students' performance in the nal exam.
Most published data comparing traditional to e-learning methods in medical education has focused primarily on students' acceptance, motivation and satisfaction [13,19,20]. However, few reported on the effect of e-learning on the medical students' performance in nal exams. Our results not only supports the idea that e-learning could be as effective as face-to-face medical education in the preparation of the students for better performance in nal exams, but it also proves that shifting from one method to another even within the same academic year is possible and has no drastic effect on students' performance in nal exams.

Strengths And Limitations
Strengths of the study the evaluating of the same cohort of students in two units with different teaching methods but with the same instructor. The focus on students' performance in the nal exam remains an important topic this study challenged.
Limitation of the study includes the unequal weight of the physiology component in both units, and therefore the unequal number of physiology questions in each nal exam.

Conclusion
COVID-19 pandemic raised the necessity to incorporate technology into the medical education process. In fact, without virtual learning facilities, the drastic impact of the global crisis on medical education could have been greater. Our results demonstrated that students' performance in nal exams was not different in face-to-face and online environments. Having said that, online learning cannot totally substitute conventional learning in medical education. Psycho-motor skills neither can be optimally conducted nor assessed via virtual interface. Unquestionably, a blended system seems to be the inevitable future of medical education.

Declarations
Ethics approval and consent to participate This study was approved by the Research and Ethics Committee of the Faculty of Medicine and Medical Sciences, Arabian Gulf University, Bahrain (E002-PI-10/20). The need for written or verbal consent was waived as deemed unnecessary. All methods were performed in accordance to the regulations and guidelines of the Research and Ethics Committee of the Faculty of Medicine and Medical Sciences, Arabian Gulf University, Bahrain.

Availability of data and materials
All the data supporting ndings are presented within the manuscript.

Competing interests
No con ict of interests.

Funding
This research did not receive any speci c grant from funding agencies in the public, commercial, or notfor-pro t sectors.
Authors' contributions TAA participated in the study design and coordination, reviewed the literature, and drafted the manuscript. RLA participated in the study design and performed the statistical analysis. YMN participated in the study design, interpreted, and discussed the results. FAA participated in the study design and collected the data. All authors read and approved the nal version of the manuscript.