Impact of COVID-19 pandemic on students’ performance in a problem-based learning system: comparative study between face-to-face-and virtual learning.

DOI: https://doi.org/10.21203/rs.3.rs-114856/v1

Abstract

Background

Amongst all other educational institutions, medical schools suffered the consequences of the COVID-19 pandemic. Medical education requires a great deal of interaction between instructors and students, and in the final years, patients as well. In response to the pandemic, the College of Medicine and Medical Sciences at the Arabian Gulf University has applied virtual teaching/learning since March 2020 as an alternative to face-to-face teaching. The college used Moodle and Zoom as online methods for education. The aim of the present study was to evaluate the effectiveness of virtual medical education by comparing students’ performance in final exams in face-to-face and virtual settings.

Methods

Following the college's ethical approval, this longitudinal study was performed on 183 medical students. Those students experienced 2 different successive methods of teaching/learning; Unit V as face-to-face followed by Unit VI as virtual settings. Students' performance in the final exams of both units was analyzed and compared.

Results

There was no significant difference in student performance between Units V and VI. Students' performance in the physiology part was equally effective in both units, while the difficulty index of both exams was insignificantly different.

Conclusion

Our results demonstrate that students’ performance in final exams could serve as an objective parameter when comparing different educational settings. Our results also support the idea that, in certain aspects, virtual is equal to face-to-face medical education strategies.

Introduction

In January 2020, the World Health Organization (WHO) announced worldwide the outbreak of COVID-19, which is a new strain of coronavirus. Two months later, in March 2020, the disease was declared as a pandemic [1]. COVID-19 has thrown its immense impact on all areas of our life. It affected our communication with each other, our social life, our rituals, and ways of worships and even our way of dressing and clothing, with wearing a mask becoming a must and a necessary piece of attire. The field of learning and teaching is of no exception. Schools and universities found themselves compelled to adopt new ways of teaching, as students were unable to come to classes and share the space and place with other colleagues. Almost all educational institutions applied distant or virtual learning, i.e., online learning as a substitute to face-to-face learning. The current situation is unique and hopefully unrepeatable, at least in the near future, so it is important to evaluate the impact of this phenomenon on different aspects of higher education industry including students’ performance.

In response to this unprecedented event, the government of the Kingdom of Bahrain imposed strict and draconian precautious measures to hinder the spread of the coronavirus disease. All walks of life were suspended or at least were put to a bottom-low-level. All educational institutions all over the Kingdom of Bahrain were put on suspension including the regional Arabian Gulf University-College of Medicine and Medical Sciences (AGU-CMMS). AGU is a regional university under the financial support of 6 Arabian Gulf countries, i.e., United Arab Emirates, Kingdom of Bahrain, State of Kuwait, Sultanate of Oman, State of Qatar, and Kingdom of Saudi Arabia. CMMS at AGU adopted the Problem Based Learning (PBL) method for its medical program [2, 3]. The AGU students are mainly from this region of the world. After the suspension of classroom teaching, AGU students went back to their countries in the hope of returning soon, but the news broke out that that they should stay home and resume their classes by e-learning or what is recently known as virtual or online learning. Hence, virtual classes for medical undergraduate students at AGU were commenced on 21st March 2020, as almost all AGU students have no access to AGU campus or face-to-face learning.

Virtual or online learning made the process of communication between students and their lecturers easier and more convenient. Students started receiving their medical education while staying at home [47]. For the last twenty years distant learning has played a significant role and created a severe competition among universities to expand their scope of teaching and their financial income [811]. Previous studies on the efficiency of online learning stated that this method of teaching should be consistently evaluated to potentially enhance e-learning [12]. As this experience of teaching is a novel method of teaching at AGU, it was necessary to assess and examine the outcome of the last six months of online learning at AGU. Recent studies were carried out on the impact of COVID- 19 on clinical practice [13, 14], and on how e-learning is a conducive way and an interactive method during the COVID-19 pandemic [15].

The aim of the present study was to evaluate the effectiveness of virtual teaching-learning strategies on the students’ performance in final exams at AGU-CMMS amid COVID-19 pandemic, when compared to their performance in conventional teaching-learning environment.

Methods

The Arabian Gulf University - College of Medicine and Medical Sciences (AGU-CMMS) has applied the virtual teaching since March 2020 as an alternative to face-to-face teaching. The university used Moodle and Zoom as an online way of teaching.

AGU-CMMS medical curriculum consists of three phases, phase I, as year one with basic and fundamental courses in science and English. Phase II, which consists of 9 units: unit I, II and III are taught in year two, units IV, V and VI in year three, and units VII, VIII and IX are given in year four. Each unit contains several cases or disease-scenarios that tackle a specific system, i.e. cardiovascular, respiratory, gastrointestinal, renal...etc. From the weekly cases, students can come up with learning needs or objectives that they are obliged to find the answers for and discuss them with their tutors.

In this study during the academic year 2019–2020, year 3 students took unit V as face-to-face learning. However, after the COVID-19 pandemic, the same group of students were forced to take their resources and lectures for Unit VI online, which provoked our team to carry out a study to compare the students’ performance under these two ways of learning.

This is a longitudinal study that attempts to shed some light on the efficiency of AGU experience in online learning for year 3 students. The study is an evaluation of the outcome of online method of teaching as compared to face-to-face traditional learning. The group of students (192 students, 20–22 years old) experienced face-to face teaching from the end of December 2019 to mid-March 2020 studying gastrointestinal and renal concepts and cases (unit V). The same group of students after the outbreak of the pandemic were compelled to take unit VI, concepts and cases of hematology, in an online method of learning.

For each student out of a cohort of 183 students, and for both Unit exams, the number of A-Type multiple choice questions (MCQs) that each student answered correctly was calculated, and then transformed to a percentage of the total number of questions. Data is expressed as mean ± SD. Kolmogorov-Smirnov test was used to test for normality of data. Statistical tests were selected according to the normality of data.

Results

Unit V (Gastrointestinal and Renal System) was delivered with face-to-face educational methods, while Unit VI (Hematopoietic and Immune System) was conducted remotely online due to the restrictions of the CoVid-19 pandemic,

For each student out of a cohort of 183 students, and for both Unit exams, the number of A-Type multiple choice questions (MCQs) that each student answered correctly was calculated, and then transformed to a percentage of the total number of questions. Data is expressed as mean ± SD.

For both Units, the data on percentage of MCQs answered correctly (scores) for each student passed the normality test (Kolmogorov-Smirnov test), so parametric analysis was utilized. The score data ranged between 41.3% to 95.8% for Unit V and 45.8–95.8% for Unit VI. The mean MCQ score was 76.0 ± 10.4% and 75.6 ± 10.9% for Units V and VI, respectively (Fig. 1). A paired t test revealed that the mean MCQ scores were not different between the two Units (t (182) = 0.66, P = 0.51). Additionally, Chi-square test of independence revealed no association between Unit and percentage of MCQs answered correctly: X2 (1, 4) = 5.88, P = 0.21.

The Physiology component in the end of Unit exam comprised of 21 MCQs in Unit V and 9 MCQs in Unit VI. One Physiology MCQ in Unit V had a negative discrimination index and was discarded from the analysis. The data of scores for correctly answered Physiology MCQs by each student failed the normality test in Unit V. The mean score was 64.9 ± 13.5% (Range: 28.6–95.2%) and 77.4 ± 18.4% (Range: 22.2–100.0%) for Units V and VI, respectively. A Wilcoxon Matched-pairs Signed-Ranks Test showed that the median Unit VI score ranks were significantly higher than the median Unit V score ranks (P < 0.0001). Chi-square test of independence revealed a strong association between Unit and percentage of MCQs answered correctly: X2 (1, 4) = 85.45, P < 0.0001. This is obvious in Fig. 2, which reveals that the number of students answering 85–100% of the Physiology MCQs correctly was high (n = 79/183 or 43% of the total student cohort).

We examined other factors that may have culminated in such a result. We compared mean MCQ difficulty index for Physiology MCQs between the two Units. Difficulty index data passed the normality test for both Units. The mean difficulty index for Unit VI Physiology MCQs was 0.67 ± 0.24 (Range: 0.30–0.95) and 0.77 ± 0.14 (Range: 0.58–0.97), however unpaired t test analysis revealed no significant difference between the means (t(27) = 1.09; P = 0.28). However, Fig. 3 shows that for Unit V, there were 7 MCQs (35%) with difficulty index well below the minimum value for Unit VI (value of 0.58).

Discussion

The rapid transition of COVID-19 into a worldwide pandemic has had serious consequences on the higher education institutions with special concerns for the medical schools. The necessity of commitment, discipline, and acquisition of clinical skills underlie the challenging nature of medical professionalism. The traditional face-to-face learning format have stood the test of time providing direct interaction with the instructor, commitment to participate, and attention in small group teaching [16]. The responsibility of the medical schools to follow the advices of the local health authorities, as well as the accountability of the institutions to execute students’ assessment and graduation timely, have seriously impacted decisions to promptly adopt online medical education strategies.

In the present study, the same cohort of the 3rd year medical students was followed up during units V (gastrointestinal and renal systems) and VI (hematopoietic and immune systems), with the earlier being delivered face-to-face before the COVID-19-induced shut down of university premises. This provided an almost optimal scenario to compare students’ achievement independent of the method of teaching. Our results demonstrated that student’s overall performance in both units was not statistically different implying no negative effect of shifting to online teaching strategies even within the same academic year. Our results were in agreement with previously published data which demonstrated that the effectiveness of e-learning in different medical education settings appeared to be as efficient as [17] or even superior to [18] traditional face-to-face learning approaches.

We focused then on the physiology component in both units for the interest of the physiology department. The overall students’ performance in the physiology questions followed the pattern seen in the whole exam; no significant difference was observed between both groups. No doubt that there was fear of the shift from face-to-face to on-line methods, specially that physiology typically deals with mechanisms and interpretations which need proper student-instructor interaction. Previous reports demonstrated the effectiveness of online-physiology teaching methods in mandatory physiology modules when compared to the traditional instructor-dependent methods [13]. We notice substantial increase in the number of students answering 85–100% of the physiology questions correctly in unit VI when compared to the corresponding value in unit V. To eliminate the possibility of relative easiness of unit VI exam, we compared the difficulty indices of both physiology exam questions which were insignificantly different. We concluded that although there was no statistical difference in student’s performance in the physiology component between both units, yet student’s performance in unit VI tended to be superior to their performance in unit V, and this was not due to easiness of the unit VI exam.

Our results highlighted an objective parameter denoting the students’ performance in the final exam. Most published data comparing traditional to e-learning methods in medical education has focused primarily on students’ acceptance, motivation and satisfaction [13, 19, 20]. However, few reported on the effect of e-learning on the medical students’ performance in final exams. Our results not only supports the idea that e-learning could be as effective as face-to-face medical education in the preparation of the students for better performance in final exams, but it also proves that shifting from one method to another even within the same academic year is possible and has no drastic effect on students’ performance in final exams.

Strengths And Limitations

Strengths of the study the evaluating of the same cohort of students in two units with different teaching methods but with the same instructor. The focus on students’ performance in the final exam remains an important topic this study challenged.

Limitation of the study includes the unequal weight of the physiology component in both units, and therefore the unequal number of physiology questions in each final exam.

Conclusion

COVID-19 pandemic raised the necessity to incorporate technology into the medical education process. In fact, without virtual learning facilities, the drastic impact of the global crisis on medical education could have been greater. Our results demonstrated that students’ performance in final exams was not different in face-to-face and online environments. Having said that, online learning cannot totally substitute conventional learning in medical education. Psycho-motor skills neither can be optimally conducted nor assessed via virtual interface. Unquestionably, a blended system seems to be the inevitable future of medical education.

Declarations

Ethics approval and consent to participate

This study was approved by the Research and Ethics Committee of the Faculty of Medicine and Medical Sciences, Arabian Gulf University, Bahrain (E002-PI-10/20). The need for written or verbal consent was waived as deemed unnecessary. All methods were performed in accordance to the regulations and guidelines of the Research and Ethics Committee of the Faculty of Medicine and Medical Sciences, Arabian Gulf University, Bahrain.

Availability of data and materials

All the data supporting findings are presented within the manuscript.

Competing interests

No conflict of interests.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Authors’ contributions

TAA participated in the study design and coordination, reviewed the literature, and drafted the manuscript. RLA participated in the study design and performed the statistical analysis. YMN participated in the study design, interpreted, and discussed the results. FAA participated in the study design and collected the data. All authors read and approved the final version of the manuscript.

Acknowledgments

Authors would also like to acknowledge the Faculty of Medicine and Medical Sciences for providing all the necessary requirements. 

References

  1. Liu Y-C, Kuo R-L, Shih S-R: COVID-19: The first documented coronavirus pandemic in history. Biomedical Journal 2020, 43(4):328–333.
  2. Bindayna KM, Deifalla A: The Curriculum at the College of Medicine and Medical Sciences at Arabian Gulf University: A Way Forward to Meet the Future Medical Education Needs. Journal of Medical Education and Curricular Development 2020, 7:2382120520932904.
  3. Hamdy H, Greally M, Grant IN, El-Shazali H, Nayar U, Rajab K, Al-Roomi K, Mohammad AM, Hamza AA, Al-Awadhi MA et al: Professional skills programme in a problem-based learning curriculum: experience at the College of Medicine & Medical Sciences, Arabian Gulf University. Med Teach 2001, 23(2):214–216.
  4. Childs S, Blenkinsopp E, Hall A, Walton G: Effective e-learning for health professionals and students—barriers and their solutions. A systematic review of the literature—findings from the HeXL project. Health Information & Libraries Journal 2005, 22(s2):20–32.
  5. Colace F, Santo MD, Pietrosanto A: Evaluation Models for E-Learning Platform: an AHP approach. In: Proceedings Frontiers in Education 36th Annual Conference: 27–31 Oct. 2006 2006; 2006: 1–6.
  6. Lee SJ, Srinivasan S, Trail T, Lewis D, Lopez S: Examining the relationship among student perception of support, course satisfaction, and learning outcomes in online learning. The Internet and Higher Education 2011, 14(3):158–163.
  7. Prober CG, Heath C: Lecture Halls without Lectures — A Proposal for Medical Education. New England Journal of Medicine 2012, 366(18):1657–1659.
  8. Blouin RA, Riffee WH, Robinson ET, Beck DE, Green C, Joyner PU, Persky AM, Pollack GM: Roles of innovation in education delivery. Am J Pharm Educ 2009, 73(8):154–154.
  9. Bolliger D, Erichsen E: Student Satisfaction with Blended and Online Courses Based on Personality Type. Canadian Journal of Learning and Technology / La revue canadienne de l’apprentissage et de la technologie 2013, 39(1).
  10. Buchanan T, Sainter P, Saunders G: Factors affecting faculty use of learning technologies: implications for models of technology adoption. Journal of Computing in Higher Education 2013, 25(1):1–11.
  11. Chapman TA: Evaluation in Distance Education and e-Learning: The Unfolding Model. Am J Pharm Educ 2010, 74(1):17a.
  12. Pham L, Limbu YB, Bui TK, Nguyen HT, Pham HT: Does e-learning service quality influence e-learning student satisfaction and loyalty? Evidence from Vietnam. International Journal of Educational Technology in Higher Education 2019, 16(1):7.
  13. Felder E, Fauler M, Geiler S: Introducing e-learning/teaching in a physiology course for medical students: acceptance by students and subjective effect on learning. Advances in Physiology Education 2013, 37(4):337–342.
  14. Tabatabai S: COVID-19 impact and virtual medical education. J Adv Med Educ Prof 2020, 8(3):140–143.
  15. Gaber DA, Shehata MH, Amin HAA: Online team-based learning sessions as interactive methodologies during the pandemic. Medical Education 2020, 54(7):666–667.
  16. Ward R, Stevens C, Brentnall P, Briddon J: The attitudes of health care staff to information technology: a comprehensive review of the research literature. Health Information & Libraries Journal 2008, 25(2):81–97.
  17. Ruiz JG, Mintzer MJ, Leipzig RM: The Impact of E-Learning in Medical Education. Academic Medicine 2006, 81(3):207–212.
  18. George PP, Papachristou N, Belisario JM, Wang W, Wark PA, Cotic Z, Rasmussen K, Sluiter R, Riboli-Sasco E, Tudor Car L et al: Online eLearning for undergraduates in health professions: A systematic review of the impact on knowledge, skills, attitudes and satisfaction. J Glob Health 2014, 4(1):010406–010406.
  19. Keis O, Grab C, Schneider A, Öchsner W: Online or face-to-face instruction? A qualitative study on the electrocardiogram course at the University of Ulm to examine why students choose a particular format. BMC Medical Education 2017, 17(1):194.
  20. Popovic N, Popovic T, Dragovic IR, Cmiljanic O: A Moodle-based blended learning solution for physiology education in Montenegro: a case study. Advances in Physiology Education 2018, 42(1):111–117.