Effectiveness and Preparedness of Institutions’ E-learning Method During COVID-19 Pandemic for residents’ medical training in Saudi Arabia: A pilot study

doi:10.21203/rs.3.rs-270326/v1

Background: Under the urgent circumstances of COVID-19 pandemic, higher education institutions of an international scale have resorted to online education methods, exclusive or not. Among those, medical institutions are under double pressure, fighting the pandemic’s effects and, at the same time providing efficient clinical training to their residents.

Methods: This is an observational cross-sectional study design. The survey’s sample included 300 medical students and residents of multiple training levels and specialties, coming from more than 15 different cities of Saudi Arabia. Filling the questionnaire required specific inclusion criteria and all obtained data were secured by Saudi Commission of Health specialty. Main objective was to evaluate the quality of e-learning methods, provided by medical universities.

Results: The study found that the frequency of digital education use increased by approximately 61% during the coronavirus crisis, while almost 9 out of 10 residents have used some e-learning platform. It was reported that before the pandemic, participants’ online training was deemed to be rather ineffective, given the rate of 3.65 out of 10. However, despite the increase in e-learning use after COVID-19, many obstacles arose during adaptation process, according to our survey: lectures and training sessions were not conducted as per curriculum (56.33%); both students and instructors’ academic behavior and attitude changed (48.33%); engagement, satisfaction and motivation in class was rated low (5.93, 6.33 & 6.54 out of 10 accordingly), compared to the desired ones. Still, participants accredited e-learning as a potential mandatory tool (77.67%) and pinpointed the qualifications that in their opinion will maximize educational impact.

Conclusions: The study concluded that innovative restructuring of online education should be based on defined critical success factors (technical support, content enhancement, pedagogy etc.) and if possible, set priority levels, so that a more permanent e-learning practice is achievable.

Educational Philosophy and Theory

E-learning

COVID-19

medical training

effectiveness

readiness

pilot study

The global spread of the novel coronavirus (COVID-19) has emerged in late December 2019 keeping billions locked down at home. The Covid-19 disease has been labeled by the World Health Organization (WHO) as an international public health emergency [1]. The Saudi government and the public health experts took a very tight measures to minimize the risk of uncontrollable spread of the infection. In response to this pandemic, several countries have announced the closure of all educational institutions. To ensure finish their curriculum on time, universities have moved rapidly to transition various courses and programs from on-site to online delivery model education has changed dramatically, with the distinctive rise of e-learning and on digital platforms [2].

Globally, online learning is not a new method of teaching for any university but preparedness to pandemic crisis varies between institutions. Many faculty members have been trained to use online learning platforms as an add-on to on-site teaching. Online working at home may be a difficult task due to the high demand of using computers from the whole family [3]. These rapid transformations have raised questions about faculties’ capability to deal with their existing technology and resources, but they have also provoked surprising new models of educational innovation [3].

However, medical institutions are also under pressures at this uncertain period and there are a lot of challenges to deal with in training healthcare professions with the limitation of social distancing. Some clinical training, simulations, technology sessions and laboratory and be delivered remotely and not all learners have internet access or computers at home [4]. Examinations have also transitioned to online settings. For resident and fellows, cancelling elective and un-urgent surgeries and routine appointments, reducing inpatients admission and shortage of personal protective equipment (PPE) my negatively impact the quality of their clinical practice [5].

In the Saudi Commission for the Health Specialties (SCFHS) the COVID-19 pandemic we delivered recently an integrated research related to the evaluation of Residents’ Training in COVID-19 Pandemic Times with an emphasis on Educational Process, Institutional Support, Anxiety and Depression [6]. The assessment of e-learning delivery in this study can be also exploited in the direction of informing our recently published set of Key Performance Indicators (KPIs) for Sustainable Postgraduate Medical Training: towards the Implementation of an Innovative Approach to Advance the Quality of Training Programs at the Saudi Commission for Health Specialties (SCFHS) [7]. Furthermore the analysis of quality benchamrks and the interpretations of e-learning perceptions of students in residency post graduate medical training programs is a key priority [8] with an emphasis to change management [9] and sophisticated e-learning strategies capable of dealing with the need for personalized learning services [10].

In a recent study [11] Yang et al, provided an integrated research on the preparedness of medical education in China with an emphasis on the lessons learnt from the COVID-19 outbreak, in their work reveled three significant factors that need to be address from an educational point of view: insufficient emphasis on public health emergency preparedness; unsophisticated mechanisms for interdisciplinary cooperation; and inadequate guidance in medical ethics.

Withing the challenging context of COVID-19 pandemic, educational institutions also had to adjust their learning strategies, and to adopt the educational practice to the new reality.

Shehata et al [12] in their recent work they focus on the required educational adjustment in the context of medical training in Egypt. The focus of their analysis and their discussion covered a variety of medical education aspects including among others the University preparedness the role of faculty in the transition as well as the Role of ME units/Departments/National/Regional bodies in the transition, One of the key findings of their work was the supportive opinions of participants for the integration of online learning activities to the future curriculum in post-COVID-19 times.

Along with the emphasis on the preparedness of institutions, many researches focus on aspects related to medical students preparedness for COVID-19 pandemic [13]. O'Byrne et al, analyzed the medical students’ mental health and concluded that there is a critical need for ‘pandemic preparedness’ to be embedded in the medical curriculum. Other studies in the literature also emphasize on residency postgraduate programs [14]. Chong et al, in this context contributed on a thorough analysis of the leadership requirements for radiology residency programs in order to manage effectively the residency-related impact of the pandemic with special emphasis to be paid on education. In the same context the interesting work of Choi et al [15], analyze how the COVID-19 pandemic challenges the final year medical students in the United Kingdom with emphasis on final examinations and placements. Their study including four hundred forty students from 32 UK medical schools proved that 38.4% (n = 169) of respondents had their final OSCEs cancelled while 43.0% (n = 189) had already completed their final OSCEs before restrictions. 43.0% (n = 189) of assistantship placements were postponed while 77.3% (n = 340) had electives cancelled. The main contribution of the study is to reveal the key aspects of the disruptions to students support with a key effect on the students’ preparedness. The study also concludes for the crucial role of the wellbeing of students in their mental health status.

Mukhtar et al [16] contributed to the relevant discussion by analyzing the advantages, limitations and recommendation for e-learning during COVID-19 with a focus on Pakistan. As the main advantages of the transition and the delivery of training online were clarified the comfort and the accessibility. One of the key limitations was proved to be the barriers in maintaining academic integrity. In their study also discuss the key need to reduce the cognitive load of online content and to increase the interactivity.

The key objective of sustainability is also well discussed in the relevant literature. For example, in their study Ashokka et al [17], are promoting the discussion on sustaining medical education during COVID-19 with an emphasis on responses of medical training centers. ,

Background: The Corona Virus Disease-19 (COVID-19) has been declared a pandemic by the World Health Organization (WHO). We state the consolidated and systematic approach for academic medical centres in response to the evolving pandemic outbreaks for sustaining medical education. The key conclusion is that continuous effort should be paid so that pandemics will not interrupt the delivery of medical training or disrupt the academic processes.

The issue of engagement and active learning during COVID-19 has also studied in various researches [18]. In this study the authors provide concrete ideas and leassons learnt related to the enhancement of the students’ engagement in the context of online delivery of education. Almarzooq et al [19] also consider e-learning as a disruptive technology and methodology for the graduate medical education They also make reference to the American College of Cardiology Fellowsin-Training Section Leadership Council that proposed 3 novel educational strategies: personalized learning, adaptive learning with real-world situations and feedback, and the flipped classroom Their final comment is that during pandemic the acceleration of this strategies is challenging the efficiency and the quality of medical training.

Sahi et al [20] also comment on the need during COVID-19 pandemic for continuity of medical education with emphasis on pedagogical innovations involving technology and simulation-based teaching. Arandjelovic et al [21] in a thorough study review the strategies implemented during previous global infectious disease epidemics, and suggest strategies for utilizing clinical education in times of pandemic crisis. Similar discussion and arguments for the support of medical education with e-learning can be found in [22], [23] with emphasis in multimodal multi-Institutional solution to remote medical student education and Digitalization or strategic consideration of key issues like the Strategic Deployment of Cardiology Fellows in Training Using the Accreditation Council for Graduate Medical Education Coronavirus Disease [24].

In Table 1, below, we provided the key aspects of the studies literature and in Table 2, we summarize an overview of the

Table 1: An overview of our critical literature review and key aspects of the research problem

Literature review

Author(s)

Title of article

Impact on our research model

[11]

Yang Da-Ya, Cheng Shu-Yuan, Wang Shu-Zhen, Wang Jin-Song, Kuang Ming, Wang Ting-Huai, Xiao Hai-Peng.

(2020)

Preparedness of medical education in China: Lessons from the COVID-19 outbreak.

Focus on preparedness.

Lessons Learnt

[12]

Shehata Mohamed HK, Abouzeid Enjy, Wasfy Nourhan F, Abdelaziz Adel, Wells Ray L, Ahmed Samar A.

(2020)

Medical education adaptations post COVID-19: an Egyptian reflection.

Adjustments to medical training

Provision of Assessments

[13]

O'Byrne Lorcan, Gavin Blánaid, McNicholas Fiona.

(2020)

Medical students and COVID-19: the need for pandemic preparedness.

Students’ mental health

Integration of pandemic preparedness to Curriculum

[14]

Chong Alice, Kagetsu Nolan J, Yen Andrew, Cooke Erin A.

(2020)

Radiology residency preparedness and response to the COVID-19 pandemic.

Impact of pandemic to residency programs

[15]

Choi Byung, Jegatheeswaran Lavandan, Minocha Amal, Alhilani Michel, Nakhoul Maria, Mutengesa Ernest.

(2020)

The impact of the COVID-19 pandemic on final year medical students in the United Kingdom: a national survey.

Final year medical students examinations and placements.

[16]

Mukhtar Khadijah, Javed Kainat, Arooj Mahwish, Sethi Ahsan.

(2020)

Advantages, Limitations and Recommendations for online learning during COVID-19 pandemic era.

E-learning best practices

E-learning strategies

[17]

Ashokka, Balakrishnan; Ong, Say Yang; Tay, Kwang Hui; Loh, Ne Hooi Will; Gee, Chen Fun;

(2020)

Coordinated responses of academic medical centres to pandemics: sustaining medical education during COVID-19.

Sustaining Education

Sustainability of medical training in times of pandemic

[18]

Zayapragassarazan, Zayabalaradjane

(2020)

COVID-19: Strategies for Engaging Remote Learners in Medical Education.

Engagement educational strategies

Medical Education adjustment in COVID-19 times

[19]

Almarzooq, Zaid I; Lopes, Mathew; Kochar, Ajar;

(2020)

Virtual learning during the COVID-19 pandemic: a disruptive technology in graduate medical education.

Impact of Virtual learning

Disruptive character of e-learning in medical training

[20]

Sahi, Puneet Kaur; Mishra, Devendra; Singh, Tejinder;

(2020)

Medical education amid the COVID-19 pandemic,Indian pediatrics.

Lessons learnt

[21]

Arandjelovic, Andjela; Arandjelovic, Katarina; Dwyer, Karen; Shaw, Cameron

(2020)

COVID-19: considerations for medical education during a pandemic.

Lessons learnt

[22]

Ruthberg JS; Quereshy HA; Ahmadmehrabi S; Trudeau S; Chaudry E; Hair B; Kominsky A; Otteson TD; Bryson PC; Mowry SE.

(2020)

Solution to Remote Medical Student Education for Otolaryngology during Covid-19.

Educational strategies

Best practices

[23]

Alkhowailed, M. S.; Rasheed, Z.; Shariq, A.; Elzainy, A.; El Sadik, A.; [24] Alkhamiss, A.; Alsolai, A. M.; Alduraibi, S. K.; Alduraibi, A.; Alamro, A.; et al.

(2020)

Digitalization Plan in Medical Education during Covid-19 Lockdown.

[24]

Gallagher MJ; Bloomingdale R; Berman AD; Williamson BD; Dixon SR; Safian RD.

(2020)

. Strategic Deployment of Cardiology Fellows in Training Using the Accreditation Council for Graduate Medical Education Coronavirus Disease 2019 Framework.

Table 2: Key focus of selected literature

	Key emphasis
Author	Covid-19	Medical education	Public health emergency	Preparedness	Medical ethics	Online learning
*Yang Da-Ya* *et al[6]*	x	x	x	x	x
*Shehata Mohamed HK,* *et al[5]*	x	x		X New teaching and assessment methods		x
*O'Byrne Lorcan* *et al[2]*	x	x	X volunteering	x	X Moral trauma
*Chong Alice* *et al[3]*	x	X Radiology education		x	X Anxiety and Well-being Issues of Trainees
*Choi Byung* *et al[5]*	x	x		x		x
*Mukhtar Khadijah* *et al[3]*	x	X undergraduate				x
*Ashokka Balakrishnan.* *et al[4]*		X undergraduate	x			X e-learning/computer
*Zayapragassaraza Z.*	x	x			x	x
*Almarzooq Zaid I.* *et al[2]*	x	x		X integration		X Communication Collaboration
*Sahi Puneet Kaur.* *et al[2]*	x	x		X Pedagogical innovations		x
*Arandjelovic Andjela.* *et al[3]*	x	x	X medical students on the frontline	X changes to medical schools during a pandemic
*Ruthberg JS.* *et al[9]*	x	X Otolarynology Acting internship		X multimodal		X Remote education
*Alkhowailed M. S.* *et al*	x	x		X digitalization		X Virtual classroom
*Gallagher MJ* *et al[5]*	x	X Fellows in training	X Health services

In this research study we adopted an integrated research method aiming to promote the following four primary research objectives:

To evaluate the preparedness of the educational institutions for the e-learning platform transition for the delivery of medical training
To evaluate the effectiveness of e-learning educational method among undergraduate and post graduate health professions education
To evaluate the overall satisfaction level of the participants with their e-learning experience
To review methods of assessment in e-learning platform and to understand their suitability and effectiveness.

The research setting was targeted to all training hospitals and universities across in the country. In this research paper we focus on a pilot study among 300 participants.

The inclusion criteria for the participants are related to undergraduate students and postgraduate trainees from all health disciplines (medicine, dentistry, applied health sciences, pharmacy, and nursing) from both gender in all levels of training. The exclusion criteria refer to all resident who are in leave or outside rotation.

The research design is related to an observational cross sectional study design. Participants who fulfill the inclusion criteria invited to participate in this study by filling online self-administered closed ended questionnaire about the effectiveness of online educational method and the preparedness of the institutions for such an activity.

The questionnaire has the following sections:

Section 1: Addressing the demographic data of the participants which include Age, City, Institution, College, Job title and Subspecialty.

Section 2: E- learning experience before COVID-19 pandemic

Section 3: E- learning experience during COVID-19 pandemic

Section 4: Review methods of assessment in e-learning platform

In the Results section we provide a statistical analysis summarized and analyzed using SPSS program. Simple descriptive statistics are reported in form of mean and standard deviation for frequency and percentage for qualitative data.

We also deployed qualitative analysis and sentiment analysis for analyzing the open text contributions of participants related tot heir perceptions for the adoption of e-learning in medical training.

For the Ethical consideration of our study the main rule is that participants’ confidentiality will be kept and won’t be forced to fill the questioner. Name and contact numbers were not requested. All data are secured in the Saudi Commission of Health Specialties’ computer facility. Data in the next section are summarized and presented with honesty without any falsification or fabrication.

Table 3: Training Level of participants

Training Level	Count	Percentage
F1	14	4.7%
F2	7	2.3%
F3	2	0.7%
R1	84	28.0%
R2	51	17.0%
R3	54	18.0%
R4	37	12.3%
R5	13	4.3%
I am a RTP	36	12.0%
Not available	2	0.7%
Total	300	100.0%

In our research a total number of 300 respondents participated, meeting the inclusion criteria presented in the previous section. In total 122 of them (41%) were female and 178 (59%) male. Concerning the training level of participants as indicated in Table 3 above, residents of R1, R2 and R3 level, are representing almost 65% of the respondents.

Table 4: Training City of participants

Training City	Count	Percentage
Abha	15	5.0%
Al-baha	3	1.0%
Al-hasa	21	7.0%
Al-jouf	1	0.3%
Al-khobar	25	8.3%
Al-Madina Al-Monawara	10	3.3%
Al-quaseem	4	1.3%
Dahran	5	1.7%
Dammam	27	9.0%
Jezan	12	4.0%
Jubail	3	1.0%
Makkah Al-Mokarrma	83	27.7%
Najran	4	1.3%
Qatif	9	3.0%
Riyadh	63	21.0%
Tabouk	3	1.0%
Taif	7	2.3%
Yanbu	3	1.0%
Not available	2	0.7%
	300	100.0%

As per the training city in Table 4, there is an overview of the respondent’s location. Makkah Al-Mokarrma and Riyadh is the basis for almost 49% of the participants in our survey.

One of the key objectives of our research is to understand and to interpret the effectiveness and the readiness of the medical training centers of residents to adopt e-learning platforms and online instruction during the COVID-19. In Table 5, below, we provide the key findings. Participants in our survey were asked to rate their attitude and perception in a scale of 1 to 10, where one represented the lowest perception or use and 10 the highest perception or use.

Table 5: Perceptions and attitudes of participants for e-learning experience

Question Number	Rating (1-10)
Q6. Frequency of E-learning at institution before COVID-19	3.55
Q7. How Efficient your institutional uses of e-learning platform before COVID-19	3.65
Q10. On-site education experience during COVID-19	5.50
Q11. Rating off-site education experience during COVID-19	6.05
Q14. Frequency of E-learning at institution during COVID-19	5.71

The residents’ training medical centers were using at a rather low rate e-learning methods before the COVID-19 pandemic. This reality was enhanced significantly during COVID-19 pandemic, and the frequency of e-learning use at institutions was increased at a bold rate, from 3.55 to 5.71, as seen in Table 5, above (increase by approximately 61%).

Another significant finding is that the participants in our survey rate rather low the efficiency of the institutional use of e-learning platform before COVID-19. The average rating of 3.65 out of 10 indicates that most of the participants were not satisfied with the use of the e-learning platform in their institutions before the COVID-19 pandemic.

As per the evaluation of respondents for the on-site and the off-site education experience during COVID-19, the key findings permit interpretation. The off-site education experience during COVID-19 was rated 10% higher than the on-site education experience (6.05/10 compared to 5.50 respectively). We will discuss the limitation of our study in the next section, but a first comment and interpretation of this perception is that trainees prefer the off-site training over the on-site. For sure, this simplistic assumption requires further analysis. We do have to understand which components of the e-learning educational approach are valued by trainees and our respondents.

The main facts related to the transition of instruction during COVID-19 are presented in Table 6, below.

Table 6: Adjustment to e-learning environment during the pandemic

Question Number	YES	NO
Q12. Have you used e-learning platform during COVID-19 era?	89.33%	10.67%
Q13. Since COVID-19 pandemic, do your education activities still conducted as per curriculum?	56.33%	43.67%
Q23. Have you observed any changes in student's/faculty behavior or personality during e-learning experience?	48.33%	51.67%
Q24. Do you think e-learning should be implemented as a mandatory educational tool in future?	77.67%	22.33%

The need of medical training institutions in Saudi Arabia to adopt fast to the new reality of COVID-19 resulted in the extensive use of e-learning platforms. Almost 9 out of 10 respondents acknowledged the use the e-learning platform during the pandemic (YES: 89.33%, NO: 10.67%).

The disruption in educational activities per curriculum was also significant. According to 44% of our respondents, the educational activities stopped being conducted as per curriculum (YES: 56.33%, NO: 43.67%). This fact requires further investigation concerning the real impact of the e-learning mode on the quality of education and the skills and competencies of the residents.

Additionally, it is extremely important to understand the impact of the e-learning delivery on the behavior and personality of students and faculty. The psychological pressure and requirement for the adoption of digital and other soft skills, related to online delivery of education, need to be analyzed further. Our survey participants confirmed at a very high rate (almost 50%) that they observed some changes in students’ or faculty’s behavior during e-learning experience (YES: 48.33%, NO: 51.67%).

Our respondents also assure that e-learning practice should be an inevitable or useful pillar of any future training approach. Despite the difficulties and barriers of the on-site e-learning mode, trainees realized the additional value of the online approach; not as an exclusive mode of medical training but as a supporting, enhancing enabler. Thus, in the relevant question on their opinion if the e-learning should be implemented as a mandatory educational tool in the future, 77.67% of them foresee its future potential (Table 6).

In Table 7 below, we present the results related to the attitude and perceptions of trainees for the value-adding components of the e-learning mode. It is evident that the trainees during COVID-19 confirm a rather moderate engagement in lectures through e-learning platforms. Their evaluation on this factor is rated with 5.93 out of 10. This key finding can initiate educational strategies that can be facilitated by digital means and can promote the active participation of students. Various propositions for active learning and medical training can be found in the relevant literature. Also, it is extremely important to analyze and to provide creative ideas for the simulation of clinical and surgical training and practice in times of pandemics like COVID-19. It is somehow this period a test bed for best practices, lessons learnt and creative policy making for the future.

Table 7: Attitude of participants for the value adding components of the E-learning mode during COVID-19 for medical training

Question Number	Rating (1-10)
Q17. Rating of engagement during Covid-19	5.93
Q18. Rating of overall satisfaction related to the use of e-learning platform during COVID-19	6.33
Q19. Rating of your motivation to start -e-learning during COVID-19	6.54
Q20. Rating of barriers faced during E-learning mode of instruction during COVID-19	4.80

Additionally, as it is indicated in Table 7, the overall satisfaction related to the use of e-learning platforms during COVID-19, equals to 6.33 out of 10. A first interpretation of this key figure is that trainees understand and appreciate the value of the alternative mode of the e-learning platform during COVID-19. Thus, their satisfaction seems to be adequate. A logical association would be to investigate if sacrificing the traditional educational approach due to COVID-19 has an impact on this rate. On the other side, the current rate of satisfaction permits us to verify e-learning mode’s positive educational impact.

A supplementary finding of our research is that trainees claim a significant rate for their motivation (6.54 out of 10) to start e-learning during COVID-19 (Table 7). Without empirical data to support such claims, we assume that a significant part of their satisfaction is related to their willingness to achieve their educational objectives without interruption from COVID-19 pandemic or even due to their motivation to support their mental health with the continuation of the training.

Furthermore, the average measure for barriers faced during online mode of training delivery equals to 4.80 out of 10 (Table 7). This also indicates that the fast transition to online e-learning platforms for medical training, challenged various aspects of the strategy of training. As an average rate, 4.80 indicates that many things could be designed or supported better; on the other hand it explains the overall satisfaction rate previously discussed.

In Table 8, we summarize some key aspects related to the efficiency and the effectiveness of the e-learning platform during COVID-19 for medical training. The institutional use of the platform was rated rather low with an absolute value of 50.17 out of 100. The overall understanding is that trainees faced difficulties while they were trying to accommodate themselves to the new reality for the delivery of the training due to the COVID-19 pandemic.

We plan to use this pilot study for designing a new survey targeted to a greater sample in Saudi Arabia for investigating qualitative aspects of this rather fair efficiency. It seems that there is a lot of space for improvements. In the Discussion section, we also use the constructive comments of respondents towards the launch of a theoretical abstraction for the lessons learnt and best practices for effective educational strategies.

Table 8: Efficiency and Effectiveness of the E-learning platform during COVID-19 for medical training

Question Number	Rating (1-100)
Q15. Efficient institutional use of e-learning platform during COVID-19	50.17
Q21. How educative was the use of e-learning platform for you during COVID-19?	57.58
Q22. How effective do you think you can apply the newly learned knowledge, skills or attitude to your daily practice?	59.20

The evaluation of trainees for the educative use of the e-learning platform is rather satisfactory: 57.58 out of 100 (Table 8). The interpretation of this finding must take into consideration the key qualitative comments of trainees provided in our survey, summarized in the Discussion section.

Table 9: Critical success factors according to the participants’ viewpoint

To your opinion	Disagree and Strongly Disagree	Agree and Strongly Agree
Provision of high quality hardware equipment for e-learning from the medical training institutions	28.3%	36.7%
Provision of required software needed for e-learning mode from the institution	33.6%	39.2%
The e-learning platform incorporates the necessary tools for communication and collaboration	18.9%	54.9%
The institutions secured connectivity to e-learning servces for trainees	31.5%	35.7%
The institution provided antivirus for secured navigation and use of online training	31.5%	33.2%
Cloud services were provided to the trainees by the institutions	33.6%	38.1%
The institution provided immediate assistance to trainees on case of technical matters on e-learning platform.	31.5%	37.4%
The e-learning platform has many options for flexibility and adaptation to trainees needs.	24.5%	37.8%
I feel confident in my ability to use e-learning in education	10.5%	63.6%
I would feel better about using e-learning if I knew more about it	8.4%	66.1%

Next, we will look into particular -mainly technical- parameters, which our trainees deemed as crucial for guaranteeing digital education’s productivity. These parameters are assessed by our introducing “to-your-opinion” statements that participants rated with “agree”, “strongly agree”, “disagree”, “strongly disagree” or “neutral” comments (Table 9 and Figure 1).

First, there are some infrastructure issues under the responsibility of medical institutions. The survey’s participants were generally dissatisfied or neutral (about two thirds of the sample consisted of “disagree”, “total disagree” or “neutral” answers) regarding the institutional provision of hardware and software equipment for e-learning use, which includes connectivity services, antivirus programs or internet apps.

We must note that the high rate of neutral answers (approximately 30-35%) is quite concerning and must be further investigated. Most probably, it does not refer to lack of judgement, but on the contrary reflects residents’ current incomplete satisfaction, disappointment or even lack of trust in institutional management. In any case, it is fairly evident that “technical support” is considered as a fundamental quality factor for e-learning process and thus, we intend to keep that in mind when planning our recommended model in the discussion section.

Another factor that interests us is the e-learning platform’s system fast flexibility, in case of any type of technical problems or future upgrades. About 37% of the residents were fulfilled with the system’s aid to such issues. However, there is still plenty of development potential, also given the fact that online education should be advanced enough to handle medical training in long-lasting situations, like the current pandemic. Finally, a little less than two thirds (agree, strongly agree) of trainees are already confident in their competencies required for online lessons, but at the same time would prefer a more thorough education when it comes to skill-building – so that their training is mostly productive.

In the Discussion section that follows, we will also present an integrated interpretation of the lessons learnt and a synthesis of attitudes and opinions communicated by trainees for their medical training COVID-19 experience.

Critical success results and interpretation

After completing a brief interpretation of our research’s results, we should now target the most essential findings that through discussion will lead us to creating a consistent strategy to strengthen educational effect of e-learning. So, according to our survey, even though during coronavirus crisis online education’s frequency increased, medical institutions were proved to be generally unprepared for fully coordinating the transition process, resulting in:

Being unable to comply with academic curriculum and thus, postponing or skipping over scheduled lectures or training sessions (44%)
Changes in professional behavior and attitude of both trainees -due to their lack of active interaction- and faculty members, due to their unsuccessful adaptation to e-learning methods (50%)

We also noticed that residents’ overall assessment of e-learning experience factors (participation, satisfaction, barriers) was mostly average, however they clearly acknowledged digital education’s future potential. From these findings, we deduce that although institutional practice of e-learning was rapid and fulfilled some basic educational standards, the problem lies within regular establishment and training effectiveness of e-learning in the long term.

Key pillars of online education

In regard of these problematic aspects, in this point we can comment that the basic perceptions of trainees for the e-learning platforms and the online mode of training can be categorized in nine (9) key dimensions and pillars. These can serve as the basis for a meta-model for Best Practices for E-learning for Residents Training in COVID-19 pandemic period.

Technical Support: In this category fall various complementary opinions of trainees. Most common concerns related to the availability of internet connection, hardware devices and applications for sophisticated medical activities both in hospital and university grounds. Further technical considerations refer to the availability and cost of different platforms for e-learning and online communication. There is, in general, a demand for offers for both infrastructure and internet service.

Pedagogy: One of the most significant pedagogical issues for residents’ training during COVID-19 is related to the provision of clear structure of curriculum related topics through e-learning education. This is also associated with the need of residents to have available well defined and clear training and learning objectives. Additionally, the enhancement of motivations and active engagement of trainees to meaningful learning content is valued as significant. Thus, practical themes related to the availability and planning of more Lectures, pedagogical activities and practical sessions are becoming significant priorities. The same stands for the need of the trainees to be provided with more clear and more structured presentations. For a Smart technology point of view there is a request for enhancements to improve punctuality, interaction with students during lectures and online exams platforms.

Institutional Support: The majority of our survey’s questionees were either neutral or disappointed (about two thirds in total), when asked about the e-learning support provided by their medical training centers. More specifically, there is a high demand for commonly available facilities, combined with more detailed and flexible time management of the residents’ weekly activities. Taking into consideration the physicians’ hectic workload, it is quite common that their lecture and working hours may overlap, so under those circumstances they should be excused from their learning sessions or compensate the missed work time on another occasion. Healthcare institutions’ e-learning managers should be able to foresee such developments and adjust the existing time schedule. Another issue to be tackled by training centers is the sufficiency and quality of online education software, including modern teaching tutorials, online study guides and antivirus security programs, with the aim of ensuring trainees’ professional and efficient training. In parallel to better software provision, there is an additional need for institutions to handle the increasing information load, such as by separating junior from senior’s e-learning or establishing platforms in which all educational content can be stored.

E-learning platform: Careful thought and planning must be put into the operational advancement of the selected e-learning platform. Especially in the COVID-19 era, during which the conference hall is principally substituted by the platform, methodical actions should be taken so as that the learning experience is not degrading. Our study’s sample expressed an approximately average satisfaction regarding the use of educational platforms. This fact indicates there is still a lot to be done when it comes to the functionality of the e-learning platform. Above all, we need internet connection stability which can be assessed with numerous webtools and is a must for establishing an uninhibited and time-saving educational process. Residents should, also, be able to attend their training sessions on a daily basis by enrolling easily on the e-learning platform in an automatized manner, if possible. A very promising prospect for the extension of the potential capacities of the e-learning platform would be the establishment of online examination or even medical operations’ simulation -using the existing platform, altogether arranged by Saudi Commission for Health Specialties. Last, even when not highly needed, e-learning platforms can always be used in combination with traditional teaching methods.

E-learning strategy: Next, even if all the essential requirements pertaining to the technical use of e-learning software are met, we must also apply some basic principles of how and under which limitations to use the e-learning platform, thus we ought to structure an accurate and specified “e-learning strategy”. In order to clarify and explain further the composition of such a strategy, we will mention a few examples of which components we deem important. First, the teaching process must always take place only during academic-engaged time, that is the time spent at hospital or university grounds or by extension the time aimed strictly at learning; in other words there should be no intentional involvement in training physicians’ private time at home. Second, it is necessary that we ensure there is a minimum set breaktime period between working hours and lecture hours, which will be used for physical and digital transportation to the e-learning environment, as well as for the residents to have a decent rest. By the same logic, various more standards -including academic recognition of e-learning and importance of physical education- can be established, always to guarantee educational efficacy.

Educational enhancement: Considering that teaching is a form of bidirectional communication, we aim to preserve this particular characteristic when “passing through the screen” in online education. More specifically, as shown in this study, residents would like and basically need to know more about what they can do and how they can interact via e-learning. These requests pose the challenge of interactive e-learning and skill-building activities, which are essential for simulating physical education. Towards this direction, special workshops, webinars, and tutorials can be realized, educating physicians in terms of technical skills, e-learning literacy, mentorship and communication. The contribution of state-of-the-art technology to educational enhancement is also of the essence.

Human Factors: In order to optimize e-learning experience, our research also focuses on dealing with particular human factors, which are rather subjective and vary among trainees and healthcare institutions, however they offer a general concept of the trainees’ needs and how we must proceed to sufficiently satisfy them. Human factors mostly incorporate great demand of direct social interaction and free online courses within the academic duty hours, as well as fairer distribution of workload and time schedule. There has also been notice of institutional deficiency, in relation to providing residents with practical medical knowledge, which is independently obtained most of the times, as reported. Such issues must be tackled, by systematically and directly addressing the trainees’ anxieties and problems, e.g. using e-learning platform’s questionnaires.

Content Enhancement: The goal of online education, as it currently works in COVID-19 pandemic, is to temporarily substitute -in the most efficient way possible- physical education, due to an emergency situation. Nevertheless, future prospects should aim at e-learning quality and viability by fundamentally rearranging and enriching educational content. Main themes of content enhancement are the improvement of teaching material and applications, coupled with running more innovative lessons, based on group discussion, active participation, instructive medical tutorials and above all free access to educational material. The topics of the lectures should also be more precise, to-the-point and relevant to trainees’ needs -particularly to their selected specialty and up-to-date medical data (articles, webinars etc.)- so that the lesson has a practical use and the knowledge provided is well-rounded.

Soft Issues: Similar to human factors, soft issues include experiential concerns of highly subjective nature, that residents express about the academic behavior of their medical trainers or instructors during both lectures and training sessions. Problematics situations, in which trainers exhibit professional deficiency, negligence and indifference towards their duties or even overcharge their subordinates in terms of routine medicals tasks, should be addressed. Online education must also be rid of more severe issues, such as work intimidation and harassment of any type.

The pillars above are more analytically presented in Table 10 below.

Table 10: Key dimensions of online education

Technical Support	Institutional Support
Improve internet connection (on and off site) Ensure free subscription to or offers for internet applications Provide most suitable platforms Improve technical setup of e-learning Provide modern tablets	Ensure free access to medical online facilities for all academic members Ensure meeting the time schedule Provide more e-learning tutorials and applications Store all educational material provided in suitable platforms, for national used (e.g. RCOG) Provide study rooms (e.g. library) that enable e-learning while at the campus or hospital
Pedagogy	Educational enhancement
Strictly comply with academic curriculum and adjust lecture topics Define clear objectives Increase motivation More lectures and activities More practical sessions Smart technology enhancement to improve punctuality, interaction with students during lectures and online exams platforms Divide the sessions into 2 groups: junior and senior	Build skills and competencies for using e-learning platform Special workshops, webinars and training courses for digital literacy Carefully select/plan mentorships Promote interaction via technological upgrades Do feedback survey Pursue Innovation Encourage international communication
E-learning Platform	E-learning Strategy
Provide daily/weekly sessions Aim at standardizing online education, even after COVID-19, considering the platforms’ complementary impact on medical training Facilitate registration Arrange timely subscription to e-learning platforms and offer free use to trainees Encourage SCFHS interventions for provision of online exams and surgical simulation tools	Rearrange time schedule: establish minimum break times between lectures and training hours Do not violate personal (off work) time with postponed lessons Encourage e-learning practice if needed for the academic activity Do not underestimate and underachieve in person teaching approach (Integrate with e-learning) Promote active engagement in educational process Accredit e-learning courses or degrees
Human Factors	Soft Issues
Ensure residents’ effective clinical training in relevance to selected subspecialty Do not overload trainees with unreasonable medical duties Ensure that training practice does not intervene with medical education and skill building Respect trainees’ employment rights Increase social interaction between colleagues Address work pressure issues In any case, provide free courses Additionally, complement in person training with e-learning methods if preferred by trainees	Instructors: focus on training students with emphasis on motivation, mentorship, acknowledgment, and experiential education Exhibit professional conduct Radically address soft issues and adjust academic behavior Eradicate pathological phenomena in workplace: discrimination, psychological pressure, abuse, harassment, and any type of misconduct
Content enhancement
Keep the lectures to-the-point Focus on group discussion, and interactive learning Provide medical tutorials Pose practical questions Prioritize lecture topics, according to selected specialties Improve the materials & the applications used for comfortable, efficient & flexible e-learning Point out more case studies Purchase more question banks Seek feedback from trainees Exploit SCFHS learning material Hold more webinars Experiment with new applications Invest in making the content interesting and archived into free access library

Our contribution: E-learning Quality enhancement model & Maturity Model

Based on the analytical categorization of the nine key pillars, mentioned above, in this section we will proceed with introducing a new, multi-functional, strategic model for “Quality enhancement of e-learning process in medical institutions during COVID-19 pandemic”. This model, displayed at Figure 2, consists of six priority levels, the necessity of each decreases from bottom to top. Level 1 contains technical support; level 2 includes e-learning platform along with e-learning strategy; level 3 contains content enhancement and so on. For levels that include more than one parameter, each one has equal value to the other.

The above model is also one of our research team’s main contributions in context of restructuring online education during COVID-19 pandemic and is realized by analyzing our research goal into key dimensions and then classifying those into defined levels. As for its value, the recommended strategy can be utilized in various ways. First of all, it can be implemented by medical institutions and medical training centers as an operational framework for improving the existing e-learning methods and maximizing effectiveness not only of typical teaching with lectures but also of healthcare simulation courses.

Towards this purpose, the model can, also, serve as a foundation for complementary restructuring actions, aimed at preserving the accomplished enhancement of e-learning process. It would be very beneficial that medical directors or instructors in charge of e-learning arrange a regular assessment of the model’s key parameters through open dialogue or indirect interaction with the trainees – using for example the existing polling tools of e-learning platforms or even questionnaires that maintain residents’ anonymity. As a result, any complaints or unfulfilled goals will be reported and if possible addressed, so that there is a tactical review of the model.

Another critical effort would be the establishment of online education as a mandatory tool not only during the COVID-19 era but also in the long-term, as our findings also confirmed. Undoubtedly, physical education is irreplaceable regarding its educational value, still e-learning can be: first, necessary in emergency situations such as the current pandemic; and second, plenty helpful as a supplemental teaching method under normal circumstances.

However, redeveloping online learning structure and strategy cannot possibly be an immediate change, because of a lack of infrastructure, resources, human force and probably time availability. Still, we acknowledge the frequent necessity of instant e-learning use, as it is at present due to coronavirus crisis. Thus, we need an additional theoretical model, which accompanies the one introduced at Figure 2. To be more specific, in the previous model we examined e-learning enhancement, according to defined levels (one to six), which in fact are comprised of one or two specific key pillars (technical support, institutional support, etc.). In this point, we need to look into the development of e-learning setting priority measures in each “level”, thus defining “stages”.

In other words, the improvement of online education in medical institutions can be studied as a temporally-evolving developmental process and be divided into “stages”. Since there is a need for fast development, any upgrading efforts should start with “stage 1” for example -so that there is a prompt result- and, if finished, move on to the next, less crucial stages. Based on this reasoning, we suggest a “Maturity Model for E-learning Effectiveness in Medical training” (Figures 3), capable of optimizing e-learning impact and value and offering practical guidelines on how to proceed to each stage.

In Figure 3, we can see our Maturity Model’s basic components:

We define three (3) “Maturity stages”, all of which require fulfilling specific measures of every key pillar (levels one to six).
When the model is applied, priority is given to the earlier stage and lowest level, that is stage 1, level 1. Moving on to the next stages, theoretical effectiveness of online education for medical trainees increases.

In order to demonstrate our strategy’s concept in a more comprehensive way, we provide two indicative models (Figure 4 and 5) which refer to specific measures for the development of e-learning platform and strategy (L2) and content enhancement (L3) accordingly.

To sum up, we firmly believe that combining Quality enhancement model with Maturity Model can serve as a management tool and therefore, turn residents’ experience of online education into optimal medical practices.

In conclusion, this research investigated into the effectiveness of e-learning approach, used by numerous medical institutions in Saudi Arabia, in relation to all types of emerging obstacles in the medical training and teaching process: limited institutional readiness, technical difficulties, behavioral and psychological concerns, soft issues, lack of e-learning content and tools and —above all— academic efficacy. Our resolution strategy was based on accurate definition of e-learning factors —according to residents’ viewpoint, and further analysis on nine key dimensions. After processing the analysis’s results, we constructed two practical models, which associated online education’s effectiveness firstly, with priority levels (Quality enhancement Model) and secondly, with time stages (Maturity Model). Main purpose of both is time and cost-effective implementation of e-learning in medical institutions of higher education.

However, we clearly acknowledge this study’s limitations. First of all, it is evident that any conclusions made were based on the sample of 300 survey participants. Thus, results regarding e-learning support, engagement and overall satisfaction are subject to modulations among training centers and even individual trainees. Furthermore, we recognize that the suggested models have not directly considered financial parameters that restrict transition from one step to another. Still, we provide practical guidance on e-learning priorities, that may facilitate institutions’ cost management. Other limitations of this study are the issues of online data protection against piracy or illegal recording of provided lectures, most of which should be addressed in cooperation with governmental legislature.

Despite the limitations, the research’s strategy could be a foundation for converting online education into a more consistent, independent, and long-lasting learning approach, capable of fully supporting in person education, if needed. This would also be one of our main future concerns, along with expanding our research to a larger population.

SCFHS: Saudi Commission for Health Specialties

COVID-19: Coronavirus disease 2019

All manuscripts must contain the following sections under the heading 'Declarations':

Ethics approval and consent to participate: Ethical approval obtained from the Etthics Committee of the Saudi Commission for Health Specialties and complied with code of ethics for relevant studies. All participants provided informed consent'.

Consent for publication: Not applicable

Availability of data and materials: All data are available in the Research and Development Department (RDD) of the Saudi Commission for Health Specialties. For any inquiry about the data you can contact Ms Maha Abuzenada at the Saudi Commission for Health Specialties, email: [email protected]

Competing interests: The authors declare that they have no competing interests.

Funding: Not applicable

Authors' contributions: Abdulrahman Housawi (AH) contributed to the conceptualization of the survey, the research methodology specification, the analysis of results and authoring of first draft. Basim Alsaywid (BA) contributed to the conceptualization of the survey, the research methodology specification, the analysis of results and authoring of first draft. Miltiadis D. Lytras (MDL) contributed in the, the analysis of results, the authoring of first draft, and the interpretation of the results and the specification of the research contribution. Maha Abuzenada (MA) ) contributed to the conceptualization of the survey, the research methodology specification, the data collection and editing of the first draft. Hara Lytra (HL) contributed in the, the analysis of results, the authoring of first draft, and the interpretation of the results and the specification of the research contribution. Wesam Abuznadah (WA) contributed to the conceptualization of the survey, the research methodology specification, and the analysis of results. Sami A. Alhaidar (SA). contributed to the conceptualization of the survey, the research methodology specification, and the analysis of results. Areti Apostolaki (AA) contributed to the literature review and editing of first draft. Lama Sultan (LS) contributed to the literature review and editing of first draft. Hala Badawoud, (HB) contributed to the literature review and editing of first draft.Acknowledgements: We would like to thank all the participants in our survey and our colleagues in the SCFHS and the RDD.

Authors' information (optional)

Basim Alsaywid