Can Web Based Learning be the New Normal for the National Radiology Didactics in Singapore during COVID-19?

Objectives: The objective of this study aims to quantify and share the experiences of the residents and tutors during the transition to web-based learning during this time of COVID-19. Normally radiology residents would visit the different public hospitals to attend the different didactic teachings. However this is no longer possible and web-based learning has been implemented instead. Methods: Online anonymized surveys were given to the residents and tutors to gather feedback regarding the web-based learning exercise to help quantify the effectiveness of the sessions and to gather suggestions for improvement going forward. Results: The feedback from the web-based learning experience was generally positive from both the tutors and the residents and quality of learning was not compromised. Issues faced gradually improved as participants get more accustomed to web-based learning. The quality of learning also increased as tutors started to get more familiar and integrated their teaching material with the tools available in the software. Conclusions: As such, it is strongly recommended that the use of web-based learning should be considered the new norm going forward, even as the end of social distancing measures locally and worldwide remain uncertain.


Introduction
During this time of COVID-19, many unprecedented measures have been implemented to reduce the potential spread of the virus.Some such measures include social distancing 1 and the creation of new hybrid cohort teams in Radiology 2 .Singapore's Ministry of Health (MOH) has also implemented measures to prevent potential spread between healthcare workers by limiting the number of public hospitals they can work in 3 .In normal times, the residents across the island, from the 3 sponsoring institutions (SI) would take turns attending lectures at the different public hospitals and have face-to-face teaching with the faculty as part of a coordinated national cross SI programme.Understandably, this is no longer possible due to the risk of virus transmission.A rapid midstream switch to conduct the remaining lectures via web-based learning was implemented to maintain resident learning 4 .
This study aims to quantify and share the experiences of the residents and the tutors during this transition to web-based learning.As it appears that COVID-19 may persist in the community for some time and social distancing measures may need to come into effect intermittently or for extended periods 5 , this study will help inform us if web-based learning is a viable alternative for the long-term future.

Materials & Methods
The web-based exercise was conducted over a 3-month period from February 2020 to May 2020.Cisco Webex and Zoom were the platforms of choice for web-based learning.All sessions were conducted using Cisco Webex save for the nal one where Zoom was used instead due to data compliance reasons under Singapore's Personal Data Protection Act (PDPA).We created an online anonymized questionnaire of approximately 10 questions which had a combination of multiple choice and open-ended responses.The nal survey also included further questions such as asking residents for their choice of learning medium moving forward.These were given to the 1 st year residents (R1), 4 th year residents (R4) and the respective tutors.For the residents, the surveys were given at the commencement of web-based learning, once every month subsequently and once at the very end of the web-learning exercise.For the tutors, surveys were given for same sessions that they conducted.The surveys were spaced out to avoid survey fatigue 6 and to give enough time to track any longitudinal change in their responses.

Results
The feedback from the web-based learning experience was generally positive from both the tutors and the residents.
Portable devices were favored with the mobile phone and laptop as the most commonly used devices.
Most residents found that web-based learning was effective compared to the usual face-to-face didactic sessions.At least 75% of the residents chose "all of the time" and "most of the time" with the remaining quarter choosing "some of the time" and even less respondents picking "very occasionally" or "none of the time".[Figure 1a][Figure 1b] The tutors had a similar positive impression of the web-based learning experience.
The biggest advantages listed were that there was no need to travel to other locations, ease of access in joining the online teaching material and respondents could still attend sessions even when they were on sick leave or off-duty.Close to 100% of respondents picked "no need to travel to other locations" with about 70% of respondents picking "ease of access in joining online teaching material" and "can even attend when off-duty or sick".Of note was that one of the tutors was on "leave of absence" due to compliance with COVID-19 isolation protocol.In previous circumstances, the lecture would normally have been postponed or canceled but was able to proceed via web-based learning.[Figure 2] The biggest disadvantages faced were mostly related to technical issues, with about 20-30% of respondents having di culty in answering in-class questions and 23-25% nding no face-to-face contact as a disadvantage.However these disadvantages did not have a major impact on learning quality.[Figure 3] The technical issues faced were wide-ranging such as web browser compatibility and user interface, with the most common being audio.The frequency of these technical issues reduced subsequently with increased familiarity with the software and web-based learning experience.For audio, 35% of the R1s and 39% of the R4s had issues which dropped to 19% and 23% subsequently.
Overall the results showed great support for web-based learning.Furthermore in the nal survey where residents were asked for their nal preference moving forward, 92% of the respondents replied that they would prefer web-based learning instead of the previous face-to-face national didactics system.[Table 1] Yes we should have this most of the time unless person-to-person interaction is de nitely required

38%
No we should only have this when "social distancing" measures are needed 8% No we should go back to didactic lectures as soon as possible 0%

Discussion
Moving away from the usual method of face-to-face learning might be di cult for some of the lecturers who have had little experience with web-based learning 7 .It might also be challenging for some residents to adapt to a new didactic structure.However the results have overall been positive.
Before the implementation of web-based learning, Radiology residents would be distributed over the 3 healthcare clusters -National Healthcare Group (NHG), National University Health Services (NUHS) and SingHealth Services (SHS) -covering the whole of Singapore.Each cluster would have a few hospitals under their care and Radiology residents would be posted to each of these hospitals.At a single given time, it is conceivable that the national residents might spread over 12 different hospitals or tertiary care institutions.In the past, there would be issues with scheduling and commute time required for residents to travel from their workplace to the lecture venue.

Devices Used
The results showed that portable devices were the favored with the mobile phone and laptop being the most commonly used devices.As some residents had chosen to remain on-site in their respective healthcare institution for the web-based learning, at times wireless connectivity may not be available.In such circumstances, residents would use their mobile phones instead of a laptop.
There are a few reasons for the lack of wireless connectivity in public institutions.Firstly, even though "Wireless@SG" was a community initiative launched in December 2006 by the Infocomm Media Development Authority (IMDA) to bring high-speed wireless broadband to all Singapore citizens in public areas 8 , there are still some locations where connectivity and bandwidth is insu cient for video.
Furthermore even if residents could nd an area where the connection is better such as the public seating areas, the area might be non-conducive for web-learning.Secondly, all workstations in the healthcare institutions used to have internet connectivity.However after a data breach incident leaking personal and medical information of about 160,000 patients, the decision was made for internet separation from all public healthcare IT terminals.Subsequently only a few selected terminals with no access to patient record information would be allowed to have internet access 9 .
As some residents might be using their mobile phones, tutors might need to consider altering their web learning materials to have larger text for display or distributing it to residents beforehand so that they can still have o ine access.

Security
This brings us to the issue of security for web-based learning.Initial sessions were held using Cisco Webex but were eventually switched to Zoom due to data privacy reasons.Singapore introduced the Personal Data Protection Act (PDPA) in 2012 to govern the collection, use and disclosure of personal information to organizations 10,11 .As such, care needs to be taken to not reveal any identifying information when sharing clinical cases and preventing unauthorized individuals from gaining access to the same information.There have also been cases in Singapore where "Zoombombing" has occurred, which is the practice of uninvited users disrupting Zoom meetings 12 .This subscription to Zoom is bought via Singtel, one of the broadband providers in Singapore, which helps ensure PDPA compliance.Other measures have been taken such as having a "waiting area" before allowing access into the meeting which is controlled by the administrative assistant, the removal of all identifying features in any teaching material that is distributed and meeting codes only distributed via o cial emails.

Teaching
Overall the response to the quality of teaching was positive.It was effective "almost all the time" when compared to regular face-to-face teachings.Furthermore in the nal survey given, most of the residents felt that this should be the "new normal" moving forward with teachings.The second most common response was that this should be the "new normal" with exceptions for topics where face-to-face interactions are essential such as viva sessions.

Teaching -Disadvantages
The biggest issues faced were related to technical issues which will be discussed later.Besides technical issues, the next most common issue raised was the lack of interactivity.Some of the reasons raised included the time taken for residents to respond either via the chat function or turning on their microphone to reply.Residents would also occasionally use the chat function to ask questions, which would be sometimes missed by the tutors as their chat window might be obscured during screen sharing.One possible solution is to establish rules of interaction for answering questions posed.One of the tutors used the "raise the hand" virtual function to check who had questions or responses to a question she had raised.Alternatively they could have "checkpoints" in the presentation where questions can be elded or the built-in poll function.Another method to improve interactivity and to simulate a near-physical viva-like experience could be to use the built-in Zoom ability to transfer remote cursor control to other participants.All these issues will likely improve with time and familiarity with the software.This will eventually help integrate the use of these functionalities and as tutors adapt their presentations to a web-based learning environment.
Another related issue was that there was no opportunity for the residents to ask the tutors any questions individually after the session is over.This can be overcome by requesting tutors stay in the video session for a short while after the o cial end, allowing people who want to ask questions to remain.Alternatively residents could contact the tutor privately via email.
Other issues that a few residents faced were such as the inability to nd a conducive environment which is also related to the occasional lack of wireless connectivity in certain sites.

Teaching -Advantages
One major advantage was that residents did not need to travel to other locations.As the residents are possibly posted up to 12 different locations, this is clearly advantageous in reducing commute time and to avoiding missing part of the teaching.Another signi cant advantage was that residents could still attend sessions even when they were on sick leave or off-duty, such as being post overnight call.As mentioned above, one of the tutors was able to conduct their scheduled session despite being on "leave of absence" due to the COVID-19 situation.The lecture would otherwise have been postponed or cancelled if not for the availability of web-based learning.
One of the unexpected advantages was that the use of web-based learning has allowed participants to zoom into the images displayed.This is greatly advantageous as Radiology national didactics are more image-intensive compared to other residency programs.Typically in large group face-to-face sessions residents would likely have to interrupt the session to ask the tutor to zoom in to region of interests.
An interesting point to note is that the responses obtained for the nal session, which was conducted using Zoom instead of Cisco Webex, was more positive.Respondents said that Zoom was the preferred choice.As Zoom has recently been used more often than Cisco Webex for departmental meetings and clinician rounds, it is unclear whether the increased positive response is due to reduced technical issues from a preference of Zoom's user interface or due to familiarity with web-learning in general after repeated sessions.
Technical most technical issues faced were related to audio, other issues were faced such as di culty in navigating the user interface and video.The main issue with audio was feedback loops occurring causing echoes due to the unmuted microphones.This has been largely overcome by reminding participants to mute their microphones upon entry of the room and auto-muting all the other participants besides the lecturer.Furthermore as web-based mediums are being used more frequently for other matters such as combined rounds with clinicians and department meetings, familiarity has increased which has helped reduce the number of technical issues over time.Residents and tutors have also been teaching each other on how to navigate the software's interface.
Notably there was one resident who appeared to have particular di culty with technical issues during one of the web-learning sessions.More useful information may be gained by asking the resident if he or she would be willing to volunteer more feedback.Some suggestions raised include having a technical person on site with the tutor to help directly troubleshoot any issues.However, this is impractical as the tutors are from many different hospitals and occasionally some from private institutions.It would be di cult to have a person to travel down to where the tutor is located to provide technical support.This is especially during the COVID-19 situation, where physical interaction of staff between different hospitals is highly discouraged.
One possibility could be to have a short illustrated guide on how to troubleshoot the most common issues faced or having one of the administrative assistants available to answer phone calls during the session for any technical issues.Another possibility would be to have a short trial run with the tutor before the actual session to iron out any technical issues that might occur.

Limitations
One of the limitations was that there was no comparison with the previous year's residents who had underwent the same lectures, but in an in-person setting.
Another limitation was that this study was only conducted over a period of a few months in the middle of the residency year.This study would been more ideally conducted over the course of a full residency year.However this was not possible due to the sudden onset of COVID-19 and related restrictions.As such a fast switch to web-based learning was required in order to minimise disruption to residency teachings.

Conclusion
In conclusion, the study has shown that web-based learning is positively received by residents and tutors, and quality of learning is not compromised.Furthermore with social distancing measures which may be in effect for an indeterminate period of time, it is recommended that web-based learning should be the way forward for the majority of lectures.
The biggest advantages listed were eliminating the need to travel to other locations, ease of accessing the online teaching material and the ability to attend sessions even when on sick leave or off-duty.
All data generated or analysed during this study are included in this published article and in further detail in its supplementary information les.

Figures
Figures

Table 1 :
Now that you have experienced web learning, what should we do moving forward?