Virtual adaptation of healthcare quality improvement training programmes in response to COVID-19: A rapid narrative review

Information and communication technology is playing a major role in ensuring continuity of healthcare services during the COVID-19 pandemic. The pandemic has also disrupted healthcare quality improvement (QI) training and education for healthcare professionals and there is a need to rethink the way QI training and education is delivered. The purpose of this rapid evidence review is to quickly but comprehensively collate studies to identify what works and what does not in delivering QI training and education using distance-learning modalities. Three healthcare databases were searched along with grey literature sources for studies published between 2015 and 2020. Studies with QI training programmes or courses targeting healthcare professionals and students with at least one component of the programme being delivered online were included. A total of nineteen studies were included in the review. Most studies had a mixed methods design and used blended learning methods, combining online and in-person delivery modes. Most of the included studies reported achieving desired outcomes, including improved QI knowledge, skills and attitudes of participants and improved clinical outcomes for patients. Some benets of online QI training delivery include fewer required resources, reducing the need for on-site instructors, increased programme reach, and more control and exibility over learning time for participants. Some limitations of online delivery modes include limited learning and networking opportunities, functional and technical problems and long lead time for content adaptation and customization. The review highlights that distance learning approaches to QI help in overcoming barriers to traditional QI training. Virtual training environment holds great potential in delivering standardised training and distance collaborations which will be of utmost importance in the post-COVID environment. The review offers guidance on important questions to be considered in delivering QI programmes through distance learning modes.


Background
The COVID-19 pandemic is rapidly transforming the landscape of the healthcare system and virtual healthcare solutions are playing a key role in this change [1]. It has also presented unique challenges in the healthcare quality improvement (QI) sphere and highlighted the need for a dynamic approach that enables QI structures and policies to adapt to the pandemic environment [2]. QI principles offer useful strategies for implementing and sustaining meaningful change [3] and staff trained in QI principles can play a critical role in responding to these emerging challenges by accelerating the pace of learning [4]. QI is increasingly being recognised as an important skill for healthcare professionals [5] and is an important component of medical education and training [6]. The immediate focus of the COVID-19 response of the healthcare sector has focused on ensuring continuity of care for patients and communities. However, the pandemic has also had a profound impact on medical education and how healthcare professionals will be educated in future [7]. As researchers and practitioners rush to explore ways to support healthcare professionals during the pandemic, it is also important to rethink the way QI training and education is delivered to healthcare professionals.
Rather than viewing COVID-19 as a disruption to healthcare QI education and training, it can be considered as an opportunity to improve distance learning techniques and bene t from digital hyper-connectivity to enhance education delivery that can extend into the post-pandemic environment [8]. Application of technology-enhanced learning is often cited as a pedagogical advancement for a curricular transformation to prepare healthcare professionals to adapt to the complexity of the healthcare system [9]. However, the pandemic has made it an inescapable necessity for the healthcare system to adapt to virtual ways of working.
Online learning platforms have the potential to bring healthcare professionals together to share knowledge and collaborate in QI teaching, learning and education [10]. Beyond the pandemic situation, the nature of knowledge is continuously evolving in healthcare and self-directed and e-learning could lead to better knowledge retention as compared to passive knowledge absorption through didactic lectures [11]. There is much to be understood about the usefulness of distance-learning modalities in effectively delivering QI training. The purpose of this rapid narrative review is to collate studies to identify what works and what does not in delivering QI training and education using distance-learning modalities. A rapid evidence assessment summarises research ndings in a systematic manner, within time and resource constraints and is suited to the current situation [12].
The review aims to answer the following questions: What distance learning modalities are being used to train healthcare staff and students in QI methods?
What is the e cacy of distance learning QI programmes?
What were the advantages and limitations in delivering QI programmes using distance-learning modalities?
By answering these questions, we aim to synthesise guidelines and recommendations for those who are dealing with the challenge of adapting and delivering QI training to healthcare professionals though distance learning modes.

Methods
Three databases (PubMed, Web of Science and Scopus) were searched to identify studies published between 2015 and 2020. The inclusion criteria were QI training programmes or courses targeting healthcare professionals and students with at least one component of the programme being delivered online. Only studies with primary data were included. Conference proceedings, editorials, protocols, and book sections were excluded. Studies that did not explicitly teach QI principles in the programme were excluded. Papers with no full text available or no English translation available were also excluded. Reference lists of included papers and grey literature search of Google Scholar was also conducted to identify further papers. The complete search strategies and screening process are documented in Figure 1 (Prisma Diagram) and search strategy is presented in Supplemental File 1. The critical appraisal of the studies was conducted using the Quality Improvement Minimum Quality Criteria Set (QI-MQCS) -Version 1.0 tool which possess acceptable psychometric properties for critical appraisal to support systematic reviews containing diverse quality improvement intervention(QII) evaluations [13].

Results
The summary of the nineteen studies included in this evidence synthesis is presented in Table 1. Although, the overarching aim of the included studies was to improve QI skills of healthcare professionals and students, the studies differed in design, evaluation, and analytical methods used. Most studies had a mixed methods design and 18 of the 19 studies were based in the United States. Design of the interventions was also variable; most studies used a blended learning method combining online and in-person modes while only six studies [17,22,27,30,32,33] were entirely delivered online. Some blended learning modules conducted classroom-based sessions followed by support through online modules and QI project completion, while others used a ipped curriculum approach where participants completed online modules prior to the in person sessions such as seminars, workshops, lectures and QI project completion.

Quality Assessment
All included studies were deemed to be of good quality even though some studies did not report on all areas evaluated by the QI-MQCS tool. All studies discussed the rationale behind the intervention, organizational motivation, description of the intervention and implementation approach. Some studies included limited information about describing sustainability or the potential for sustainability of the interventions and explicitly naming the study design. None of the studies were excluded based on quality assessment and a detailed quality assessment is attached in Supplemental File 2.

Distance Learning Modes
The online delivery modes used by studies included online modules [14-16, 20, 23, 24, 28-31, 34], access to web-based curricula [17], virtual learning environments [18], webinars [19,30,32], calls [19,32], web-based QI portals [22], smartphone apps [27], emails [27,33], access to package of tools [30], virtual whiteboard [34] and video lessons [25]. Instead of developing their own distance learning content, most studies relied on the completion of the Institute of Healthcare Improvement's (IHI) online modules,[15, 16, 20, 23, 31, 32] many of which are free to use. The rationale behind using IHI's modules is that it provides a standardised methodology which does not require prior faculty pro ciency or entail an increase in educational time commitment [20]. The IHI methodology is designed to help organizations in identifying and closing gaps via a standard improvement methodology [32]. Another advantage is that an institutional subscription to the IHI programme provides access to comprehensive QI training and allows tracking the progress of participants [24]. Some studies adapted IHI modules [29,32,35] to their local context while a minority used self-developed content [17-19, 22, 27, 28, 30]. The major online modalities used are summarised in Table 2. Interactive online delivery Didactic lectures delivered live online allowing participants to participate in real time [17,20] Video lectures Pre-recorded didactic lectures made available to participants [25] Phone/app/email-based methods QI questions sent out to participants through text messages, phone apps or email [27,33] Online modules to supplement classroom delivery IHI QI modules [14,15,24,31] Self-developed QI modules [19,29] Legend: Summary of major online modes used by studies in delivering QI training and education Only a few studies discussed the tools/software used to deliver the online QI training components. One study used Adobe Connect and blackboard for delivering a web-based QI curriculum [17] while for a digital storytelling pedagogy, researchers recommended participants to use freely available software such as Windows Moviemaker or Apple i-movie [18]. Another QI collaborative used iLab which is a secure, online workspace [19] while a study that developed a webbased QI portal used WordPress CMS platform, social media account integration and a network management site called Hootsuite [22]. A microlearning app called Qstream was used in another training programme [27].
How participants implemented QI skills and knowledge also constituted a part of outcome evaluation in various studies. This included development of action and improvement plans [18], number of QI projects developed [20] and completed [28] and results attained from these projects [19]. Improved results for the patients were also used as a proxy for outcome evaluation [14, 15, 17-19, 27, 30, 32].
Most of the included studies reported achieving desired outcomes such as increased QI knowledge & skills [14,24,25,27,28,34,35], positive reaction from participants towards the training [17,19,20,32], implementation of QI knowledge by participants [18,29] and con dence to use the learned skills in future [22,28]. One study reported no improvement in the measure being tracked [32]. In the ve studies with control groups, the intervention participants demonstrated improved QI knowledge [16], improved comfort with QI methods [31] and greater participation than the control groups [33]. One study did not demonstrate any signi cant difference in post-intervention safety indicators between intervention and comparison group however it was a positive outcome showing that nursing home residents' safety was not compromised during the QI programme [30].
This shows that majority of the interventions were successful in demonstrating the desired results. However, there was scant information around the role played by mode of delivery in the attainment of these outcomes. One study using a control group concluded that use of online content in conjunction with inperson sessions as being more effective in improving QI knowledge than only relying on online content [23]. The ipped curriculum [16] and utilization of webbased platforms to deliver advanced QI training [17] proved to be effective methods for teaching QI. Since there is a shortage of comparable prior studies on web-based tools for QI education, it is challenging to compare results across similar interventions and more longitudinal studies may be required to analyse outcome trends over time [22].

Bene ts of Online QI Education
An online QI programme can virtually connect users and provide them with an environment that balances training and practice [22]. Online delivery of QI training programmes requires fewer resources [19], reduces the burden on sites and instructors [17] and the organizations do not need to maintain QI faculty [17]. It is useful in delivering a centralised QI curriculum to distributed learners [17] hence increasing the reach of the programme [19,27]. Additionally, online learning seemed to balance the educational time constraints and clinical responsibilities in educating healthcare professionals [16]. Moreover, the IHI teaching modules used in this study are widely recognized and accessible to all programmes and learners [16]. Interactive, distance learning models which occur in real time with multi-way communication and feedback and tailored education proved to be effective [17]. Online modules which are interactive were preferred by participants over static computer-based modules [16].
Participants in online QI programmes have more control over their learning time [27], allowing them to complete much of the curriculum at a time convenient to them [31]. Programmes that are interactive and real-time in nature lead to better and personalised engagement from participants [17]. In the case of a ipped classroom where participants complete online modules prior to in-person sessions, the online component enables maximisation of in-class time [34]. However, many participants expressed that in-class sessions and in-class application were more effective than online content in enhancing their QI knowledge [16]. Similarly, viewing didactic material on videos beforehand enables participants to use classroom time to clarify concepts [25] and reduces the overall time required for the curriculum [20]. On the other hand, quiz based online courses provide real-time feedback, engagement and healthy competition but are more suited to reinforce concepts taught in classrooms and supplement other QI activities rather than as standalone activities [27].
Another advantage of online programmes using innovative methods such as digital story telling is that it engages the participants in learning a new skill and creates a level playing eld in terms of the anxiety associated with a new experience [18]. Virtual discussion boards also have the advantage of providing a safe space where participants can freely express their opinions and ideas which they might not feel comfortable doing face-to-face [18]. Similarly, using game mechanics and team-based competition in a safe virtual environment is an effective participant engagement strategy [33].
Participants were generally positive about features of online programmes such as open discussion forums, closed groups, private messaging, and feedback submission forms [22]. Using tools such as leader boards motivates participants to engage and provides a sense of status [33]. Group size between two to eight participants worked best [34]. Virtual formats also allow for easier modi cations in the curriculum length, content, and level which are important considerations in training design [17].
Open communication, stakeholder buy-in, and continuous feedback were necessary in developing a shared vision and QI site ownership [22]. Educational content developed by faculty with practical and teaching QI experience strengthens the programme [27]. The early involvement of key stakeholders and SMART (speci c, measurable, attainable, relevant, and time-bound) goals proved to be critical to the success [29]. QI coaches also play an important role in distance learning programmes as well and a study concluded that having tailored coaching support for each team was a useful aspect of the programme [14]. In the case of students, providing an opportunity for experiential learning through QI project completion alongside a faculty member was also important [31].

Limitations of Online Methods
Evidence suggests that mobile and asynchronous educational technologies have the potential to overcome barriers related to teaching QI methods [33].
However, studies have also identi ed some limitations to such approaches. Participants often valued the learning application sessions conducted in person, more than the online components [16]. In the same way, although learners enjoyed asynchronous learning and online delivery, they preferred assessment questions that focused on application of concepts rather than information acquisition [27]. Online modes offer limited learning and networking opportunities [19].
Customizing didactic materials to suit programme and participant needs is a time intensive task [28]. Developing an online site can be resource-intensive and lead to functional problems [19]. Since data and reporting systems are external and independent form the QI education sites, it is di cult to integrate these as site resources [22]. Online programmes also require facilitators who have QI knowledge [32] as well as technical support in case participants face any technical challenges [20] such as phone line chatter as reported in a study [32]. Additionally, the adoption and use of a new technology requires signi cant run-time [22].
Even though devices such as mobile phones and computers are already used regularly by the learners [27], there are limitations to their usage such as content and character restrictions in text messages and email fatigue for participants [27]. Although the IHI Open School modules are widely used and effective, one study recommended augmenting the content to suit local needs [23] and online content such as videos should be at an appropriate level and pace suitable to the participants [25].
With blended learning programmes, a major challenge was group session scheduling so that participants could attend without disruption of clinical responsibilities [28]. Some participating teams raised concerns speci c to collecting and reporting the measurement data and perceived the measures to be complex and not well-matched to the teams' goals [19]. Apart from the challenges associated with online delivery, teams also experience other challenges such as demand of other work duties and inability to meet as a team during implementation [14].

Discussion
The training modalities discussed in the included studies can be broadly categorised into e-learning programmes and blended learning programmes. A previous systematic review comparing online with face-to-face education for healthcare professionals concluded that online programmes had comparable knowledge gains and bene ts to onsite or face-to-face training [36]. However, there is a gap in literature evaluating the success of QI training delivery online. The purpose of this evidence review is to focus on e cacy of QI training and education interventions being delivered online to synthesise recommendations for adapting QI training content into e-learning materials The review highlights that distance learning approaches to QI help in overcoming barriers to traditional QI training such as shortage of trained faculty and de ciencies in organizational structure to support QI education (24) and in many participants preferred blended approaches to traditional approaches (32). This review highlights important lessons for future programmes including balancing virtual and non-virtual methods, improving the technology and providing resources and support speci c to learners [19]. Like other QI programmes, distance learning QI education also requires substantial commitment from the organization, collaboration among participants, faculty, and leaders for success [14].
Another recommendation for those considering online delivery of QI programmes is to build relationships with institutional QI units to identify resources and link the programme to an institutional network of QI education [28]. Instead of developing home-grown solutions, future collaboratives can purchase or customize existing applications for their technical infrastructure [19]. Programmes aiming to deliver QI curriculum online also have the option of collaborating with other programmes that already have faculty with QI expertise or use publicly available online QI courses [28].
Coaches play an important role in the success of the initiatives and should connect with teams early on to provide supplemental support through coaching calls [19]. In terms of content, programme developers should also explore utilizing psychological learning effects of spaced learning and testing [33] and identify more conceptually and methodologically appropriate performance measures [19]. An important consideration for future programme evaluation is that reactions to the training do not necessarily correspond to actual knowledge increase therefore QI curricula should be evaluated on multiple criteria rather than just on participant reaction [17]. Institutional leadership and environment play an important role in the effectiveness of QI programmes and should be considered in programme design [29].
There is a need for curriculum and training designs to evolve to the needs of the new generation of healthcare professionals with an increasing emphasis on technological tools to overcome the generational difference between educators and learners [37]. Some important questions to be considered in delivering QI programmes through distance learning modes are summarised in Table 3. Are there any already available external platforms or QI resources that the organization can use for delivering training?
What distance learning modes does the training organization currently incorporates in traditional QI training?
What will be the nancial impact for the make vs. buy decision and does the training organization have the required budget?
What is the opinion of the key stakeholders regarding distance learning delivery and make vs buy decision?
Does the training organization have the resources to conduct pilot testing of the e-delivery prior to the launch?

Structure and Content
Are the training objectives, topics, and activities suitable for distance delivery?
Does the organization want to develop or use their own content or adapt from already available content?
Would the programme be delivered completely online, or a blended learning approach would be used?
How can the social and networking aspects of face to face programmes adapted to online delivery?
How will the facilitators provide distance learning support and coaching?
How will contextual factors be incorporated into programme design?

Programme Evaluation and Implementation
What is important to the training organization in terms of evaluation?
(For example: programme attendance, training reaction, patient level outcome, organizational outcomes, content knowledge testing, QI project outcomes, feedback on training mode) How will feedback and evaluation data be collected and who will be responsible?
Will the evaluation be short term or long term?
Will the training organization provide extended coaching online/distance coaching support and how?
What implementation support will be provided and how? Table 3 Legend: Important questions for adapting QI programme delivery to online modes LIMITATIONS This rapid evidence review aimed to synthesise research and recommendations quickly and comprehensively to adapt traditional QI programmes for online delivery in response to the COVID-19 pandemic which has disrupted healthcare systems and QI training programmes. The review employed a systematic search strategy with robust and transparent screening processes, however, owing to the time and resource constraints, only one reviewer was involved in the screening process. However, two additional reviewers critically reviewed the study synthesis to ensure quality. It is recognised that a systematic review is neither appropriate nor possible in every situation [38]. This rapid narrative reviews serves a speci c and time-sensitive purpose and will be useful for policy makers and QI programme designers seeking to make quick decisions about adapting training to meet needs of healthcare staff during the COVID-19 pandemic.

Conclusion
The ndings of this review have important implications for those looking to adapt traditional QI programmes to be delivered virtually. Virtual training environment holds great potential in delivering standardised training and distance collaborations which will be of utmost importance in the post COVID environment. However, caution should be practiced, and a realistic evaluation of capabilities and needs should be conducted before making adaptation decisions. Factors such as programme design, mode of delivery, technical skills and support, implementation support and contextual factors are important consideration. If adapted appropriately, virtual QI and training of healthcare professionals and students can provide an alternative to traditional QI education and play a vital role in building competence and con dence to improve the health care system in post-COVID environment.

Declarations
Ethics approval and consent to participate:

Not Applicable
Consent for publication:

Nor applicable
Availability of data and materials: The studies analysed as part of the narrative literature review are all referenced in the manuscript. No additional data and material are included.
Competing interests: No con icts of interest to declare.

Funding
The corresponding author is receiving a PhD funding from the Health Service Executive Ireland (Project reference 57399). The study is also supported by the Irish Health Research Board (RL-2015-1588).
Author's Contributions: GM suggested the design while ZK led the literature review. ZK, ADB and EM worked on writing the manuscript and GM also provided feedback. All authors contributed to and read and approved the nal manuscript.