Virtual adaptation of healthcare quality improvement training programmes in response to COVID-19: A rapid narrative review

doi:10.21203/rs.3.rs-56911/v1

Download PDF

Short Report

Virtual adaptation of healthcare quality improvement training programmes in response to COVID-19: A rapid narrative review

https://doi.org/10.21203/rs.3.rs-56911/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Information and communication technology is playing a major role in ensuring continuity of healthcare services during the COVID-19 pandemic. The pandemic has also disrupted healthcare quality improvement (QI) training and education for healthcare professionals and there is a need to rethink the way QI training and education is delivered. The purpose of this rapid evidence review is to quickly but comprehensively collate studies to identify what works and what does not in delivering QI training and education using distance-learning modalities. Three healthcare databases were searched along with grey literature sources for studies published between 2015 and 2020. Studies with QI training programmes or courses targeting healthcare professionals and students with at least one component of the programme being delivered online were included. A total of nineteen studies were included in the review. Most studies had a mixed methods design and used blended learning methods, combining online and in-person delivery modes. Most of the included studies reported achieving desired outcomes, including improved QI knowledge, skills and attitudes of participants and improved clinical outcomes for patients. Some benefits of online QI training delivery include fewer required resources, reducing the need for on-site instructors, increased programme reach, and more control and flexibility over learning time for participants. Some limitations of online delivery modes include limited learning and networking opportunities, functional and technical problems and long lead time for content adaptation and customization. The review highlights that distance learning approaches to QI help in overcoming barriers to traditional QI training. Virtual training environment holds great potential in delivering standardised training and distance collaborations which will be of utmost importance in the post-COVID environment. The review offers guidance on important questions to be considered in delivering QI programmes through distance learning modes.

Health Policy

Quality improvement

Online learning

medical education

Quality improvement training

The COVID-19 pandemic is rapidly transforming the landscape of the healthcare system and virtual healthcare solutions are playing a key role in this change[1]. It has also presented unique challenges in the healthcare quality improvement (QI) sphere and highlighted the need for a dynamic approach that enables QI structures and policies to adapt to the pandemic environment[2]. QI principles offer useful strategies for implementing and sustaining meaningful change[3] and staff trained in QI principles can play a critical role in responding to these emerging challenges by accelerating the pace of learning[4]. QI is increasingly being recognised as an important skill for healthcare professionals[5] and is an important component of medical education and training[6]. The immediate focus of the COVID-19 response of the healthcare sector has focused on ensuring continuity of care for patients and communities. However, the pandemic has also had a profound impact on medical education and how healthcare professionals will be educated in future[7]. As researchers and practitioners rush to explore ways to support healthcare professionals during the pandemic, it is also important to rethink the way QI training and education is delivered to healthcare professionals.

Rather than viewing COVID-19 as a disruption to healthcare QI education and training, it can be considered as an opportunity to improve distance learning techniques and benefit from digital hyper-connectivity to enhance education delivery that can extend into the post-pandemic environment [8]. Application of technology-enhanced learning is often cited as a pedagogical advancement for a curricular transformation to prepare healthcare professionals to adapt to the complexity of the healthcare system[9]. However, the pandemic has made it an inescapable necessity for the healthcare system to adapt to virtual ways of working.

Online learning platforms have the potential to bring healthcare professionals together to share knowledge and collaborate in QI teaching, learning and education[10]. Beyond the pandemic situation, the nature of knowledge is continuously evolving in healthcare and self-directed and e-learning could lead to better knowledge retention as compared to passive knowledge absorption through didactic lectures[11]. There is much to be understood about the usefulness of distance-learning modalities in effectively delivering QI training. The purpose of this rapid narrative review is to collate studies to identify what works and what does not in delivering QI training and education using distance-learning modalities. A rapid evidence assessment summarises research findings in a systematic manner, within time and resource constraints and is suited to the current situation[12].

The review aims to answer the following questions:

What distance learning modalities are being used to train healthcare staff and students in QI methods?
What is the efficacy of distance learning QI programmes?
What were the advantages and limitations in delivering QI programmes using distance-learning modalities?

By answering these questions, we aim to synthesise guidelines and recommendations for those who are dealing with the challenge of adapting and delivering QI training to healthcare professionals though distance learning modes.

Three databases (PubMed, Web of Science and Scopus) were searched to identify studies published between 2015 and 2020. The inclusion criteria were QI training programmes or courses targeting healthcare professionals and students with at least one component of the programme being delivered online. Only studies with primary data were included. Conference proceedings, editorials, protocols, and book sections were excluded. Studies that did not explicitly teach QI principles in the programme were excluded. Papers with no full text available or no English translation available were also excluded. Reference lists of included papers and grey literature search of Google Scholar was also conducted to identify further papers. The complete search strategies and screening process are documented in Figure 1 (Prisma Diagram) and search strategy is presented in Supplemental File 1. The critical appraisal of the studies was conducted using the Quality Improvement Minimum Quality Criteria Set (QI-MQCS) – Version 1.0 tool which possess acceptable psychometric properties for critical appraisal to support systematic reviews containing diverse quality improvement intervention(QII) evaluations[13].

The summary of the nineteen studies included in this evidence synthesis is presented in Table 1.

Table 1: Summary of Studies

Study Characteristics			Population Description		Intervention Description			Outcomes
Study ID	Location	Study Design	Population	Sample size	Training purpose	Intervention type	Delivery modes	Evaluation of Outcomes	Outcomes Achieved
Baernholdt-2017[14]	United States	Mixed methods	Interprofessional health care teams	40	Training interprofessional health care teams to lead QI projects using PDSA methodology	Interprofessional Quality Improvement Training Program	seminars, online modules, bimonthly meetings, QI project work	· Participation · Learner reactions to training · Participants’ QI knowledge, attitudes, behaviours · Patient safety outcomes	· 19 out of 22 teams completed the programme · Higher QI self-efficacy post-programme · Program and sessions rated favourably · Improvements in clinical settings
Baxley-2016[15]	United states	Mixed methods	Interprofessional group of faculty	27	Preparing faculty to lead frontline clinical transformation	Teachers of Quality Academy Professional development program	Online, didactic, small-group, experiential learning, QI project, QI symposium	· Progress of QI initiatives · Incorporation of educational modules into curriculum · production of scholarly products by participants · Participants’ QI knowledge, attitudes, behaviours · Patient safety outcomes · Interprofessional practice	· All participants completed QI projects · 70% participants engaged in design and delivery of curriculum · Participants applied new knowledge and skills in educational initiatives development
Bonnes-2017[16]	United States	Prospective validation study	Internal medicine residents	143	Educating trainees on how to successfully improve health care quality	Flipped QI curriculum	Online modules, facilitated small group discussions	· Preferences for mode of delivery · Past experiences with delivery mode · Completion of online modules · Participants’ QI knowledge, attitudes, behaviours	· Improved perception of FC · Participants of FC demonstrated improved QI knowledge compared to the control group · FC associated with greater engagement in online modules
Gregory-2018[17]	United states	Quantitative descriptive	Postdoctoral nurses, post-residency physicians, clinical psychologist	54	Training health care professionals to become leaders in QI	Veterans Affairs Quality Scholars Curriculum	web-based curriculum delivered in real time	· Participants’ QI knowledge, attitudes, behaviours · Transfer of training · Learner reactions to training	· Learners satisfied with training · Improvements in QI knowledge, attitudes, behaviours · Significant improvement in affective transfer but no significant change in cognitive or skill-based transfer
Hafford-Letchfield-2018[18]	United Kingdom	Mixed methods	Social workers, midwives, community nurses, occupational therapists, dieticians, general and mental health nurses	62	Using digital storytelling method to encourage collaboration for identifying and developing plans for service improvements	Service Development and Quality Improvement module	Digital storytelling pedagogy with online activities and half-day workshops taught face-to-face	· Developing digital story · Developing action plan to address selected issue · Writing improvement plan · Experience with delivery mode · Patient safety outcomes	· Levelling effect in interprofessional collaboration · Programme content should focus on communicating service user/patient needs · Virtual learning pedagogies encourage co-construction of shared solutions across disciplines · Nearly all students created innovative and informative digital stories with genuine practical utility
Hargreaves-2017[19]	United States	Mixed methods	primary care, public health, and community leaders & project managers, faculty, project staff	11 teams	Sharing and spreading, evidence-based QI practices to prevent and treat obesity	National Initiative for Children’s Healthcare Quality (NICHQ) Healthy Weight Collaborative	In-person networking events and virtual learning sessions, webinars, coaching calls, peer networking calls, technical assistance calls	· Implementation of activities · Developing action plans · Engagement of community teams · Project results · Patient outcomes · Online module usage patterns · Documents submitted by teams	· Developed collaborative capacity among teams · 34% of Phase 2 teams had an “above average” level of engagement · Use of QI methods and performance measures helped teams make progress · All teams adopted a healthy weight message, 59% implemented community-wide assessments and plans
Jamal-2017[20]	United States	Quantitative descriptive	Otolaryngology residents	11	Integrating patient safety and quality improvement into resident education	Patient Safety and Quality Improvement (PSQI) curriculum	Interactive online modules, classroom group discussions, lectures by PSQI experts, self-directed workshops to develop projects	· Online Module Content and Quality · Number of projects developed · Confidence in using QI	· IHI online modules are appropriate for patient safety and QI beginners and well accepted by participants · Over half of residents found these modules to be ‘‘extremely’’ or ‘‘very’’ worthwhile
Keefer-2016[21]	United States	Quantitative descriptive	House Officers	80	Training house staff about basic QI techniques	Flipped classroom quality improvement curriculum	Online modules and in-person workshops	· Participants’ QI knowledge, attitudes, behaviours	· Improved QI content knowledge · Improved perceived readiness to participate in QI projects
Kennedy-2017[22]	United States	Mixed methods	Faculty, staff, administrators, supervisors, data managers	60	Undertaking and sharing Continuous Quality improvement techniques	Online quality improvement information exchange	web-based portal/website	· Experience with delivery mode · QI delivery mode effectiveness, efficiency, satisfaction	· Results were overall positive and desirable · Majority reviewers reported they would use the learning materials, complete quality improvement projects and reported the site would help address quality improvement challenges
Maxwell-2016[23]	United States	Pretest/posttest control group design	Baccalaureate nursing students	64	Improving knowledge, skills, and attitudes regarding QI and safety	QSEN competencies	Online modules, flipped classroom	· Participants’ QI knowledge, attitudes, behaviours, and comfort · Safety knowledge, comfort, and attitude	· Statistically significant effect between the groups for QI · Experimental group had slightly higher knowledge scores than the control group for safety and QI · Use of online modules in conjunction with the flipped classroom had a greater effect on increasing QI knowledge than the use of online modules only
Potts-2016[24]	United States	Mixed methods	Family Medicine residents	23	Integrating residents to actively participate in quality improvement and patient safety activities	Integrated Quality Improvement Residency Curriculum	Web-based tutorials, quality improvement projects, small-group sessions	· Quality improvement skills · Patient safety skills · Chronic care management	· Participants of full curriculum reported higher use of knowledge · Chronic care management and patient safety skill significantly improved for majority items · Only one item (designing prospective chart reviews) was significantly improved for the QI skills category
Ramar-2015[25]	United States	Quantitative	Fellowship trainees	7	Incorporating a QI curriculum into a training program	Flipped classroom (FC) model	Video lessons, half-day session, case examples, a hands-on workshop	· Learner reactions to training · Participants’ QI knowledge, attitudes, behaviours	· Significant improvement in post-FC QI knowledge · Overall positive reaction towards FC model
Scales-2016[26]	United States	Randomized control trial	Resident physicians	422	Increasing learner participation in quality-improvement education	QI curriculum	Spaced delivery of interactive healthcare quality questions via email	· Participation · Participant Engagement	· Residents in the intervention arm demonstrated greater participation than the control group · Percentage of questions attempted at least once was greater in the intervention group versus control group · Response time was faster in intervention group · Team competition increases resident participation in an online course delivering QI content
Shaikh-2017[27]	United States	Quantitative descriptive	Residents and faculty	500	Increasing resident and faculty knowledge in QI, patient safety, and care transitions	University of California Health’s Enhancing Quality in Practice online course	Three modules, questions sent on smartphones using an app, or on computers using e-mail.	· Course completion · QI Knowledge · Patient safety outcomes · Preferences for mode of delivery	· Learners rated quiz-based system as an effective teaching modality and preferred it to classroom-based lectures · Course completion rate between 66–86% · Knowledge acquisition scores for, QI, patient safety and care transitions increased after course completion · Course best utilized to supplement classroom and experiential curricula
Shelgikar-2017[28]	United States	Mixed methods	Sleep medicine fellows	7	Developing skills to systematically analyse practice using quality improvement methods, and implement changes	QI curriculum using a flipped classroom	Online modules and group sessions	· QI Knowledge · Confidence in QI application · Participation · Project completion	· All participants completed the curriculum · Knowledge of QI concepts and confidence in performing QI activities increased · QI projects improved timeliness and quality of care for patients.
Sorita-2015[29]	Canada	Mixed methods	secretaries, clinical assistants, registered nurses, nurse practitioners, physician assistants, physicians	Not stated	Running Plan-Do-Study-Act cycles to streamline examination process	QI curriculum	Didactics, workshop, online modules, and experiential learning	· Improvement in care process	· Residents successfully applied QI methods to improve the efficiency of the DOT examination process · Total visit time successfully reduced Accuracy of certificate issuance, as proxy for examination quality improved after intervention
Tappen-2018[30]	United States	Randomized, controlled trial	Nursing Facility Residents	264	Improving the identification, evaluation, and management of acute changes	INTERACT Quality Improvement Program	INTERACT tools, online training programme, webinars, an intensive initial training programme, monthly follow-up webinars	· Patient safety outcomes	· No adverse effects on resident safety · No significant differences in safety indicators between intervention and comparison group · Intervention NFs with high levels of INTERACT tool use reported significantly lower rates of severe pain
Tartaglia-2015[31]	United States	Observational study with control group	Fourth-year medical students	34	Improving QI knowledge	QI Curriculum	Online modules, reflective writing, discussion with content expert, mentored QI project	· Comfort with QI principles · Participants’ QI knowledge, attitudes, behaviours · Projects completion	· Students in the intervention group reported more comfort with their skills in QI · Curriculum strength included effective use of classroom time, faculty mentorship, reliance on pre-existing online modules · Curriculum is expandable to larger groups and transferable to other institutions
Zubkoff-2019[32]	United States	Mixed methods	Team leader, senior level support person, nurse, physician, nurse practitioner champion, pharmacist, and physical therapist	60	Enhancing knowledge, infrastructure, and capacity for QI	Virtual Breakthrough Series Collaborative	webinar-based educational format, open discussion sessions, Meet and Greet” call with coaches, Pre-work calls,	· Learner reactions to training · Report submission · Patient safety outcomes	· No statistically significant decrease in total fall rates or major injury rates · Significant improvement in fall related injury rate · Majority were satisfied with the educational calls · Minor injury rate decreased significantly · Monthly report submission between 65 to 85%

Legend: A summary of study characteristics, intervention descriptions, and outcomes of included studies

Study Characteristics

Although, the overarching aim of the included studies was to improve QI skills of healthcare professionals and students, the studies differed in design, evaluation, and analytical methods used. Most studies had a mixed methods design and 18 of the 19 studies were based in the United States. Design of the interventions was also variable; most studies used a blended learning method combining online and in-person modes while only six studies[17, 22, 27, 30, 32, 33] were entirely delivered online. Some blended learning modules conducted classroom-based sessions followed by support through online modules and QI project completion, while others used a flipped curriculum approach where participants completed online modules prior to the in person sessions such as seminars, workshops, lectures and QI project completion.

Quality Assessment

All included studies were deemed to be of good quality even though some studies did not report on all areas evaluated by the QI-MQCS tool. All studies discussed the rationale behind the intervention, organizational motivation, description of the intervention and implementation approach. Some studies included limited information about describing sustainability or the potential for sustainability of the interventions and explicitly naming the study design. None of the studies were excluded based on quality assessment and a detailed quality assessment is attached in Supplemental File 2.

Distance Learning Modes

The online delivery modes used by studies included online modules[14-16, 20, 23, 24, 28-31, 34], access to web-based curricula[17], virtual learning environments[18], webinars[19, 30, 32], calls[19, 32], web-based QI portals[22], smartphone apps[27], emails [27, 33], access to package of tools[30], virtual whiteboard[34] and video lessons[25]. Instead of developing their own distance learning content, most studies relied on the completion of the Institute of Healthcare Improvement’s (IHI) online modules,[15, 16, 20, 23, 31, 32] many of which are free to use. The rationale behind using IHI’s modules is that it provides a standardised methodology which does not require prior faculty proficiency or entail an increase in educational time commitment[20]. The IHI methodology is designed to help organizations in identifying and closing gaps via a standard improvement methodology[32]. Another advantage is that an institutional subscription to the IHI programme provides access to comprehensive QI training and allows tracking the progress of participants[24]. Some studies adapted IHI modules[29, 32, 35] to their local context while a minority used self-developed content [17-19, 22, 27, 28, 30]. The major online modalities used are summarised in Table 2.

Table 2: Description of online modes

Modality	Description
Flipped Curriculum/Flipped Classroom	Instructional content delivered through online modes before class and class time used for knowledge application[16, 23, 28, 34]
Virtual breakthrough series collaborative	Virtual adaptation of the Institute for Healthcare Improvement (IHI) face-to-face collaborative model through webinar-based educational delivery [32]
Dedicated web portal/QI site	In-house QI sites developed to provide access to QI tools, resources, and training[19] [22]
Interactive online delivery	Didactic lectures delivered live online allowing participants to participate in real time[17, 20]
Video lectures	Pre-recorded didactic lectures made available to participants[25]
Phone/app/email-based methods	QI questions sent out to participants through text messages, phone apps or email[27, 33]
Online modules to supplement classroom delivery	IHI QI modules [14, 15, 24, 31] Self-developed QI modules[19, 29]

Legend: Summary of major online modes used by studies in delivering QI training and education

Only a few studies discussed the tools/software used to deliver the online QI training components. One study used Adobe Connect and blackboard for delivering a web-based QI curriculum [17] while for a digital storytelling pedagogy, researchers recommended participants to use freely available software such as Windows Moviemaker or Apple i-movie[18]. Another QI collaborative used iLab which is a secure, online workspace[19] while a study that developed a web-based QI portal used WordPress CMS platform, social media account integration and a network management site called Hootsuite[22]. A microlearning app called Qstream was used in another training programme[27].

Efficacy of QI training

Studies used various evaluation methods; some focused on programme level factors such as course completion rates[16, 27], learner reaction to training[14, 17, 25, 32], engagement level of participants[14, 28, 33], participant perceptions of the online module content and quality[20], preferred training delivery mode[16, 22, 27] and document and report submission by participating teams[32]. Another area of outcome evaluation was improvement in participant knowledge. Studies evaluated improved participant comfort[31] and confidence[20, 28]in using QI. Many studies also assessed participants’ QI knowledge, skills, attitudes and behaviours[14-17, 23-25, 27, 31, 34]. Relatively few studies assessed improved patient safety skills and knowledge of participants[23, 24]. How participants implemented QI skills and knowledge also constituted a part of outcome evaluation in various studies. This included development of action and improvement plans[18], number of QI projects developed[20] and completed[28] and results attained from these projects[19]. Improved results for the patients were also used as a proxy for outcome evaluation[14, 15, 17-19, 27, 30, 32].

Most of the included studies reported achieving desired outcomes such as increased QI knowledge & skills [14, 24, 25, 27, 28, 34, 35], positive reaction from participants towards the training [17, 19, 20, 32], implementation of QI knowledge by participants[18, 29] and confidence to use the learned skills in future[22, 28]. One study reported no improvement in the measure being tracked[32]. In the five studies with control groups, the intervention participants demonstrated improved QI knowledge[16], improved comfort with QI methods[31] and greater participation than the control groups[33]. One study did not demonstrate any significant difference in post-intervention safety indicators between intervention and comparison group however it was a positive outcome showing that nursing home residents’ safety was not compromised during the QI programme [30].

This shows that majority of the interventions were successful in demonstrating the desired results. However, there was scant information around the role played by mode of delivery in the attainment of these outcomes. One study using a control group concluded that use of online content in conjunction with in-person sessions as being more effective in improving QI knowledge than only relying on online content[23]. The flipped curriculum[16] and utilization of web-based platforms to deliver advanced QI training[17] proved to be effective methods for teaching QI. Since there is a shortage of comparable prior studies on web-based tools for QI education, it is challenging to compare results across similar interventions and more longitudinal studies may be required to analyse outcome trends over time[22].

Benefits of Online QI Education

An online QI programme can virtually connect users and provide them with an environment that balances training and practice[22]. Online delivery of QI training programmes requires fewer resources[19], reduces the burden on sites and instructors[17] and the organizations do not need to maintain QI faculty [17]. It is useful in delivering a centralised QI curriculum to distributed learners[17] hence increasing the reach of the programme[19, 27]. Additionally, online learning seemed to balance the educational time constraints and clinical responsibilities in educating healthcare professionals[16]. Moreover, the IHI teaching modules used in this study are widely recognized and accessible to all programmes and learners[16]. Interactive, distance learning models which occur in real time with multi-way communication and feedback and tailored education proved to be effective[17]. Online modules which are interactive were preferred by participants over static computer-based modules[16].

Participants in online QI programmes have more control over their learning time[27], allowing them to complete much of the curriculum at a time convenient to them[31]. Programmes that are interactive and real-time in nature lead to better and personalised engagement from participants[17]. In the case of a flipped classroom where participants complete online modules prior to in-person sessions, the online component enables maximisation of in-class time[34]. However, many participants expressed that in-class sessions and in-class application were more effective than online content in enhancing their QI knowledge[16]. Similarly, viewing didactic material on videos beforehand enables participants to use classroom time to clarify concepts[25] and reduces the overall time required for the curriculum[20]. On the other hand, quiz based online courses provide real-time feedback, engagement and healthy competition but are more suited to reinforce concepts taught in classrooms and supplement other QI activities rather than as standalone activities[27].

Another advantage of online programmes using innovative methods such as digital story telling is that it engages the participants in learning a new skill and creates a level playing field in terms of the anxiety associated with a new experience[18]. Virtual discussion boards also have the advantage of providing a safe space where participants can freely express their opinions and ideas which they might not feel comfortable doing face-to-face[18]. Similarly, using game mechanics and team-based competition in a safe virtual environment is an effective participant engagement strategy[33].

Participants were generally positive about features of online programmes such as open discussion forums, closed groups, private messaging, and feedback submission forms[22]. Using tools such as leader boards motivates participants to engage and provides a sense of status[33]. Group size between two to eight participants worked best[34]. Virtual formats also allow for easier modifications in the curriculum length, content, and level which are important considerations in training design[17].

Open communication, stakeholder buy-in, and continuous feedback were necessary in developing a shared vision and QI site ownership[22]. Educational content developed by faculty with practical and teaching QI experience strengthens the programme[27]. The early involvement of key stakeholders and SMART (specific, measurable, attainable, relevant, and time-bound) goals proved to be critical to the success[29]. QI coaches also play an important role in distance learning programmes as well and a study concluded that having tailored coaching support for each team was a useful aspect of the programme[14]. In the case of students, providing an opportunity for experiential learning through QI project completion alongside a faculty member was also important[31].

Limitations of Online Methods

Evidence suggests that mobile and asynchronous educational technologies have the potential to overcome barriers related to teaching QI methods[33]. However, studies have also identified some limitations to such approaches. Participants often valued the learning application sessions conducted in person, more than the online components[16]. In the same way, although learners enjoyed asynchronous learning and online delivery, they preferred assessment questions that focused on application of concepts rather than information acquisition[27]. Online modes offer limited learning and networking opportunities[19].

Customizing didactic materials to suit programme and participant needs is a time intensive task[28]. Developing an online site can be resource-intensive and lead to functional problems[19]. Since data and reporting systems are external and independent form the QI education sites, it is difficult to integrate these as site resources[22]. Online programmes also require facilitators who have QI knowledge[32] as well as technical support in case participants face any technical challenges[20] such as phone line chatter as reported in a study[32]. Additionally, the adoption and use of a new technology requires significant run-time[22]. Even though devices such as mobile phones and computers are already used regularly by the learners[27], there are limitations to their usage such as content and character restrictions in text messages and email fatigue for participants[27]. Although the IHI Open School modules are widely used and effective, one study recommended augmenting the content to suit local needs[23] and online content such as videos should be at an appropriate level and pace suitable to the participants[25].

With blended learning programmes, a major challenge was group session scheduling so that participants could attend without disruption of clinical responsibilities[28]. Some participating teams raised concerns specific to collecting and reporting the measurement data and perceived the measures to be complex and not well-matched to the teams’ goals[19]. Apart from the challenges associated with online delivery, teams also experience other challenges such as demand of other work duties and inability to meet as a team during implementation[14].

The training modalities discussed in the included studies can be broadly categorised into e-learning programmes and blended learning programmes. A previous systematic review comparing online with face-to-face education for healthcare professionals concluded that online programmes had comparable knowledge gains and benefits to onsite or face-to-face training[36]. However, there is a gap in literature evaluating the success of QI training delivery online. The purpose of this evidence review is to focus on efficacy of QI training and education interventions being delivered online to synthesise recommendations for adapting QI training content into e-learning materials

The review highlights that distance learning approaches to QI help in overcoming barriers to traditional QI training such as shortage of trained faculty and deficiencies in organizational structure to support QI education(24) and in many participants preferred blended approaches to traditional approaches(32). This review highlights important lessons for future programmes including balancing virtual and non-virtual methods, improving the technology and providing resources and support specific to learners[19]. Like other QI programmes, distance learning QI education also requires substantial commitment from the organization, collaboration among participants, faculty, and leaders for success[14].

Another recommendation for those considering online delivery of QI programmes is to build relationships with institutional QI units to identify resources and link the programme to an institutional network of QI education[28]. Instead of developing home-grown solutions, future collaboratives can purchase or customize existing applications for their technical infrastructure[19]. Programmes aiming to deliver QI curriculum online also have the option of collaborating with other programmes that already have faculty with QI expertise or use publicly available online QI courses[28].

Coaches play an important role in the success of the initiatives and should connect with teams early on to provide supplemental support through coaching calls[19]. In terms of content, programme developers should also explore utilizing psychological learning effects of spaced learning and testing[33] and identify more conceptually and methodologically appropriate performance measures[19]. An important consideration for future programme evaluation is that reactions to the training do not necessarily correspond to actual knowledge increase therefore QI curricula should be evaluated on multiple criteria rather than just on participant reaction[17]. Institutional leadership and environment play an important role in the effectiveness of QI programmes and should be considered in programme design[29].

There is a need for curriculum and training designs to evolve to the needs of the new generation of healthcare professionals with an increasing emphasis on technological tools to overcome the generational difference between educators and learners[37]. Some important questions to be considered in delivering QI programmes through distance learning modes are summarised in Table 3.

Table 3: Important questions for virtual training adaptation

Capability Assessment

Do the trainees have access to the required resources and infrastructure to benefit from online delivery?

Do the trainees have the required technical understanding to participate in online training?

Does the training organization have the necessary QI expertise and facilitation capacity?

Does the training organization have the requisite technical support?

What are the current challenges faced by the training organization in delivering traditional QI programmes?

Make or Buy

Does the organization have the required resources, skills, and technical support to develop an online QI training solution from scratch?

Does the organization have an already available QI platform that can be adapted to deliver QI training?

Are there any already available external platforms or QI resources that the organization can use for delivering training?

What distance learning modes does the training organization currently incorporates in traditional QI training?

What will be the financial impact for the make vs. buy decision and does the training organization have the required budget?

What is the opinion of the key stakeholders regarding distance learning delivery and make vs buy decision?

Does the training organization have the resources to conduct pilot testing of the e-delivery prior to the launch?

Structure and Content

Are the training objectives, topics, and activities suitable for distance delivery?

Does the organization want to develop or use their own content or adapt from already available content?

Would the programme be delivered completely online, or a blended learning approach would be used?

How can the social and networking aspects of face to face programmes adapted to online delivery?

How will the facilitators provide distance learning support and coaching?

How will contextual factors be incorporated into programme design?

Programme Evaluation and Implementation

What is important to the training organization in terms of evaluation?

(For example: programme attendance, training reaction, patient level outcome, organizational outcomes, content knowledge testing, QI project outcomes, feedback on training mode)

How will feedback and evaluation data be collected and who will be responsible?

Will the evaluation be short term or long term?

Will the training organization provide extended coaching online/distance coaching support and how?

What implementation support will be provided and how?

Table 3 Legend: Important questions for adapting QI programme delivery to online modes

LIMITATIONS

This rapid evidence review aimed to synthesise research and recommendations quickly and comprehensively to adapt traditional QI programmes for online delivery in response to the COVID-19 pandemic which has disrupted healthcare systems and QI training programmes. The review employed a systematic search strategy with robust and transparent screening processes, however, owing to the time and resource constraints, only one reviewer was involved in the screening process. However, two additional reviewers critically reviewed the study synthesis to ensure quality. It is recognised that a systematic review is neither appropriate nor possible in every situation[38]. This rapid narrative reviews serves a specific and time-sensitive purpose and will be useful for policy makers and QI programme designers seeking to make quick decisions about adapting training to meet needs of healthcare staff during the COVID-19 pandemic.

The findings of this review have important implications for those looking to adapt traditional QI programmes to be delivered virtually. Virtual training environment holds great potential in delivering standardised training and distance collaborations which will be of utmost importance in the post COVID environment. However, caution should be practiced, and a realistic evaluation of capabilities and needs should be conducted before making adaptation decisions. Factors such as programme design, mode of delivery, technical skills and support, implementation support and contextual factors are important consideration. If adapted appropriately, virtual QI and training of healthcare professionals and students can provide an alternative to traditional QI education and play a vital role in building competence and confidence to improve the health care system in post-COVID environment.

Ethics approval and consent to participate:

Not Applicable

Consent for publication:

Nor applicable

Availability of data and materials:

The studies analysed as part of the narrative literature review are all referenced in the manuscript. No additional data and material are included.

Competing interests:

No conflicts of interest to declare.

Funding

The corresponding author is receiving a PhD funding from the Health Service Executive Ireland (Project reference 57399). The study is also supported by the Irish Health Research Board (RL-2015-1588).

Author’s Contributions:

GM suggested the design while ZK led the literature review. ZK, ADB and EM worked on writing the manuscript and GM also provided feedback. All authors contributed to and read and approved the final manuscript.

Acknowledgements:

Not Applicable

COVID-19: Coronavirus Disease 2019

QI: Quality Improvement

QI-MQCS: Quality Improvement Minimum Quality Criteria Set

QII: Quality Improvement Interventions

Mann DM, Chen J, Chunara R, Testa PA, Nov O: COVID-19 transforms health care through telemedicine: Evidence from the field. Journal of the American Medical Informatics Association 2020.
Oesterreich S, Cywinski JB, Elo B, Geube M, Mathur P: Quality improvement during the COVID-19 pandemic. Cleveland Clinic Journal of Medicine 2020.
Clarke MJ, Steffens FL, Mallory GW, Starr SR, Porter BL, Krauss WE, Dankbar EC: Incorporating Quality Improvement into Resident Education: Structured Curriculum, Evaluation, and Quality Improvement Projects. World Neurosurgery 2019, 126:e1112-e1120.
Fitzsimons J: Quality & Safety in the time of Coronavirus-Design Better, Learn Faster. International journal for quality in health care : journal of the International Society for Quality in Health Care 2020:mzaa051.
Ewins E, Macpherson R, van der Linden G, Arnott S: Training in quality improvement for the next generation of psychiatrists. BJPsych Bulletin 2017, 41:45-50.
Herman DD, Weiss CH, Thomson CC: Educational Strategies for Training in Quality Improvement and Implementation Medicine. ATS Scholar 2020, 1:20-32.
Rose S: Medical Student Education in the Time of COVID-19. JAMA 2020, 323:2131-2132.
Kanneganti A, Lim KMX, Chan GMF, Choo S-N, Choolani M, Ismail-Pratt I, Logan SJS: Pedagogy in a pandemic – COVID-19 and virtual continuing medical education (vCME) in obstetrics and gynecology. Acta Obstetricia et Gynecologica Scandinavica 2020, 99:692-695.
Grainger R, Liu Q, Geertshuis S: Learning technologies: A medium for the transformation of medical education? Medical Education, n/a.
Mehta R, Sharma KA: Use of Learning Platforms for Quality Improvement. Indian Pediatrics 2018, 55:803-808.
Suliman S, Hassan R, Athamneh K, Jenkins M, Bylund C: Blended learning in quality improvement training for healthcare professionals in Qatar. International journal of medical education 2018, 9:55-56.
Noble H, Smith J: Reviewing the literature: choosing a review design. Evidence Based Nursing 2018, 21:39.
Rubenstein LV, Hempel S, Liu JL, Danz MJ, Foy R, Lim Y-W, Motala A, Shekelle PG: The Minimum Quality Criteria Set (QI-MQCS) for critical appraisal: advancing the science of quality improvement. Implementation Science : IS 2015, 10:A19-A19.
Baernholdt M, Feldman M, Davis-Ajami ML, Harvey LD, Mazmanian PE, Mobley D, Murphy JK, Watts C, Dow A: An Interprofessional Quality Improvement Training Program That Improves Educational and Quality Outcomes. American Journal of Medical Quality 2019, 34:577-584.
Baxley EG, Lawson L, Garrison HG, Walsh D, Lazorick S, Lake D, Higginson J: The Teachers of Quality Academy: A Learning Community Approach to Preparing Faculty to Teach Health Systems Science. Academic Medicine 2016, 91:1655-1660.
Bonnes SL, Ratelle JT, Halvorsen AJ, Carter KJ, Hafdahl LT, Wang AT, Mandrekar JN, Oxentenko AS, Beckman TJ, Wittich CM: Flipping the Quality Improvement Classroom in Residency Education. Academic Medicine 2017, 92:101-107.
Gregory ME, Bryan JL, Hysong SJ, Kusters IS, Miltner RS, Stewart DE, Polacek N, Woodard LD, Anderson J, Naik AD, Godwin KM: Evaluation of a Distance Learning Curriculum for Interprofessional Quality Improvement Leaders. American Journal of Medical Quality 2018, 33:590-597.
Hafford-Letchfield T, Dayananda A, Collins D: Digital storytelling for interprofessional collaborative practice to develop quality and service improvements. Social Work Education 2018, 37:804-812.
Hargreaves MB, Orfield C, Honeycutt T, Vine M, Cabili C, Coffee-Borden B, Morzuch M, Lebrun-Harris LA, Fisher SK: Addressing Childhood Obesity Through Multisector Collaborations: Evaluation of a National Quality Improvement Effort. Journal of Community Health 2017, 42:656-663.
Jamal N: Patient Safety and Quality Improvement Education in Otolaryngology Residency: Preliminary Look at a Module-Based Approach. OTO Open 2017, 1:2473974x17698647.
Keefer P, Orringer K, Vredeveld J, Warrier K, Burrows H: Developing a Quality Improvement and Patient Safety Toolbox: The Curriculum. MedEdPORTAL 2016, 12.
Kennedy DR, Boren SA, Kapp JM, Simoes EJ: Building and launching an online quality improvement information exchange for home visiting programs in Missouri. Online J Public Health Inform 2017, 9:e189.
Maxwell KL, Wright VH: Evaluating the effectiveness of two teaching strategies to improve nursing students' knowledge, skills, and attitudes about quality improvement and patient safety. Nursing Education Perspectives 2016, 37:291-292.
Potts S, Shields S, Upshur C: Preparing Future Leaders: An Integrated Quality Improvement Residency Curriculum. Fam Med 2016, 48:477-481.
Ramar K, Hale CW, Dankbar EC: Innovative model of delivering quality improvement education for trainees – a pilot project. Medical Education Online 2015, 20:28764.
Scales CD, Moin T, Fink A, Berry SH, Afsar-Manesh N, Mangione CM, Kerfoot BP: A randomized, controlled trial of team-based competition to increase learner participation in quality-improvement education. International Journal for Quality in Health Care 2016, 28:227-232.
Shaikh U, Afsar-manesh N, Amin AN, Clay B, Ranji SR: Using an online quiz-based reinforcement system to teach healthcare quality and patient safety and care transitions at the University of California. International Journal for Quality in Health Care 2017, 29:735-739.
Shelgikar AV, Priddy C, Harrison RV: Meeting ACGME and ABMS Quality Improvement Requirements in a Sleep Medicine Fellowship Program. J Clin Sleep Med 2017, 13:1177-1183.
Sorita A, Raslau D, Murad MH, Steffen MW: Teaching quality improvement in occupational medicine: Improving the efficiency of medical evaluation for commercial drivers. Journal of Occupational and Environmental Medicine 2015, 57:453-458.
Tappen RM, Newman D, Huckfeldt P, Yang Z, Engstrom G, Wolf DG, Shutes J, Rojido C, Ouslander JG: Evaluation of Nursing Facility Resident Safety During Implementation of the INTERACT Quality Improvement Program. Journal of the American Medical Directors Association 2018, 19:907-913.e901.
Tartaglia KM, Walker C: Effectiveness of a quality improvement curriculum for medical students. Medical Education Online 2015, 20.
Zubkoff L, Neily J, Delanko V, Young-Xu Y, Boar S, Bulat T, Mills PD: How to Prevent Falls and Fall-Related Injuries: A Virtual Breakthrough Series Collaborative in Long Term Care. Physical & Occupational Therapy In Geriatrics 2019, 37:234-246.
Scales CD, Jr., Moin T, Fink A, Berry SH, Afsar-Manesh N, Mangione CM, Kerfoot BP: A randomized, controlled trial of team-based competition to increase learner participation in quality-improvement education. Int J Qual Health Care 2016, 28:227-232.
Keefer P, Orringer Ke, Vredeveld J, Warrier K, Burrows H: Developing a Quality Improvement and Patient Safety Toolbox: The Curriculum. MedEdPORTAL 2016, 12.
Worsham C, Swamy L, Gilad A, Abbott J: Quality Improvement Virtual Practicum: The QI Simulator. MedEdPORTAL : the journal of teaching and learning resources 2018, 14:10670.
Sullivan LG: A systematic review to compare the effectiveness of face-to-face versus online (including blended learning) delivery of CME/CPD for healthcare practitioners (HCPs). 2017.
Chaudhuri JD: Stimulating Intrinsic Motivation in Millennial Students: A New Generation, a New Approach. Anatomical Sciences Education 2020, 13:250-271.
Greenhalgh T, Thorne S, Malterud K: Time to challenge the spurious hierarchy of systematic over narrative reviews? European Journal of Clinical Investigation 2018, 48:e12931.

SupplementalFile1.docx
Search strategy
SupplementalFile2.xlsx
Study quality assessment

Download PDF

Version 1

posted

You are reading this latest preprint version

Virtual adaptation of healthcare quality improvement training programmes in response to COVID-19: A rapid narrative review

Status:

Version 1

Abstract

Figures

Background

Methods

Results

Discussion

Conclusion

Declarations

Abbreviations

References

Supplementary Files

Status:

Version 1