Effectiveness of the Extension for Community Healthcare Outcomes (ECHO) Model of continuing tele-education and in uence of key learning conditions on the development of competencies in healthcare professionals: Protocol for a mixed methods systematic review

Gabrielle Chicoine (  gabrielle.chicoine@umontreal.ca ) Universite de Montreal Faculte des sciences in rmieres https://orcid.org/0000-0003-3179-5806 José Côté University of Montreal Faculty of Nursing: Universite de Montreal Faculte des sciences in rmieres Jacinthe Pepin Universite de Montreal Faculte des sciences in rmieres Guillaume Fontaine Université de Montréal Faculté des sciences in rmières: Universite de Montreal Faculte des sciences in rmieres Marc-André Maheu-Cadotte Universite de Montreal Faculte des sciences in rmieres Quan Nha Hong University College London Geneviève Rouleau Women's College Hospital Daniela Ziegler University of Montreal Hospital Centre Library: Centre Hospitalier de l'Universite de Montreal Bibliotheque Didier Jutras-Aswad Université de Montréal Faculté de médecine: Universite de Montreal Faculte de medecine


Abstract Background
The Extension for Community Healthcare Outcomes (ECHO) Model of continuing tele-education is an innovative guided-practice model aiming at amplifying healthcare professionals' competencies in the management of chronic and complex health conditions. While data on the effectiveness of the ECHO Model is increasingly available in the literature, the in uence of key learning conditions on the model effectiveness remains unclear. Therefore, the overarching aim of this systematic review is to identify, appraise and synthesize the available quantitative (QUAN) and qualitative (QUAL) evidence regarding the effectiveness of the ECHO Model and the in uence of key learning conditions on the development of competencies in healthcare professionals.

Methods
This proposed systematic review will be conducted in accordance with the Joanna Briggs Institute (JBI) methodology for Mixed Methods Systematic Reviews (MMSR) and will follow a convergent segregated approach. A systematic search will be undertaken using QUAN, QUAL and mixed methods (MM) studies of ECHO-a liated programs identi ed in ve databases. A publication date lter will be applied to nd the articles published from 2003 onwards. Sources of unpublished studies and grey literature will be searched as well. Retrieved citations will be screened by two review authors independently.
Disagreements will be resolved through discussion until a consensus is reached, or by including a third reviewer. Studies meeting the prede ned inclusion criteria will be assessed on methodological quality and the data will be extracted using standardized data extraction forms. Separate QUAN and QUAL synthesis will be performed, and ndings will be integrated using a matrix approach for the purpose of comparison and complementarity.

Discussion
This MMSR will ful ll important gaps in the current literature on the ECHO Model, as the rst to provide estimates on its effectiveness and consider simultaneously the in uence of key learning conditions on the development of competencies in healthcare professionals. As implementations of the ECHO Model greatly vary depending on the context, topic and targeted professional group, a better understanding of the conditions that contribute to competencies' development in healthcare professionals is crucial to inform the design and implementation of the model. Background Page 4/28 Innovation in continuing education: the ECHO Model In the current of change and uncertainty, healthcare professionals are expected to develop high levels of competencies in order to effectively manage complex health conditions and to respond to populations' needs [1]. In the health education sciences literature, the de nition of competency remains polysemous due to varying conceptions and underlying philosophical assumptions [2]. However, authors mostly agree that a competency: (1) requires the e cient mobilization and orchestration of a cluster of internal resources (e.g., knowledge, attitudes, values, skills, abilities) and external resources (e.g., material, human, organizational) in clinical practice; (2) is constantly contextualized to a speci c situation; and (3) evolves throughout a professional's lifetime [3][4][5][6][7][8]. Hence, a competency can be understood as complex and systemic knowledge-in-action to solve real-life situations effectively [9,10]. The development of competencies refers to a dynamic and ongoing process of learning and practice renewal requiring engagement at an individual and collective level [5,11,12]. The development of competencies in healthcare professionals is crucial in ensuring they practice within the full scope of their role and in improving patients' health outcomes [13,14].
In the healthcare professions, continuing education (CE) is recognized as an essential aspect of competency development [15][16][17]. CE can be described on a continuum, from informal learning experiences and practices to formal educational interventions held through a diversity of modalities and sources of media, in both academic and clinical practice settings [18]. In all cases, CE focuses on meaningful learning experiences that are conducive to the development of competencies in healthcare professionals.
In recent decades, a number of continuing educational programs using information and communication technologies (ICTs) have been developed to overcome barriers related to healthcare professionals' participation in CE activities (e.g., staff shortages, cost and travel time) [19][20][21][22][23][24][25]. Advantages of ICT-based programs include increased accessibility, lower costs and personalization compared to large-group, inperson instruction [26]. One of these ICT-based programs is the Extension for Community Healthcare Outcomes (ECHO) Model [27], a continuing tele-education program that provides ongoing support and clinical supervision to healthcare professionals in the management of complex and chronic health conditions.

Description and conceptual representation of the ECHO Model
Launched in 2003 at the University of New Mexico Health Center, ECHO aims to facilitate knowledge sharing, capacity building and expanded access to best practice care to reduce treatment disparities in underserved populations. The ECHO Model was rst developed under the name Project ECHO (© 2020, The University of New Mexico, Albuquerque, NM, USA; http://www.hsc.unm.edu/echo) to support primary care providers in rural and carceral settings in managing patients infected by the hepatitis C virus [28][29][30][31][32][33]. Since then, the model has been replicated for dozens of diseases and health conditions and it now operates at more than 100 academic medical health centres across multiple continents [7]. The ECHO Model involves establishing a network between front-line healthcare professionals located in remote areas-i.e., "spokes"-with a multidisciplinary team of specialists at academic medical centres-i.e., a "hub"-using video conference technology. The model typically includes a 6-to-12-month curriculum of weekly "ECHO clinics", in which a case-based discussion about a real patient situation and a short didactic presentation are held over two hours.
To offer a meaningful understanding of the ECHO Model, we developed a conceptual representation designed to explore the intended function of the model in context-meaning the examination of surfacelevel components with the conceptual learning conditions they were designed to foster. This conception was built from Cianciolo and Regehr's Learning Theory and Educational Intervention Framework [34], a layered perspective based on the premise of enabling a rich examination of the interplay between the pedagogic intention of an educational intervention (educational theories and principles) and its adaptation in a speci c context (educational methods, contextual and personal conditions). The authors claim that this examination helps to discern whether a given educational intervention "worked" as intended on an anticipated outcome and to draw any plausible conclusions about which components of the intervention this effect can be attributed to. Figure 1 depicts the proposed layered conceptualization of the ECHO Model and is summarized in the paragraphs below.
As shown in Fig. 1, our conception excavates the layers of the ECHO Model: methods at the surface, principles in the middle, and theories at the core. Together, these three layers depict the favorable educational conditions that must be established in a program for learning to occur. This layered classi cation illustrates that the ECHO Model has a unique identity, which preserves its essence despite being adapted in a speci c context. Also, the lack of clear delineations between layers re ects the absence of clear boundaries between them.
At the bottom of Fig. 1 [34], this foundational theory layer represents a contextindependent and idealized statement of the educational conditions that must hold for a given intervention to be what its designer claims it is-i.e., the pedagogic intention.
The middle layer of educational principles illuminates the general underlying postulates that are engaged throughout an intervention and clarify the structural learning aspects of the ECHO Model-i.e., assumptions of how participants will learn [39]. These principles may be adjusted to context, but nevertheless they re ect relatively stable approaches to learning. The top layer of the gure, the most context-sensitive, comprises educational methods that account for context, allowing each ECHO model implementation to be tailored to a speci c local setting (modes of delivery, functioning and characteristics of educational program). The educational methods used in the ECHO Model are summarized in three main categories, as shown in Fig. 1, each of them specifying the type of learning experience a participant might be exposed to-meaning the intervention components. Concrete examples of how these educational methods are delivered in ECHO implemented programs [28] can be found in additional le 1.
Our conceptualization also suggests that implementation comprises a complex and uncontrolled set of conditions that in uence the adaptation of a given educational program in a speci c context. Hence, the arrows surrounding Fig. 1 imply that personal and contextual conditions, as an overlay to the maintenance of the conceptual educational conditions of the ECHO model, may in uence its effectiveness, sometimes in unanticipated ways. Importantly, our conception re ects the fact that personal and contextual conditions are seen as processes of in uence on learning. Speci cally, personal conditions refers to in uences from an individual perspective, while contextual conditions range from the proximal in uences (interpersonal) to increasingly distal in uences (institutional, organizational, community and sociopolitical) [40,41]. In summary, the proposed conception supports the contention that a deeper investigation of the ECHO Model's effectiveness must be undertaken within a holistic and rich examination of the key learning conditions under which an effect is observed (or not observed)-i.e., the educational, contextual and personal conditions in uencing competency development.
Current knowledge gaps about the ECHO Model and added value of the proposed systematic review There is substantial evidence showing the impacts of the ECHO Model on healthcare professionals' learning outcomes [7,42]. For instance, increases in healthcare professionals' perceived knowledge and con dence in their ability to manage complex cases without referring to specialists, as well as improvements in their ability to perform new behaviours have been reported [7,42]. Moreover, there is evidence in support of the acceptability and feasibility of the model, notably for reducing healthcare professionals' feelings of isolation and regarding its cost-effectiveness [42]. The relevance of the didactic presentations' topics to practice, personal interest and motivation to learn, peer-to-peer interactions and positive reinforcement were reported as favorable conditions in extending healthcare professionals' knowledge and skills [7,43]. Additionally, mentorship or supervision led by a quali ed and competent team of healthcare professionals-the "hub"-in a helping environment has been found to support ECHO's participants in a di cult situation and allow the opportunity to re ect on their own practice from feedback and positive reinforcement [43].
While Project ECHO has been successful at expanding its scope and scale internationally, there remains a paucity of evidence regarding the effectiveness of the ECHO Model on the development of healthcare professionals' competencies [44]. Considering that the model has been implemented in diverse settings, each implementation bringing variations to the original Project ECHO design, and that the model is intended for a wide range of clinical conditions and populations, our current understanding of the key learning conditions that may contribute to its success is limited. A second contributor to this knowledge gap is substandard reporting of implemented ECHO-a liated programs' components in primary studies (such as number and duration of intended and actual videoconferencing sessions conducted, professions of participants, retention rates, targeted topics in didactic presentations and adaptations in educational methods) [44]. In order to address these gaps, it has been suggested that there is a need to not only assess the effectiveness of the ECHO Model on the development of healthcare professionals' competencies, but also to examine in which key learning conditions the model is more effective [44].
A preliminary search of MEDLINE, PROSPERO, the Cochrane Database of Systematic Reviews and the Joanna Briggs Institute (JBI) Database of Systematic Reviews and Implementation Reports was conducted to identify potentially relevant reviews. Only two systematic reviews focusing on the impact of the ECHO Model on healthcare professionals' learning outcomes and patients' health outcomes were found [7,42]. To the best of our knowledge, no previous or in-progress reviews have focused on investigating the in uence of key learning conditions on the effectiveness of ECHO-a liated programs on the development of competencies in healthcare professionals.

Aim and research questions
The aim of this mixed methods systematic review (MMSR) is to identify, appraise and synthesize the available quantitative (QUAN) and qualitative (QUAL) evidence regarding the effectiveness of the ECHO Model and the in uence of key learning conditions on the development of competencies in healthcare professionals. This systematic review seeks to answer the following three research questions:

Methods
This systematic review protocol was developed in accordance with the JBI methodology for MMSR [45], and is reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols (PRISMA-P) 2015 checklist [46,47] together with the PRISMA 2020 updated guidance [48] (see additional le 2). The review protocol was submitted to the PROSPERO database on October 21, 2020 (ID Number: 197579) and is pending registration.

Approach
We will use a convergent segregated approach [45,49] to extract and synthesize data from the QUAN, QUAL and MM included studies (see Fig. 2). With this type of design, the QUAN and the QUAL extracted data will rst be analyzed separately using different synthesis methods (QUAN descriptive statistics and synthesis, intervention effect estimates and QUAL thematic synthesis) and then the ndings of both syntheses will be merged for the purposes of comparison and complementarity [50].

Eligibility criteria
The eligibility of studies will be screened based on the following criteria structured according to the PICO (Population, Intervention, Comparator, Outcomes) mnemonic for the QUAN component, and the PICo (Phenomenon of Interest, Context) mnemonic for the QUAL component [45].

Population
We will include studies conducted with healthcare professionals, regardless of their profession group (such as community health providers, long-term care providers, pharmacists, physicians, psychologists, primary care physicians, nurse practitioners and social workers) and area of practice (such as family medicine and general practices, geriatric care, addiction and psychiatric services, pain management and pediatric care). We will exclude studies that involve pre-licensure healthcare professional students only, since entry-to-practice competencies may differ from the standards and levels of competencies in clinical practice.

Intervention
For inclusion in the QUAN component of this review we will consider studies of implemented a liated ECHO programs targeting healthcare professionals, where the effectiveness of the program was investigated. We de ned ECHO programs as "a technology-enabled learning model in which a mentor with specialized knowledge and expertise provides interactive and case-based guidance to a group of mentees for the purpose of strengthening their skills and knowledge to provide high-quality healthcare" [7]. The following six inclusion criteria will be used for considering programs as "ECHO-a liated": (1) using a technology-enabled platform (videoconferencing sessions); (2) having a health-focused objective; (3) implementing a hub-and-spoke framework with generalists in one or more locations (spokes) and specialists at a different location (hub) with a ratio of more than 1:1; (4) using case-based learning (case presentations and discussions); (5) using interactive mentorship; and (6) including a didactic component. Technology-enabled learning models that are not "ECHO-a liated" will be excluded. We will include studies of implemented ECHO programs addressing any type of health conditions and topics (e.g., chronic pain, co-occurring disorders of mental health and substance use disorders, delirium, diabetes, infection diseases).

Comparator
We will consider for inclusion studies with all types of comparator(s), including active and passive comparators.

Outcomes
The New World Kirkpatrick Model [51], an evaluation model for educational programs that is frequently used in the healthcare professions, was chosen to operationalize and categorize the outcomes of this systematic review. The model includes the following four levels of outcomes: (1) reaction-i.e., the degree to which participants nd the educational program favorable, engaging and relevant to their practice; (2) knowledge-i.e., the degree to which participants acquire the intended knowledge based on their participation in the educational program; (3) behaviour-i.e., the degree to which participants apply what they learned in the educational program in their practice; and (4) results-i.e., the degree to which targeted outcomes occur as a result of the educational program.
Based on this categorization and the focus given to the development of competencies in healthcare professionals, the QUAN component of this review will consider studies that include at least one of the following critical or important [52] outcome measures:

Critical outcome
Competency development (i.e., the e cient mobilization and orchestration of a cluster of internal resources in clinical practice): studies assessing changes in healthcare professionals' competencies on the basis of their participation in ECHO will be considered for inclusion. This may include objective outcome measures (e.g., simulation, observations in educational settings) or subjective outcome measures (e.g., self-reported competence, perceived competence, self-reported con dence, self-reported abilities, perceived skills, self-e cacy) of competencies (or overall competence) [53].

Important outcomes
Reaction (Kirkpatrick's Level 1): healthcare professionals' views and reactions to ECHO including outcome measures of satisfaction (e.g., general satisfaction, program acceptability, perceived usefulness, perceived bene ts) and participation (e.g., number of online participations and degree of retention). Results (Kirkpatrick's Level 4): changes in patients' health due to changes in the practice behaviour of healthcare professionals participating in ECHO, including objective outcome measures (e.g., measures recorded in patient charts or administrative databases and access to care) or subjective outcome measures (e.g., measures from patient self-reports) of health indicators. The health indicators targeted for this review will include outcomes such as health behaviours, health status and wellbeing, including physical and psychological health, social functioning and treatment outcomes.

Phenomena of interest
For the QUAL component of this review, we will consider for inclusion studies in which conditions in uencing-positively or negatively-the development of competencies in healthcare professionals participating in ECHO are investigated. In line with our conceptual representation of the ECHO Model, these conditions may include educational (educational intervention components), personal (e.g., sociodemographic, motivation, engagement, ability to use technology, anterior knowledge, satisfaction regarding programs and workplaces, openness to change, attitude and feeling of social recognition) and contextual conditions (interpersonal, institutional, organizational, community and sociopolitical in uences) of learning.
It may also include the bene ts and challenges of the program (e.g., resources constraints, change in patient health, integrating new knowledge in work practices, challenging clinical situations) in the views of ECHO participants (e.g., experiences, perceptions, perspectives). Studies exploring factors in uencing competency development, facilitators and barriers to learning, knowledge transfer in practice (e.g., changes in clinical practices, behaviours or interventions) and lifelong learning processes (e.g., developmental stages, transition, unexpected or critical moments in learning processes) will be considered as well.

Context
For inclusion in this review (QUAN and QUAL components), we will consider studies conducted in any type of clinical setting (e.g., ambulatory clinics, community health centres, home care, hospitals, longterm care facilities, primary care services), geographic location (e.g., rural or remote areas, urban located services and carceral healthcare) or country.

Type of studies
This review will consider QUAN, QUAL and MM studies. Regarding QUAN studies, we will include both experimental studies (randomized controlled trials [RCTs], cluster RCTs, crossover RCTs) and quasiexperimental studies (e.g., non-randomized controlled trials, cluster non-randomized controlled trials, cohort study with control group). QUAL studies will include designs such as phenomenology, grounded theory, ethnography, narrative inquiry, interpretative description, exploratory and action research. All types of QUAL data sources will be included in this review (e.g., individual semi-structured interviews, observations, eld notes, focus groups). MM studies will be considered for inclusion if the QUAN or the QUAL or both components meets the inclusion criteria mentioned above. All types of MM designs will be considered for inclusion (e.g., convergent, sequential exploratory, sequential explanatory). This review will be limited to empirical studies in peer-reviewed journals as well as in the grey literature. Only full-text papers of English or French-language studies will be included. Case reports, study protocols, discussion papers, editorials and knowledge synthesis papers (e.g., MMSR, narrative reviews, rapid reviews, realist reviews, systematic reviews, scoping reviews) will be excluded. We will consider for inclusion studies published from 2003 onwards as the initial pilot-ECHO program was launched in 2003 [32].
Search strategy for identi cation of studies A systematic search strategy was developed in consultation with a scienti c health librarian (DZ) and was reviewed by a second librarian. The search strategy was built with the objective of locating studies on ECHO-a liated programs exclusively. Therefore, the search combined speci c words and expressions related to the ECHO Model (e.g., Extension for community healthcare outcomes, ECHO, Project ECHO, SCAN-ECHO, TeleECHO). Given the absence of standardized indexing, we exploded the ECHO speci c terminology with search term groups covering the following three domains: (1) healthcare professionals (population); (2) technology-enabled learning model (intervention); and (3) hub-and-spoke model linking specialists with general healthcare professionals (context). The ECHO-speci c terminology and the search terms used for each domain were developed based on a previous systematic review on the impact of the ECHO Model [7]. A pilot search was executed using the search terminology to ensure its allinclusiveness and the accuracy of the original articles retrieved.
The following bibliographic databases were searched: CINAHL COMPLETE (from 1937 onwards), All EBM Reviews (from 1991 onwards), EMBASE (from 1974 onwards), MEDLINE (from 1946 onwards) and PsycINFO (from 2002 onwards). The search strategy was translated and adapted for each database using controlled vocabulary (MeSH, EMTREE and others) and free text searching. A publication date lter was applied to nd the articles published from 2003 onwards. Additional le 3 presents the complete search strategy.
A forward citation tracking procedure-i.e., search articles that cited the included studies-will also be performed in Google Scholar. Duplicates will be removed with EndNote X9 (© 2020 Clarivate Analytics, Boston, MA, USA; www.endnote.com) using a Bramer method for de-duplication of database search results for systematic reviews [54]. Automated search updates will be set up in each database to ensure the inclusion of the latest publications in the eld of the ECHO Model. Sources of unpublished studies and grey literature will include ProQuest Dissertations and Theses and DART Europe E-theses Portal. For QUAN studies only, ClinicalTrials.gov and WHO International Clinical Trials Registry Platform will be searched via the Cochrane Library. The list of references of existing reviews on the ECHO model [7,42] will be manually scrutinized.

Study selection, appraisal, and data extraction
A three-stage process including study selection, appraisal and data extraction will be followed. Each stage of the process will be conducted by teams of two independent reviewers. Teams will be formed based on the experience of each reviewer in a eld (e.g., screening titles and abstracts, conducting a systematic review of effectiveness, assessing heterogeneity, coding studies, using qualitative research software).
During the selection of studies, quality appraisal and data extraction, the study authors will be contacted for additional information regarding eligibility criteria if necessary. Also, any disagreements that arise between the reviewers will be resolved through discussion until consensus is reached. In the event of a persistent disagreement, a third reviewer will be convened to make a decision.

Selection of studies
Following the search, all identi ed records will be uploaded into the Covidence systematic review software (© 2019 Veritas Health Innovation Ltd, Melbourne, Australia; www.covidence.org). Reviewers will rst independently screen titles and abstracts according to eligibility criteria. Then, full-text articles of all studies deemed eligible will be retrieved and assessed in detail against the eligibility criteria by two independent reviewers. The results of the search will be presented in a PRISMA ow diagram [48].
Excluded studies with reasons for exclusion will be reported in table form as well.

Assessment of methodological quality
All included studies will be assessed by two independent reviewers for methodological quality using the The MMAT was speci cally developed to assess the methodological quality of various study designs, including MM studies, and proposes a list of 25 criteria based on ve categories of empirical studies. In the current literature on critical appraising tools, authors discourage using an overall numerical score since the latter does not shed light on any methodological issues and provides equal weight to all criteria [52,56,58]. The MMAT was revised in accordance with these recommendations and therefore offers guidance on presenting details of the ratings for each criterion of included studies.
All studies, regardless of their methodological quality, will undergo data extraction and synthesis. The results of critical appraisal will be reported in narrative form and will also be taken into consideration when discussing the nal integrated ndings of the review. A detailed presentation of the ratings for each criterion of the included studies will be reported in table form.

Data extraction and management
Based on the JBI standardized QUAN [59] and QUAL [60] data extraction tools, two separate data extraction forms were developed speci cally for this systematic review. These data extraction forms will be iteratively validated by the entire team of reviewers to ensure their completeness and clarity. Before data extraction, the forms will be tested on a total of six randomly selected articles from the search strategy (two studies of QUAN method only, two studies of QUAL method only and two studies of MM) and amended accordingly.
For the QUAN component, data will be extracted from QUAN and MM studies (QUAN component only) included in the review by two independent reviewers and will be managed with the Covidence systematic review software (© 2019 Veritas Health Innovation Ltd, Melbourne, Australia; www.covidence.org). Data extraction will include the following speci c details: First and corresponding author(s) information, publication year and country.
Study funding source(s).

Study objective(s) and design.
Study population, and health care setting.

Time of study, method(s) of data collection.
Planned and actual sample sizes.
Participation and response rate.
Results of signi cance to the QUAN review objective (outcomes measures of competency development and Levels 1 to 4 of Kirkpatrick's Model), including details on outcomes (de nition, time points measured, missing data) and measurement (name of tool, unit of measurement, scales).
In addition, we will use the Guideline for Reporting Evidence-based practice The QUAL data extracted will include speci c details about rst and corresponding author(s), study funding source(s), publication year and geographical location. We will also extract data regarding study aim and research question(s), population(s), context, philosophical or theoretical foundations, methodology and method(s) for data collection. Study results that are relevant to answer the QUAL question of this review on key learning conditions will be extracted for further analysis. Where possible and if appropriate, we will extract gures, images or schemas and their related summarized text as well.
Data extraction of QUAL studies and MM studies (QUAL component only) included in the review will be performed by two reviewers, with each of them subject to repeated independent readings.

Synthesis and integration of QUAN and QUAL ndings
In accordance with the JBI convergent segregated approach to MMSR [45], a fourth-step procedure will be performed at the synthesis and integration stage. This procedure will involve separate QUAN and QUAL synthesis followed by integration of the QUAN ndings and QUAL ndings. Table 1 summarizes this procedure and is detailed in the sections below. Second step: Quantitative synthesis

Summary intervention effects and meta-analyses
To evaluate the effectiveness of the ECHO Model on competency development in healthcare professionals, we will synthesize all intervention effect estimates for each outcome of interest using meta-analyses. Meta-analyses will be undertaken using the Cochrane Collaboration Review Manager RevMan version 5.4.1 (© 2020 The Cochrane Collaboration, London, UK; www.training.cochrane.org). All results will be expressed with 95% con dence intervals (CI). Statistically signi cant results will be de ned with a two-sided alpha of 0.05.
At least two studies will need to contribute to a meta-analysis for it to be conducted. To minimize clinical heterogeneity, we will favor the pooling of studies in which the comparators (active or passive) and the outcomes of interest (objective or subjective outcome measures) are similar. Based on existing systematic reviews on the ECHO Model [7,42], we currently expect that all outcomes of interest were mostly measured as continuous variables and using different instruments. As such, we will use an inverse variance approach for continuous outcomes and random effect models in all meta-analyses. All results will be expressed as standardized mean differences. We will interpret the signi cance of effect sizes using Cohen's classi cation (< 0. In each meta-analysis, we will assess statistical heterogeneity, which is the inconsistency in intervention effect estimates between studies that is not due to chance, using the X 2 test and the I 2 statistic. A statistically signi cant p value at the X 2 test or an I 2 statistic > 50% will be considered as indicative of high statistical heterogeneity. Where statistical pooling is not possible the ndings will be presented in narrative form including tables and gures to aid in data presentation.

Subgroup and sensitivity analysis
We plan to carry out subgroup analyses to investigate potential statistical heterogeneity sources when four or more studies are included in a single meta-analysis (two in each subgroup). If there are a su cient number of studies, we will explore the following potential effect modi ers: Intervention: topic(s) or health condition(s) targeted in the program; Population: professional group(s) participating in the program; Context: practice setting of participating healthcare professionals.
Sensitivity analyses will also be conducted to exclude studies deemed at high risk of bias.

Assessment of reporting biases
Based on Cochrane recommendations [52], we will assess reporting biases using funnel plots if more than 10 studies are included in a single meta-analysis. We will follow the guidelines regarding funnel plot asymmetry as described in the Cochrane Handbook for Systematic Reviews of Interventions version 6.1 [52]. To ensure consistency and coherence between each reviewer's coding, we will rst use a deductive approach to data reduction (organization of the mass of QUAL data and discarding of irrelevant data) [66]. To achieve this, a set of three conceptual categories that depict the nature of key learning conditions (educational, personal and contextual conditions) [34] will serve as an initial path for organizing the QUAL raw data. However, this stage of the review will remain somehow iterative as these broad categories may be modi ed in case of similarity and recurrence of data that do not match the conceptual categories.
During this process, each study will be read and reread to enable the reviewer to familiarize themselves with the study results and the methods used. Then, two independent reviewers will scrutinize the results of each study for meaningful units with regards to the QUAL review question. Data will be coded line by line to assign the content of each line or sentence under one of the established conceptual categories. Any changes or differences arising from the coding system will be resolved with two reviewers and we will bring in a third reviewer in case of a persistent disagreement or uncertainty. Irrelevant information will be kept in a in an independent category to ensure that we have access to it later if required, as unexpected ndings may call for reexamination of some data previously considered unnecessary.
In the following step, the lead review author (GC) will examine each code to identify recurrence or patterns in the data. Subsequently, a rst set of themes and subthemes will be created, through assembling the data and displaying the data into the form of a hierarchy. These initial sets of themes and subthemes will then be subjected to a synthesis by examining the themes within and across each study, based on similarity in meaning. Themes will be re ned and renamed with the review authors (JP and GR) until the synthesized ndings provide an answer to the QUAL review question. The QUAL ndings will be presented in narrative form including each emergent theme with supporting quotes drawn from the included studies.
Fourth step: Integration of quantitative and qualitative evidence At the nal stage, ndings of each synthesis will be compared and contrasted to produce an overall con gured synthesis and interpretation of the effectiveness of the ECHO Model and the in uence of key conditions on competency development in healthcare professionals. This will involve QUAN and QUAL evidence being juxtaposed for the purpose of interrogating simultaneously the extent to what, and under which key learning conditions, the ECHO Model is most effective in developing competencies in healthcare professionals. Integration of both sets of evidence will be attained by performing a comparison integration strategy [67], which will assist in considering where the QUAN and QUAL ndings of the review agree (correspondence, similarities), offer complementary information or are in contradiction (disagreement or dissonance).
The comparison integration strategy will be performed using a matrix approach [68,69]. A matrix will allow us to closely map the ndings of the review on a same table and to conduct a side-by-side comparison to identify matches and mismatches [69]. We will organize the matrix in a theme-by-effect size con guration [70]. This juxtaposition will assist the review team in exploring heterogeneity between the QUAN ndings (educational components and effect size measures) and in interpreting, on the basis of the QUAL ndings (themes), under which key learning conditions some ECHO programs were effective -or more effective-and some were not. Where juxtaposition is not possible, the ndings will be presented in narrative form. An example of the planned matrix inspired by Candy et al MMSR [71] can be found in additional le 4.
To support the use of the review ndings in informed decision-making, we will apply the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach [72] jointly with the GRADE-CERQual (Con dence in the Evidence from Reviews of Qualitative research) approach [73], to assess and transparently communicate how much con dence can be placed in the cumulative evidence.

Discussion
This protocol outlines the process to be undertaken for a MMSR aiming to gather evidence on the ECHO Model. This review is necessary to establish in which key learning conditions the model is most effective in developing competencies in healthcare professionals. The proposed review was developed in accordance with the current recommendations in the literature on MMSR [45,49,68,69,74], entailing a rigorously and thoughtfully articulated convergent segregated design. The evidence from QUAN, QUAL and MM studies drawn from the literature will be utilized to illustrate the best ways to implement the ECHO Model as an effective intervention and will be useful in guiding future research and educational practice in this area. These ndings may be applied internationally across all disciplines. It is expected that the review ndings will be valuable to researchers, academicians and other stakeholders (e.g., policymakers, healthcare professionals, educators) in improving aspects of further implementations of the ECHO Model.
To the best of our knowledge, no review has selected, appraised and synthesized evidence from QUAN (and the QUAN component of MM studies) and QUAL (and the QUAL component of MM studies) studies for the overarching objective of comparison and complementarity between both strands of ndings. We therefore believe that the synthesis and integration of evidence from a range of diverse methodologies in a systematic way should help shed new light on the ECHO Model. Furthermore, no review has focused on extracting evidence on how ECHO-a liated programs are implemented in various contexts, sometimes re ecting the particular health conditions and learning objectives addressed by a given program [44]. This MMSR protocol was developed in order to elucidate these variations of the ECHO Model in practice and to provide a clear understanding of which critical components may lead to better outcomes in healthcare professionals and patients.
This MMSR has some potential limitations. First, the search strategy of this review was built with the objective of including English and French-language empirical studies exclusively, which means that other sources of existing information and language will be excluded from the outset. While other forms of evidence might be an interesting addition to the state of knowledge in terms of comprehensiveness, this decision was made on the grounds that there is a considerable amount of eligible and primary studies. Indeed, based on a previous review that focused on a similar body of literature [7], 52 empirical studies from peer-reviewed articles were included by these authors.
Second, all studies meeting the eligibility criteria will be included in the review, meaning that no studies will be excluded on the basis of low methodological quality. Since risk of bias and lack of rigour are primary concerns when undertaking a MMSR [45], all included studies will be critically appraise using the latest version of the MMAT. To ensure transparency and enhance rigour, a table clearly indicating the ratings for each criterion of all included studies will be developed using the MMAT and will be reported in full.
Third, although this review will be restricted to ECHO-a liated programs only in order to limit clinical diversity, we anticipate that programs' characteristics of included QUAN studies will vary in terms of population (targeted professional groups), topics (targeted health condition or disease), participants' exposure to the intervention (frequency and duration of the program) and context of care delivery (e.g., community services, primary care and hospital). However, the richness of the QUAL ndings on learning, individual and contextual conditions that we expect to gather will aid in explaining any potential variation in the program effect on our critical and important QUAN outcomes. Another downside of including studies of ECHO-a liated programs exclusively is that the ndings generated from this review may not be generalizable to other technology-enabled learning models. However, we believe that these models, which differ in their identity and layers, do not meet the aim and scope of this review.
Finally, considering that this review builds on a MM approach, a potential challenge to consider is the complexity associated with the incorporation of evidence derived from a range of research designs into one single synthesis [74]. To address any practical issues during synthesis and integration, the process will follow the available guidance for MMSR, and the overall interpretation of the evidence will be reviewed independently by each member of the review team. Full immersion of the lead author in the entirety of the evidence base, extended re ection with potential explanations in case of divergences between QUAN and QUAL ndings, and transparency in the reporting of the integration process will provide greater insight into and understanding of the evidence.
In conclusion, the dissemination plan will include standard and innovative (e.g., website portals, social media, Project ECHO Networks, knowledge exchange events with clinical administrators, healthcare professionals, key stakeholders) means of ensuring that the ndings of the review are communicated regionally, nationally and internationally, and that they are accessible to a broad audience. Results of this MMSR-including the QUAN, QUAL and MM ndings-will be disseminated through publication in relevant peer-reviewed journals, and will be presented at suitable fora including academic, scienti c and professional conferences in the eld of ICTs and CE in the health professions. The strengths, limitations, and recommendations to improve the development and evaluation of further implementations of ECHOa liated programs will be discussed in the completed review. Availability of data and material

List Of Abbreviations
No data are yet available.

Competing interests
Review authors have no competing interests.

Funding
This review received no speci c grant from any funding agency in the public, commercial or not-for-pro t sectors.
Authors' contributions GC conceptualized the review, designed the review, drafted the article and is the guarantor of the review. JC, GF and MAMC conceptualized the review, designed the review, and drafted the article. All other authors helped design the review, draft and revise the article. All authors read and approved the nal manuscript.

Figure 1
Layers of Cianciolo and Regehr's Framework [34] applied to the ECHO Model of continuing tele-education