Study Design
A mixed methods systematic review will be conducted following the Joanna Briggs Institute (JBI) methodology for mixed methods reviews [18]. Mixed methods reviews allow for the inclusion of both qualitative and quantitative study designs and are best suited to broad research questions or phenomena of interest [18]. The review has been registered online with PROSPERO (CRD42022293348). To ensure transparency of search results, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines will be used to report the mixed methods review results [19], while the PRISMA extension for review protocols (PRISMA-P) checklist guided the reporting of this protocol paper [20].
Inclusion criteria
Following JBI guidelines for mixed methods systematic reviews [18], this review will use a PICo (Population; Phenomenon of Interest; Context) format for question development and to determine inclusion criteria. This review will aim to answer the following research question: How do health care organizations implement and evaluate the transformation of learning health systems?
Population
This review will consider studies that include any mention of LHS, including rapid learning systems, rapid learning health care, learning health care systems and learning health systems. Due to the inconsistent terminology around LHS in the literature, only studies utilizing one of these terms will be included. Studies which attempt to describe LHS, without using this terminology will be excluded.
Phenomenon of Interest
This review will consider studies reporting on implementation strategies and outcome measures associated with the adoption of LHS across health care settings. Implementation strategies include any procedure or method to implement, assess or evaluate the uptake of LHS in any health care setting. The ERIC taxonomy of 73 strategies will be used to determine what classifies as an implementation strategy [14]. This review will also consider any implementation outcome measures described in studies.
Context
This review will consider studies that focus on any health care setting. This includes hospitals, academic medical centres, primary care clinics, community health centres, practice based health networks, health care organizations, and individual units or wards, that provide in-patient or out-patient services. Any country, size of health care organization and other setting characteristics are appropriate for this review. Any non-health care settings, such as academic centres, government or non-government organizations where care is not directly provided to patients, will be excluded.
Types of Studies
This mixed methods review will consider quantitative, qualitative and mixed methods study designs for inclusion. Mixed methods studies will be included if the individual qualitative and quantitative data can be extracted separately. Included studies will be limited to peer-reviewed, published, full text papers. No restrictions will be placed on language or year of publication. Conference proceedings, dissertations, commentaries, news articles and other grey literature sources will be excluded. Protocol papers will be excluded, but forward citation searching for any published intervention studies stemming from an identified protocol will be conducted. Similarly, reviews will be excluded but the reference list will be manually searched for relevant papers. The reference lists of all included studies will be manually searched for additional, relevant papers.
Search Strategy
A comprehensive search strategy was developed by a research librarian (MR). Key terms related to learning health systems, implementation and health care organizations formed the basis of an initial limited search of two databases, CINAHL and Medline. After reviewing terms from titles, abstracts and index terms and checking search results against target articles a final, more open search strategy was developed. An example search strategy for CINAHL can be found in Appendix 1. The search strategy will be adapted for each chosen database, which will include: CINAHL (EBSCOhost), Medline (Ovid), Embase (Elsevier), Nursing at Allied Health Database (ProQuest), PAIS (ProQuest) and Scopus (Elsevier). Boolean operators and MESH terms will be used as needed for each database search.
Study Selection
Following a search of all databases, retrieved titles will be uploaded to the data management software, Covidence (Veritas Health Innovation, Melbourne, Australia). Duplicates will be removed electronically. Three team members (MS, CC, JC) will independently screen all titles and abstracts, making decisions using the pre-determined inclusion criteria. Following title and abstract screening, the full texts of relevant papers will be sourced and assessed for inclusion by three independent reviewers (MS, CC, JC), using Covidence. A fourth reviewer will be consulted when conflicts arise between reviewers. Reviewers will discuss the suitability of the paper for inclusion, based on inclusion/exclusion criteria, until a consensus is reached. Papers that do not meet the study criteria will be excluded with reasons for exclusion reported in the 2020 PRISMA flow diagram [19].
Data Extraction
Following full text screening, data will be extracted from each included study using a standardized extraction form. Data extraction will be pilot tested by two reviewers on a sub-sample of studies, and then completed by one reviewer (MS). Data extraction will be verified by a second reviewer to ensure consistency and reliability of the data. Regular team meetings will be held throughout the data extraction phase to discuss any arising concerns with included studies.
This mixed methods review will follow a convergent integrated approach, where quantitative and qualitative data is extracted separately and then synthesized together [18]. Qualitative data will be extracted as themes and sub-themes as related to the research question. Data from qualitative studies will include findings related to LHS and implementation strategies. Evaluation measures and outcome data will also be extracted. This may include a description of how a LHS was implemented or qualitative findings on the LHS impact. Quantitative data will include descriptive or analytical findings related to the implementation and evaluation of LHS. This may include changes in health outcomes, impact on health system, or cost effectiveness. All studies will be extracted for study characteristics including year of publication, country, study design, description of LHS, types of implementation strategy(ies) employed, types of outcome measure(s) used, objective, study setting, population, sample size. An example of the data extraction tools can be found in Appendix 2. The data extraction tools will be modified as necessary during the data extraction process, as details about included studies are refined. The finalized data extraction tools will be included in the published systematic review.
Critical Appraisal
All studies will undergo critical appraisal using the relevant JBI critical appraisal checklist, based on study design [21]. Two independent reviewers will complete critical appraisal of all studies. The two scores for each study will be averaged and a final score will be reported as a percentage. When significant differences exist between appraisal scores, a third reviewer will be consulted until a consensus about each scoring item has been reached.
Data Synthesis and Triangulation
This mixed methods systematic review will use a convergent integrated approach to data synthesis, where quantitative and qualitative data are assessed separately first, followed by integration of data [18]. First, tables will be used to present the extracted data, categorized based on study design (quantitative, qualitative or mixed methods). An accompanying overview of all study characteristics will be presented in text. Next, a narrative synthesis will be provided for each study to describe the key findings and contextual data. This will include reporting similar findings together within each study design category. Implementation outcome data will be categorized using the ERIC taxonomy of 73 implementation strategies [22], Tierney et al.’s list of 10 implementation outcome measures [15] and Allen et al.’s LHS logic model components [13].
Following data synthesis, a triangulation protocol will be used to integrate the qualitative and quantitative findings, which will be presented as themes and sub-themes. A triangulation protocol is a detailed approach to examine meta-themes across findings from different data sources that have already been analyzed individually [23]. We will create a convergence-coding matrix to cross-tabulate the quantitative results with the qualitative themes [24]. To determine common patterns across the data, the review team will meet to discuss the convergence-coding matrix, including separate quantitative and qualitative findings and similarities and differences in outcomes. The resulting integrated data will be presented in a table and as a narrative synthesis.