This scoping review will be conducted according to recommendations specified by Arksey and O’Malley (14), and enhanced by Levac and colleagues (15). The conduct and reporting of this scoping review with be further guided by the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guide review reporting (16).
Stage 1: Identification of the research question
We designed the research question to be broad in order to capture all potentially relevant information sources, as recommended by Arksey and O’Malley (14). This scoping review aims to answer the question: “what is the state of knowledge on methods to incorporate and evaluate citizen engagement in health research?”. The proposed research question provides direction to the subsequent steps, and defines the scope of inquiry with respect to population, concept, and outcomes of interest (14).
For the purpose of this scoping review, our target population “citizens” will be defined as consumers of health services, informal caregivers, advocates and representatives from community organizations, and members of the general public. Our target concept is engagement or participation in health research. For the purposes of this scoping review, we will adopt the CIHR definition of health research, which includes research in the biomedical, clinical, health systems and services, and social, cultural, environmental and population health fields (17). We recognize that engagement may take many forms, including but not limited to: priority-setting, planning, decision-making, research conduct, implementation, evaluation, or dissemination. Additionally, citizen engagement may occur when citizens are joint grant holders or co-applicants on research grants, help to identify research priorities, hold membership in a project advisory or steering group, comment and develop research materials, interact with research participants, and/or help carry out research activities.
Stage 2: Identification of relevant studies
As recommended by Arksey & O’Malley and Levac and colleagues (14, 15), we will identify relevant literature using a pre-determined plan for data sources and search strategy, including search terms, languages, and dates of search. The search strategies will be designed to maximize the comprehensiveness and breadth of the search, while considering time and personnel workload as limiting factors (14, 15).
A search strategy will be developed by the study team, including a medical librarian (DLL), who have led multiple initiatives to develop, adapt, and evaluate approaches to incorporate citizen engagement into research (18–22). The following databases will be searched from inception: MEDLINE (1879-present), EMBASE (1947-present), Cochrane Library (1996-present), CINAHL (1961-present), PsycINFO (1927-present), Scopus (1970-present), and Web of Science (1964-present). Our search will be broad to encompass relevant terminology in this area, including subject headings and keywords and relevant synonyms related to three concepts: (1) citizens (e.g., community member, lay person, public, stakeholder), (2) engagement (e.g., collaboration, engagement, participation), (3) health research (e.g., biomedical research, health research, public health research). No exclusion criteria will be placed on language, though we will exclude studies published prior to the year 2000. Prior to implementation, the search strategy will be independently reviewed using the Peer Review of Electronic Search Strategies (PRESS) checklist (23). A draft search strategy (MEDLINE) is shown in Additional File 1.
A targeted search of the grey literature will also be conducted, searching relevant local, provincial, national, and international organizations’ websites and related scientific or national funding organizations (i.e., OpenGrey, Trip, Involve, CIHR, Patient-Centered Outcomes Research Institute [PCORI]) for additional studies, frameworks, and reports on citizen engagement. Finally, the reference lists of included studies and related systematic reviews will also be screened to identify potentially relevant literature in health research.
Stage 3: Study selection
Following developing a strategy to identify studies, we will screen and select relevant studies for inclusion in the scoping review. As recommended by Levac and colleagues, we will develop inclusion and exclusion criteria a priori, with additional meetings with the study team to refine the study selection process at the beginning, midpoint, and endpoint of the citation screening process in case there are any unforeseen factors to be considered (15).
Studies will be eligible for inclusion if they: (1) are primary (e.g., observational or interventional studies) or secondary (e.g., systematic or scoping reviews) research, frameworks, reviews, or reports, (2) report citizen engagement in health research (including biomedical, clinical, health systems and services, and social, cultural, environmental and population health, as defined by CIHR) and (3) report method(s) to conduct, measure, or evaluate citizen engagement. No restrictions will be placed on language or study design. Studies published prior to the year 2000 will be excluded.
Retrieved articles will be imported to Covidence (Veritas Health Innovation, Melbourne, Australia) for title & abstract screening, completed independently and in duplicate by two reviewers. Reviewers will pilot screen the titles and abstracts of 50 articles to ensure consistency of inclusion and exclusion criteria and once a Kappa coefficient of inter-rater agreement ≥ 0.8 is achieved, proceed to screen the remaining articles. If one reviewer indicates an article as potentially relevant at the title & abstract screening phase, the article will proceed to full-text review to ensure inclusivity. Following title & abstract screening, the full- text of selected studies will be screened independently and in duplicate by two reviewers. Reviewers will pilot screen the full text of 20 articles, and proceed to screen the remaining articles once a Kappa coefficient of inter-rater agreement ≥ 0.8 is achieved. Both reviewers must agree on inclusion status and reason for exclusion, if relevant. Any disagreements between the reviewers will be resolved by discussion or the involvement of a third reviewer, if required.
Stage 4: Charting the data
We will chart the data according to Arksey & O’Malley and Levac’s recommendations (14, 15), beginning with defining the information to be extracted from the studies by developing an initial data charting form and then refining this form iteratively and through regular discussion with the study team.
The study team will develop a standardized data extraction form in Microsoft Excel (version 16.29.1). The data extraction form will be piloted by two reviewers with ten included studies, and revised as needed. Once the final data abstraction form is developed, all relevant articles will then be abstracted independently and in duplicate by two reviewers.
Extracted variables in the initial data extraction form will include study characteristics, participants, type and goals of citizen engagement, engagement frameworks and other strategies, engagement uptake, and findings and outcomes of citizen engagement in the context of health research. A draft list of variables to be extracted is shown in Table 1.
Draft list of variables to be extracted
List of authors (surnames)
Year of publication
Publisher (e.g., journal, institution)
Research area (e.g., health sciences, social sciences, and natural sciences and engineering)
Framework OR Model OR Report
Primary study OR Secondary research
Characteristics of intended participants
Population of focus (patient, family, healthcare provider, researcher, knowledge-user, decision-maker)
Definition of citizen engagement
Type and goals of engagement (e.g., priority setting, decision-making, research conduct, implementation, evaluation, dissemination)
Engagement frameworks used
Results and Discussion
Findings and outcomes of citizen engagement
Barriers to citizen engagement
Facilitators of citizen engagement
Benefits of citizen engagement
Challenges and risks of citizen engagement
Strategies to mitigate challenges and risks of citizen engagement
Author-stated conclusions (if applicable)
Stage 5: Collating summarizing and reporting the results
We will collate, summarize, and report the results according to the Arksey & O’Malley and Levac guidelines, which recommend undertaking synthesis of results by i) analyzing the data, ii) reporting results, ii) and evaluating their meaning (14, 15). More specifically, these guidelines suggest that researchers execute a detailed thematic analysis of their data guided by qualitative analytical techniques before reporting results in a suitable form such as a narrative synthesis, a table of barriers/facilitators or knowledge gaps, or a framework depending on the goal of the review (14, 15). Arksey & O’Malley and Levac further recommend that researchers aim to contextualize their findings within existing literature and research, practice, and policy (14, 15).
We will use a PRISMA-ScR checklist to guide reporting and the flow diagram to report the number of unique articles identified, excluded articles, reasons for exclusion, and included articles in the final scoping review (16). We will present a descriptive summary of characteristics of the included documents, and the characteristics of the intended participants or audiences for these documents. If applicable, we will conduct a narrative synthesis of extracted data variables, including, but not limited to: definition of citizen engagement, goal of citizen engagement, impact of citizen engagement, outcome measurement, author stated conclusions, barriers, facilitators and benefits, risks, and challenges of citizen engagement.
Stage 6: Consultation
As recommended by Arksey and O’Malley (14), we will involve citizens (NF, BS) in study conception and design as well as interpretation and contextualization of the data. This stakeholder consultation (i.e., citizen engagement) will allow for the incorporation of perspectives and insights beyond the literature and is necessary to improve the rigour and uptake of findings (14, 15).
Results from this scoping review will be synthesized to inform the future development or refinement of a framework for incorporating meaningful citizen engagement in health research. The barriers, facilitators, risks and challenges to citizen engagement identified through this scoping review will be used by our team in future framework development or refinement. Specific focus will be placed on evaluation of citizen engagement in order to improve upon existing methods of incorporating citizen engagement in health research. Once developed, a draft of this framework will be circulated through surveys to stakeholders including health researchers, decision and policy-makers, and citizens who indicate interest to solicit feedback. Focus groups will be conducted to discuss specific items in the framework which will lead the reiterative refinement of this work. While the limitations to citizen engagement have been outlined before (24–26), this applied framework will chart barriers and facilitators to citizen engagement systematically, allowing our team and stakeholders to effectively utilize this knowledge to initiate, perform and carry out an evidence-based process of framework building.