Study selection
Figure 1 shows the study flow diagram which summarizes the selection process. Out of the 33,339 citations identified through the electronic databases search, 10 papers met our inclusion criteria. We excluded 898 articles based at the full text screening stage for the following reasons: not paper type of interest (n=49), not describing a prioritization approach (n=324), not about practice guidelines (n=525). We provide a detailed tabular description of each of the 10 included prioritization approaches in Supplementary file 4.
General characteristics
The general characteristics of the 10 distinct approaches for prioritizing guideline topics described in the papers are reported in Table 1. Most of the papers (7 out of 10, 70%) focused on guidelines for clinical practice (12-18); one paper proposed a prioritization approach that is applicable to the clinical, public health and health systems fields (8); and the two remaining papers proposed prioritization approaches respectively for World Health Organization (WHO) healthcare recommendations (19) and for public health guidelines (20). All of the identified prioritization approaches focused on prioritizing guideline topics; none on prioritizing guideline recommendation questions or outcomes. None of the approaches were specific to the update or adaptation of guidelines; all focused on the de novo development of practice guidelines.
Table 1: General characteristics of the approaches for prioritizing guideline topics
Paper
|
Lead entity
|
Target audience
|
Field (specific domain)
|
Focus of prioritization
|
Type of guideline development
|
Battista, 1995 (13)
|
Independent researchers
|
Canadian guideline developing groups
|
Clinical
|
Topics
|
De novo
|
Field, 1995 (14)
|
Institute of Medicine
|
Agency for Health Care Policy and Research
|
Clinical
|
Topics
|
De novo
|
McClarey, 1999 (16)
|
Royal College of Nursing (RCN)
|
RCN guideline developers
|
Clinical, nursing
|
Topics (e.g., hyperplasia, colon cancer, breast cancer, wound care, etc.)
|
De novo
|
Oxman, 2006 (19)
|
WHO Advisory Committee on Health Research
|
WHO entities developing guidelines
|
Health care
|
Topics or interventions
|
De novo
|
Ketola, 2007 (15)
|
‘Current Care’
|
Guideline developers
|
Clinical
|
Topics (e.g., benign prostatic hypertrophy, brain injuries in adults, atrial fibrillation, cataract, etc.)
|
De novo
|
Reveiz, 2010 (18)
|
Independent researchers
|
Guideline developers within developing countries
|
Clinical
|
Topics
|
De novo
|
Atkins, 2012 (12)
|
Independent researchers
|
Guideline developers in respiratory diseases
|
Clinical, respiratory diseases
|
Topics
|
De novo
|
Schünemann, 2014 (8)
|
Independent researchers
|
Guideline developers
|
Clinical, Public health
and Health systems
|
Topics
|
De novo
|
Reddy, 2014 (20)
|
Independent researchers
|
National Institute
for Health and Care Excellence (NICE)
|
Public health
|
Topic (e.g., sickle cell screening,
substance misuse,
water fluoridation, etc.)
|
De novo
|
Mounesan, 2016 (17)
|
Tehran University of Medical Sciences
|
Guideline developers
|
Clinical, family medicine
|
Topics (e.g., anemia, osteoporosis, indigestion/dyspepsia, pneumonia, etc.)
|
De novo
|
Development process
Table 2 shows steps of the development process of each of the 10 included approaches for prioritizing guideline topics. The two steps most frequently reported to be used in the development process were: reviewing the grey literature (e.g., databases of guideline developing organizations) (9 out of 10, 90%) (8, 12-19), and engaging various stakeholders (9 out of 10, 90%) (8, 13-20). Patient and public involvement was reported to be used in the development of only one prioritization approach (16). Conducting primary research was reported in the development of four out of the 10 approaches (40%) (13, 15, 17, 18). The primary aim of conducting this type of research was to rate the importance of the suggested prioritization criteria and to assess the participants’ views regarding existing prioritization approaches in their respective organizations. Two studies followed all of the steps in the development process and were thus the most comprehensive and detailed (17, 18).
Table 2: Steps of the development process of the approaches for prioritizing guideline topics
Paper
|
Peer-reviewed literature
|
Grey literature
|
Consensus building
|
Ranking of proposed prioritization criteria
|
Pilot testing
|
Conducting primary research
|
Stakeholder involvement
|
% papers reporting the step
|
70%
|
90%
|
60%
|
50%
|
40%
|
40%
|
90%
|
Battista, 1995 (13)
|
P
|
P
|
|
P
|
|
P
Mailed survey
|
P
|
Field, 1995, 1995 (14)
|
P
|
P
|
P
|
|
|
|
P
|
McClarey, 1999 (16)
|
|
P
|
|
|
|
|
P
|
Oxman, 2006 (19)
|
P
|
P
|
|
|
|
|
P
|
Ketola, 2007 (15)
|
|
P
|
P
|
P
|
P
|
P
Phone interviews
|
P
|
Reveiz, 2010 (18)
|
P
|
P
|
P
|
P
|
P
|
P
Online survey
|
P
|
Atkins, 2012 (12)
|
P
|
P
|
|
|
|
|
|
Schünemann, 2014 (8)
|
P
|
P
|
P
|
|
|
|
P
|
Reddy, 2014 (20)
|
|
|
P
|
P
|
P
|
|
P
|
Mounesan, 2016 (17)
|
P
|
P
|
P
|
P
|
P
|
P Interviews
|
P
|
Aspects of prioritization
Table 3 shows the aspects proposed to be addressed when prioritizing guideline topics. Only one study highlighted the need to conduct prioritization during the various steps of guideline development, such as prioritizing the target audience, scope of guideline, questions of potential interest, effort of synthesizing evidence, recommendations, and recommendations for research (12). Six studies (60%) included steps to generate an initial list of topics (8, 12, 15-18). Table 4 represents the steps proposed for generating an initial list of topics when prioritizing guideline topics. All of the studies incorporated the use of prioritization criteria as an aspect of the prioritization approach. Most of the studies (9 out of 10, 90%) included the involvement of stakeholders as one aspect of prioritization (8, 12-19). Table 5 shows the proposed types of stakeholders to involve in prioritizing guideline topics and the method for their involvement. Three studies covered the highest number of aspects of prioritization, that is four out of the five aspects (8, 12, 17).
Table 3: Aspects proposed to be addressed when prioritizing guideline topics
Paper
|
When to conduct prioritization?
|
How to generate an initial list of topics?
|
What criteria to use?
|
What stakeholders to involve?
|
Documentation
|
% papers reporting the aspect
|
10%
|
60%
|
100%
|
90%
|
40%
|
Battista, 1995
|
|
|
P
|
P
|
P
|
Field, 1995 (14)
|
|
|
P
|
P
|
|
McClarey, 1999 (16)
|
|
P
|
P
|
P
|
|
Oxman, 2006 (19)
|
|
|
P
|
P
|
P
|
Ketola, 2007 (15)
|
|
P
|
P
|
P
|
|
Reveiz, 2010 (18)
|
|
P
|
P
|
P
|
|
Atkins, 2012 (12)
|
P
|
P
|
P
|
P
|
|
Schünemann, 2014 (8)
|
|
P
|
P
|
P
|
P
|
Reddy, 2014 (20)
|
|
|
P
|
|
|
Mounesan, 2016 (17)
|
|
P
|
P
|
P
|
P
|
Table 4: Steps proposed for generating an initial list of topics when prioritizing guideline topics
Study
|
Description
|
Battista, 1995
|
Not reported
|
Field, 1995 (14)
|
Not reported
|
McClarey, 1999 (16)
|
1. Collect data using questionnaire from RCN professional groups and other RCN databases.
2. Collect information on patient priorities from representative groups and the literature.
3. Group topics by themes and accept that some might be arbitrary.
|
Oxman, 2006 (19)
|
Not reported
|
Ketola, 2007 (15)
|
1. Need for a new guideline arises in a specialist society or other source.
2. PRIO-tool from the ‘Current Care’ web site (http://www.kaypahoito.fi) is used to make a topic suggestion to the ‘Current Care’ board.
|
Reveiz, 2010 (18)
|
A thematic team (experts in the field and methodological consultant) would suggest three to five clinical topics that could potentially be selected for developing a clinical practice guideline.
|
Atkins, 2012 (12)
|
1. Survey clinicians, experts, and patients for candidate topics.
2. Create a list of topics using formal or informal (e.g., review of other guidelines).
3. Allow stakeholders to comment on scope and specific questions.
4. Identify issues arising from new and emerging technologies and treatments.
|
Schünemann, 2014 (8)
|
1. Decide who will oversee the process (e.g., priorities of the government, funding agency or professional society).
2. Apply specific criteria and use a transparent and systematic process to guide the suggestions of guideline topics.
|
Reddy, 2014 (20)
|
Not reported
|
Mounesan, 2016 (17)
|
1. Topic identification should be informed by evidence including: scientific evidence, available reports, expert opinion and/or needs assessment
2. Topic identification should be done separately for: prevention, diagnosis and treatment
|
Table 5: Common framework of the guideline topics prioritization criteria and their respective domains
Items
Paper
|
Disease-related factors
|
Interest
|
Practice
|
Guideline development
|
Potential impact of the intervention
|
Implementation considerations
|
Health burden
|
Economic burden
|
Burden on healthcare system
|
Equity relevance
|
Urgency
|
Health professional level
|
Consumer level
|
National level
|
Practice variation
|
Uncertainty or controversy about best practice
|
Absence of guidance
|
Unsatisfactory guidance
|
Availability of evidence
|
Potential for changing existing guidance
|
Impact on health outcomes
|
Economic impact
|
Impact on the healthcare system
|
Impact on equity/access
|
Feasibility of intervention implementation
|
Availability of resources
|
% papers reporting the criterion
|
100
|
50
|
30
|
50
|
10
|
40
|
40
|
20
|
80
|
40
|
50
|
50
|
50
|
50
|
70
|
50
|
40
|
20
|
40
|
30
|
Battista, 1995 (13)
|
P
|
|
|
|
|
|
|
|
|
|
|
|
P
|
|
P
|
P
|
|
|
|
|
Field, 1995 (14)
|
P
|
P
|
|
|
|
|
|
|
P
|
|
|
|
|
|
P
|
P
|
|
|
|
|
McClarey, 1999 (16)
|
P
|
|
P
|
P
|
|
P
|
|
|
P
|
|
P
|
|
P
|
P
|
P
|
|
|
|
P
|
P
|
Oxman, 2006 (19)
|
P
|
|
|
|
|
|
|
|
|
|
|
P
|
|
P
|
|
|
P
|
|
P
|
P
|
Ketola, 2007 (15)
|
P
|
P
|
P
|
P
|
|
|
P
|
|
P
|
P
|
P
|
|
|
|
P
|
P
|
P
|
|
|
|
Reveiz, 2010 (18)
|
P
|
P
|
P
|
P
|
|
P
|
P
|
P
|
P
|
P
|
P
|
P
|
P
|
P
|
P
|
|
P
|
P
|
P
|
P
|
Atkins, 2012 (12)
|
P
|
P
|
|
|
|
P
|
P
|
|
P
|
P
|
P
|
|
P
|
|
P
|
P
|
P
|
P
|
|
|
Schünemann, 2014 (8)
|
P
|
P
|
|
|
|
|
|
|
P
|
P
|
P
|
P
|
|
P
|
|
|
|
|
|
|
Reddy, 2014 (20)
|
P
|
|
|
P
|
|
|
|
|
P
|
|
|
P
|
P
|
P
|
|
|
|
|
|
|
Mounesan, 2016 (17)
|
P
|
|
|
P
|
P
|
P
|
P
|
P
|
P
|
|
|
P
|
|
|
P
|
P
|
|
|
P
|
|
Prioritization criteria
We identified 118 prioritization criteria; 68% (80 out of 118) of the criteria were either defined or categorized under specific domains. 8% (9 out of 118) were supplied with data sources. The studies included a mean of 12 criteria (range 5-41). We derived from the 118 criteria a common framework of guideline prioritization criteria and of the domains they fall under. The framework is composed of 20 prioritization criteria clustered in six domains (Table 6): (1) disease-related factors; (2) interest; (3) practice; (4) guideline development; (5) potential impact of the intervention; (6) implementation considerations. The most frequently reported criteria were related to the health burden of disease which was included in all of the prioritization approaches, practice variation (8 out of 10, 80%) (8, 12, 14-18, 20) and impact on health outcomes (7 out of 10, 70%) (12-18). Urgency was included in only one of the approaches (17), while very few approaches reported criteria on interest at the national level (2 out of 10, 20%) (17, 18) and potential impact of the intervention on equity/access (2 out of 10, 20%) (12, 18).
Table 6: Proposed types of stakeholders to involve in prioritizing guideline topics and the method for their involvement
Paper
|
Number
|
Involvement method
|
Type
|
Battista, 1995 (13)
|
Not reported
|
Not reported
|
· Members of guideline developing organizations
· Potential end users
· Patient representatives
· Public
|
Field, 1995 (14)
|
Not reported
|
Delphi or Delphi-like techniques
|
· Experts
· Potential end users (clinicians or patient representatives)
|
McClarey, 1999 (16)
|
Not reported
|
Not reported
|
· Professional guideline groups
· Health care professionals
· Patient representatives
|
Oxman, 2006 (19)
|
Not reported
|
Delphi technique
|
· Experts
· Potential end users
· Public
· Others
|
Ketola, 2007 (15)
|
Not reported
|
Not reported
|
· Specialist society
· Board members of guideline developing organization
|
Reveiz, 2010 (18)
|
>12
|
Workshop, consensus meeting
|
· Experts
· External guideline developers
· Methodologist
· End users
|
Atkins, 2012 (12)
|
Not reported
|
Not reported
|
· Clinicians
· Professional organizations
· Policymakers
· Payers (e.g., health plans)
· Government bodies
· Quality organizations
· Patient representatives
|
Schünemann, 2014 (8)
|
Not Reported
|
Not reported
|
· Clinicians
· Professional societies
· Policymakers
· Payers
· Public
|
Reddy, 2014 (20)
|
Not Reported
|
Not reported
|
Not reported
|
Mounesan, 2016 (17)
|
Range
(5-15)
|
Face-to-face meeting
|
· Experienced family physicians
· Management representatives
|