Umbrella reviews effectively summarize, compare and contrast existing evidence syntheses (i.e., systematic reviews or CPGs) on a specific topic (39–41), and are an important research approach to inform healthcare planning and future research directions (41–44). In umbrella reviews, data abstraction, quality appraisal and synthesis are conducted on the existing synthesized evidence versus individual studies (39–42, 45). Umbrella reviews are also commonly referred to as ‘overviews of reviews’ and ‘systematic reviews of systematic reviews (or CPGs)’ (35, 41–44). To optimize the quality of reporting of this umbrella review protocol, the PRISMA-P checklist was followed (46).
The present study design is informed by Johnston et al.’s (2019) recommendations for conducting a ‘systematic review of CPGs’ (35). First, a multi-phased search of the literature will be conducted to retrieve relevant CPGs. Next, critical appraisals will be done to determine the quality of the CPGs and recommendations. Lastly, recommendations contained in CPGs appraised to be high-quality will be synthesized and a narrative analysis will be completed.
Eligibility Criteria
Eligibility criteria will be applied iteratively during all phases of the review (Table 1). In recognition of the unique considerations when synthesizing CPGs, the ‘PICAR’ criteria (35) have been modified from the traditional ‘PICOS/T’ criteria used to guide traditional evidence syntheses such as systematic reviews (41, 42). The first four components (PICA) will be applied during evidence selection. The last component (R) will be applied along with the other criteria after the critical appraisal phase to identify recommendations eligible to include in the synthesis and narrative analysis. In order to retain all potentially relevant CPGs, the eligibility criterion for population was broadened to include all adults because it is anticipated that some delirium CPGs may not limit their population to older adults but identify them as a high-risk group. This review will exclude: (1) summaries, audits, or quick guides of CPGs; (2) CPGs adapted from the original (i.e., one organization adapts a pre-existing CPG to another context for use); (3) CPGs based solely on expert opinion and/or consensus (i.e., no evidence-based process presented and no formal process for rating the strength of recommendations); (4) CPGs addressing delirium in specific conditions or populations (e.g., cancer, HIV, or pediatrics), or not related to the ED context (e.g., long-term care or rehabilitation); and (5) CPGs for other types of delirium that have a different pathophysiology and care trajectory (e.g., delirium tremens from alcohol misuse (47) and excited delirium from psychoactive substance use or new onset psychosis (48)).
Table 1
PICAR component
|
Study criteria
|
P: Population, clinical indication(s), and condition(s)
|
Older Adults (≥ 65 years)*
Delirium
|
I: Intervention(s)
|
Any intervention (due to unknown relevance)
|
C: Comparator(s), comparison(s), (key) content
|
Any comparator/comparison (due to unknown relevance)
|
A: Attributes of the CPG
|
Original full-text CPG published/updated in past 10 years
English language (or translation available)
Evidence-based development process presented
Relevant to the general care of delirium (i.e., not setting or condition specific), or relevant to the acute care setting
|
R: Recommendation characteristics and other considerations
|
Recommendations only extracted from CPGs attaining a quality score ≥ 70% in the AGREE-II rigour of development domain
|
Note: *, CPG can be for the All Adult population, but must be inclusive of Older Adult population.
|
Search Strategy
The proposed search strategy was iteratively developed and refined through consultation with a health science librarian (JL). A multi-phased process will be conducted to locate and retrieve delirium CPGs that are published in English (or English translation available). Guidelines and their recommendations have been shown to remain up to date for a median of 5 years (49–51), and approximately 25% of CPGs are still pertinent after 8 years (50, 51). Given the dearth of knowledge in this area and to ensure that we capture all potentially relevant scientific literature, our authorship team will identify all CPGs published or updated within the last 10 years.
The first phase of the search will involve a search of the bibliographic databases Scopus (includes Medline and EMBASE), and EBSCOhost (CINAHL, Ageline, and Academic Search Complete) using a combination of keywords and Subject Headings (see Table 2 for example search strategy). Next, the Guidelines International Network (G-I-N) Library and the ECRI Guidelines Trust® databases will be searched using the keyword “delirium”. To be indexed in one of these international guideline databases, CPGs must meet the minimum criteria for quality (32, 52, 53), therefore will be the two main databases searched. Supplementally, Google Advanced will be searched for records published in the last 10 years from a regional or national professional healthcare organization (e.g., Canadian Medical Association). Lastly, a snowball search will be conducted if summaries, audits, or quick guides are identified by searching for the full-text original CPG in the referenced location if it has not been retrieved already. If multiple versions of a CPG are retrieved (e.g., older versions, summaries, etc.) only the latest full-text version will be retained for screening. Retrieved citations will be merged into a reference manager (Zotero) and duplicate citations will be removed.
Table 2
Example Search Conducted in Scopus
#
|
Searches
|
1
|
TITLE-ABS-KEY (delirium OR delirious)
|
2
|
TITLE-ABS-KEY (“clinical practice guideline*” OR “practice guideline*” OR “practice recommendation*” OR “evidence synthesis”
|
3
|
(TITLE-ABS-KEY (delirium OR delirious) AND TITLE-ABS-KEY (“clinical practice guideline*” OR “practice guideline*” OR “practice recommendation*” OR “evidence synthesis”))
|
4
|
(TITLE-ABS-KEY (delirium OR delirious) AND TITLE-ABS-KEY (“clinical practice guideline*” OR “practice guideline*” OR “practice recommendation*” OR “evidence synthesis”)) AND PUBYEAR > 2011
|
5
|
(TITLE-ABS-KEY(delirium OR delirious) AND TITLE-ABS-KEY("clinical practice guideline*" OR "practice guideline*" OR "practice recommendation*" OR "evidence synthesis")) AND PUBYEAR > 2011 AND ( LIMIT-TO ( DOCTYPE,"ar" ) OR LIMIT-TO ( DOCTYPE,"re" ) ) AND ( LIMIT-TO ( LANGUAGE,"English" ) )
|
Evidence Selection
Evidence selection will be conducted independently by two reviewers using a two-step process. First, all titles and abstracts will be screened. Some CPGs may not provide a structured abstract (35), in these situations, reviewers will use the scope and purpose statements to help inform eligibility screening. Citations will be uploaded to ‘Covidence’, an online collaboration platform for conducting evidence syntheses (54). Once title and abstract screening is complete, websites for the organizations that authored potentially eligible CPGs will be searched to ensure all relevant and up-to-date documentation is retrieved (e.g., evidence surveillance and/or health technology assessment summaries conducted during CPG review and update process). Second, full-text documents for potentially eligible CPGs will be screened against the same eligibility criteria. The screening process, including most common reasons for exclusion at each stage, will be summarized using a modified PRISMA flowchart (see Fig. 1) (35, 55).
Critical Appraisals Of Cpgs And Recommendations
The critical appraisals of CPGs and recommendations will be conducted independently by three reviewers using the Appraisal of Guidelines, Research, and Evaluation (AGREE-II) and (33, 56) and Appraisal of Guidelines Research and Evaluation – Recommendations EXcellence (AGREE-REX) instruments (57, 58), respectively. The AGREE-II instrument is endorsed by the Equator Network (59) and has been used internationally to appraise CPG quality for over 10 years. The AGREE-REX instrument was recently developed to compliment the AGREE-II, recognizing the need to ensure that CPG-specific recommendations have also been rigorously developed (37, 57, 58, 60).
The “My AGREE Plus” online platform will be used to appraise the CPGs using the AGREE-II instrument (56). The AGREE-II is a 23-item instrument that assesses the quality of CPGs according to their scope and purpose (3 items), stakeholder involvement (3 items), rigour of development (8 items), clarity of presentation (3 items), applicability (4 items), and editorial independence (2 items) (33, 56). The last item assesses the overall quality of the CPG. Each item is rated on a 7-point scale from 1 (strongly disagree) to 7 (strongly agree). Quality scores for each domain and for the overall rating are automatically computed for each CPG by the “My AGREE Plus” platform (56). Quality scores are calculated by summing the item scores in each domain and scaling the summative score as a percentage of the maximum possible score for that domain (56). Once the independent appraisals are complete, appraisers will meet to discuss scores and compare items with large discrepancies (i.e., point difference ≥ 3) (56). Appraisers will modify their scores based on the discussion and summative scores will be recalculated where required.
The updated AGREE-II manual provides guidance to select a score threshold to differentiate between high, moderate and low quality practice guidelines (56). A threshold of 70% or greater in the rigour of development domain will be used in this study. Only CPGs obtaining a score at or above the established threshold, as well as meeting all other eligibility criteria, will be included for further critical appraisal, data abstraction, and synthesis of CPG recommendations. Practice guidelines that meet all other eligibility criteria but have a score below 70% in the rigour of development domain will be retained to abstract the general characteristics of these CPGs to compare with those that meet the quality criteria. The scores from the other quality domains will be used to facilitate the description of CPGs.
Recommendation of eligible CPGs will be critically appraised using the AGREE-REX instrument (57, 58). The AGREE-REX is a 9-item instrument that assesses the quality of CPG recommendations according to their clinical applicability (3 items), values and preferences (4 items), and implementability (2 items) (57). Each item is rated on a 7-point scale from 1 (strongly disagree) to 7 (strongly agree). Items from the instrument will be entered into Excel to facilitate the appraisal process. Quality scores will be calculated in Excel by summing scores for the items in each domain and scored using the same process as previously defined for the AGREE-II. Appraisers will meet to discuss and reach consensus on final quality scores. There has yet to be a defined domain score to quantify high- or low-quality CPG recommendations (57), therefore, in this study we will describe the range of AGREE-REX scores (overall and by quality domain) and assess the extent to which quality scores vary by recommendation type.
Data Abstraction And Synthesis
All data will be extracted independently by two reviewers. First, the general characteristics of the included CPGs (e.g., title, year of publication and last update, name and location of publishing organization, intended audience, and database retrieved from) will be extracted. Next, general data on recommendations from CPGs that meet the quality criterion will be extracted and categorized according to target population (i.e., older adults vs. all adults) and setting (e.g., non-setting specific, ED, intensive care unit, hospital ward, or operating room). Descriptive analysis will be used to facilitate the organization, characterization, and interpretation of data extracted on CPG and recommendation characteristics (35).
Data abstraction matrices will be created to facilitate the data synthesis process for eligible recommendations. The main elements of focus for the data synthesis include: the assessed quality of the CPG and recommendation, reported strength of the evidence, intended target population and setting, and aspect of delirium care addressed (i.e., delirium assessment, identification, prevention, or management). If more than one evidence grading system (e.g., the Grading of Recommendations Assessment, Development, and Evaluation [GRADE] system (61)) is used by CPG developers a standardized evidence matrix will be created and applied to each recommendation. Some of the elements can be identified a priori, however some important elements may only become evident once data is abstracted (35). Data abstraction and synthesis will be iteratively discussed to resolve any discrepancies in results. Unresolved discrepancies and/or suggestions to modify the abstraction form will be discussed with a third reviewer (PI).
The synthesized information will be examined to identify areas of similarity and discrepancy. If delirium care recommendations are identified that are specific to older ED patients, they will be assessed to gain understanding about areas of agreeance and discordance with other acute care settings.