Study Design
The present study will use the Appraisal of Guidelines, Research and Evaluation II (AGREE II) instrument for the quality assessment of CPGs focused on TI use and/or the management of adult patients with COVID-19 [10]. The systematic review will be conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [14]. A protocol was prospectively registered on the international prospective register of systematic reviews (PROSPERO) — CRD42020219944. If significant deviations are made from this protocol, then they will be reported and published with the results of the review.
Protocol
The protocol for the systematic review will be developed in accordance with the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) [15]. See Additional File 1.
Eligibility Criteria
The criteria for eligible COVID-19 CPGs was developed in accordance with the Population, Intervention, Comparison and Outcomes (PICO) framework [16]. We will include CPGs that focus on adults (18 years or older) with COVID-19. With respect to the interventions, we will include CPGs that primarily discuss and make recommendations for TI use and/or the management of COVID-19. We will include CPGs that are evidence-based, publicly available, published after 2019, and in English. We will exclude articles that focus on the diagnosis or screening of COVID-19. CPGs that are not in English or have a more recent update available from each organization will also be excluded. Furthermore, we will also exclude articles that are abstracts, editorials, letters, protocols, consensus documents, conference proceedings, and summaries of CPGs.
Outcomes and Prioritization
The primary outcome of this study is the assessed quality of eligible CPGs. The assessed quality will be determined through the calculation of the scaled domain percentages for each CPG, as outlined in the AGREE II instrument [10]. We will also report the average appraiser scores and average overall assessments, in addition to the scaled domain percentages. The secondary outcomes of this study are the types of management and/or TI recommendations made. We will determine which recommendations are inconsistent, unique, or duplicate across CPGs.
Search Methods for Identification of Studies
We will search databases including: Medical Literature Analysis and Retrieval System Online (MEDLINE), Excerpta Medica Database (EMBASE), Cumulative Index to Nursing and Allied Health Literature (CINAHL) and Web of Science from 2019 up to December 2020. As the databases have differing capabilities, we will adapt the search for each accordingly. The searches will use appropriate Medical Subject Headings (MeSH) terms to ensure inclusion of relevant titles and abstracts [17]. We will also hand search the websites of guideline developing organizations for the most recent guidelines that satisfy the eligibility criteria. A sample search strategy to be run on MEDLINE is presented in Table 1. This search strategy may be modified to increase specificity and sensitivity.
Study Selection Process
Two reviewers will independently perform title and abstract screening using Rayyan [18]. Articles that meet the eligibility criteria will be included and their respective full-texts will be retrieved for full-text screening. Full-text screening will be similarly performed independently and in duplicate. Any discrepancies regarding the eligibility of an article will be resolved through discussion and consensus between reviewers, or recruitment of another co-author for resolution. Supplementary materials and related documents for CPGs that pass full-text screening will be retrieved thereafter. Refer to Figure 1 for the proposed CPG selection process.
Data Collection and Extraction
The reviewers will independently and in duplicate perform data collection and extraction in data extraction spreadsheets prepared a priori. Any discrepancies in the data extracted will be resolved through discussion and consensus between reviewers, or recruitment of another co-author for resolution. For eligible CPGs, we will data extract demographic information including the author, title, year of publication, publishing organization, country, and funding source. Any management and/or TI recommendations made will also be extracted in addition to their corresponding level of evidence and grade of recommendation.
Quality Assessment
We will assess eligible CPGs using the AGREE II instrument which is a well-established and widely-validated international tool used to assess the quality and reporting of CPGs [10]. It is composed of 23 items over six quality domains which include: (1) scope and purpose, (2) stakeholder involvement, (3) rigor of development, (4) clarity and presentation, (5) applicability, and (6) editorial independence [10]. The instrument uses a seven-point Likert scale that ranges from strongly disagree (1) to strongly agree (7) that each item is met [10]. Each eligible CPG will be evaluated by four appraisers as recommended by the AGREE II instrument [10]. Prior to assessing the CPGs, each reviewer will complete the AGREE II Tutorial + Practice Exercise on the ‘My AGREE PLUS’ platform [19]. The reviewers will also perform a minimum of three rounds of calibration with CPGs that vary in quality. The reviewers will then score the CPGs for each of the 23 items over 6 domains. The overall quality of the CPG (1 to 7) will sequentially be evaluated by the reviewers and used to recommend for or against the use of a CPG. Any major discrepancies in the scoring of a CPG (greater than 3 points) will be resolved through discussion and consensus between reviewers, or recruitment of another co-author for resolution. We will then calculate the average overall assessment scores and average appraisal scores for each CPG. The average overall assessment scores will be calculated by averaging the scores each appraiser gave for “overall guideline assessment.” The average appraisal scores will be calculated by averaging the score an appraiser gave for all 23 items, and then subsequently averaging this value calculated for each of the 4 appraisers. The quality of each domain will be compared within and across CPGs using the scaled domain percentages. The scaled domain percentages will be calculated through the summation of the appraisers’ scores for the items within each domain, and subsequently scaling this value as a percentage of the maximum possible score for the given domain.
Data Synthesis and Analysis
Descriptive statistics will be used to quantify the quality of the CPGs. We will calculate the average overall assessment scores, average appraisal scores and the standard deviation for each of these two scores. The scaled domain percentages will also be calculated. We will tabulate these results for ease of quality comparison both across and within the domains of the CPGs.
Description and Comparison of Recommended Interventions
We will tabulate the recommendations made in each CPG for the management of COVID-19. We will indicate whether a therapy was recommended for or against use. Additionally, we will compare inconsistent and duplicate TI and/or management recommendations made between CPGs.