Critical Appraisal of Clinical Practice Guidelines for the Management of COVID-19: Protocol for a Systematic Review

DOI: https://doi.org/10.21203/rs.3.rs-125457/v1

Abstract

Background

In December 2019, a novel coronavirus, severe acute respiratory syndrome coronavirus 2 was identified as the cause of an acute respiratory disease, coronavirus disease 2019 (COVID-19). Given the lack of validated treatments, there is an urgent need for the high-quality management of COVID-19. Clinical practice guidelines (CPGs) are one tool that healthcare providers may use to enhance patient care. As such, it is necessary that they have access to high-quality evidence-based CPGs upon which they may base decisions regarding the management and use of therapeutic interventions (TI) for COVID-19. The purpose of the proposed study is to assess the quality of CPGs that make management or TI recommendations for COVID-19 using the AGREE II instrument.

 

Methods‌

The proposed systematic review will identify CPGs for TI use and/or the management of COVID-19. MEDLINE, EMBASE, CINAHL and Web of Science will be searched from December 2019 up to December 2020. The primary outcome of this study is the assessed quality of the CPGs. The quality of eligible CPGs will be assessed using the Appraisal of Guidelines, Research and Evaluation II (AGREE II) instrument. Descriptive statistics will be used to quantify the quality of the CPGs. The secondary outcomes of this study are the types of management and/or TI recommendations made. Inconsistent and duplicate TI and/or management recommendations made between CPGs will be compared across guidelines.

 

Discussion

The results of this study will be of utmost importance to enhancing clinical decision-making among healthcare providers caring for patients with COVID-19. Moreover, the results of this study will be relevant to guideline developers in the creation of CPGs or improvement of existing ones, researchers who want to identify gaps in knowledge, and policy-makers looking to encourage and endorse the adoption of CPGs into clinical practice. The results of this review will be published in a peer-reviewed journal and presented at conferences. 

 

Systematic Review ‌Registration‌

International‌ ‌Prospective‌ ‌Register‌ ‌for‌ ‌Systematic‌ ‌Reviews‌ ‌(PROSPERO)‌ ‌—‌ ‌CRD42020219944

Background

In December 2019, a novel coronavirus, severe acute respiratory syndrome coronavirus 2 was identified as the cause of an acute respiratory disease, coronavirus disease 2019 (COVID-19) [1]. COVID-19 was subsequently declared a pandemic by the World Health Organization on March 11, 2020 [1]. Patients with COVID-19 are typically adults who present with symptoms such as fever, dry cough, myalgia, dyspnea and fatigue [2]. In contrast, severe cases may present with viral pneumonia, severe acute respiratory distress syndrome, and death [3]. In November 2020, there were over 57 million cases of COVID-19 and more than 1.3 million deaths globally [4]. Given the lack of successful treatments [5], there is an urgent need for the high-quality management of COVID-19.

As a result of the relative novelty of COVID-19, there is a lack of data to help guide prognosis. As such, healthcare providers require high-quality evidence-based tools that can support clinical decision-making and inform medical practice. Clinical practice guidelines (CPGs) are one such tool that healthcare professionals may use to guide clinical practice. CPGs are developed systematically and formulate recommendations based on the highest quality of supporting evidence [6]. They provide healthcare professionals with explicit recommendations on how to proceed and help improve the consistency of care [6]. In response to this pandemic, CPGs have been developed to guide the management of adult patients with COVID-19. High-quality, evidence-based CPGs have the potential to enhance the care of adults with COVID-19; however, this depends on the quality of the CPG and its adoption in clinical practice. Several studies have found that CPGs vary in quality for various health conditions [79], and consequently, may affect the healthcare providers’ ability to deliver high-quality care and service. Therefore, in order to enhance care, it is crucial that healthcare providers have access to high-quality evidence-based CPGs upon which they may base decisions regarding the management and use of TIs for COVID-19.

The well-established and widely-validated Appraisal of Guidelines, Research and Evaluation II (AGREE II) instrument is one tool commonly used to assess the methodological quality of CPGs [10]. To date, only 3 published studies have examined the quality of CPGs for COVID-19 using the AGREE II instrument [1113]. These studies found that the recommendations formulated within the CPGs were lacking evidence and that the overall quality of CPGs was poor and variable [1113]. However, these studies are limited in that they assessed CPGs published very soon after the first confirmed case of COVID-19 [11], CPGs addressing very specific interventions [12], or CPGs with very specific populations [13]. To our knowledge, this study will be the first to assess the quality of evidence-based CPGs focused on the use of TIs and/or management of COVID-19. In doing so, the study will inform healthcare providers on which CPGs should be trusted the most and inform CPG developers of areas that need improvement. Hence, the primary objective of this study is to comprehensively assess the quality of evidence-based CPGs that make management or TI recommendations for adult patients with COVID-19 using the AGREE II instrument. The secondary objective is to determine the types of management and/or TI recommendations made, as well as which recommendations are inconsistent, unique, or duplicate across CPGs.

Methods

Study Design

The present study will use the Appraisal of Guidelines, Research and Evaluation II (AGREE II) instrument for the quality assessment of CPGs focused on TI use and/or the management of adult patients with COVID-19 [10]. The systematic review will be conducted ‌in‌ ‌accordance‌ with ‌the‌ ‌Preferred‌ ‌Reporting‌ ‌Items‌ ‌for‌ ‌Systematic‌ ‌Reviews‌ ‌and‌ ‌Meta-Analyses‌ ‌(PRISMA)‌ [14].‌ ‌A protocol was prospectively registered on the ‌international‌ prospective ‌register‌ ‌of‌ systematic‌ ‌reviews‌ ‌(PROSPERO)‌ ‌— CRD42020219944. ‌If ‌significant‌ deviations are made from this protocol, then they will be ‌reported‌ ‌and‌ ‌published‌ ‌with‌ ‌the‌ ‌results‌ ‌of‌ ‌the‌ ‌review.

Protocol

The protocol for the systematic review will be developed in accordance with the ‌Preferred‌ ‌Reporting‌ ‌Items‌ ‌for‌ ‌Systematic‌ ‌Review ‌and‌ ‌Meta-Analysis Protocols (PRISMA-P) [15]. See Additional File 1.

Eligibility‌ ‌Criteria‌

The criteria for eligible COVID-19 CPGs was developed in accordance with the Population, Intervention, Comparison and Outcomes (PICO) framework [16]. We will include CPGs that focus on adults (18 years or older) with COVID-19. With respect to the interventions, we will include CPGs that primarily discuss and make recommendations for TI use and/or the management of COVID-19. We will include CPGs that are evidence-based, publicly available, published after 2019, and in English. We will exclude articles that focus on the diagnosis or screening of COVID-19. CPGs that are not in English or have a more recent update available from each organization will also be excluded. Furthermore, we will also exclude articles that are abstracts, editorials, letters, protocols, consensus documents, conference proceedings, and summaries of CPGs. 

‌Outcomes‌ ‌and Prioritization

The primary outcome of this study is the assessed quality of eligible CPGs. The assessed quality will be determined through the calculation of the scaled domain percentages for each CPG, as outlined in the AGREE II instrument [10]. We will also report the average appraiser scores and average overall assessments, in addition to the scaled domain percentages. The secondary outcomes of this study are the types of management and/or TI recommendations made. We will determine which recommendations are inconsistent, unique, or duplicate across CPGs. 

Search‌ ‌Methods‌ ‌for‌ ‌Identification‌ ‌of‌ ‌Studies‌ ‌ ‌ ‌

We will search databases including: Medical Literature Analysis and Retrieval System Online (MEDLINE), Excerpta Medica Database (EMBASE), Cumulative Index to Nursing and Allied Health Literature (CINAHL) and Web of Science from 2019 up to December 2020. As the databases have differing capabilities, we will adapt the search for each accordingly. The searches will use appropriate Medical Subject Headings (MeSH) terms to ensure inclusion of relevant titles and abstracts [17]. We will also hand search the websites of guideline developing organizations for the most recent guidelines that satisfy the eligibility criteria. A sample search strategy to be run on MEDLINE is presented in Table 1. This search strategy may be modified to increase specificity and sensitivity. 

Study‌ ‌Selection‌ ‌Process

Two reviewers will independently perform title and abstract screening using Rayyan [18]. Articles that meet the eligibility criteria will be included and their respective full-texts will be retrieved for full-text screening. Full-text screening will be similarly performed independently and in duplicate. Any discrepancies regarding the eligibility of an article will be resolved through discussion and consensus between reviewers, or recruitment of another co-author for resolution. Supplementary materials and related documents for CPGs that pass full-text screening will be retrieved thereafter. Refer to ‌Figure 1 for the proposed CPG selection process.

Data‌ Collection and Extraction‌

The reviewers will independently and in duplicate perform data collection and extraction in data extraction spreadsheets prepared a priori. Any discrepancies in the data extracted will be resolved through discussion and consensus between reviewers, or recruitment of another co-author for resolution. For eligible CPGs, we will data extract demographic information including the author, title, year of publication, publishing organization, country, and funding source. Any management and/or TI recommendations made will also be extracted in addition to their corresponding level of evidence and grade of recommendation. ‌

Quality Assessment

We will assess eligible CPGs using the AGREE II instrument which is a well-established and widely-validated international tool used to assess the quality and reporting of CPGs [10]. It is composed of 23 items over six quality domains which include: (1) scope and purpose, (2) stakeholder involvement, (3) rigor of development, (4) clarity and presentation, (5) applicability, and (6) editorial independence [10]. The instrument uses a seven-point Likert scale that ranges from strongly disagree (1) to strongly agree (7) that each item is met [10]. Each eligible CPG will be evaluated by four appraisers as recommended by the AGREE II instrument [10]. Prior to assessing the CPGs, each reviewer will complete the AGREE II Tutorial + Practice Exercise on the ‘My AGREE PLUS’ platform [19]. The reviewers will also perform a minimum of three rounds of calibration with CPGs that vary in quality. The reviewers will then score the CPGs for each of the 23 items over 6 domains. The overall quality of the CPG (1 to 7) will sequentially be evaluated by the reviewers and used to recommend for or against the use of a CPG. Any major discrepancies in the scoring of a CPG (greater than 3 points) will be resolved through discussion and consensus between reviewers, or recruitment of another co-author for resolution. We will then calculate the average overall assessment scores and average appraisal scores for each CPG. The average overall assessment scores will be calculated by averaging the scores each appraiser gave for “overall guideline assessment.” The average appraisal scores will be calculated by averaging the score an appraiser gave for all 23 items, and then subsequently averaging this value calculated for each of the 4 appraisers. The quality of each domain will be compared within and across CPGs using the scaled domain percentages. The scaled domain percentages will be calculated through the summation of the appraisers’ scores for the items within each domain, and subsequently scaling this value as a percentage of the maximum possible score for the given domain.

Data Synthesis and Analysis

Descriptive statistics will be used to quantify the quality of the CPGs. We will calculate the average overall assessment scores, average appraisal scores and the standard deviation for each of these two scores. The scaled domain percentages will also be calculated. We will tabulate these results for ease of quality comparison both across and within the domains of the CPGs. 

Description and Comparison of Recommended Interventions

We will tabulate the recommendations made in each CPG for the management of COVID-19. We will indicate whether a therapy was recommended for or against use. Additionally, we will compare inconsistent and duplicate TI and/or management recommendations made between CPGs. 

Discussion

COVID-19 has affected over 57 million people and accounted for more than 1.3 million deaths globally [4]. Classified as a pandemic by the World Health Organization, individuals with COVID-19 suffer from symptoms such as fever, dry cough, myalgia, dyspnea, and fatigue [1, 2]; where critical cases may result in severe acute respiratory distress syndrome and death [3]. Many of those individuals affected must turn to healthcare professionals for the high-quality care and management of the condition. Evidently, with the current lack of validated treatments, there is a greater emphasis placed on the management of COVID-19. As such, it is necessary that healthcare professionals have access to high-quality evidence-based CPGs which contain recommendations pertaining to the use of TIs and the management of COVID-19. The purpose of this systematic review will be to investigate the quality of existing CPGs designed for the management of adult patients with COVID-19 using the AGREE II instrument.

To date, few published studies have assessed the quality of COVID-19 CPGs [1012]; however, they have significant differences from the proposed study. One rapid review assessed the quality of COVID-19 CPGs developed by February 14, 2020 [11]. This study included consensus-based CPGs and was conducted before COVID-19 was declared a pandemic by the World Health Organization; therefore, several CPGs developed afterwards have not been assessed. Another study examined COVID-19 CPGs relevant to anesthetic management [12], while Yeo et al. examined CPGs for managing neonates born to mothers with COVID-19 [13]. Due to the highly specific interventions and populations, these studies would not have assessed all evidence-based CPGs relevant to COVID-19 management or TI use [1213]. Thus, to our knowledge, this study will be the first to assess the quality of evidence-based CPGs for the management of adult patients with COVID-19.

Our proposed systematic review and assessment will aide healthcare professionals who wish to improve the standard of care for patients with COVID-19 by incorporating CPGs into their practice, guideline developers in the creation of CPGs or improvement of existing ones, researchers who want to identify gaps in knowledge, and policy-makers looking to encourage and endorse the adoption of CPGs into clinical practice. Additionally, the evidence gathered through this systematic review will provide insight into gaps in available CPGs which may result in the identification of other research avenues.

There are several notable strengths to this study, including the use of a comprehensive systematic review to determine eligible CPGs pertaining to TI use and the management of COVID-19; use of multiple appraisers; and the use of the well-established and widely-validated AGREE II instrument.

Abbreviations

AGREE II: Appraisal of Guidelines for Research & Evaluation II

COVID-19: Coronavirus Disease 2019 

CPG: Clinical Practice Guideline

PICO: Patients, Intervention, Comparison and Outcomes 

PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses

PRISMA-P: Preferred‌ ‌Reporting‌ ‌Items‌ ‌for‌ ‌Systematic‌ ‌Review ‌and‌ ‌Meta-Analysis Protocols

TI: Therapeutic Intervention

Declarations

Ethics Approval and Consent to Participate

The‌ ‌study‌ ‌will‌ ‌not‌ ‌require‌ ‌ethics‌ ‌approval. ‌The results of this study will be published in a peer-reviewed journal and may be presented at local, national or international conferences. 

Consent for Publication

All authors gave consent to publish this manuscript

Availability of Data and Materials 

Not applicable 

Competing Interests

The authors declare that they have no competing interests

Funding

The authors of this research declare that they have received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

Authors’ Contributions 

TD made substantial contributions to the conception and design of the study, wrote the first draft of the manuscript, and significantly revised it. UA wrote the first draft of the manuscript, and substantially revised it for important intellectual content. FB wrote the first draft of the manuscript, and substantially revised it for important intellectual content. LA and OC substantially reviewed the drafts and revised it for important intellectual content. TD, UA, LA, FB, and OC agreed on the final version of the manuscript. 

TD and UA contributed equally to this paper.

Acknowledgements

Not applicable

References

  1. Rothan HA, Byrareddy SN. The epidemiology and pathogenesis of coronavirus disease (COVID-19) outbreak. Journal of Autoimmunity Journal of Autoimmunity. 2020;Vol. 109:102433. https://doi.org/10.1016/j.jaut.2020.102433.
  2. Huang C, Wang Y, Li X, Ren L, Zhao J, Hu Y, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. Vol. 395, The Lancet. The Lancet; 2020. p. 497–506. https://doi.org/10.1016/S0140-6736(20)30183-5.
  3. Lovato A, De Filippis C. Clinical Presentation of COVID-19: A Systematic Review Focusing on Upper Airway Symptoms. Vol. 99, Ear, Nose & Throat Journal. Ear, Nose & Throat Journal; 2020. p. 569–76. https://doi.org/10.1177/0145561320920762.
  4. COVID-19 Map [Internet]. Johns Hopkins Coronavirus Resource Center. https://coronavirus.jhu.edu/map.html. Accessed 26 Nov 2020.
  5. Sanders JM, Monogue ML, Jodlowski TZ, Cutrell JB. Pharmacologic Treatments for Coronavirus Disease 2019 (COVID-19). JAMA. JAMA; 2020. Available from: http://doi.org/10.1001/jama.2020.6019.
  6. Woolf SH, Grol R, Hutchinson A, Eccles M, Grimshaw J. Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines. BMJ (Clinical research ed). 1999 Feb 20;318(7182):527–30. Available from: https://doi.org/10.1136/bmj.318.7182.527.
  7. Burda BU, Norris SL, Holmer HK, Ogden LA, Smith ME. Quality varies acrossclinical practice guidelines for mammography screening in women aged40-49 years as assessed by AGREE and AMSTAR instruments. J ClinEpidemiol. 2011;64(9):968–76.22.
  8. de Haas ER, de Vijlder HC, van Reesema WS, van Everdingen JJ. NeumannHA. Quality of clinical practice guidelines in dermatological oncology. J EurAcad Dermatol Venereol. 2007;21(9):1193–8.23.
  9. Burgers JS, Fervers B, Haugh M, Brouwers M, Browman G, Philip T, et al. International assessment of the quality of clinical practice guidelines inoncology using the Appraisal of Guidelines and Research and EvaluationInstrument. J Clin Oncol. 2004;22(10):2000–7.
  10. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, Fervers B, Graham ID, Grimshaw J, Hanna SE, Littlejohns P. AGREE II: advancing guideline development, reporting and evaluation in health care. Can Med Assoc J 2010, 182(18):E839–42. https://dx.doi.org/10.1503%2Fcmaj.090449.
  11. Dagens A, Sigfrid L, Cai E, Lipworth S, Cheng V, Harris E, et al. Scope, quality, and inclusivity of clinical guidelines produced early in the covid-19 pandemic: rapid review. BMJ. BMJ; 2020. p. M1936. https://doi.org/10.1136/bmj.m1936.
  12. Ong S, Lim WY, Ong J, Kam P. Anesthesia guidelines for COVID-19 patients - A narrative review and appraisal. Korean Journal of Anesthesiology Korean Journal of Anesthesiology. 2020. https://doi.org/10.4097/kja.20354.
  13. Yeo KT, Oei JL, De Luca D, Schmölzer GM, Guaran R, Palasanthiran P, et al. Review of guidelines and recommendations from 17 countries highlights the challenges that clinicians face caring for neonates born to mothers with COVID-19. Vol. 109, Acta Paediatrica. Acta Paediatrica; 2020. p. 2192–207. https://doi.org/10.1111/apa.15495.
  14. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264–9. https://doi.org/10.1016/j.ijsu.2010.02.007.
  15. Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ; 2015;349(jan02 1):g7647–. Available from: https://dx.doi.org/10.1136/bmj.g7647.
  16. Schardt C, Adams MB, Owens T, Keitz S, Fontelo P. Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC medical informatics and decision making. 2007 Dec 1;7(1):16. Available from: http://doi.org/10.1186/1472-6947-7-16.
  17. Lowe HJ, Barnett GO. Understanding and using the medical subject headings (MeSH) vocabulary to perform literature searches. Jama. 1994 Apr 13;271(14):1103-8. http://doi.org/10.1001/jama.1994.03510380059038.
  18. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan—a web and mobile app for systematic reviews. Systematic reviews. 2016 Dec 1;5(1):210.
  19. My AGREE PLUS [Internet]. AGREE Enterprise website. https://www.agreetrust.org/resource-centre/agree-plus/. Accessed 26 Nov 2020.

Tables

Table 1: Sample MEDLINE search for COVID-19 Clinical Practice Guidelines executed using the OVID interface

#

Searches

1

exp RNA, Viral/ or exp Pneumonia, Viral/ or exp Coronavirus Infections/ or exp Pandemics/ or SARS-CoV-2.mp. or exp Betacoronavirus/ or exp SARS Virus/

2

COVID-19.mp.

3

exp Coronavirus/ or coronavirus.mp.

4

2 or 3

5

1 and 4

6

exp Practice Guideline/ or guideline.mp. or exp Guideline/

7

limit 6 to (english language and yr="2019 -Current")

8

5 and 7