Reporting of the certainty of evidence through GRADE in systematic reviews in dentistry indexed during late 2019 - late 2020

doi:10.21203/rs.3.rs-3408361/v1

Download PDF

Research Article

Reporting of the certainty of evidence through GRADE in systematic reviews in dentistry indexed during late 2019 - late 2020

https://doi.org/10.21203/rs.3.rs-3408361/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Objectives: To assess how systematic reviews of interventions in dentistry report the certainty of evidence using the Grading of Recommendations, Assessment, Development, Evaluation approach (GRADE).

Methods: A search on MEDLINE/PubMed database from September 2019 to September 2020 was performed. We included systematic reviews of intervention in dentistry using the GRADE approach to assess the certainty of the evidence. Data was narratively synthesized and summarized using relative frequencies.

Results: From 449 retrieved systematic reviews of interventions in dentistry, 23.6% used the GRADE approach and were finally included. Among these, 92.5% used the approach as recommended by the GRADE developers. Regarding the GRADE individual components, 73.6% of studies reported how the risk of bias, 61.3% inconsistency, 16% indirectness, 41.5% imprecision, and 48.1% publication bias domains were assessed. Overall, 41.5% of reviews clearly stated the main outcome assessed and, from these, 6.8% reported high, 38.6% moderate, 25% low, and 29.5% very low certainty of the evidence. For the remaining reviews (50.9%), the main outcome was not possible to be identified, and in 7.5% the judgement about the certainty of evidence found was not adequately reported. Finally, 59.4% of systematic reviews used the certainty of evidence assessment to support their conclusion.

Conclusions: Only around one in four systematic reviewsof interventions in dentistry identified in this study have used the GRADE approach to assess the certainty of evidence. In most of those reviews that applied it, GRADE was correctly applied. In more than half, the certainty of the evidence was judged to be low or very low.

Systematic review

Dentistry

Evidence-Based Dentistry

GRADE Approach

The dissemination of evidence-based practice has increased the need to develop solid research in several areas, such as health sciences, to better support clinical decisions and improve the decision-making capacity of well-informed professionals [1, 2]. Systematic reviews (SRs) became essential tools to solve these challenges due to their ability to summarize a large quantity of evidence on a particular subject, explaining the differences between the available studies and punctuating positive and negative points [2, 3].

The SRs are developed based on evidence with the same central topic, with interest in clinical practice [3–5]. Through a rigorous methodology, SRs allows the summarization and interpretation of a significant quantity of data and information on a specific field of knowledge [6–8]. Since 2000, the number of published SRs in health and, consequently, in dentistry has increased [8]. Considering only MEDLINE/PubMed database, 157 SRs in dentistry were indexed in 2000 [9]. In 17 years, this number grew to 495 SRs per year [10], adding up to about 16,500 SRs of all types in the dentistry area indexed by 2020 [9].

However, the reporting quality of SRs has decreased over time [8, 11]. As adequately conducted and reported SRs are valuable in clinical practice, guidelines and normative tools are being developed and improved for their reporting, evaluation, and methodology application [2, 12–14]. Therefore, SRs should follow standards that will directly affect their quality to better support the confidence in the sum of the accumulated evidence [3, 15].

In SRs, the confidence in the final estimate is called the certainty of the evidence, which means how close to the reality is the final effect estimative of the combination of multiple studies with a common outcome [16, 17]. To summarize outcomes in an SR, scientific and methodological rigor is expected [1]. Thus, to standardize the certainty of the evidence assessment, a working group developed the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach [18].

The GRADE approach consists of a general assessment of the outcome studied, considering five domains: risk of bias, inconsistency, imprecision, indirectness, and publication bias. For each domain, the outcome can be judged according to its impact on the certainty of evidence and can be downgraded by one or two levels: being "not serious" with no change in the certainty; "serious", the certainty is downgraded by one level; and "very serious" by two levels. The certainty starts with high for randomized clinical trials (RCTs), going through moderate certainty, low certainty, and possibly reaching very low evidence certainty [12, 19]. Information about the GRADE domains is necessary for decision-makers, clinicians, and patients to understand and trust the assessment of certainty in an effect size estimation [20].

Although the quantity of SRs published is increasing, the GRADE approach is not widespread among SRs in dentistry [8, 10]. Considering all types of SRs, the GRADE approach is used in over 20% of them [8, 10]. However, the frequency and how SR authors apply the approach is unknown in the most common types of SR, the SRs of interventions that evaluate the effectiveness of therapies and preventive methods in clinical practice. Thus, this study aimed to assess how recent SRs of interventions in dentistry report the certainty of evidence using the GRADE approach.

Protocol and registration

The protocol of this study was registered in the Open Science Framework (OSF), under the registration number: 10.17605/OSF.IO/CY7Z6 [21].

Eligibility criteria

Inclusion criteria

Systematic reviews of interventional studies in dentistry, including RCT and non-randomized clinical trials published in English between September 2019 and September 2020 were included. Within this group, we had only SRs that reported the use of the GRADE approach to assess the certainty of the evidence, regardless of the nomenclature used by the authors of the original study ("evidence certainty" or "certainty of the evidence" or "certainty of the available evidence" or "quality of the evidence" or other).

We used the criteria proposed by the Cochrane Handbook for Systematic Reviews of Interventions (2011) to classify the study as SRs when including:(1) a clearly stated set of objectives with pre-defined eligibility criteria for studies; (2) a systematic search that attempts to identify all studies that meet the eligibility criteria; a detailed, reproducible methodology; (3) an assessment of the validity of the findings of the included studies, for example through the assessment of the risk of bias; and (4) a systematic presentation, and synthesis, of the characteristics and findings of the included studies [22].

The SRs in the field of dentistry was considered reviews that aimed to test hypotheses in humans involving interventions of interest, such as the use of medications, treatments, techniques, materials, or preventive methods, in areas as the oral cavity, maxillofacial, and/or adjacent area and associated structures [10].

Exclusion criteria

The following exclusion criteria were applied: (1) Other types of SRs that include observational studies; (2) narrative/non-systematic literature reviews, comments opinions, letters, umbrella reviews (or overviews), brief SRs, network meta-analysis, scoping and rapid reviews; (3) methodology articles evaluating quality of studies; (4) protocols or summaries of SRs; and (5) articles in fields other than dentistry, previously defined by the authors.

Information sources and search strategy

A literature search strategy was developed with an experienced health science librarian (Table 1). The strategy was applied to MEDLINE through PubMed using filters of language ("English") and time ("last year"). The search strategy was applied on September 1^st, 2020. The file was exported to a reference software manager (EndNote X9, Thomson Reuters, Philadelphia, PA, USA).

Study selection and data collection

As established in the eligibility criteria, the selection process was performed in 3 steps: (1) the identification of SR in dentistry, (2) the selection of SR of interventions, and (3) the selection of SRs of intervention that assessed the certainty of the evidence using the GRADE approach.

Firstly, titles and abstracts were screened independently by two pairs of reviewers according to defined eligibility criteria. The online software Rayyan® (Qatar Computing Research Institute, Qatar) was used. The remaining articles were independently read (full text) by another two pairs of reviewers that classified the SRs according to their types, following the Munn et al. typology [23]. Any discrepancies were resolved by another author, making a final decision of both phases.

The data extraction was conducted by the first and second authors independently, in a form previously prepared using Microsoft Excel 2019 for Mac version 16.29.1 (Microsoft® Corporation, Redmond, Washington, United States). Data were extracted only for SRs of interventions in dentistry that assessed the certainty of evidence using the GRADE approach of the studied outcomes. The following characteristics were extracted: author, publication year, country, journal and impact factor (following the Journal Citation Reports 2019/2020 by Clarivate Analytics), dentistry specialty (following the classification of American Dental Association adopted by the National Commission on Recognition of Dental Specialties and Certifying Boards) [24],PICOS question's items, the final certainty for each outcome evaluated, how authors judged the GRADE domains, report on the use of GRADE in the main sections (abstract, methods, results, conclusions) of the article, and if authors used the certainty of evidence to interpret and report the conclusion, considering the primary outcome evaluated in the SRs (Supplementary tables S1 and S2).

Data analysis

A descriptive analysis of the data was performed with categorical items summarized as frequencies and described as percentages.

The considered outcomes were: (1) the frequency of conclusions that based the interpretation on the certainty of evidence; (2) the presence of the Summary of Findings (SoF) table; (3) how authors judge each GRADE domain; and (4) how authors use the nomenclatures to define the "certainty of the evidence".

Study selection

The titles and abstracts from 3,283 studies identified were read; 943 were classified generally as SRs and remained full-text reading. Next, 449 SRs were identified as SRs of interventions, and 106 studiesthat assessed the certainty of evidence using the GRADE approach were included, representing 23.6% of the total SRs of interventions.

The process of identification and selection of the studies is presented in a detailed flowchart diagram in Figure 1.

Studies characteristics

The included SRs of interventions that used the GRADE (n=106) were conducted in 28 countries of five continents (Figure 2). They involved ten of the twelve specialties adopted by the American Dental Association [24], mainly in Orthodontics and Dentofacial Orthopedics (23.6%), Periodontics (20.8%), Prosthodontics (17%), and Oral and Maxillofacial Surgery (17%) (Table 2).

All the SRs included at least one primary study in their methodology: 60 SRs included only randomized controlled trials (RCTs), 37 SRs included RCTs and non-randomized studies of interventions (NRSI), and nine studies included only NRSI. The included SRs were published in 60 journals with impact factors ranging from 0.953 ("Journal of Craniofacial Surgery") to 9.613 ("Sleep Medicine Reviews"). Of these, two journals ("European Journal of Orthodontics" [impact factor=2.202] and "Orthodontics & Craniofacial Research" [impact factor=1.455]) were the most frequent journals within the sample, with seven studies published in each one of them. Six journals were not on Clarivate's list.

Additional information on included studies' characteristics and references are available in Supplementary Table S1 and S2.

GRADE approach characteristics in the selected SRs

Most SRs used the nomenclature "certainty of the evidence" (50.9%) and "quality of the evidence" (42.5%) to express the results of the GRADE approach. Four (3.8%) SRsnamed the GRADE assessment as "level of the evidence," and two(1.9%) used the term "strength of the evidence".

Out of the 106 SRs included, 98 (92.5%) applied the GRADE approach recommended by the GRADE Working Group. Among all the included studies, 44 (41.5%) clearly stated the main outcome or assessed certainty of the evidence for only one outcome. From these, 3 (6,8%) judged the main outcome with high certainty of the evidence, 17 (38,6%) with moderate, 11 (25%) with low, and 13 (29,5%) with very low certainty.

It was impossible in 54 studies (50.9%) to identify the main outcome, which was evaluated through multiple outcomes, or for different follow-up periods, or other comparisons or interventions. For these, the number of considered outcomes/comparisons evaluated through the GRADE approach varied from 2 to 20 (mean 6.35±4.01, mode 8). Considering the highest certainty of the evidence for at least one evaluated outcome for these SR, 8 (14.8%) were judged as high, 32 (59.3%) as moderate and 14 (25.9%) as low. On the other hand, considering the lowest certainty of the evidence for at least one evaluated outcome, 24 (44.4%) were judged as low and 30 (56.6%) as very low.

From the remaining eight (7.5%) SR, one (0.94%) judged the considered main outcome as “moderate to high”, another (0.94%) considered the amount of information “insufficient” to evaluate the certainty of the evidence. The remaining six (5.7%) used the GRADE approach as a risk of bias tool, judging each included study individually instead of the body of evidence.

Regarding the methodology of the GRADE approach, 22 (20.8%) SRsreported in their methodology that at least two independent reviewers assessed the certainty of the evidence. However, most SRs (n=84/79.2%) did not detail how many authors assessed the certainty of evidence and whether another author cross-checked it.

Most SRs (n=85/80.2%) used the certainty of the evidence to interpret the conclusion and the results by reporting the main certainty of the evidence with GRADE's interpretation for the level of the evidence found. Complementary, to present the certainty of evidence assessment graphically, 40 (37.7%) SRs used the online software GRADEpro GDT to create a SoF table (GRADE working group, McMaster University) [25], one SR used a modified SoF table, and other 40 SRs did not report, or the information was unclear about the use of the GRADEpro software (Table 3).

Amongst the studies that mentioned the certainty of the evidence in the conclusion section of the paper (n=63/59.4%), 29 (46%) reported the overall certainty as "very low"; 24 (38.1%) as "low"; 7 (11.1%) as "moderate"; two (3.2%) as "high"; and one (1.6%) SR graded the certainty of the evidence as "insufficient".

Rationales given by SR authors to judge each domain

The risk of bias domain assessment was described in 78 (73.6%) of the 106 included SRs. Inconsistency domain assessment was described in 65 (61.3%) articles, and heterogeneity was reported in 55 (51.9%). The domain about the indirectness was described by 17 (16%) SRs. The imprecision domain was described in 44 (41.5%) SRs, being evaluated through sample size in 29 (27.4%) SRs and confidence interval in 17 (16%) SRs (six studies considered both characteristics while other four studies considered other characteristics). Funnel plot and/or Egger's test were used to assess publication bias in 31 (29.2%), and in 20 (18.9%), no test was performed because of the inclusion of few studies in the SR (Table 4).

Additionally, detailed information regarding the GRADE approach assessment characteristics in the included articles can be found in Supplementary Table S2.

In the present study, the prevalence of certainty of the evidence assessed through the GRADE approach was 23.6%. These findings highlight the information present in the literature that the GRADE approach has not been widespread among SRs in dentistry. Previous methodological studies showed that the approach utilization rate was less than 50% [8, 10, 26]. For example, in a study performed by Page et al. [8], from 300 SRs published in the biomedical field in one month, only 32 (11%) assessed the certainty of the evidence using the GRADE approach. In another study with SRs in dentistry, it was observed that only 19% of the SRs published over one year (SRs indexed in PubMed in 2017) used the approach to assess the certainty of the evidence [10]. However, these studies [8, 10] evaluated other types of SRs besides intervention and, for this reason, the use of the GRADE approach was lower than found in our study (23.6%) that evaluated only SRs of interventions, considering that the GRADE approach was initially developed for use in SRs of interventions [27].

Furthermore, comparing the present study's findings, another previous methodological survey performed only with Cochrane SRs of medical and health-related interventions found a prevalence of 43.6% on using the GRADE approach to appraise the certainty of the evidence [26]. However, despite the higher prevalence found in the study of Fleming et al. [26] for the use of the GRADE approach in health sciences SRs, it is worth mentioning that it is in common agreement in the literature that Cochrane SRs have more rigorous methods when comparing to non-Cochrane SRs, explaining the greatest adherence of the SRs included in these results to the GRADE approach for the assessment of the certainty of the evidence [28–31].

Considering the information about the main application of the GRADE approach, it is important to highlight that it was not created to be adapted to other types of studies. The GRADE's developers emphasize that the approach comprises validated modifications only for diagnostic questions and accepts the assessment of observational SRs only when the SRs also include intervention studies (and starting with significant initial low certainty of the evidence) [27]. Since there are not many approaches to assess the quality of the evidence in other types of SRs, SRs authors are wrongly induced to adapt and/or modify the GRADE approach corroborating with this methodological gap.

Also, it is essential to consider that using the GRADE approach is not easy or intuitive, and SR’s authors may face difficulties and doubts. Indeed, in the present study, six SR erroneously used it individually for each included study and were considered inappropriate by the authors. The GRADE guidelines state that it should be used to assess the quality of the body of evidence to support specific outcomes gathered in SRs of intervention research and not to assess individual studies [20, 32, 33]. Besides, with the incorporation of the certainty of the evidence in the 2020 update of the PRISMA statement [34], an increase in the number of SRs assessing the certainty evidence evaluation is expected. However, for this increase to occur in association with the correct application of the approach, SR authors in dentistry must be trained to use the GRADE approach and its methodology, limitations, and judgments. Therefore, it is expected that the new PRISMA 2020 and future GRADE guidelines may contribute to making the reporting process of the assessment of the certainty of evidence clearer and more objective [15, 34].

It is essential to highlight that after assessing the five domains of the GRADE approach, authors should clearly state the reasons for their judgments. For example, in the SRs included in the present study, assessing the risk of bias domain of the GRADE approach was evaluated through general observation of the individual risk of bias judgements of each included study. If these were mainly studies with a high risk of bias, in agreement with the recommendations of the GRADE guidelines for this domain, the domain could be downgraded by one or two levels [35]. As well, inconsistency, as indicated by the guidelines for the SRs, must consider the heterogeneity of the results across the included studies of the SR, which can be performed through heterogeneity tests such as I², Cochran Q test, Tau² test, and chi-square test; the significance for the heterogeneity tests; similar effect estimates among studies and overlap of 95%CI [36].

In addition, the imprecision domain, described in some SRs included in this study, was evaluated by the sample size, the number of events, confidence interval and inaccurate. Since the guidelines also have recommendations regarding the confidence interval, after the number of events, as a criterion for judging precision, the percentage found about this criterion in the present study was considered low among the literature [37]. The indirectness domain also agreed with the guidelines, since the recommendation is to assess this domain by judging when there are differences between the key points of the research question related to the studies where the evidence comes from, making it difficult to link the evidence to the research question. All SRs included in the present study evaluated if the evidence where the studies come from are related or applicable to the research question (PICO) [38]. Finally, the SRs that assessed publication bias evaluated the Funnel plot and/or Egger's test, as recommended [39]. Additionally, few SRs also reported that it was impossible to estimate publication bias through the Funnel plot and/ or Egger’s test as the number of studies included in the SR was less than ten included studies.

It is important to discuss the prevalence found in our study for each level of the certainty of the evidence for the studies that presented just one clearly stated the main outcome assessed by the GRADE approach (n = 44/41.5%) (29.5% "very low"; 25% "low"; 38.6% "moderate"; and 6.8% "high"), that also agrees with other evidence found in the literature. For example, Pandis et al. [41] evaluated 91 Cochrane and non-Cochrane SRs in dentistry, published from 2008 to 2013. In the Cochrane SRs (n = 41), 27% of the studied outcomes were judged as " moderate" certainty of the evidence, 47% were judged as "low," and 27% as "very low" certainty of the evidence. On the other hand, in the group of the non-Cochrane SRs, 30% of the outcomes studied were judged as "very low" certainty of the evidence, 48% were judge as "low" certainty, 18% were judged as "moderate" certainty, and 4% were judged as "high" certainty of the evidence. Our results confirm Pandis et al. results that currently, over 50% of clinical decisions are still based on SR's findings with low and very low certainty of the evidence, highlighting the need to improve the studies in the dentistry area [40].

Furthermore, although the GRADE Working Group states that SRs should only assess the certainty of the evidence [20, 27], some SRs included in the present study (n = 21/19.8%) used the overall certainty of the evidence to recommend some results. However, according to the GRADE Working Group, recommendations regarding a health care decision should be restricted solely to guidelines and public policy guidance [12, 27]. Therefore, although the approach might differ among reviewers, it is essential to correctly and carefully follow GRADE approach instructions published by its developers since clinical practices and health care decisions might be influenced by the results of SRs assessed by the GRADE approach [20].

Finally, there are some similarities between our results and the study of Bassani et al. [10], which evaluated SRs in the dentistry area. In agreement, both studies found SRs published predominantly from Brazil, being 23.6% in the 2019 study [10] and 35.8% in our study. However, about the specialty’s areas of publication of the SRs included in both studies, there was a high prevalence of SRs of Oral and Maxillofacial Surgery, Oral and Maxillofacial Pathology/stomatology, and Implantology and Periodontology in the study of Bassani et al. [10], while the most studied specialties areas in our study were Orthodontics and Dentofacial Orthopedics, Periodontics, Oral and Maxillofacial Surgery and Prosthodontics, respectively.

Strengths and Limitations

The present study was conducted with a limited search of one database over one year of SRs of interventions in dentistry. Although MEDLINE/PubMed is one of the most relevant and large databases for health sciences [41], it is possible that some studies that would be eligible for inclusion were not identified for this reason. However, we believe that the SR sample included in this methodological survey study represents the SRs of dentistry interventions. Nevertheless, it is important to emphasize that the objective of the present study was not to analyze all the SR of dentistry published in one year but to have a glimpse of what is happening on the certainty of evidence analysis in SR of interventions in dentistry.

Lastly, it is essential to highlight that the goal of this study was not to judge the conclusions of the included studies directly but to map out how the evaluation of the certainty of the evidence in the included SRs of interventions in dentistry using the five domains of the GRADE approach. Our findings highlight how SR authors in the dental field can improve its application and reporting rigorousness. In addition, this study is the first to be carried out with a specific focus on dentistry.

Less than one in four SRs of interventions in dentistry used the GRADE approach to assess the certainty of the evidence. Among those that applied it, most applied GRADE appropriately. However, more than half of the reviews judged the certainty of the evidence as "low" or "very low". Incorporating and adequately assessing the certainty of evidence using the GRADE approach by systematic reviews authors shall be encouraged.

GRADE - Grading of Recommendations, Assessment, Development, Evaluation approach

SRs - Systematic Reviews

RCTs - Randomized Clinical Trials

OSF - Open Science Framework

NRSI - Non-randomized studies of interventions

Ethics approval and consent to participate

Not applicable

Consent for publication

Not applicable

Availability of data and materials

The datasets used and/or analysed during the current study are available from the Supplementary Material in this article and from corresponding author on reasonable request.

Competing interests

The authors declare that they have no competing interests.

Funding

Not applicable

Authors' contributions

JMDO: Conceptualization, formal analysis, data curation, investigation, methodology, and Writing - original draft. LBO: Conceptualization, formal analysis, data curation, investigation, methodology, and Writing - original draft. PP: Conceptualization, formal analysis, data curation, investigation, methodology, and Writing - original draft. CMS: Conceptualization, formal analysis, methodology, visualization, and Writing - review & editing. CM: Conceptualization, formal analysis, methodology, visualization, and Writing - review & editing. CCM: Conceptualization, methodology, visualization, and Writing - review & editing.

GJMP: Conceptualization, methodology, visualization, and Writing - review & editing. MAP: Conceptualization, methodology, visualization, supervision, and Writing - review & editing.

CFM: Conceptualization, methodology, visualization, and Writing - review & editing.

GDLC: Conceptualization, methodology, visualization, supervision, and Writing - review & editing.

Acknowledgments

The authors thank the researchers, Dr. Silvio Bellini-Pereira, Dr. Leandro Chambrone, and Dr. Jayakumar Jayaraman, who kindly answered the electronic contacts for more information about their studies. We also thank the librarian MSc Karyn Munyk Lehmkuhl, for her assistance regarding the search strategy of this study.

Whittemore R. Combining evidence in nursing research: methods and implications. Nurs Res. 2005;54(1):56–62.
Urra Medina E, Barría RM, Pailaquilén. Systematic review and its relationship with evidence-based practice in health. Rev Lat-Am Enferm. 2010;18(4):824–31.
Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Annals Intern Med. 1997;126(5):376–80.
Murad MH, Montori VM. Synthesizing evidence: shifting the focus from individual studies to the body of evidence. J Am Med Assoc. 2013;309(21):2217–8.
Murad MH, Montori VM, Ioannidis JP, Jaeschke R, Devereaux P, Prasad K, Neumann I, Carrasco-Labra A, Agoritsas T, Hatala R. How to read a systematic review and meta-analysis and apply the results to patient care: users’ guides to the medical literature. J Am Med Assoc. 2014;312(2):171–9.
Galvão CM, Sawada NO, Trevizan MA. Systematic review: a resource that allows for the incorporation of evidence into nursing practice. Rev Lat-Am Enferm. 2004;12(3):549–56.
Newell R, Burnard P. Vital Notes for Nurses: Research for evidence-based practice, Wiley-Blackwell (2006).
Page MJ, Shamseer L, Altman DG, Tetzlaff J, Sampson M, Tricco AC, Catala-Lopez F, Li L, Reid EK, Sarkis-Onofre R. 2016. Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med 13(5), e1002028.
PubMed. http://pubmed.ncbi.nlm.nih.gov. (accessed 26 December 2020).
Bassani R, Pereira GKR, Page MJ, Tricco AC, Moher D. Sarkis-Onofre, Systematic reviews in dentistry: Current status, epidemiological and reporting characteristics. J Dent. 2019;82:71–84.
Ioannidis JP. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. The Milbank Q. 2016;94(3):485–514.
Guyatt GH, Sackett DL, Sinclair JC, Hayward R, Cook DJ, Cook RJ, Bass E, Gerstein H, Haynes B, Holbrook A. Users' guides to the medical literature: IX. A method for grading health care recommendations. J Am Med Assoc. 1995;274(22):1800–4.
L'abbé KA, Detsky As. O'rourke, Meta-analysis in clinical research. Annals Intern Med. 1987;107(2):224–33.
Oxman AD, Cook DJ, Guyatt GH, Bass E, Brill-Edwards P, Browman G, Detsky A, Farkouh M, Gerstein H, Haines T. Users' guides to the medical literature: VI. How to use an overview. J Am Med Assoc. 1994;272(17):1367–71.
Schünemann H, Brozek J, Guyatt G, Oxman A. Handbook for grading the quality of evidence and the strength of recommendations using the GRADE approach, updated October 2013 [online], 2013.
Atkins D, Eccles M, Flottorp S, Guyatt GH, Henry D, Hill S, Liberati A, O'Connell D, Oxman AD, Phillips B. Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches The GRADE Working Group. BMC Health Serv Res. 2004;4(1):38.
Hultcrantz M, Rind D, Akl EA, Treweek S, Mustafa RA, Iorio A, Alper BS, Meerpohl JJ, Murad MH, Ansari MT. The GRADE Working Group clarifies the construct of certainty of evidence. J Clin Epidemiol. 2017;87:4–13.
Ebell MH, Siwek J, Weiss BD, Woolf SH, Susman J, Ewigman B, Bowman M. Strength of recommendation taxonomy (SORT): a patient-centered approach to grading evidence in the medical literature. The J Am Board of Family Pract. 2004;17(1):59–67.
De Luca Canto G, Porporatti A, Massignan C, Flores-Mir C, Casett E, Porfírio GJM, Polmann H, Porto de Toledo I, Réus JC, Oliveira JMD, Dutra-Horstmann KL, Mezzomo LA, Savi MGM, Bolan M, Pauletto P, Mendes BD, de Souza C, Pacheco-Pereira CC, Martins E, Guerra G, Melo. Revisões Sistemáticas da Literatura: Guia Prático, Editora Brazil Publishing, Brazil, 2020.
Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R, Brozek J, Vist GE, Falck-Ytter Y, Meerpohl J. Norris, GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011;64(4):401–6.
Oliveira JMD, Oliveira LB, Stefani CM, Pauletto P, Massignan C, Porfírio GJM, Flores-Mir C, Peres MA, Canto GDL. The use of the Grading of Recommendations, Assessment, Development and Evaluation approach in systematic reviews in Dentistry, 2020.
Higgins J, Thomas J, Chandler J, Cumpston M, Li T, Page M, Welch V. Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (updated September 2020), Cochrane, 2020.
Munn Z, Stern C, Aromataris E, Lockwood C, Jordan Z. What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences, BMC Med. Res Methodol. 2018;18(1):1–9.
Specialty Definitions. https://www.ada.org/en/ncrdscb/dental-specialties/specialty-definitions. (accessed 26 December 2020).
GRADEpro GDT. https://gradepro.org. (accessed 26. December 2020).
Fleming PS, Koletsi D, Ioannidis JP, Pandis N. High quality of the evidence for medical and other health-related interventions was uncommon in Cochrane systematic reviews. J Clin Epidemiol. 2016;78:34–42.
Grading quality of evidence and strength of recommendations, in. : GRADE Working Group (Ed.) 2004, p. 1490.
Olsen O, Middleton P, Ezzo J, Gøtzsche PC, Hadhazy V, Herxheimer A, Kleijnen J, McIntosh H. Quality of Cochrane reviews: assessment of sample from 1998. BMJ. 2001;323(7317):829–32.
Shea B, Moher D, Graham I, Pham B, Tugwell P. A comparison of the quality of Cochrane reviews and systematic reviews published in paper-based journals. Eval & the Health Prof. 2002;25(1):116–29.
Collier A, Heilig L, Schilling L, Williams H, Dellavalle R. Cochrane Skin Group systematic reviews are more methodologically rigorous than other systematic reviews in dermatology. Br J Dermatol. 2006;155(6):1230–5.
Moseley AM, Elkins MR, Herbert RD, Maher CG, Sherrington C. Cochrane reviews used more rigorous methods than non-Cochrane reviews: survey of systematic reviews in physiotherapy. J Clin Epidemiol. 2009;62(10):1021–30.
Andrews J, Guyatt G, Oxman AD, Alderson P, Dahm P, Falck-Ytter Y, Nasser M, Meerpohl J, Post PN, Kunz R. GRADE guidelines: 14. Going from evidence to recommendations: the significance and presentation of recommendations. J Clin Epidemiol. 2013;66(7):719–25.
Guyatt G, Oxman AD, Sultan S, Brozek J, Glasziou P, Alonso-Coello P, Atkins D, Kunz R, Montori V, Jaeschke R. GRADE guidelines: 11. Making an overall rating of confidence in effect estimates for a single outcome and for all outcomes. J Clin Epidemiol. 2013;66(2):151–7.
Page MJ, McKenzie J, Bossuyt P, Boutron I, Hoffmann T, Mulrow C, Shamseer L, Tetzlaff J, Moher D. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews, MetaArXiv (2020).
Guyatt GH, Oxman AD, Vist G, Kunz R, Brozek J, Alonso-Coello P, Montori V, Akl EA, Djulbegovic B. Falck-Ytter, GRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias). J Clin Epidemiol. 2011;64(4):407–15.
Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, Alonso-Coello P, Glasziou P, Jaeschke R. Akl, GRADE guidelines: 7. Rating the quality of evidence—inconsistency. J Clin Epidemiol. 2011;64(12):1294–302.
Guyatt GH, Oxman AD, Kunz R, Brozek J, Alonso-Coello P, Rind D, Devereaux P, Montori VM, Freyschuss B, Vist G. GRADE guidelines 6. Rating the quality of evidence—imprecision. J Clin Epidemiol. 2011;64(12):1283–93.
Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, Alonso-Coello P, Falck-Ytter Y, Jaeschke R. Vist, GRADE guidelines: 8. Rating the quality of evidence—indirectness. J Clin Epidemiol. 2011;64(12):1303–10.
Guyatt GH, Oxman AD, Montori V, Vist G, Kunz R, Brozek J, Alonso-Coello P, Djulbegovic B, Atkins D. Falck-Ytter, GRADE guidelines: 5. Rating the quality of evidence—publication bias. J Clin Epidemiol. 2011;64(12):1277–82.
Pandis N, Fleming PS, Worthington H, Salanti G. 2015. The quality of the evidence according to GRADE is predominantly low or very low in oral health systematic reviews, PLoS ONE 10(7), e0131644.
Halladay CW, Trikalinos TA, Schmid IT, Schmid CH, Dahabreh IJ. Using data sources beyond PubMed has a modest impact on the results of systematic reviews of therapeutic interventions. J Clin Epidemiol. 2015;68(9):1076–84.

Table 1. Search strategy in PubMed database.

("Dentistry"[MeSH Terms] OR "dental"[All Fields] OR "dentistry"[All Fields] OR "oral"[All Fields]) AND ("Systematic Review"[Title/Abstract] OR "meta-analysis"[Title/Abstract] OR "metanalysis"[Title/Abstract] OR "meta analysis"[Title/Abstract]) AND ((y_1[Filter]) AND (fft[Filter]) AND (english[Filter]))

Table 2. Characteristics of the included SRs of intervention in dentistry that used the GRADE approach to assess the certainty of evidence (n=106).

VARIABLE		N (%)
Specialties definitions adopted by the American Dental Association*²²	Orthodontics and Dentofacial Orthopedics	25 (23.6)
	Periodontics	22 (20.8)
	Prosthodontics	18 (17.0)
	Oral and Maxillofacial Surgery	18 (17.0)
	Endodontics	8 (7.5)
	Pediatric Dentistry	8 (7.5)
	Orofacial Pain	5 (4.7)
	Oral and Maxillofacial Pathology	5 (4.7)
	Dental Anesthesiology	3 (2.8)
	Oral Medicine	3 (2.8)
Data synthesis	Meta-analysis	88 (83.0)
Data synthesis	Descriptive synthesis	18 (17.0)

*Nine articles were classified into more than one specialty following the definitions adopted by the American Dental Association.

Table 3. Reporting the certainty of evidence in the paper (n=106).

ARTICLE'S SESSION	Reporting the Certainty of Evidence	N (%)
Abstract	Y	67 (63.21)
Methods	Y	103 (97.17)
Results	Y	86 (81.13)
Summary of Finding Table	Y	91 (85.85)
Discussion	Y	73 (68.87)
Conclusion	Y NA	63 (59.43) 1 (0.94)
	Y NA	63 (59.43) 1 (0.94)

N, no; NA, not applicable; Y, yes.

Table 4. Rationales gave by the SRs to downgrade the certainty of evidence per GRADE domain (n=106).

DOMAINS*	AUTHORS's parameterS	N (%)
Risk of Bias	Not reported	26 (24.5)
Inconsistency	Not reported	38 (36.8)
	Heterogeneity (I²)	55 (51.9)
	Cochran Q test	9 (8.5)
	Tau²	4 (3.8)
	Confidence Interval (CI)	1 (0.94)
	Chi-square (chi²) test	5 (4.7)
Indirectness	Not reported	86 (77.4)
	PICO studied is not exactly the same as the review question	10 (9.4)
	Methodological heterogeneity ("no control group"; "great differences between included studies"; “no direct statistical data measuring the outcome”)	6 (5.7)
Imprecision	Not reported	59 (55.7)
	Confidence Interval (CI)	17 (16)
	Sample size	29 (27.4)
	Estimates not precise ("information bias"; "Different follow-up times / Insufficient information relative to prognosis"; “Different clinical measurement between included studies”; “Imprecision not serious as there was no serious adverse effect and cause minimal inconvenience”; and “No statistical difference observed between groups”)	6 (5.7)
	Number of events	4 (3.8)
Publication bias	Not reported	52 (49.1)
	Not applicable (<10 included studies)	20 (18.9)
	Egger's test	12 (11.3)
	Funnel plot	27 (24.8)

*All parameters presented in the table were described by the authors of the 106 included SRs and, therefore, counted independently according to the number of parameters used (duplication) for each domain of the GRADE approach.

Download PDF

Version 1

posted

You are reading this latest preprint version

Reporting of the certainty of evidence through GRADE in systematic reviews in dentistry indexed during late 2019 - late 2020

Status:

Version 1

Abstract

Figures

INTRODUCTION

METHODS

RESULTS

DISCUSSION

Strengths and Limitations

CONCLUSION

Abbreviations

Declarations

Ethics approval and consent to participate

Authors' contributions

Acknowledgments

References

Tables

Supplementary Files

Status:

Version 1