Topic duplication and research waste at the ‘Overviews of systematic reviews’ level: Survey of overlapping overviews

Multiple overviews of systematic reviews conducted on the same topic (“overlapping overviews”) represent a waste of research resources and can confuse or mislead clinicians and policymakers. We aimed to assess the frequency and characteristics of published overviews addressing the same clinical question or topic.


Background
High-quality 'overviews of reviews' and systematic reviews give the best perspective of our current state of evidence on a subject. 'Overviews of systematic reviews' (henceforth called overviews) synthetize the results of multiple systematic reviews and help inform evidence-based clinical practice. Overviews are one of the multiple types of evidence syntheses. They are growing in number and popularity, and our Research is needed to establish whether overviews are performed on unique topics; represent updates of previous overviews done by the same team of authors; or pertain to topics that attract attention of different teams who independently perform and publish their overviews. In this second of two companion papers, we aimed to assess how common it is to have overviews of systematic reviews (overviews) published on the same topic, henceforth referred to as overlapping overviews. Methods

Study design
This is a methods study in the knowledge synthesis eld. We followed systematic review guidance for the searching, study selection, and data extraction stages of our review [20].

Eligibility criteria
As described in our rst companion paper, we performed a bibliometrics study of overviews published in MEDLINE (Ovid), Epistemonikos, and the Cochrane Database of Systematic Reviews. We included overviews meeting the following selection criteria: • Synthesises systematic reviews with or without meta-analyses (but the overview may also include primary studies) as a primary focus.
• Searches the literature systematically, and with a search strategy section found in the main body of the paper (i.e. search strategy includes text words and MeSH terms in at least two databases).
• Methods section located in the main body of the paper (not just in the abstract).
• Focuses on the effects of health interventions or clinical treatments.
We excluded overviews that based their results exclusively on primary studies and methodological studies. Reports that were editorials, letters, or comments were excluded. Overviews of risk, exposure, prevention, measurement instruments, quality indicators, diagnostic, screening or prognostic research were also excluded. We excluded protocols of overviews.
We included overviews published in any language and published from January 1, 2000 to December 31, 2018. Given that the Cochrane Handbook chapter on overviews was rst published in 2009, we did not expect to identify overviews published prior to 2000. Reports were translated by one of the authors (French, Spanish, German, Mandarin), when needed.
We reasoned that overviews may exist as a stand-alone report or also packaged as part of a clinical practice guideline and health technology assessment; accordingly, we developed eligibility criteria for both circumstances.
Inclusion criteria for clinical practice guidelines and health technology assessments (HTAs): Clinical practice guidelines or HTA aims to primarily include, synthesise and present the results of the systematic reviews; but may also include primary studies.
Overlapping overviews eligibility criteria We determined if two or more overviews were overlapping in topic if they investigated the same population(s), intervention(s), and at least one outcome. We did not consider an update by the same authors, or if the authors stated the overview was a replication, as an overlapping overview.

Search
Overviews were retrieved using a validated search lter [21] from MEDLINE (Ovid), Epistemonikos and the Cochrane database of systematic reviews (Appendix A). The Epistemonikos search was limited to the "Broad Syntheses" category which includes overviews of systematic reviews, HTAs and clinical practice guidelines.

Overview screening and study selection
Page 6/22 The initial search results were imported into Excel 2010 for screening. A pilot screening of the rst 19 papers was conducted in duplicate by all screeners to ensure high levels of agreement and common de nitions of coding. We screened the titles and abstracts against the stated eligibility criteria rst, then eligible full-text articles were reviewed for inclusion. Two independent reviewers screened reports at the title and abstract, and then again at the full text stage, then compared their results. Discrepancies were resolved by consensus, and arbitration by a third reviewer when necessary.
In a previously published study, 187 overviews (2013 and 2016) were screened [21,22] using identical methods (Appendix A). We therefore included these 187 studies and categorized them as "other sources".

Data extraction
We extracted data on overviews published between 2000 and 2018. Data extraction was piloted on 20 studies by all authors independently to identify any missing variables, come to agreement on coding de nitions, and re ne/reword the items. Discrepancies in the piloting phase were discussed and consensus reached by two authors. Full data extraction was performed independently by one investigator, and checked by a second reviewer.
We categorized the medical classi cation of each overview using the 10th revision of the International Statistical Classi cation of Diseases and Related Health Problems (ICD), a medical classi cation list by the World Health Organization (WHO) (https://www.who.int/standards/classi cations/classi cation-ofdiseases). The main condition or intervention in the title of the overview was entered into the search function of the WHO ICD-10 site (https://icd.who.int/browse10/2016/en) to determine its classi cation).
For example, the title "Nonpharmacological treatment for behavioural and psychological disturbances in older adults with dementia" was catagorised in the ICD-10 classi cation "Mental and behavioural disorders" because the intervention was treatment for behavioural disturbance in a dementia population.
In addition to ICD-10 classi cation, the topic of each overview was determined through review of the clinical question, including study population(s), intervention(s), and outcome(s).
For overlapping overviews, we additionally extracted whether an author was involved in more than one overview on the same topic. We also noted the journal of publication, number of included systematic reviews, search date; inclusion of meta-analysis; and funding status.

Data analysis
Descriptive analysis using frequencies and percentages were performed for categorical data and median and interquartile range (IQR) for continuous data.
The distribution of total overviews by medical classi cation was plotted in a bubble chart using Excel. The x axis represents medical classi cation, y axis the number of overviews pertaining to that medical classi cation and the size of the bubble (third variable) represents the cumulative number of systematic reviews in all overviews included in that classi cation. We described the gaps in ICD-10 medical classi cations covered by all overviews published between 2000 and 2018. We also charted the number of overviews covering a topic related to complementary and alternative medicines (CAM).

Search results
After searching MEDLINE, Cochrane, Epistemonikos and other sources we retrieved 10,145 records ( Fig. 1). After removal of duplicates, 8220 records remained, 6733 were excluded at the title/abstract stage, and 946 were excluded at the full text stage. A total of 541 overviews published between 2000 and 2018 were included (Appendix B). Many of the citations were excluded because they did not have a methods section, did not conduct a systematic search, and did not search for and include systematic reviews (Fig. 1).

WHO ICD-10 Medical Classi cations
The 541 overviews covered 20 of the 22 WHO ICD-10 Medical Classi cations (Fig. 2). The most frequent ICD-10 classi cation for retrieved overviews (92/541 [17%]) was "factors in uencing health status and contact with health services." Another 62/541 (11.5%) focused on diseases of the musculoskeletal system and connective tissue, 56/541 (10.4%) were about mental and behavioural disorders, 42/541 (7.8%) were on diseases of the circulatory system, and 34/541 (6.3%) were focused on neoplasms  Table 1). Overviews with overlapping patient(s), intervention(s) and/or outcome(s) A total of 65 topics involving one, or a combination of, overlapping patient(s), intervention(s) and outcome(s) were covered more than once by the 178 overlapping overviews ( Table 2). The most prevalent topic that overlapped across seven overviews pertained to interventions for smoking cessation (pharmacologic and non-pharmacologic). Five overviews focused on acupuncture for pain, ve on the topic of cannabinoids for pain and symptoms, and ve on exercise therapy for bone and muscle health. Ten topics were covered by four overviews each, 15 topics were covered by three overviews each, and 36 topics were covered by two pairs of overviews (Table 2). for various conditions. Appendix C lists the 178 overviews that overlapped for the 65 topics with their WHO ICD-10 medical classi cation, population, interventions, and outcomes.

Authors who publish overviews on similar topics
In 11/65 (17%) topics, at least one author was involved in two or more of the overviews that overlapped.
These duplicated overviews had partially overlapping participants, interventions, outcomes, and indications/settings. Of two broad overviews on spinal manipulations [23,24], the 2011 overview was written by two authors, and a year later, one of these authors published an overview with the same populations, intervention, and outcomes.
Characteristics of overlapping overviewsSome overlapping overviews were particularly broad in scope and considered several drugs or interventions, while some of the other overviews on the same topic considered only one of these interventions (e.g. [25,26]). For example, an overview by Kelly et al. entitled "Exercise and sleep: a systematic review of previous meta-analyses" was conducted which included studies with participants with certain conditions (obstructive sleep apnea, cancer) [27], and the second overview by the same authors focused on cancer-related fatigue entitled "Exercise and cancer-related fatigue in adults: a systematic review of previous systematic reviews with meta-analyses" [28]. Many overlapping overviews contained overlapping outcomes such as all-cause mortality, body weight, different de nitions of pain, and adverse events.
We also found that a speci c population in one overview could overlap with an overview considering a broader patient population on the same topic (e.g. [29,30]). For spinal manipulations, there were two overviews that considered any type of pain [23,24] and two that focused on speci c pain (headache [31] and chronic low back pain [32]).

Summary of results
This is the rst study to examine overlap across a cohort of overviews. Our methodological assessment identi ed 33% of overviews dated between 2000 and 2018 that overlapped in content across 13 WHO ICD-10 medical classi cations. Sixty-ve topics or clinical questions had two or more overlapping overviews. As many as 7 overviews (median of 2) were completed on the same topic (e.g. regarding smoking cessation) representing signi cant unnecessary and redundant overlap. We found that it was common for some overlapping overviews to cover broad topic areas, whereas others considered only subsets of the evidence. We conclude that some of the observed overlap in our sample seems unnecessary and re ects wasted effort, resources, and ine ciency.

Coverage and gaps in ICD-10 Medical Classi cations
The 541 overviews we included pertained to 20 of the 22 WHO ICD-10 Medical Classi cations. The most prevalent classi cation was "factors in uencing health status and contact with health services," "diseases of the musculoskeletal system and connective tissue", and "mental and behavioural disorders". Gaps in overview topics were found in two of the WHO ICD-10 classi cations "congenital malformations, deformations and chromosomal abnormalities", and "diseases of the ear and mastoid process". This study and database of overviews can provide a guide to authors about which topics are covered, and gaps in the evidence for future analysis.

Legitimate reasons for observed overlap
Observed overlap can be legitimately justi ed for several reasons. Overviews may be out-of-date and therefore an update including more recent studies published is needed. This reason is justi ed especially when systematic reviews on the topic are inconclusive, and their synthesis may reconcile discrepancies in their results, interpretation and conclusions. A duplicate overview may be warranted when an older overview used inappropriate or invalid methods, or was of low methodological quality. Other factors that lend to decisions to conduct an overview on a duplicated topic can include methodological considerations (e.g. if whether re-assessments of risk of bias, re-analyses of data, or (re)evaluation of GRADE assessments are required), or overviews with a broader scope than narrowly focused overviews A rationale for why an overlapping overview is needed should be provided by the authors.
Finally, replication is also a good reason to conduct an overview with the same or similar PIO.
Reproducibility of research by independent and con ict-free academics to obtain the same (or similar) results when repeating an experiment or test is one of the hallmarks of good science [33]. With replication and duplication of the same research results, decision makers, healthcare workers and patients can be con dent in the consistency and trustworthiness of the research [34]. Dr Moher recently suggested two to three systematic reviews were a su cient number to ensure replicability but four or more represented unnecessary duplication and research waste [34].

Potential for discrepant results and conclusions across overlapping overviews
We would hope that overviews on the same topic would nd the same results and come to similar conclusions, but this is often not the case with systematic reviews [12,[35][36][37]. Potential discrepancies can cause endless debates (e.g. [38]), and confuse and possibly mislead clinicians and policymakers [5].
Avoidance of this problem is the responsibility of researchers thinking about conducting overviews, funders, and publishers of this type of research.

Potential solutions and recommendations
To avoid unnecessary redundancy, several strategies can be used. First, protocols of overviews should be registered in a targeted database; second, journal editors and peer reviewers, funders and commissioners should require acknowledgement of other similar overviews and scrutinize the rationale where a de novo overview is proposed, rather than repurposing or updating an existing overview; and third, authors should cite the other known overviews that overlap in scope with a clinical or methodological rationale as to why the study is needed [39]. Authors wishing to conduct an overlapping overview should be clear about why a duplicate overview is needed. We do not advocate that authors conduct 'meta-overviews' (i.e. overviews of overviews) to try and explain differences in results across multiple overviews.
Currently, there is no dedicated database registry for protocols of overviews, such as there is for systematic reviews (i.e. PROSPERO). A dedicated database for overview protocols, as well as the development of a MeSH term for overviews would help in their identi cation by prospective overview authors, who when nding a similar overview, could choose a different topic to explore. A published search lter for overviews [21] can help in their identi cation, as would imbedded lters in databases such as Epistemonikos, MEDLINE, and Embase. The Cochrane Database of Systematic Reviews attempts to avoid duplication of effort by publishing only one overview and systematic review per topic of interest.
Other journals should follow suit and avoid duplicate publication of overviews with similar PIOs.
During the protocol phase and conduct of the overview, guidance [40] and methodological studies [41][42][43][44] should be consulted by to ensure rigor and a consistently high level of quality. Overviews of high quality will reduce the need for overlapping overviews and aid in avoiding wasting researchers' time, effort and money.

Strengths and limitations
Despite the growing popularity of overviews as a method to synthesise systematic reviews, to our knowledge this is the rst study to examine overlap across a cohort of overviews. A strength of our research is that we based our methods on systematic review guidance, and searched using a validated search strategy for overviews. We selected the overviews based on stringent eligibility criteria using two independent reviewers, who then compared their results and identi ed and resolved discrepancies.
One of our major limitations was classifying overlapping topics in overviews. Many broad overview topics could have been classi ed under several ICD-10 classi cations. For example, the overview entitled "Melatonin for health" was classi ed under "Mental and behavioural disorders" due to the primary theme of its outcomes (sleep latency, pre-operative anxiety, prevention of agitation and risk of breast cancer).
This same overview could have also been classi ed under the ICD-10 classi cation "Factors in uencing health status and contact with health services".

Conclusions
Our empirical evaluation identi ed duplication in 33% of overviews dated between 2000 and 2018. Sixtyve topics across 13 WHO ICD-10 medical classi cations had overlapping overviews. As many as 7 overviews addressed the same topic (e.g. interventions for smoking cessation) representing signi cant unnecessary and redundant overlap. A multiplicity of overviews on the same topic adds to the ongoing waste of research resources, time and effort across medical disciplines. To avoid duplication of research and redundancy, protocols of overviews should be registered in a targeted database, and overviews should cite other studies on similar topic with a rationale. This study and database of overviews provides a guide to authors about which topics are covered, and gaps in the evidence for future analysis.

Highlights
Overviews synthesise the results of multiple systematic reviews. This is the rst study to examine overlap across a sample of overviews.
Of 541 overviews published from 2000-to 2018, 178 (33%) were considered overlapping in topic with another overview.
Over 11/65 (17%) overlapping health topics, at least one author was involved in two or more overlapping overviews.