Summary of results
This is the first study to examine overlap across a cohort of overviews. Our methodological assessment identified 33% of overviews dated between 2000 and 2018 that overlapped in content across 13 WHO ICD-10 medical classifications. Sixty-five topics or clinical questions had two or more overlapping overviews. As many as 7 overviews (median of 2) were completed on the same topic (e.g. regarding smoking cessation) representing significant unnecessary and redundant overlap. We found that it was common for some overlapping overviews to cover broad topic areas, whereas others considered only subsets of the evidence. We conclude that some of the observed overlap in our sample seems unnecessary and reflects wasted effort, resources, and inefficiency.
Coverage and gaps in ICD-10 Medical Classifications
The 541 overviews we included pertained to 20 of the 22 WHO ICD-10 Medical Classifications. The most prevalent classification was “factors influencing health status and contact with health services,” “diseases of the musculoskeletal system and connective tissue”, and “mental and behavioural disorders”. Gaps in overview topics were found in two of the WHO ICD-10 classifications “congenital malformations, deformations and chromosomal abnormalities”, and “diseases of the ear and mastoid process”. This study and database of overviews can provide a guide to authors about which topics are covered, and gaps in the evidence for future analysis.
Legitimate reasons for observed overlap
Observed overlap can be legitimately justified for several reasons. Overviews may be out-of-date and therefore an update including more recent studies published is needed. This reason is justified especially when systematic reviews on the topic are inconclusive, and their synthesis may reconcile discrepancies in their results, interpretation and conclusions. A duplicate overview may be warranted when an older overview used inappropriate or invalid methods, or was of low methodological quality. Other factors that lend to decisions to conduct an overview on a duplicated topic can include methodological considerations (e.g. if whether re-assessments of risk of bias, re-analyses of data, or (re)evaluation of GRADE assessments are required), or overviews with a broader scope than narrowly focused overviews A rationale for why an overlapping overview is needed should be provided by the authors.
Finally, replication is also a good reason to conduct an overview with the same or similar PIO. Reproducibility of research by independent and conflict-free academics to obtain the same (or similar) results when repeating an experiment or test is one of the hallmarks of good science [33]. With replication and duplication of the same research results, decision makers, healthcare workers and patients can be confident in the consistency and trustworthiness of the research [34]. Dr Moher recently suggested two to three systematic reviews were a sufficient number to ensure replicability but four or more represented unnecessary duplication and research waste [34].
Potential for discrepant results and conclusions across overlapping overviews
We would hope that overviews on the same topic would find the same results and come to similar conclusions, but this is often not the case with systematic reviews [12, 35–37]. Potential discrepancies can cause endless debates (e.g. [38]), and confuse and possibly mislead clinicians and policymakers [5]. Avoidance of this problem is the responsibility of researchers thinking about conducting overviews, funders, and publishers of this type of research.
Potential solutions and recommendations
To avoid unnecessary redundancy, several strategies can be used. First, protocols of overviews should be registered in a targeted database; second, journal editors and peer reviewers, funders and commissioners should require acknowledgement of other similar overviews and scrutinize the rationale where a de novo overview is proposed, rather than repurposing or updating an existing overview; and third, authors should cite the other known overviews that overlap in scope with a clinical or methodological rationale as to why the study is needed [39]. Authors wishing to conduct an overlapping overview should be clear about why a duplicate overview is needed. We do not advocate that authors conduct ‘meta-overviews’ (i.e. overviews of overviews) to try and explain differences in results across multiple overviews.
Currently, there is no dedicated database registry for protocols of overviews, such as there is for systematic reviews (i.e. PROSPERO). A dedicated database for overview protocols, as well as the development of a MeSH term for overviews would help in their identification by prospective overview authors, who when finding a similar overview, could choose a different topic to explore. A published search filter for overviews [21] can help in their identification, as would imbedded filters in databases such as Epistemonikos, MEDLINE, and Embase. The Cochrane Database of Systematic Reviews attempts to avoid duplication of effort by publishing only one overview and systematic review per topic of interest. Other journals should follow suit and avoid duplicate publication of overviews with similar PIOs.
During the protocol phase and conduct of the overview, guidance [40] and methodological studies [41–44] should be consulted by to ensure rigor and a consistently high level of quality. Overviews of high quality will reduce the need for overlapping overviews and aid in avoiding wasting researchers’ time, effort and money.
Strengths and limitations
Despite the growing popularity of overviews as a method to synthesise systematic reviews, to our knowledge this is the first study to examine overlap across a cohort of overviews. A strength of our research is that we based our methods on systematic review guidance, and searched using a validated search strategy for overviews. We selected the overviews based on stringent eligibility criteria using two independent reviewers, who then compared their results and identified and resolved discrepancies.
One of our major limitations was classifying overlapping topics in overviews. Many broad overview topics could have been classified under several ICD-10 classifications. For example, the overview entitled “Melatonin for health” was classified under “Mental and behavioural disorders” due to the primary theme of its outcomes (sleep latency, pre-operative anxiety, prevention of agitation and risk of breast cancer). This same overview could have also been classified under the ICD-10 classification “Factors influencing health status and contact with health services”.