A new taxonomy was developed for overlap across ‘Overviews of systematic reviews’: a meta-research study of research waste

doi:10.21203/rs.3.rs-665009/v4

Download PDF

Method Article

A new taxonomy was developed for overlap across ‘Overviews of systematic reviews’: a meta-research study of research waste

https://doi.org/10.21203/rs.3.rs-665009/v4

This work is licensed under a CC BY 4.0 License

Journal Publication

published 19 Dec, 2021

Read the published version in Research Synthesis Methods →

Version 4

posted

You are reading this latest preprint version

Background

Multiple ‘overviews of reviews’ conducted on the same topic (“overlapping overviews”) represent a waste of research resources and can confuse clinicians making decisions amongst competing treatments. We aimed to assess the frequency and characteristics of overlapping overviews.

Methods

MEDLINE, Epistemonikos and Cochrane Database of Systematic Reviews were searched for overviews that: synthesised reviews of health interventions and conducted systematic searches. Overlap was defined as: duplication of PICO eligibility criteria, and not reported as an update nor a replication. We categorized overview topics according to 22 WHO ICD-10 medical classifications, overviews as broad or narrow in scope, and overlap as identical, nearly identical, partial, or subsumed. Subsummation was defined as when broad overviews subsumed the populations, interventions and at least one outcome of another overview.

Results

Of 541 overviews included, 169 (31%) overlapped across similar PICO, fell within 13 WHO ICD-10 medical classifications, and 62 topics. 148/169 (88%) overlapping overviews were broad in scope. Fifteen overviews were classified as having nearly identical overlap (9%); 123 partial overlap (73%), and 31 subsumed (18%) others.

Conclusions

One third of overviews overlapped in content and a majority covered broad topic areas. A multiplicity of overviews on the same topic adds to the ongoing waste of research resources, time and effort across medical disciplines. Authors of overviews can use this study and the sample of overviews to identify gaps in the evidence for future analysis, and topics that are already studied which do not need to be duplicated.

Statistical Epidemiology

overviews of reviews

overview of systematic reviews

umbrella reviews

meta-reviews

‘reviews of reviews’

overlap

duplication

redundancy

waste

This is the first study to examine overlap across a sample of overviews of reviews.
Of 541 overviews published from 2000 to 2018, 169 (31%) were considered overlapping in topic with one or more overviews.

We classified overlap as identical, nearly identical, partial, or subsumed. Subsummation was defined as when broad overviews subsumed the populations, interventions and outcomes of another overview.
The majority of overlapping overviews covered broad topic areas, whereas others considered only subsets of the evidence. Most frequently, broad overviews had targeted populations for which multiple interventions were addressed, and least frequently broad overviews addressed a targeted intervention for multiple populations.
A limitation of this research was that the coding of clinical topics was prone to subjectivity when determining the overlap in PICOs. Performing this process in duplicate was an attempt to ensure accuracy.

Overlapping overviews, without warranted rationale, represent wasted efforts on the part of investigators and can confuse and mislead clinicians and policymakers.

Rigorous ‘overviews of systematic reviews’ (henceforth called overviews) and high quality ‘systematic reviews with or without meta-analysis’ give the best perspective of our current state of evidence on a subject. Overviews synthetize the results of multiple systematic reviews and help inform evidence-based clinical practice. Overviews, also called umbrella reviews, meta-reviews, or reviews of reviews, are one of the multiple types of evidence syntheses. They are growing in number and popularity, and our bibliometric study of the prevalence of overviews found an 8-fold increase in the number of overviews published between 2009 (n = 25) and 2020 (n = 332) [1]. The growth in overviews is unlikely to decrease [1].

Several author teams have expressed concerns about the volume of systematic reviews published which are overlapping in content [2-4]. Conflicting results or conclusions across systematic reviews on the same topic can confuse or create uncertainty for policymakers and clinicians who are required to choose among all competing treatments, and may impact and delay clinical decision-making [5]. These concerns can also be extrapolated to overviews reporting discordant findings. Another downside to the publication of multiple overviews on the same topic is that the efforts of the investigators, journal editors and peer reviewers may be unnecessarily duplicated.

Overviews have the potential for overlap because many are broad in scope, thus covering several individual topics. These broad overviews often address less specific research questions than their constituent systematic reviews, including a wider range of study populations and conditions, interventions, and contexts. Overviews that are broad in scope allow for policy relevance; for example, overviews have informed clinical practice guidelines (e.g. [6]) and government health policies (e.g. Australian Government Department of Health, 2015 [7]). Broad overviews can make a large volume of evidence accessible to clinicians and policymakers [8], but may necessitate extensive screening, data extraction, and the synthesis of a large number of systematic reviews.

Overviews may alternatively aim to answer narrow, focused clinical questions, and to identify and explore reasons for variation in the results, interpretation, or conclusions of systematic review analyses [9-18]. As an example, an overview that compared surgical versus conservative treatments for clavicle fractures aimed to determine which systematic review provided the most trustworthy evidence for treatment, and explored reasons for differences in review-level results [16]. Overviews with narrowly focused research questions can be completed more quickly as compared with broad research questions, but may have limited generalizability to different populations and settings [19].

Research is needed to establish if and how overviews overlap in content. For example, overviews may be performed on unique topics which only partially overlap; they may represent updates of previous overviews done by the same team of authors (similar to updated Cochrane reviews); replications; or may be redundant/salami slicing publications on the same topic [20].

This paper is the second of two companion papers. The first paper evaluated the bibliometric characteristics of overviews, and factors affecting citation rates and journal impact factors [21]. In this second paper, we aimed to determine if the overviews overlap in eligibility criteria according to their populations and interventions and at least one outcome, henceforth referred to as overlapping overviews. In a post hoc analysis we also aimed to categorise overviews as being narrow or broad in scope, and classify the overlap into four categories: partial, nearly identical, identical, or subsumed.

2.1. Study design

This is a meta-research study, which aims to evaluate research practices [22]. We followed systematic review guidance for the searching, study selection, and data extraction stages of our study [23].

2.2 Eligibility criteria

As described in our first companion paper [1], we performed a bibliometric study of overviews published in MEDLINE (Ovid), Epistemonikos, and the CDSR from 2000-2018.

Inclusion criteria for overviews including systematic reviews:

• Synthesises systematic reviews with or without meta-analyses (but the overview may also include primary studies) as a primary focus.

• Searches the literature systematically, and with a search strategy section found in the main body of the paper (i.e. search strategy includes text words and MeSH terms in at least two databases).

• Methods section located in the main manuscript or within a supplementary file (not just in the abstract).

• Focuses on the effects of health interventions (e.g., clinical treatments like medication or therapies).

We excluded overviews that based their results exclusively on primary studies and methodological studies. Reports that were editorials, letters, or comments were excluded. Overviews of risk, exposure, prevention, measurement instruments, quality indicators, diagnostic, screening or prognostic research were also excluded. We excluded protocols of overviews.

We included overviews published in any language and published from January 1, 2000 to December 31, 2018. Given that the Cochrane Handbook for Systematic Reviews of Interventions chapter on overviews was first published in 2009, we did not expect to identify overviews published prior to 2000. Reports were translated by one of the authors (French, Spanish, German, Mandarin), when needed.

We reasoned that overviews may exist as a stand-alone report or also packaged as part of a clinical practice guideline and health technology assessment; accordingly, we developed eligibility criteria for both circumstances.

Inclusion criteria for clinical practice guidelines and health technology assessments (HTAs):

Clinical practice guidelines or HTAs aim to primarily include, synthesise and present the results of the systematic reviews; but may also include additional primary studies.

Inclusion criteria for overlapping overviews

We determined two or more overviews were overlapping in topic if they investigated the same population(s) or condition(s), intervention(s), and at least one outcome (PICO). When reported, we did not consider an update by the same authors, or if the authors stated the overview was a replication, as an overlapping overview.

2.3 Search

Overviews were retrieved using a validated search filter [24] from MEDLINE (Ovid), Epistemonikos and the CDSR (Appendix A). The Epistemonikos search was limited to the “Broad Syntheses” category, which includes overviews of systematic reviews, HTAs and clinical practice guidelines.

2.4 Overview screening and study selection

The initial search results were imported into Excel 2010 for screening. A pilot screening of the first 19 papers was conducted in duplicate by all screeners to ensure high levels of agreement and common definitions of coding. We screened the titles and abstracts against the stated eligibility criteria first, then eligible full-text articles were reviewed for inclusion. Two independent reviewers screened reports at the title and abstract, and then again at the full text stage, then compared their results. Discrepancies were resolved by consensus, and arbitration by a third reviewer when necessary.

In two previously published studies categorising all methods used in overviews (2013 and 2016) [24, 25], 187 overviews were screened using identical methods (Appendix A). We therefore included these 187 studies and categorized them as “other sources”.

2.5 Data extraction

We extracted data on overviews published between 2000 and 2018. Data extraction was piloted on 20 studies by all authors independently to identify any missing variables, come to agreement on coding definitions, and refine/reword the items. Discrepancies in the piloting phase were discussed and consensus reached by two authors. Full data extraction was performed independently by one investigator and checked by a second reviewer.

We categorized the medical classification of each overview using the 10th revision of the International Statistical Classification of Diseases and Related Health Problems (ICD), a medical classification list by the World Health Organization (WHO) (https://www.who.int/standards/classifications/classification-of-diseases). The main condition or intervention in the title of the overview was entered into the search function of the WHO ICD-10 site (https://icd.who.int/browse10/2016/en) to determine its classification). For example, the title “Nonpharmacological treatment for behavioural and psychological disturbances in older adults with dementia” was catagorised in the ICD-10 classification “Mental and behavioural disorders” because the intervention was treatment for behavioural disturbance in a dementia population. In addition to ICD-10 classification, the topic of each overview was determined through review of the clinical question, including study population(s), intervention(s), and outcome(s).

For overlapping overviews, we additionally extracted whether an author was involved in more than one overview on the same topic. We hypothesised that if an overview was conducted by the “same team of authors” or had some authors in common, then it might be reported as an update of a previous overview, or might represent two forms of self-plagiarism: (a) redundant (duplicate) publication, or (b) salami slicing or a salami publication [20]. Redundant/duplicate publication can be defined as a reporting identical or very similar data in two or more papers without explicitly stating that the data are recycled [20]. Salami publication, aka salami slicing, is a form of redundant publication where different papers from the same data set are published. We also extracted the journal of publication, number of included systematic reviews, search date, inclusion of meta-analysis, and funding status.

2.6 Classification of overviews as broad or narrow in scope

In a post hoc analysis, we classified overviews as being broad or narrow in scope (Figure 1). We hypothesised that broad overviews were more prevalent than overviews with a narrow scope. We defined a broad overview as addressing (a) more than one distinct population (e.g. individuals with cancer, menopause, and lower back pain) or a generalized population (e.g. humans of all ages) and/or (b) multiple interventions (e.g. aerobic exercise, resistance training) for outcomes of interest. Broad overviews could therefore be further sub-classified as being non-targeted (multiple populations and interventions), as having a targeted population (with multiple interventions) or as having a targeted intervention (for multiple populations). We defined narrow overviews as covering only one population and one intervention or comparison (e.g. aripiprazole for patients with schizophrenia).

The scope of outcomes within overviews was not specifically addressed within the classification system in this study. We did not consider the broad or narrow scope of the overviews to mean a broad or narrow scope of the evidence but rather intended this system to be a means to characterize the breadth of the topic investigated.

This schema provides a general system to classify the scope of overviews and can be applied to overviews with or without topic overlap. When specifically assessing how two or more overviews overlap, the schema is helpful as a first step from which the overlap type can be more easily determined.

2.7 Classification of overlapping overviews

We classified the overlap across overviews based on the degree of similarity of their PICO eligibility criteria elements. We classified the overlapping overviews into four categories:

Identical overlap occurred when an overview’s eligible populations, interventions, comparisons, and all outcomes were identical to another overview. Identical overviews had to have the same aims and include the same study designs.
Nearly identical overlap was when an overview’s eligible populations, interventions, comparisons, and at least one outcome were identical to another. Nearly identical may have included overviews with the same PICO elements but allowed for inclusion of different study designs.
Partial overlap was defined as two or more overviews with at least one component of each of their populations, interventions, and comparisons in common, along with at least one common outcome (example in section 3.7).
Subsumed overlap was when the full scope of populations, interventions, comparisons and at least one outcome in an overview was addressed in full by a second (broader) overlapping overview.

A broad overview that subsumed another overview (broad or narrow) was classified as having partial overlap with the overview it encompassed, and the overview encompassed by the broader overview was considered subsumed. One overview could be classified in multiple categories depending on the number of overviews it overlapped with and the nature of the overlap (e.g. Wells et al. [26] is subsumed by Geneen [27]; and has partial overlap with Swinkels [28]). Two authors independently coded the overlapping overviews which was then checked by a second reviewer.

2.8 Data analysis

Descriptive analysis using frequencies and percentages were performed for categorical data and median and interquartile range (IQR) for continuous data.

The distribution of total overviews by medical classification was plotted in a bubble chart using Excel. The x axis represents medical classification, y axis the number of overviews pertaining to that medical classification and the size of the bubble (third variable) represents the cumulative number of systematic reviews in all overviews included in that classification. We described the gaps in ICD-10 medical classifications covered by all overviews published between 2000 and 2018.

3.1. Search results

After searching MEDLINE, CDSR, Epistemonikos and other sources, we retrieved 10,145 records (Appendix B Figure). After removal of duplicates, 8220 records remained, 6733 were excluded at the title/abstract stage, and 946 were excluded at the full text stage. A total of 541 overviews published between 2000 and 2018 were included (Appendix C). Many of the citations were excluded because they did not have a methods section, did not conduct a systematic search, and did not search for and include systematic reviews (Appendix B Figure). For example, of the 873 citations that might have been included as overviews (as they searched for, and included systematic reviews/meta-analyses, guidelines or HTAs), 122 (14%) did not contain a systematic search strategy.

3.2 WHO ICD-10 Medical Classifications

The 541 overviews covered 20 of the 22 WHO ICD-10 Medical Classifications (Figure 3). The most frequent ICD-10 classification for retrieved overviews (92/541 [17%]) was “factors influencing health status and contact with health services.” Another 62/541 (11.5%) focused on diseases of the musculoskeletal system and connective tissue, 56/541 (10.4%) were about mental and behavioural disorders, 42/541 (7.8%) were on diseases of the circulatory system, and 34/541 (6.3%) were focused on neoplasms (Figure 3). A little under half of the overviews focused on 15 other ICD-10 medical classification (Table 1).

Table 1: Frequency of WHO ICD-10 Medical Classifications in 541 overviews

WHO ICD-10 Medical Classifications	Frequency of overviews	Percent (%)
Blood and blood-forming organs	3	0.6
Certain conditions originating in the perinatal period	7	1.3
Certain infectious and parasitic diseases	22	4.1
Codes for special purposes	12	2.2
Congenital malformations, deformations and chromosomal abnormalities	0	0.0
Diseases of the circulatory system	42	7.8
Diseases of the digestive system	24	4.4
Diseases of the ear and mastoid process	0	0.0
Diseases of the eye and adnexa	5	0.9
Diseases of the genitourinary system	26	4.8
Diseases of the musculoskeletal system and connective tissue	62	11.5
Diseases of the nervous system	27	5.0
Diseases of the respiratory system	27	5.0
Diseases of the skin and subcutaneous tissue	7	1.3
Endocrine, nutritional and metabolic diseases	19	3.5
External causes of morbidity and mortality	23	4.3
Factors influencing health status and contact with health services	92	17.0
Injury, poisoning and certain other consequences of external causes	19	3.5
Mental and behavioural disorders	56	10.4
Neoplasms	34	6.3
Pregnancy, childbirth and the puerperium	9	1.7
Symptoms, signs and abnormal clinical and laboratory findings, not elsewhere classified	25	4.6
Total	541	100.00

3.3 Gaps in WHO ICD-10 Medical Classifications across 541 overviews

The WHO ICD-10 medical classification has 22 classifications in total. No overviews were found for two WHO ICD-10 Medical Classifications, namely ‘congenital malformations, deformations and chromosomal abnormalities’ and ‘diseases of the ear and mastoid process’ (Table 1).

3.4 Overlapping overviews according to medical classification

Of 541 overviews published from 2000 to 2018, 169 (31%) had at least one other overlapping overview (median 2 overviews per topic, interquartile range 2-3, maximum 6; listed in Appendix D). Of the 169 overlapping overviews, 39 (23%) were published on or before 2010, 61 (36%) from 2011 to 2015, and 70 (41%) from 2016 to 2019. These numbers show how incidence of overlapping overviews has changed chronologically.

The 169 overlapping overviews fell under 13/22 (59%) WHO ICD-10 Medical Classifications. The greatest number of overlapping overviews were classified under “factors influencing health status and contact with health services” (40/169 [24%]), followed by “musculoskeletal system and connective tissue diseases” (35/169 [21%]), and “symptoms, signs and abnormal clinical and laboratory findings, not elsewhere classified” (23/169 [14%]).

3.5 Overlap in overview topics

A total of 62 topics involving a combination of the same population(s), intervention(s) and outcome(s) were covered by two or more of the 169 overlapping overviews (Table 2). One topic was covered by six overviews, namely behavioral counseling and pharmacotherapy interventions for tobacco cessation. The topics of acupuncture for pain, cannabinoids for pain and symptoms, acupuncture for management of pregnancy-related symptoms, and exercise therapy for bone and muscle health overlapped across five overviews each. Nine topics were covered by four overviews each, 13 topics were covered by three overviews each, and 34 topics were covered by two pairs of overviews (Table 2).

Table 2: Overview topics covered by 169 overlapping overviews (n = 62)

Overview topics	Frequency	Percentage by topic (%)
Behavioral Counseling and Pharmacotherapy Interventions for Tobacco Cessation	6	9.7
Acupuncture for management of pregnancy-related symptoms	5	8.1
Acupuncture for pain	5	8.1
Cannabinoids for pain, and symptoms	5	8.1
Exercise therapy	5	8.1
Exercise to relieve pain	4	6.5
Nonpharmacological treatment for behavioural and psychological disturbances	4	6.5
Over-the-counter analgesics for pain	4	6.5
Pharmacological, non-pharmacological, and surgical treatments of low back disorders	4	6.5
Reduction interventions of alcohol intake	4	6.5
Spinal manipulation	4	6.5
Surgical treament of low back pain	4	6.5
Treatment of venous thromboembolism with LMWH and UFH	4	6.5
Vitamin D supplementation	4	6.5
Acupuncture for palliative care of cancer	3	4.8
Antipsychotic drugs for schizophrenia	3	4.8
Complementary and alternative procedures for fibromyalgia	3	4.8
Diets to reduce weight and obesity	3	4.8
Influenza vaccination	3	4.8
Interventions to treat complex wounds	3	4.8
Non-pharmacological interventions for osteoarthritis	3	4.8
Pharmacologic treatment of low back pain	3	4.8
Pharmacological and non-pharmacological treatments for depression	3	4.8
Physical activity promotion in children and adolescents	3	4.8
Preterm birth	3	4.8
Psychotherapy and non-medication-based interventions	3	4.8
Urinary incontinence	3	4.8
Acute asthma management in children	2	3.2
Assisted reproductive technologies (ARTs)	2	3.2
Childhood obesity interventions	2	3.2
Chronic treatment in childhood asthma	2	3.2
Cupping	2	3.2
Effects of coffee	2	3.2
Effects of financial arrangements for health systems in low-income countries	2	3.2
Exercise to relieve fatigue	2	3.2
Food supplements for body weight reduction	2	3.2
Ginkgo biloba for dementia	2	3.2
Hip fracture pre-op management and rehabilitation	2	3.2
Interventions for improving patient quality of life	2	3.2
Knee osteoarthritis - physical therapy	2	3.2
Lung cancer cost effectiveness analysis	2	3.2
Lung cancer treatment	2	3.2
Lupus nephritis treatment	2	3.2
Mammography screening	2	3.2
Management of hip and knee osteoarthritis	2	3.2
Manual therapy for the treatment of migraine	2	3.2
Neuraminidase inhibitors for influenza	2	3.2
Non-pharmacological interventions for insomnia	2	3.2
Non-pharmacological treatment for cancer-related fatigue	2	3.2
Opioid use in non-cancer pain	2	3.2
Periodontal treatment and glycaemic control	2	3.2
Pharmacological interventions for smoking cessation	2	3.2
Pharmacological treatments for major depressive disorder	2	3.2
Physiotherapy exercise	2	3.2
Pysiotherapy and exercise	2	3.2
Rheumatoid arthritis - nonparmacological interventions	2	3.2
Rheumatoid arthritis - pharmacological interventions	2	3.2
Rotator cuff repair surgery rehabilitation	2	3.2
Therapy for fibromyalgia syndrome	2	3.2
Treatment for multiple myeloma	2	3.2
Web-based interventions for weight loss	2	3.2

Appendix D lists the 169 overviews that overlapped for the 62 topics with their WHO ICD-10 medical classification, population, interventions, and outcomes.

3.6 Classification of overlapping overviews as broad or narrow in scope

Our hypothesis that broad overviews were more prevalent than overviews with a narrow scope was supported, as 148/169 (88%) overlapping overviews were characterized as broad in scope (Appendix D). Most frequently, broad overviews had targeted populations for which multiple interventions were addressed [65/148 (44%)], and least frequently broad overviews addressed a targeted intervention for multiple populations [27/148 (18%)]. Broad overviews categorised as non-targeted (56/148) accounted for 38% of broad overviews.

3.7 Classification of overlap as identical, nearly identical, partial or subsumed

The 169 overviews overlapped such that a similar portion, major component(s), or complete representation of an overview was duplicated across one or more different overviews. The following characterizes the type of overlap found:

0 identical (0%)
15 nearly identical (9%)
31 subsumed (18%), and
123 partial (73%)

We did not identify any overviews for which overlap was identical to another overview according to our definition. The 15 overviews with nearly identical overlap spanned across seven topics (periodontal treatment and glycaemic control (n = 2); food supplements for body weight reduction (n = 2); vitamin D supplementation (n = 3); acupuncture for managing gynaecologic conditions (n = 2); acupuncture for management of pregnancy-related symptoms (n = 2); acupuncture for pain (n = 2); rotator cuff repair surgery rehabilitation (n = 2)). A theme amongst nearly identical overviews was for slightly different outcomes to be investigated for the same patient population(s) and intervention(s). For example, overviews by Hasuike [29] and Botero [30] both addressed patients with Type 1 or Type 2 diabetes with periodontitis, receiving periodontal treatment, with a primary outcome of impact on hemoglobin A1c (HbA1c). Botero [30] includes an additional secondary outcome of fasting blood glucose, which Hasuike [29] does not. Hasuike [29] was published after Botero [30] but the dates of searches were comparable.

Two clusters of overviews (i.e. [31, 32] and [33, 34]) representing nearly identical overlap were from the same author group. In the latter example, the 2011 overview by Lee and Ernst [34] included Cochrane reviews, whereas the 2011 overview by Ernst, Lee and Choi [33] included reviews of all types [Cochrane and non-Cochrane]).

A notable example of nearly identical overlap were three vitamin D overviews by Autier [35], Mateussi [36], and Rejnmark [37]. They were categorised as nearly identical based on their similarly general populations and the intervention of vitamin D supplementation. Mateussi [36] explored a broad range of outcomes which included skeletal outcomes. Interestingly, the aim of the two other overviews [28, 35] was to examine specifically non-skeletal outcomes. Because there is overlap in at least one outcome (in fact, many are common to the three overviews) they are classified as nearly identical. However, the aims of the three are clearly different.

Subsummation of overviews could happen in different contexts. Some broad overviews covering an intervention for multiple patient populations subsumed one or multiple overviews investigating the intervention for just a portion of these populations. This situation is exemplified by the overlapping reviews by Posadzki [50-52], “Is spinal manipulation effective for pain? An overview of systematic reviews,” “Spinal manipulation: an update of a systematic review of systematic reviews,” and “Systematic reviews of spinal manipulations for headache: and attempt to clear up the confusion.” All three overviews address the targeted intervention of spinal manipulation for pain outcomes in varying populations. The former [50] is a broad overview focusing on patients with any types of pain, which subsumes the more specific populations in the broad overview focusing on patients with pain from any type of clinical condition [52]. Both of these overviews subsume the scope of the narrow overview investigating spinal manipulation in the specific sub-population with headache [51].

Sometimes overviews investigating multiple interventions in a targeted population subsumed others covering just a subset of these interventions. An example of this type of overlap happened with Chen’s [38] broad overview entitled “Treatment for lupus nephritis: an overview of systematic reviews and meta-analyses,” and Mac-Namara [39], “Is rituximab effective for induction of remission in lupus nephritis?” Both overviews investigated pharmacologic treatments in patients with lupus nephritis at various stages. Chen’s [43] overview broadly encompassed all interventions and more diverse regimens, which subsumed Mac-Namara’s [39] overview which targeted any medication regimen involving rituximab specifically.

Overviews were most often partially overlapping in various combinations of their populations, interventions, and outcomes. An example are the overviews by Wu, Towler, and Ezzo [34-36] which are visually represented in Figure 2. Wu (Overview A in Figure 2) [36] addressed any type of acupuncture therapy for cancer patients receiving palliative care. Towler (Overview B in Figure 2) [35] addressed a narrower intervention, acupuncture excluding any acupressure therapy, for a slightly broader population of cancer patients receiving palliative or supportive care. Ezzo (Overview C in Figure 2) [34] had an even narrower targeted intervention of acupuncture on point P6, and investigated multiple populations including cancer patients with chemotherapy-induced nausea and vomiting, post-operative nausea and vomiting, and pregnancy-related nausea and vomiting. Nausea and vomiting were outcomes common to all three overviews.

Figure 2 visually represents three broad, partially overlapping overviews from our sample.

3.8 Authors who publish overviews on similar topics

Amongst groups of overlapping overviews, 28/169 (17%) had at least one author involved in each of the overviews that overlapped. Of these 28, seven were self-reported as updates of older overviews (Appendix D). Subsummation was exemplified by three overviews published by a common author between 2011 and 2012 about the targeted intervention of spinal manipulation for pain [40-42]. In another instance, one of the same authors published two studies on acupuncture for pain which represented nearly identical overlap [33, 34].

3.9 Updated overviews or replications

Twelve studies (12/169; 7%) were reported as updates of previous overviews (Appendix D), and we found no studies self-reporting as a replication.

3.10 Overlap in Cochrane reviews

We identified 11 (7%) Cochrane overviews in our sample of 169 overlapping overviews. The majority (10/11; 91%) were partially overlapping with non-Cochrane overviews. One overview by Moore 2015 [43] was subsumed by a broader 2015 Cochrane overview by the same authors [44], and was partially overlapping with Els 2017 [45].

Summary of results

This is the first study to examine overlap across a sample of overviews. Our methodological assessment identified 31% of overviews dated between 2000 and 2018 that overlapped in content across 13 WHO ICD-10 medical classifications and 62 topics. If we would have included overviews without systematic searches/methods sections, we believe there would have been more overlapping overviews identified, which points to why systematic methods are needed. As many as six overviews (median of 2) were completed on the same topic (e.g. behavioral counseling and pharmacotherapy interventions for tobacco cessation). We found that it was common for some overlapping overviews to cover broad topic areas, whereas others considered only subsets of the evidence. Though slight differences in the scope of the overlapping overviews were observed, we feel that at least some of the multiplicity represents unnecessary overlap, although we address legitimate reasons for overlap below. This study and the database of overviews found in Appendix D can provide a guide to authors about which topics are covered, and gaps in the evidence for future analysis. Overview authors can use the appendix to determine if the topic they are wanting to examine is already published. Our definitions of overlap can be used by other methodologists studying overlap across randomised controlled trials, systematic reviews, and overviews.

Legitimate reasons for observed overlap

Observed overlap can be legitimately justified for several reasons. Overviews may be out-of-date and therefore an update, including more recently published, reviews is needed. This reason is justified especially when systematic reviews on the topic are inconclusive, and their synthesis may reconcile discrepancies in their results, interpretation and conclusions. For example, the overview by Doll [46] published in 2017 addressing the safety and effectiveness of neuraminidase inhibitors for influenza treatment, prophylaxis, and outbreak control provides and update of the literature reviewed in the overview published by Michiels in 2013 [47], after which more systematic review evidence became available. A duplicate overview may be warranted when an older overview used inappropriate or invalid methods, or was of low methodological quality (e.g. if whether re-assessments of risk of bias, re-analyses of data, or (re)evaluation of GRADE assessments are required). Broader, more comprehensive overviews may be warranted if existing overviews are narrow in focus. A rationale for why an overlapping overview is needed should be provided by the authors and citing existing overviews known to the authorship team to make the case.

Finally, replication is also an appropriate reason to conduct an overview on the same topic with the same or similar PICO. Reproducibility of research by independent and conflict-free academics to obtain the same (or similar) results when repeating an experiment or test is one of the hallmarks of good science [48]. As defined by Karunananthan et al. [49] for reviews, one type of replication involves repetition of PICO, using the same or very similar methods as a previous study to determine whether similar results can be obtained [49].

With replication of the same research results, decision makers, healthcare workers and patients can be confident in the consistency and trustworthiness of the research [50]. David Moher and colleagues have suggested two to three replicated systematic reviews with similar eligibility criteria would help ensure reliability of the findings, but four or more might represent unnecessary duplication and research waste [50]. Postulating from the replication of systematic reviews literature, we hypothesised that the number of appropriate replications for overviews where there has been no change otherwise in the literature should be one to two. The overview replication(s) would have to be accompanied by a strong rationale such as re-analyzing studies from included reviews when this was not done in the original overview.

Estimating the cost of research waste

Global research output is growing rapidly [51], as is the number of systematic reviews being produced [52] which means that evidence syntheses now take longer and cost more to conduct than they did over forty years ago [53]. In 1999, it was estimated that meta-analyses took on average of 1139 hours to complete [54], which has more than doubled in 2017 to approximately 2700 hours (ie. 67.5 weeks) [55]. The cost of conducting a systematic review is considerable, recently estimated at $141,000 USD [56]. For Canadian context, the Canadian Institutes of Health Research would previously allocate $100,000 CAD for a knowledge synthesis grant for which a systematic review would be the intended output (e.g. [57]). Due to the challenges in searching for, collecting and synthesing evidence in broad overviews, we estimate an overview could cost between $100,000 to $600,000 CAD. As a conservative estimate, of the 15 overviews that were nearly identical, two could be considered wasted. This would equate to a loss of between $200,000 to $1,200,000 Canadian research dollars.

Potential for discrepant or conflicting results and conclusions across overlapping overviews

We would hope that overviews with the same eligibility criteria on the same topic would find the same results and come to similar conclusions, but this is often not the case with systematic reviews [12, 58-60]. For example, two nearly identical overviews [61, 62] in our sample distinguished by additional outcomes other than range of motion (ROM), pain, functional improvements, with 6/9 meta-analyses in common, came to discordant conclusions. The overview by Mazuquin et al [62] found that of seven meta-analyses, early motion improves ROM after rotator cuff repair but increases the risk of rotator cuff retear. The overview by Houck et al [61], with nine included meta-analyses, came to an opposing conclusion; namely that early motion does not improve ROM and does not cause a higher retear rate. Potential discrepancies like these can cause endless debates (e.g. [63]), and confuse and possibly mislead clinicians and policymakers [5]. Avoidance of this problem is the responsibility of researchers thinking about conducting overviews, funders, and publishers of this type of research.

Potential solutions and recommendations

To avoid unnecessary overlap and redundancy, several strategies can be used. First, protocols of overviews should be registered in a targeted database; second, journal editors and peer reviewers, funders and commissioners should require acknowledgement of other similar overviews and scrutinize the rationale where a de novo overview is proposed, rather than repurposing or updating an existing overview; and third, authors should cite the other known overviews that overlap in scope with a clinical or methodological rationale as to why the study is needed [64]. Authors should clearly outline in their study protocol how their overview is different than earlier similar overviews based on: (a) the search strategy, (b) the results of the search, (c) the screening and inclusion criteria, (d) re-analysis of review data, etc. and then justify why a new overview is needed. Authors should also define why the choice of an overview design is preferable over other synthesis types such as scoping reviews, systematic reviews with meta-analyses, or network meta-analyses.

We do not advocate that authors conduct ‘meta-overviews’ (i.e. overviews of overviews) to try and explain differences in results across multiple overviews, as this would produce more redundant and potential useless research. Our list of 541 overviews have undergone some basic quality checks for inclusion (reported methods, conducted a systematic search, synthesised systematic review with or without meta-analysis), and can be used in policy decisions. There is currently no quality appraisal tool for overviews, and hence overlapping overviews cannot at this time be chosen based on a quality assessment. To choose one overview among several on the same topic, we suggest policymakers choose the overview with the closest research question in terms of population, interventions and outcomes. In addition, we suggest at the very least using our stated eligibility criteria for the overviews examined in our report to ensure a minimum level of rigour. Recency and comprehensiveness of the evidence accumulated should be next examined to determine which overview should be chosen by policymakers.

Currently, there is no dedicated database registry for protocols of overviews, such as there is for systematic reviews (i.e. PROSPERO). A dedicated database for overview protocols, as well as the development of a MeSH term for overviews would help in their identification by prospective overview authors, who when finding a similar overview, could choose a different topic or new scope to explore. A published search filter for overviews [24] can help in their identification, as would imbedded filters in databases such as Epistemonikos, MEDLINE, and Embase. The CDSR attempts to avoid duplication of effort by publishing only one overview and systematic review per topic of interest [65], although this has not been successfully empirically defended. In fact, in our study, we found subsummation and partial overlap across three Cochrane overviews. Other journals should follow suit and avoid duplicate publication of overviews with similar PICOs.

During the protocol phase and conduct of the overview, guidance [66] and methodological studies [67-70] should be consulted to ensure rigor and a consistently high level of quality. Overviews of high quality will reduce the need for overlapping overviews and aid in avoiding wasting researchers’ time, effort and resources.

Principles of transparency and best practice in scholarly publishing

Finally, we are against the practice of duplicate publications of the same research in different journals. We speculate that this was done by the authors of two overviews on acupuncture for pain [33, 34] as the 2011 overview was written by three authors, and a year later, two of these authors published an overview with the same populations, intervention, and efficacy outcomes, but this time focusing solely on Cochrane Reviews. The second paper references the earlier 2011 paper which was in print at the time, while providing a subsumed, subset of information from the earlier paper which we feel represents redundancy. Redundant publication is defined as findings that have previously been published elsewhere without proper attribution to previous sources or disclosure to the editor, permission to republish, or justification [71]. These same Principles of Transparency and Best Practice in Scholarly Publishing [71] state that “in cases of partial overlap (ie, when authors present new findings in an article that contains a substantial amount of previously published information) editors should consider whether the entire article is retracted or whether to issue a correction.” Retractions may be used to alert readers to cases of redundant publication, or partial overlap.

Strengths and limitations

Despite the growing popularity of overviews as a method to synthesise systematic reviews, to our knowledge this is the first study to examine overlap across a sample of overviews. A strength of our research is that we based our methods on systematic review guidance, and searched using a validated search strategy for overviews. We selected the overviews based on stringent eligibility criteria using two independent reviewers, who then compared their results and identified and resolved discrepancies. We acknowledge, however, that our methods were not outlined in a protocol and registered prior to undertaking this meta-research study. We recognize that this reduces the transparency of our pre-planned analyses and created the potential for redundancy in the efforts of this and other similar studies. We did, however, acknowledge transparently our post hoc analyses and declare that our categorization of overlap is therefore exploratory. A second study that uses our classification system and methods to pre-plan the analysis could be used to validate our findings.

A limitation to our study was the subjective nature of classifying overlapping topics in overviews. Many broad overview topics could have been classified under several ICD-10 classifications. For example, the overview entitled “Melatonin for health” was classified under “Mental and behavioural disorders” due to the primary theme of its outcomes (sleep latency, pre-operative anxiety, prevention of agitation and risk of breast cancer). This same overview could have also been classified under the ICD-10 classification “Factors influencing health status and contact with health services”. There was also room for error or subjectivity when extracting the overview eligibility criteria, i.e. PICOs to identify overlap, and classifying the type of overlap represented across overlapping overviews. Performing these step in duplicate or with a double-check were our attempts to ensure accuracy but we recognize that some of the overview topics were specialized and nuanced. Another limitation is that we excluded protocols of overviews; including them would have given us a broader sense of overlap.

Our empirical evaluation identified overlap in 31% of overviews dated between 2000 and 2018. Sixty-two topics across 13 WHO ICD-10 medical classifications had overlapping overviews. As many as 6 overviews addressed the same topic. We found that it was common for overlapping overviews to cover broad topic areas. Most frequently, broad overviews had targeted populations for which multiple interventions were addressed, and least frequently broad overviews addressed a targeted intervention for multiple populations. A multiplicity of overviews on the same topic adds to the ongoing waste of research resources, time and effort across medical disciplines.

To avoid duplication of research and redundancy, protocols of overviews should be registered in a open registry like the Open Science Framework or as a preprint, and overviews should cite other overviews on similar topic with a rationale as to why they are undertaking the overviews despite existence of others. Authors of overviews can use this study and the sample of overviews to identify topics that are already covered, and gaps in the evidence for future analysis.

1. Lunny, C., et al., Bibliometric study of ‘overviews of systematic reviews’ of health interventions: evaluation of prevalence, citation and journal impact factor. Research Synthesis Methods, 2021. n/a(n/a).

2. Ioannidis, J.P., The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta-analyses. Milbank Q, 2016. 94(3): p. 485-514.

3. Siontis, K.C. and J.P.A. Ioannidis, Replication, Duplication, and Waste in a Quarter Million Systematic Reviews and Meta-Analyses. Circ Cardiovasc Qual Outcomes, 2018. 11(12): p. e005212.

4. Moher, D., The problem of duplicate systematic reviews. BMJ : British Medical Journal, 2013. 347: p. f5040.

5. Ioannidis, J.P.A., The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta-analyses. The Milbank quarterly, 2016. 94(3): p. 485-514.

6. Zhang, W., et al., OARSI recommendations for the management of hip and knee osteoarthritis, part I: critical appraisal of existing treatment guidelines and systematic review of current research evidence. Osteoarthritis Cartilage, 2007. 15(9): p. 981-1000.

7. NHMRC, The Australian government's natural therapies review 2019-20 and the role of the national health and medical research council's natural therapies working committee. Australian Journal of Herbal and Naturopathic Medicine, 2019. 31(4): p. 140-141.

8. Caird, J., et al., Mediating policy-relevant evidence at speed: are systematic reviews of systematic reviews a useful approach? Evidence & Policy: A Journal of Research, Debate and Practice, 2015. 11(1): p. 81-97.

9. Chalmers, P.N., et al., Do arthroscopic and open stabilization techniques restore equivalent stability to the shoulder in the setting of anterior glenohumeral instability? a systematic review of overlapping meta-analyses. Arthroscopy, 2015. 31(2): p. 355-63.

10. Fountoulakis, K.N., R.S. McIntyre, and A.F. Carvalho, From Randomized Controlled Trials of Antidepressant Drugs to the Meta-Analytic Synthesis of Evidence: Methodological Aspects Lead to Discrepant Findings. Curr Neuropharmacol, 2015. 13(5): p. 605-15.

11. Lucenteforte, E., et al., Discordances originated by multiple meta-analyses on interventions for myocardial infarction: a systematic review. J Clin Epidemiol, 2015. 68(3): p. 246-56.

12. Osnabrugge, R.L., et al., A systematic review and critical assessment of 11 discordant meta-analyses on reduced-function CYP2C19 genotype and risk of adverse clinical outcomes in clopidogrel users. Genetics in Medicine, 2015. 17(1): p. 3-11.

13. Pantoja T, O.N., Ciaponni A, Herrera C, Lewin S, Oxman A, Paulsen E, Rada G, Wiysonge C Strategies for improving health systems in low-income countries: lessons learnt from four overviews of systematic reviews of health systems interventions. in Cochrane Colloquium. 2015. Vienna.

14. Pollock A, F.S., Brady M, Langhorne P, Mead G, Mehrholz J, van Wijck F Completing the first Cochrane overview of stroke reviews: experiences of the Cochrane Stroke Group in Cochrane Colloquium. 2015. Vienna.

15. Useem, J., et al., Systematic Differences between Cochrane and Non-Cochrane Meta-Analyses on the Same Topic: A Matched Pair Analysis. PLoS One, 2015. 10(12): p. e0144980.

16. Zhao, J.G., J. Wang, and L. Long, Surgical Versus Conservative Treatments for Displaced Midshaft Clavicular Fractures: A Systematic Review of Overlapping Meta-Analyses. Medicine (Baltimore), 2015. 94(26): p. e1057.

17. Zhao, J.G., et al., Intramedullary nail versus plate fixation for humeral shaft fractures: a systematic review of overlapping meta-analyses. Medicine (Baltimore), 2015. 94(11): p. e599.

18. Siontis, K.C., T. Hernandez-Boussard, and J.P.A. Ioannidis, Overlapping meta-analyses on the same topic: survey of published studies. BMJ : British Medical Journal, 2013. 347: p. f4501.

19. Gurevitch, J., et al., Meta-analysis and the science of research synthesis. Nature, 2018. 555(7695): p. 175-182.

20. Ding, D., et al., Duplicate and salami publication: a prevalence study of journal policies. International journal of epidemiology, 2020. 49(1): p. 281-288.

21. Lunny, C., et al., Bibliometric study of ‘overviews of systematic reviews’: evaluation of prevalence, citation and impact factor (preprint). 2021: Researchsquare.com.

22. Ioannidis, J.P., et al., Meta-research: evaluation and improvement of research methods and practices. PLoS biology, 2015. 13(10): p. e1002264.

23. Higgins, J.P.T., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M.J., Welch, V.A. (editors). , Cochrane Handbook for Systematic Reviews of Interventions version 6.0. . 2019, Cochrane. Available from www.training.cochrane.org/handbook: London, United Kingdom.

24. Lunny, C., J.E. McKenzie, and S. McDonald, Retrieval of overviews of systematic reviews in MEDLINE was improved by the development of an objectively derived and validated search strategy. J Clin Epidemiol, 2016. 74: p. 107-18.

25. Lunny, C., et al., Overviews of reviews incompletely report methods for handling overlapping, discordant, and problematic data. Journal of Clinical Epidemiology, 2020. 118: p. 69-85.

26. Wells, C., et al., Effectiveness of Pilates exercise in treating people with chronic low back pain: a systematic review of systematic reviews. BMC medical research methodology, 2013. 13(1): p. 1-12.

27. Geneen, L.J., et al., Physical activity and exercise for chronic pain in adults: an overview of Cochrane Reviews. Cochrane Database of Systematic Reviews, 2017(4).

28. Swinkels, A., et al., Exercise interventions for non-specific low back pain: an overview of systematic reviews. Physical Therapy Reviews, 2009. 14(4): p. 247-259.

29. Hasuike, A., et al., Systematic review and assessment of systematic reviews examining the effect of periodontal treatment on glycemic control in patients with diabetes. Medicina oral, patologia oral y cirugia bucal, 2017. 22(2): p. e167.

30. Botero, J., C. Rodríguez, and A. Agudelo‐Suarez, Periodontal treatment and glycaemic control in patients with diabetes and periodontitis: an umbrella review. Australian Dental Journal, 2016. 61(2): p. 134-148.

31. Pittler, M.H. and E. Ernst, Dietary supplements for body-weight reduction: a systematic review. The American journal of clinical nutrition, 2004. 79(4): p. 529-536.

32. Onakpoya, I.J., et al., Food supplements for body weight reduction: a systematic review of systematic reviews. Obesity, 2011. 19(2): p. 239.

33. Ernst, E., M.S. Lee, and T.-Y. Choi, Acupuncture: does it alleviate pain and are there serious risks? A review of reviews. PAIN®, 2011. 152(4): p. 755-764.

34. Lee, M.S. and E. Ernst, Acupuncture for pain: an overview of Cochrane reviews. Chinese Journal of Integrative Medicine, 2011. 17(3): p. 187-189.

35. Autier, P., et al., Effect of vitamin D supplementation on non-skeletal disorders: a systematic review of meta-analyses and randomised trials. The lancet Diabetes & endocrinology, 2017. 5(12): p. 986-1004.

36. Mateussi, M.V., et al., What do Cochrane systematic reviews say about interventions for vitamin D supplementation? Sao Paulo Medical Journal, 2017. 135: p. 497-507.

37. Rejnmark, L., et al., Non-skeletal health effects of vitamin D supplementation: A systematic review on findings from meta-analyses summarizing trial data. PloS one, 2017. 12(7): p. e0180512.

38. Chen, Y., et al., Treatment for lupus nephritis: an overview of systematic reviews and meta-analyses. Rheumatology international, 2017. 37(7): p. 1089-1099.

39. Mac-Namara, M. and G. Rada, Is rituximab effective for induction of remission in lupus nephritis? Medwave, 2014. 14(7): p. e6010-e6010.

40. Posadzki, P., Is spinal manipulation effective for pain? An overview of systematic reviews. Pain Medicine, 2012. 13(6): p. 754-761.

41. Posadzki, P. and E. Ernst, Systematic reviews of spinal manipulations for headaches: an attempt to clear up the confusion. Headache: The Journal of Head and Face Pain, 2011. 51(9): p. 1419-1425.

42. Posadzki, P. and E. Ernst, Spinal manipulation: an update of a systematic review of systematic reviews. NZ Med J, 2011. 124(1340): p. 55-71.

43. Moore, R.A., et al., Single dose oral analgesics for acute postoperative pain in adults‐an overview of Cochrane reviews. Cochrane Database of Systematic Reviews, 2015(9).

44. Moore, R.A., et al., Non‐prescription (OTC) oral analgesics for acute pain‐an overview of Cochrane reviews. Cochrane Database of Systematic Reviews, 2015(11).

45. Els, C., et al., Adverse events associated with medium‐and long‐term use of opioids for chronic non‐cancer pain: an overview of Cochrane Reviews. Cochrane Database of Systematic Reviews, 2017(10).

46. Doll, M., et al., Safety and effectiveness of neuraminidase inhibitors for influenza treatment, prophylaxis, and outbreak control: a systematic review of systematic reviews and/or meta-analyses. Journal of Antimicrobial Chemotherapy, 2017. 72(11): p. 2990-3007.

47. Michiels, B., et al., The value of neuraminidase inhibitors for the prevention and treatment of seasonal influenza: a systematic review of systematic reviews. PloS one, 2013. 8(4): p. e60348.

48. Popper, K., The logic of scientific discovery. 2005: Routledge.

49. Karunananthan, S., et al., When and how to replicate systematic reviews. Cochrane Database of Systematic Reviews, 2020(2).

50. Moher, D., The problem of duplicate systematic reviews. 2013, British Medical Journal Publishing Group.

51. Bornmann, L. and R. Mutz, Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references. JASIST, 2015. 66(11): p. 2215-2222.

52. Page, M.J., et al., Epidemiology and Reporting Characteristics of Systematic Reviews of Biomedical Research: A Cross-Sectional Study. PLoS Med, 2016. 13(5): p. e1002028.

53. Lau, J., Systematic review automation thematic series. Systematic reviews, 2019. 8(1): p. 1-2.

54. Allen, I.E. and I. Olkin, Estimating time to conduct a meta-analysis from number of citations retrieved. Jama, 1999. 282(7): p. 634-635.

55. Borah, R., et al., Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ open, 2017. 7(2): p. e012545.

56. Michelson, M. and K. Reuter, The significant cost of systematic reviews and meta-analyses: A call for greater involvement of machine learning to assess the promise of clinical trials. Contemp Clin Trials Commun, 2019. 16: p. 100443.

57. Canadian Institutes of Health Research (CIHR). Pre-announcement: Indigenous Peoples and COVID-19 Knowledge Synthesis, Evaluation and Assessment Grants Rapid Research Funding Opportunity. 2020.

58. Lucenteforte, E., et al., Discordances originated by multiple meta-analyses on interventions for myocardial infarction: a systematic review. Journal of Clinical Epidemiology, 2015. 68(3): p. 246-256.

59. Augustin, G., et al., Discordant outcomes of laparoscopic versus open appendectomy for suspected appendicitis during pregnancy in published meta-analyses: an overview of systematic reviews. Surgical Endoscopy, 2020. 34(10): p. 4245-4256.

60. Dettori, J.R. and D.C. Norvell, Discordant Systematic Reviews: Which to Believe? Global Spine Journal, 2020. 10(2): p. 237-239.

61. Houck, D.A., et al., Early versus delayed motion after rotator cuff repair: a systematic review of overlapping meta-analyses. The American journal of sports medicine, 2017. 45(12): p. 2911-2915.

62. Mazuquin, B.F., et al., Effectiveness of early compared with conservative rehabilitation for patients having rotator cuff repair surgery: an overview of systematic reviews. British Journal of Sports Medicine, 2018. 52(2): p. 111-121.

63. Trinquart, L., D.M. Johns, and S. Galea, Why do we think we know what we know? A metaknowledge analysis of the salt controversy. International Journal of Epidemiology, 2016. 45(1): p. 251-260.

64. Pieper, D., et al., Systematic review finds overlapping reviews were not mentioned in every other overview. Journal of clinical epidemiology, 2014. 67(4): p. 368-375.

65. Pollock, M., et al., Chapter V: overviews of reviews. Cochrane Handbook for systematic reviews of interventions version, 2018. 6.

66. Pollock, M., Fernandes, R.M., Becker, L.A., Pieper, D., Hartling, L.C, Chapter V: Overviews of Reviews. , in Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (updated September 2020). Available from www.training.cochrane.org/handbook, T.J. In: Higgins JPT, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). , Editor. 2020, Cochrane: UK.

67. Lunny, C., et al., Evidence map of studies evaluating methods for conducting, interpreting and reporting overviews of systematic reviews of interventions: rationale and design. Syst Rev, 2016. 5: p. 4.

68. Lunny, C., et al., Toward a comprehensive evidence map of overview of systematic review methods: paper 1-purpose, eligibility, search and data extraction. Syst Rev, 2017. 6(1): p. 231.

69. Lunny, C., et al., Toward a comprehensive evidence map of overview of systematic review methods: paper 2-risk of bias assessment; synthesis, presentation and summary of the findings; and assessment of the certainty of the evidence. Syst Rev, 2018. 7(1): p. 159.

70. Lunny, C., et al., Overviews of reviews incompletely report methods for handling overlapping, discordant, and problematic data. J Clin Epidemiol, 2020. 118: p. 69-85.

71. COPE Council, COPE Retraction guidelines — English. https://doi.org/10.24318/cope.2019.1.4 Committee on Publication Ethics.

Ethics approval and consent to participate

N/A

Data sharing and availability of data and materials

The full dataset in Appendix D is available on https://osf.io/q9t63/

Competing interests

The authors have declared no competing interest.

Funding

This work did not receive any funding.

Dissemination declaration

The results were disseminated in conference and university workshops, social media, and email lists.

Authors’ contributions

CL conceived of the study design, wrote the study protocol, analyzed the data, wrote and revised the final manuscript. GS, YWZ, CL, LL, PHY, MD, SS, PN screened citations and full-text articles. TN, AS, GS, ST, SA, UE, JHZ, EKR abstracted and verified data. CL, EKR, TN, BJS, and AS analysed the data. CL, EKR, and BJS interpreted results, and edited the manuscript. TN, AC, GS, AS, ST, SS, SA, YWZ, LL, PHY, MD, PN, UE, BJS, EKR, and JMW edited and approved of the final manuscript,

Acknowledgements

No acknowledgements.

AppendixABandC.docx
Appendices Appendix A – Search Strategy Appendix B - Figure 1: PRISMA flowchart of study selection Appendix C – 541 overviews included
AppendixDOverlappingoverviews20210824.xlsx
Appendix D lists the 172 overviews that overlapped for the 63 topics with their WHO ICD-10 medical classification, population, interventions, and outcomes.

Download PDF

Journal Publication

published 19 Dec, 2021

Read the published version in Research Synthesis Methods →

Version 4

posted

You are reading this latest preprint version

A new taxonomy was developed for overlap across ‘Overviews of systematic reviews’: a meta-research study of research waste

Status:

Journal Publication

Version 4

Abstract

Background

Methods

Results

Conclusions

Figures

Highlights

Background

Methods

Results

Discussion

Conclusions

References

Declarations

Supplementary Files

Status:

Journal Publication

Version 4