Characteristics of retracted systematic reviews and meta-analyses in the biomedical literature

The number of retracted reviews, including systematic reviews and meta-analyses, in the biomedical literature has increased dramatically in recent years. There has not yet been a systematic study to clarify the characteristics of retracted reviews. Present study determined trends in biomedical reviews, particularly with respect to reasons for retraction and geographical origin. Reviews were collated from Medline and Cochrane Library searches. Reasons for retraction were catalogued based on information retrieved via PubMed, Google, and journal websites. A total of 171 retracted non-Cochrane reviews and 439 Cochrane reviews were included. The most common reasons for retraction of non-Cochrane reviews were invalid review (n = 70), error (n = 36), and plagiarism (n = 20). Chinese authors produced a distinctively large number of reviews retracted for a faked review process and most retracted meta-analyses from areas of biochemistry molecular biology, oncology, genetics heredity and etc. Cochrane review retraction reasons included principally updating issues (n = 382), errors (n = 26), and sponsorship policy issues (n = 13). The number of retracted reviews in the biomedical literature, especially from Chinese authors, has surged since 2005. The major reasons for these retractions were related to updating among Cochrane systematic reviews but were related to misconduct among non-Cochrane systematic reviews. The present results highlight the need misconduct prevention measures.


Introduction
Systematic reviews and meta-analyses-collectively referred to as "reviews" in this article for simplicity-provide important support for evidence-based medicine because they summarize data quantitatively to produce composite ndings that can be used to guide future research and the clinical implementation of research ndings (Evans 2003).
The number of reviews being published has shown a consistently increasing trend over the past decade or so, and it had been argued that reviews are becoming increasingly redundant (Ioannidis 2016), with more than one metaanalysis of randomized trials for most topics and quite large numbers of meta-analyses in some elds.For example, 185 meta-analyses about treating clinical depression with antidepressants were published between 2007 and 2014 (Ioannidis 2016).Redundancies among such articles could give false con dence in the strength of their conclusions.
The output of meta-analyses produced in China in particular has increased dramatically in recently years, especially for studies about genetic associations (Ioannidis et al. 2013).
Accuracy within reviews, which depends on the quality of the data included, is critical for their interpretation.For example, a research group led by Boldt in Germany had a large number of retracted papers due to fraud; however, those retracted works may have impacted critical care medicine practices (Bellomo 2011).Speci cally, in Boldt's retracted studies, hydroxyethyl starch had been reported to bene t patients signi cantly (Boldt et al. 2007 andBoldt et al. 2010).Following exclusion of those retracted studies from a meta-analysis examining the use of hydroxyethyl starch in critically ill patients, results opposite to the ndings of Boldt's studies were obtained in the meta-analysis (Zarychanski et al. 2013).
Generally, retraction occurs when an article's integrity has been compromised and the validity and/or reliability of a study have come into question (Wager et al. 2009).The number of retractions in the biomedical literature has increased dramatically in recent years (Wager and Williams 2010;Steen 2013).Major causes for retraction include fraud/suspected fraud, error, duplicate publication, and plagiarism (Fang 2012).A recent surge in retractions of papers, including many meta-analyses, from China due to a faked review process attracted substantial attention (Fang 2012;Chen 2018;Wang 2018).Cochrane reviews, which is a highly regarded database for biomedical and healthcare related studies and treated as a key resource in evidence-based medicine, has implemented its own indexing withdrawal policy to deal with these challenges (Policy on withdrawing published Cochrane Reviews (including protocols)).
The characteristics of retracted reviews, such as country of origin, reason for retraction, and differing treatment among different indexing resources, have yet to be de ned.In particular, it would be of interest to determine whether retraction trends differ between Cochrane indexed and non-Cochrane indexed articles.The aim of the present work was to obtain a systematic outline of the characteristics of retracted systematic reviews and meta-analyses.

Collation of published reviews
On April 1, 2018, we searched PubMed for article type and publication date.To nd appropriate articles, we used the following search terms: meta-analysis, systematic review, retraction of publication, and retracted publication.To detect trends over time, we collected articles at 5-year intervals from 1990 to 2015 (1990, 1995, 2000, 2005, 2010, and 2015) as well as from 2017.We also searched for reviews in The Cochrane Database of Systematic Reviews with the aforementioned publication dates and the requirement of being in a journal that is indexed in PubMed.

Identi cation of retracted articles
On February 11, 2017, we conducted a PubMed literature search to identify all PubMed-indexed systematic reviews and meta-analyses without a limitation on the publication date.We used the publication-type search terms "retracted publication" or "retraction of publication".We also conducted a title word search with the search terms "retracted", "retraction", "withdrawal", and "withdrawn".We applied a lter to limit the results to systematic reviews and metaanalyses.
On April 11, 2017, we conducted a Cochrane Library search for articles with the terms "retracted", "retraction", "withdrawal", or "withdrawn" in the title, abstract or keywords.We re ned the results to articles classi ed as reviews.

Determined characteristics of retracted meta-analysis
On September 01, 2019, we searched the Medline database on the Web of Science for titles containing the phrase ''meta-analysis'' to determine research areas of published (or retracted) articles.For determining types of these metaanalyses according to methodology (genetics-related, treatment-related, diagnostic, and etc.), we also searched the Medline database on the PubMed using the strategy as previous study (Ioannidis et al. 2013) reported and separated the publications (retractions) in to ve groups as shown in table4.

Data processing
We examined the titles and abstracts of all identi ed papers manually, and excluded those that were redundant, had not been retracted, or were not systematic reviews or meta-analyses.We extracted and recording the following information for each of the remaining articles: journal name, journal impact factor according to the 2015 Web of Science, rst author's country of origin, publication year, and retraction year.
Because the Cochrane Library has its own withdrawal policy, we categorized the articles as Cochrane (published in The Cochrane Database of Systematic Review) and non-Cochrane (published in other journals) reviews for further analysis.Causes of retraction were de ned according to each retraction notice.For non-Cochrane reviews, the retraction reason classes were: error, duplicate publication, plagiarism, faked review process, authorship dispute, unclear, and others.For Cochrane reviews, the retraction reason classes were: update issues (out of date, replaced, superseded, updated, update pending, unable to update, split, or merged), error, sponsorship policy issue, and others.
The data were recorded and analyzed in Excel® (Microsoft Corporation, Redmond, WA).The data are presented as means ± standard deviations (SDs), numbers, and percentages.

Results
General trends in the publication of reviews From 1990 to 2017, the numbers of systematic reviews and meta-analyses published each year and indexed in PubMed increased by some 50-fold and some 30-fold, respectively (Table 1).Note that these increases occurred in the context of only a 3.1-fold increase in the overall number of all PubMed-indexed items during this period (Table 1).
Meanwhile, the Cochrane library, which is regarded as an important systematic review source, had relatively small numbers of review-type articles and did not exhibit the increasing trends seen in other resources (Table 1).With respect to percent of systematic reviews, The Cochrane Database of Systematic Reviews indexed 16.9% of the quantity of systematic reviews in Pubmed in 2000, but only 1.9% in 2017 (Table 1).We documented a dramatic increase in retractions in recent years, with less than 100 retractions per year occurring before 2005, 3-fold that number in 2010, and nearly 7-fold that number in 2015 (Table 1).
Of these meta-analyses, there were more papers about research areas of biochemistry molecular biology, oncology, genetics heredity and etc. from China.And, more meta-analyses from USA were about pharmacology pharmacy, cardiovascular system cardiology, behavioral sciences and etc. (Table 2).As shown in Table 3, there were also more gene-related and less treatment-related papers from China compared with from other countries.

Retracted non-Cochrane reviews
A total of 171 retracted non-Cochrane reviews were identi ed in the study years (Fig. 1), of which 120 were metaanalyses.There were 176 causes listed for these retractions, which were, in order from most to least common: faked review process (n = 70), error (n = 36), plagiarism (n = 20), other (n = 17), and duplicate (n = 12).The earliest retractions occurred in 1984, and the number of retracted publications increased dramatically after 2010.The rst retraction for which faked peer review process was the cited reason was in 2014.The mean (SD) time to retraction for these 171 retracted reviews was 32.4 (36.1) months.
Of these 171 articles, 155 (90.6%) were published in Science Citation Index listed journals, including 29 that were published in journals with impact factors > 5.0.The retracted articles were published in a cumulative total of 93 journals; 5 of these journals had ≥ 5 retractions (Table 1).With respect to meta-analyses, most (n = 68) were retracted for faked review process.Manipulation of the peer review process was associated with large numbers of retractions in a small number of journals, such as Tumor Biology (n = 30), Molecular Biology Reports (n = 14), European Journal of Medical Research (n = 6), and Molecular Neurobiology (n = 4).Meanwhile, the journals with the most retractions due to error (n = 5) and "other" reasons (n = 13) were Obstetrics and Gynecology and Contraception, respectively.
Of the 171 retracted non-Cochrane reviews identi ed, the rst author was from China in 99 cases, from the USA in 29 cases, from the UK in 7 cases, and from India in 6 cases.The remaining 30 retracted articles' rst authors were from 22 other countries.The distribution of retraction causes varied according to the rst author's country of origin (Fig. 2).
Notably, articles retracted for a faked review process were concentrated among retracted articles with Chinese rst authors.Conversely, of a total of nine meta-analysis retractions with rst authors from the USA, ve were retracted for the reason of error.Most of the retracted articles from Chinese rst authors (n = 96) were meta-analyses.
There were more gene-related meta-analyses retracted compared to its total publication numbers (Table 3).Most of these retracted papers were from areas of oncology, biochemistry molecular biology, or genetics heredity (Table 5).

Retracted Cochrane reviews
The impact factor of the Cochrane database of systematic reviews in 2015 was 6.10.There were 439 retracted reviews from the Cochrane library in the study years (Fig. 2).The reasons for retraction, from most to least common, were: updating issues (n = 382), errors (n = 26), other (n = 18), and sponsorship policy issues (n = 13).The major rst author countries of origin, from most to least common, were the UK (n = 184), Canada (n = 50), Australia (n = 41), the USA (n = 26), the Netherlands (n = 20), China (n = 17), and Brazil (n = 13).Most of retracted systematic reviews were classi ed as "withdrawn".

Discussion
The present study showed that among retracted non-Cochrane reviews, the most prevalent rst-author country of origin was China and the most common reason for retraction was a faked peer review process.These retracted publications were generally published in journals with low impact factors and retractions were concentrated, in majority, to a relatively limited number of journals.There were more retracted meta-analyses of gene-related types and of research areas about oncology, biochemistry molecular biology, genetics heredity and etc.Meanwhile, for the 439 retracted Cochrane reviews examined, the most prevalent rst-author country of origin was the UK and the most prevalent reason for retraction was updating-related issues, which are unrelated to error or scienti c misconduct generally.
Papers retracted for updating-related issues are considered to be "retired" due to providing guidelines or practical recommendations that are outdated and not amenable to a simple update (Alison 2018).Notably, we found that most retractions from The Cochrane Database of Systematic Reviews for updating-related issues were not tagged as "Retracted Publication" or "Retraction of Publication" type articles in Medline.The high representation of British rst authors among retracted Cochrane reviews may be a simple re ection of the output of systematic reviews by country.
Cochrane Reviews appear to be relatively underused in the USA especially, as well as in Canada and Australia, among other countries (Grimes et al. 2008).
According to rst-author country of origin, China was the source of the largest number of non-Cochrane systematic reviews identi ed in this study.There has been a surge in the number of reviews from Chinese authors in particular in recent years (Ioannidis 2013).The annual number of meta-analyses from China increased 40-fold between 2003 and 2011 versus 2.4-fold for the USA.China has become the most proli c producer of English-language, PubMed indexed meta-analyses.For example, of 9,135 articles published in 2014 and classi ed as meta-analyses in PubMed, over a third (n = 3,150, 34%) had Chinese author a liations; the USA was a remote second, producing only 822 metaanalyses in 2014 (9%) (Ioannidis et al. 2013).
The present study had shown some difference of published meta-analyses about disciplines between from China and USA.Chinese authors had published much papers about oncology, biochemistry molecular biology, and genetics heredity.While most of articles from USA were about pharmacology pharmacy, cardiovascular system cardiology, behavioral sciences and etc. Accordingly, most of retracted meta-analyses were from research areas of oncology, biochemistry molecular biology, and genetics heredity.These results were consistent with one recent study (Campos-Varela I et al. 2019), which show more retracted all types papers in PubMed published in journals of oncology and biochemistry molecular biology.There were also higher number of retracted meta-analyses related to genetics in present study.It had been reported that genetics was a discipline with high retraction rate (Dal-Ré et al. 2019).
Chinese authors had the largest numbers of retracted medical genetics articles and USA had produced more retractions about non-medical genetics (Dal-Ré et al. 2019).
It is interesting to note that most of the retracted meta-analyses related to genetics were from Chinese authors.In 2014, Chinese authors wrote 63% of the global total (1,210/1,910), whereas authors from the USA produced only 7% (136/1,910) (Ioannidis 2013).These meta-analyses may be redundant.Typically, meta-analyses from China neglected genome-wide data, and often included candidate gene studies published in Chinese-language journals (Ioannidis et al. 2016;Ioannidis et al. 2013).Meanwhile, reviews based on high-quality original reports were relatively sparse from Chinese authors (Ioannidis et al. 2013).Indeed, of the 100 most cited meta-analyses, none were produced by Chinese authors (Uthman et al. 2013).
The present ndings con rm a dramatically increasing trend of retracted publications from mainland China (Chen et al. 2018;Wang et al. 2018).There were very few retractions from Chinese authors before 2005, less than a 100 per year between 2005 and 2014, and more than 200 articles retracted per year since 2015 (Chen et al. 2018;Wang et al. 2018).Since 2012, articles with Chinese authors represent a quarter of the retractions in Medline.The common reasons for these retractions from Chinese authors overall in the last years were plagiarism (21.7%), error (18.3%), duplicate publication (16.2%), and faked review process (15.5%) (Chen et al. 2018).
Considerable manipulation of the peer review process has caused cascades of retraction of articles from China (Chen et al. 2018;Wang et al. 2018).The preeminence of peer review manipulation as a reason for retraction of reviews produced by Chinese authors was documented in this study.These retracted meta-analyses were concentrated in a small number of journals, especially Tumor Biology and Molecular Biology Reports.It has been suggested that an irrational scienti c evaluation system may be partly blame for this misconduct (Chen et al. 2018;Wang et al. 2018;Liao et al. 2018).Chinese researches have been pressured heavily to publish articles in Web of Science-indexed journals to achieve promotion and funding (Liao et al. 2018).Simultaneously, meta-analyses can be produced and published in journals with high impact factors with relatively limited nancial resources.Tragically, some editing companies made and sold meta-analyses from China for profit (Mara 2014).Numerous Obstetrics and Gynecology retractions were due to withdrawal of articles providing practical recommendations or guidelines and subsequent replacement with new reviews, rather than errors or misconduct.
This study had several limitations.Firstly, it is likely some cases of misconduct remain undetected in the literature.Th reluctance of some journals to investigate and potentially retract suspect papers may have led to an underestimation of the true frequency of retraction due to scienti c misconduct.Secondly, some systematic reviews that encompass data from awed articles have not been retracted.For example, 3 guidelines and 15 reviews in the radiation oncology literature have been reported to cite retracted articles as valid sources of data (Zietman et al. 2019).One recent study (Garmendia et al. 2019) had found that nearly half of 24 meta-analyses including falsi ed data of clinical trials had conclusions altered if excluded the awed results.

Conclusion
The rate and number of retracted reviews in the biomedical literature have increased substantially since 2005.The most common reason for retracted non-Cochrane systematic reviews, particularly from Chinese authors, was found to be a form of academic misconduct, whereas that for retracted Cochrane systematic reviews was found to be related to updating issues.China had produced most retracted meta-analyses from special research areas.The results of this study highlight the need to take measures to prevent misconduct.

Figures Figure 1
Figures

Table 1 .
of articles tagged as a "meta-analysis", "systematic review", "retraction of publication", "retracted publication" in PubMed and number of reviews Cochrane Database of Systematic Review by publication year.PubMed search conduced on March 22, 2018.Cochrane review data became accessible in PubMed in 2000.

Table 2 .
Distribution of meta-analyses in the Medline according to research areas from China, USA and others up to year of 2017 searched thorough Web of Science.

Table 4 .
Journals with most retracted reviews in the biomedical literature.

Table 5 .
Research areas of the retracted meta-analysis in the Medline up to year of 2017 searched thorough Web of Science.