Historically, researchers have shied away from publicly sharing their work until it has been published in a journal. Due to lengthy delays within the science publication process, it can take between 14 weeks to eight years for research to be published, which dramatically slows the pace of science communication. What if researchers could post their manuscripts online while the research is traversing the publication process, allowing for fast communication? Preprints offer this advantage, as a rapid information dissemination vehicle. Read on to learn about new methods of science communication and if researchers need to be concerned about the reliability of research that has not been peer reviewed and formally published in a journal.
Distinctions between preprints and postprints
Since the 17th century origins of the science publication system, a division was established between research that had passed through the verification (peer review) process and been published, a postprint, and research that was unpublished, a preprint. There are two types of postprints, with slight differences: 1) a “postprint” that has been peer reviewed and accepted by a journal for publication; and 2) a “postprint publication” that has been typeset, copyedited, and branded by the journal. Overall, these two versions are comparable. The critical distinction of a postprint is that it has been peer reviewed and accepted by a journal for publication.
In contrast, preprints are drafts of manuscripts posted online in open access repositories before officially being published. They may be early drafts or polished work. Preprints give authors control over the initial dissemination of their work, allowing them to quickly communicate their findings, claim rights to their content and protect it from being scooped, and receive community feedback without being controlled by a publisher.
Are preprints risky if they have not been peer reviewed?
In the Web 2.0 age, preprints have exploded in popularity, with biomedical and life-science preprint servers growing over 2,000% in preprint deposition from 2017 through 2022. Despite the popularity of preprints, some in the scientific community have raised concerns that preprints are not safe to be shared with a wide audience since they have not been formally verified through peer review. Disclaimers are often found on preprints, branded by the server where they have been posted, clearly stating that the manuscript is a preprint that has not been peer reviewed and therefore should be interpreted with caution.
The safety concern hinges on the argument that the peer review process is a watchdog that identifies errors, fraud, and misconduct. Thus, until peer review has been completed less faith should be invested in preprints. However, the division between postprints and preprints is much closer than commonly perceived, as the peer review process does not live up to its hype. In fact, postprint publications are rife with problems that call to question the value of peer review and if there are major differences between preprints and postprint publications aside from copy editing and typesetting.
An exploration of the peer review process reveals that postprints may be no less “safe” and reliable than preprints, which encourages continued adoption and utilization of preprints as a dissemination mechanism to speed the pace of science communication. By examining the relationship between scientific conduct and verification, outlined below, the divergence between prepints and postprints is elucidated, demonstrating that these two versions of research may be remarkably similar.
Peer reviewers don’t always identify errors
Randomized trials examining reviewers’ ability to identify errors demonstrate that ~75% of reviewers fail to detect major errors in manuscripts and 68% do not recognize when conclusions are not supported by results.
Fraudulent methods are hard to detect
Researchers have admitted to using, and knowing colleagues that used, fraudulent methods. A meta-analysis of 21 surveys asked researchers if they had falsified data. Over 14% of researchers knew colleagues who falsified data, and up to 72% knew colleagues that used questionable research practices.
Retraction rates among postprints are rising
The consequence of fraud and lack of identifying errors during peer review is that the retraction rate of postprints has increased 10-fold since 1975. While more retractions could reflect heightened efforts to identify and remove questionable research, the volume highlights the sheer abundance of research requiring revocation and warrants real concern for the magnitude of undetected problems.
Out of 2,074 biomedical and life science articles, 67.4% were retracted because of misconduct, 43.4% from fraud, 14.2% from duplicate publication, 9.8% from plagiarism, and 21.3% due to simple error. In contrast, research examining retraction rates between 2000 and 2010 demonstrates that error (73.5%) is more common than fraud (26.6%) but echoes that retractions due to fraud statistically significantly increased throughout the decade. In 2013, 467 publications were retracted, and by 2015, the number had increased to 684. Between 2020 and 2021, over 2,216 postprints were retracted (~0.35% of all biomedical and life-science publications published between 2019-2020) in comparison to a mere 49 preprints (~0.042% of all preprints published between 2019 and 2020).
Finding similar results is hard
A staggering amount of research cannot be reproduced and replicated, with estimates ranging between 35–90% across major domains such as psychology, genetics, oncology, and cardiovascular disease, and cancer. Consequently, the results from the underlying trials are called into question. The irreproducible and irreplicable research was published via postprints, further demonstrating that “safe” and “valid” peer reviewed research may not actually be safe and valid.
So are there differences in quality between preprints and postprints?
Researchers examined differences in reporting quality between preprints in bioRxiv and their counterpart postprints archived in MEDLINE. The quality of reporting was measured using commonly used tools, such as STROBE, STARD, ARRIVE, CONSORT, etc. The results demonstrate that peer reviewed postprints have a slightly better reporting score, with absolute differences ranging between 4.7–5%. These absolute differences are small and imply that peer review may identify where researchers have failed to include basic information related to the reporting of a trial. However, the results say nothing about differences between preprints and postprints in respect to fraud and bias, which these quality reporting scales are not designed to evaluate.
In our digital world where information travels in seconds across platforms, and where misinformation can spread like wildfire, it is fair to question if research that has not been peer reviewed is safe to post online. As demonstrated, an alarming amount of research that has been vetted, gone through the formal publication process, and published in a journal, still can be riddled with errors and fraud, sometimes requiring retraction.
The scientific community should not be focused on if preprints are safe to post. Rather, the community should be concerned with much deeper problems that underlie the fear of safety. These problems include: 1) improving how researchers are trained to conduct trials so that they are free from bias at the start of a study; 2) the incentive structure to conduct research and the resulting biases that have led to the dissemination crisis; and 3) the methods that are used to evaluate research during the peer review process, which need to be reimagined so that it is more standardized with a structure to assess reporting quality, bias, and fraud.
If the community can improve the front-end of the research lifecycle by conducting better studies to begin with, the upstream consequences regarding safety concerns will diminish, especially when combined with an optimized peer review process.
Ultimately, preprints and postprints are two sides of the same coin. However, in contrast to postprints, preprints empower researchers to communicate their findings when and how they desire while breaking down paywalls so that knowledge-consumers can freely access the content. All scientific research needs to be interpreted with caution, especially when such a large amount across multiple domains cannot be reproduced and replicated. The scientific community needs to do more to improve the quality of studies when they are designed before having been conducted by ensuring the trial is as air-tight as possible with the least chance to introduce bias. Optimizing the research life-cycle will improve the reliability and safety of all studies, helping to ensure that knowledge is correctly translated from research into practice.
Are your researchers ready to submit a preprint today? Start your preprint submission process here. Also learn more about the benefits of preprinting on our Research Square preprint platform, as well as our presubmission tools and services.