At Research Square, we aim to make research communication faster, fairer, and more useful. Central to this goal is improving the quality of research communication in order to facilitate rapid communication of important scientific results. Our Methods Reporting Badge is an important step towards this goal.
Research has a reproducibility crisis
Have you ever tried to reproduce the results of a report in your area of study, only to find that the methods are incomplete or that you can’t replicate the previous results? You’re not alone. A 2016 survey of over 1,500 researchers conducted by Nature reported that more than 70% of researchers had tried and failed to reproduce another scientist’s experiments.
Repeated efforts to replicate experiments drain time, funds, and resources from productive researchers. Time spent troubleshooting these failed efforts holds back the speed of publication and keeps researchers from progressing in their careers. Research based on poorly reported studies can lead researchers down the wrong path for years to come, and treatment decisions based on the conclusions of studies with poor reporting can be harmful to society as a whole.
For science communication to be effective, the ability to reproduce experiments is essential. Yet irreproducibility has become commonplace in laboratory research. Reporting in scientific studies is broken - and it needs to be fixed.
Methods reporting standards are key
For most studies, a critical gap in communication exists between the study’s initial design and its reporting in a manuscript. Information that is critical to replicating a study’s findings is often left out. For example, the source, catalog number, and lot number of antibodies and other reagents are often incomplete - or omitted entirely. Information regarding the age, sex, and strain of research animals is often lacking, as are critical details surrounding human study participants. Even simple aspects of study design, such as the number of repeats performed for each type of experiment, are often left out in the rush to publish.
Although these details are sometimes addressed at peer review, they are easy for reviewers to miss. Reviewers are typically focused on the overarching conclusions of a manuscript and its relevance and impact in the field - not how many times the flow cytometry experiments were repeated or what the lot number of the actin antibody was. Yet these are precisely the details that are indispensable to faithfully replicating a study, the act of which has the power to either reinforce or completely invalidate even the most intriguing study. When reporting standards are addressed at peer review, the revisions necessary to improve the reporting can hold up the peer review process because re-review is needed.
The Methods Reporting Badge signals thorough reporting at the preprint stage
Our Methods Reporting assessment aims to address issues in reporting standards at an early stage - before peer review is complete. An independent assessment of reporting standards will be conducted to help you improve your manuscript - making your study clearer, more useful, and easier to review. Preprints that meet the mark are tagged with a visible badge signaling their compliance with these community-established standards, adherence to which is encouraged or required by a growing number of funders and publishers.
Our Methods Reporting Badge addresses key areas including:
- Study Design: Reporting of sample size, replication, randomization, and blinding
- Reagents and organisms: Complete information about reagents, cell lines, animals, and human study participants
- Data collection and analysis: Information regarding software and data availability
Within 3-5 business days of submitting a manuscript for a Methods Reporting Badge, our editorial team completes its evaluation and sends the author a short report with guidance on the revisions that must be made in order to be compliant with the relevant standards. Once the manuscript has been verified to meet these standards, the corresponding badge icon is displayed on the public preprint along with a detailed report that can be downloaded by any reader.