In July, we posted an article about our Methods Reporting badge, which addresses gaps in methodological descriptions that can impair a study’s reproducibility. This piece focuses on the other assessment we recently rolled out - statistical reporting, which is as critical to reproducibility as it is to the appropriate interpretation of a study that relies heavily on statistics.
Inadequate statistical reporting creates unreliable results.
Replication is critical to confidence around the interpretation of study results. Researchers repeat experiments over multiple samples - or multiple experimental replicates - to ensure that their conclusions are reliable. In short, researchers want to know that what they’re reporting is what’s really happening in the universe. The results of a statistical analysis, reported with sufficient clarity, should help readers understand the robustness of a study's conclusions.
Unfortunately, the majority of studies are plagued by poor statistical reporting. A recent study found statistical analysis or reporting issues in up to 96% of papers. Many researchers who are experts in their fields may still not be adequately trained in statistics, and without this training, it can be difficult to meet reporting standards for statistical analyses. Data presentation adds an extra layer of complexity to preparing a manuscript, as the description of visual data must also meet these standards.
With the persistent pressure to publish often, it can be difficult to keep statistical reporting top of mind, but the details of data analysis are essential. Without them, readers are left unable to interpret data, and the conclusions of a study are unreliable.
Reporting standards are key.
Although issues with statistical analysis are sometimes addressed at peer review, they are easy for reviewers to miss. Reviewers, who are typically subject-matter experts, focus on the conclusions of a manuscript and the impact of the results on their field. As a result, they may overlook whether a specific p value was reported or whether adjustments were made for multiple comparisons - details that are critical to understanding and reproducing a study. When statistical reporting is addressed at peer review, the revisions necessary to improve the reporting can hold up the peer review process because re-review is needed.
The Statistics Reporting Badge signals that your study’s reporting is complete.
The Research Square Statistics Reporting Badge aims to address issues in reporting standards at an early stage - before peer review is complete. An independent assessment of statistics reporting standards will be conducted to help you improve your manuscript - making your study clearer, more useful, and easier to review.
Our Statistics Reporting Badge addresses key areas including:
- Sample Size: Precise sample sizes and/or replicate numbers provided for each type of experiment
- Statistical Tests and Values: Reporting of statistical tests performed as well as exact statistical values
- Data Presentation: Complete information about error bars, center values, plot elements, and indicators of statistical significance
Upon submitting your manuscript for a Statistics Reporting Badge, our editorial team will assess your manuscript within 3-5 business days, and you’ll receive a short report with guidance on the revisions that must be made in order to be compliant with the relevant standards. Once your manuscript has been verified to meet these standards, the corresponding badge icon will be displayed on your public preprint. You’ll also receive a downloadable certificate that can be used when submitting to a journal to demonstrate the quality of your manuscript.