We registered the protocol within the Open Science Framework platform (registration ID: https://osf.io/aj2df). No amendments were made to the protocol while conducting the study. Our review question is: What is the magnitude and direction of change in the use of P-values and Bayesian analysis methods in child health RCTs published in 2007 and 2017, if any. We reported this review in accordance with the reporting guidance provided in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) . 1.
Study eligibility criteria
Eligible studies included RCTs in health research conducted among individuals aged 21 years and below published in 2007 and 2017 . We employed identical selection criteria used in the 2007 sample to maintain consistency and comparability with earlier findings . Our final samples were limited to full-text articles published in English language. We placed no restrictions on the settings in which the study was conducted, intervention, comparator or outcome.
We leveraged a pre-existing sample of child health RCTs published in 2007 (n = 300) , used by our team in a previous study of reporting quality of pediatric RCTs. Details of the search strategy and study selection methods for the 2007 sample are available in our previous publications [10, 13]. To identify a sample of studies published in 2017, a research librarian executed an updated literature search in the Cochrane Central Register of Controlled Trials (Additional file 1). The Cochrane Central Register of Controlled Trials includes randomized and quasi-randomized controlled trials indexed in MEDLINE and EMBASE, hand-searched results, gray literature sources, and Cochrane Review Groups Specialized Registers of trials .
All retrieved records were imported into EndNote (v. X9, Clarivate Analytics, Philadelphia, PA, United States) and exported to an Excel (v. 2016, Microsoft Corporation, Redmond, WA, United States) workbook for screening. Consistent with the methods used to identify the 2007 sample [10, 13], we randomly order the citations using the random numbers generator in Excel. Next, one reviewer (A.A., A.G., A.C., S.S., or M.S.) screened the titles and abstracts to identify the first 300 child health RCTs and when a record was deemed ineligible during data extraction, we substituted it with the next relevant record. We included the first 300 eligible citations from the randomly ordered list [10, 13]. The final sample included 600 child-health RCTs, 300 published in each of 2007 and 2017 (Figure 1).
We adopted part of the data extraction form from the 2007 study , with some additions to gain the information on P values and Bayesian analysis methods. We pilot tested the form using three studies from 2007 and 2017 for completeness and accuracy. One reviewer (A.A., A.G., A.C., S.S., or M.S.) extracted the data using Excel (v. 2016, Microsoft Corporation, Redmond, WA, United States); a second reviewer verified the extractions. Disagreements between reviewers, which occurred in <2% of the studies, were resolved by discussion between reviewers. We extracted data on characteristics of the publication, study design, intervention, control, trial conduct, study sample, sample size, hypothesis, primary objective, diagnostic criteria, recruitment strategies, funding, data monitoring committee (DMC), and specific statistical attributes of frequentist and Bayesian analysis/methods that were related to the primary outcome (Additional file 2). We extracted data for the primary outcome, and when this was not clearly stated, we used the objective outcome (e.g., mortality, hospitalization), the outcome used to calculate sample size, or the first outcome reported in the results. We used trial registers and published protocols (when cited in the publication) to supplement data extraction. When not cited in the publications, we searched for trial registers in the International Clinical Trials Registry Platform and Google databases.
We performed frequentist analyses using Stata (v. 16.1; StataCorp, College Station, Texas, United States) and Bayesian analysis using the jags program, called from within R statistical software [15, 16]. The analysis was mainly descriptive, using counts and percentages to compare the characteristics of trials between 2007 and 2017. We assessed the change in the proportion of trials that reported the P-value and Bayesian analysis using Pearson/Fisher Exact tests. Multinomial distributions with non-informative Dirichlet priors were used for the Bayesian analysis . A trial was described to have used Bayesian methods if any of the Bayesian inferential statistics or characteristics was used either in the methods (including hypothesis testing and analysis) or in the results of the study (Additional file 2). We performed a descriptive analysis to examine clustering of the P-value at specific significance levels and presented it in a graph. We investigated the predictors of using any element of Bayesian analysis using a logistic regression analysis.