We found that 38% of recently published SRs of interventions reported a registered or published protocol. Protocol registration was significantly associated with a higher impact factor of the journal publishing the SR and a more frequent reporting of use of PRISMA guidelines.
Compared to other studies, we observed a high percentage (38%) of interventional SRs with registered protocol. Studies dating in 2018 and 2021 reported 17%-20% of SRs with a registered protocol (15, 25). These two studies included different types of SRs, namely SRs of randomized controlled trials (15) and SRs of environmental studies with (non-)human intervention(25). The difference in observed proportions of protocol registration might therefore be explained by the earlier publication date of study, the research topic, or a combination of both.
In line with the studies of Ge et al. (2018) and Allers et al. (2018), we showed that SRs with a registered protocol were more likely to be published in journals with a higher impact factor and revealed higher quality of reporting (13, 15). In contrast with Ge et al. (2018), we did not find a associating between financial sponsorship and protocol registration. Interestingly, the number of SRs that reported the use of PRISMA in our sample was high for both SRs with a protocol (87%) and without a protocol (71%) compared with previous research (13).
It is remarkable that 71% of SRs without a protocol reported using PRISMA, but did not follow the directive of PRISMA that SRs should report on protocol registration or publication. A possible explanation is that PRISMA guidelines are more often consulted during the manuscript writing process than during the designing (and protocol writing) phase of the SR. This is supported by a recent survey study showed that almost 45% of researchers, who have written a SR between 2010-2016, have never registered a SR protocol (26). Most common reasons for not registering were: lack of knowledge on benefits or process of protocol registration, lack of time and non-mandatory requirement (26).
In 2005, the International Committee of Medical Journal Editors (ICMJE) has introduced mandatory trial registration guidelines for clinical trials on human subjects (27). No such requirement exists for SRs. Even though some journals and organisations have endorsed protocol registration in PROSPERO, our results show that one-third of SRs published in these journals did not have a protocol. Hence, recommendations in author guidelines are insufficient to effectively implement prospective SR protocol registration. High quality journals have the responsibility to uphold and continuously increase their quality standards to ensure high publication quality. Only after mandating registration of trials by the ICMJE, other journals followed and the uptake of trial registration increased. Journal and publishers could fulfill a pioneer role in raising quality standards by making protocol registration a mandatory requirement for publication. Furthermore, they could include protocol registration part of the peer-review process. Protocol registration awareness could be enhanced by introducing a mandatory protocol registration item in the abstract.
Preregistration of studies, which is similar to the Cochrane registration process, is adopted by some journals. During this process research articles undergo peer review at the study design or protocol stage (28). For Cochrane reviews, the scope of the proposed review and skills and experience of the proposed author team have typically been evaluated beforehand (29). This provides an opportunity for feedback and support in the designing phase of the review to increase SR quality.
In the context of the Open Science movement, the scientific community and journals need to navigate between recommendation versus obligation of protocol registration to strike a balance between transparency and quality on the one hand and researchers’ time investment on the other. From the perspective of patients and guideline developers, high quality, non-biased SRs are required for optimal clinical care and patient treatment for the reason that SRs and meta-analyses, more often than single clinical trials, directly inform medical guidelines and thereby medical treatment decisions.
Together, our results stress the need for further efforts to communicate the value of SR protocol registration more effectively, not only among researchers, but also among journal editors and peer reviewers. To guide further efforts to highlight the value of SR protocol registration, research is needed to gain understanding of the benefits and informativeness of SRs protocol registration amongst different stakeholders (researchers, journal editors, peer reviewers, policy makers, the public, clinicians, patients).
Strenghts & limitations
A strength of our study is that we have assessed protocol registration from a sample of published SRs. This in contrast with most previous studies, that have predominantly assessed SR protocol registration through PROSPERO records (13, 14, 30, 31). However, a limitation of this approach is that some publications might not have reported protocol registration in their manuscript. Aller et al. found that 12.5% of SRs with a protocol, did not report protocol registration or publication. We haven’t assessed whether this is the case in our sample and whether the proportion of intervention SRs with protocol registration we observed is an underestimation. When SR publications do not refer to the protocol, some of the benefits of protocol registration may be lost.
Another limitation is the single author selection and data extraction process. Through clear in- and exclusion criteria and well-defined items for data extraction, risk for errors and potential biases was minimized. Additionally, any difficulties in selection or data extraction were discussed amongst the researchers and a sample of the extracted data was checked.
Lastly, the generalizability of our study is limited, as we only included SRs of health care interventions. Protocol registration status might be different for other types of reviews (e.g., diagnostic, prognostic) and remains to be assessed in future research.