The Score After Ten Years of Registration of Systematic Review Protocols

doi:10.21203/rs.3.rs-1431103/v1

Background: With the exponential growth of published systematic reviews (SR), there is a high potential for overlapping and redundant duplication of work. Prospective protocol registration gives the opportunity to assess the added value of a new study or review, thereby potentially reducing research waste and simultaneously increasing transparency and research quality. The PROSPERO database for SR protocol registration was launched ten years ago. This study aims to assess the proportion SRs of intervention studies with a protocol registration (or publication) and explore associations of SR characteristics with protocol registration status.

Methods: PubMed was searched for SRs of human intervention studies published in January 2020 and January 2021. After random stratified sampling and eligibility screening, data extraction on publication and journal characteristics; and protocol registration status was performed. Both descriptive and multivariate comparative statistical analysis were performed.

Results: A total of 357 SRs (2020: n=163; 2021: n=194) were included from a random sample of 1267 publications. Of the published SRs, 38% had a protocol. SRs that reported PRISMA have higher odds of having a protocol than publications that did not report PRISMA (OR 2.6; 95% CI: 1.20 to 5.69). SRs with a higher journal impact factor, have a higher odds of having a protocol (OR 1.12; 95% CI 1.02 to 1.23). Of the 19 SRs published in journals that endorse PROSPERO, 37% did not have a protocol. Most SR protocols were registered in PROSPERO (n=129; 96%).

Conclusions: We found that only 38% of recently published SRs of interventions reported a registered or published protocol. Protocol registration was significantly associated with a higher impact factor of the journal publishing the SR and a more frequent reporting of use of PRISMA guidelines. In some parts of the world, SR protocols are more often registered or published than others. Further research is needed to gain understanding of the benefits and informativeness of SRs protocols amongst different stakeholders, to guide strategies to increase the uptake of SR protocol registration.

Protocol registration: osf.io/9kj7r/

protocol registration

systematic review

implementation

Open Science

Events following the coronavirus disease 2019 (COVID-19) pandemic have underlined the realization that societies form interconnected networks that are part of a global community. This is a realization that the academic community has embraced for several decades (1). Collecting and reporting data in a transparent manner, as well as sharing data through collaborative networks, belong to the fundamental principles of Open Science (2). Closely related is the Open Access movement, which promotes free availability of research output. Both Open Science and Open Access aim to improve research quality, research data integrity, research data usage efficiency, as well as societal impact and implementation of academic research (3, 4).

Prospective protocol registration in publicly accessible repositories increases transparency and research quality. A priori protocols help researchers to prepare the research process, to choose appropriate methods, and to specify the research questions and outcomes beforehand (5). Registration of protocols facilitates thorough checking of review methods and reduces unnecessary duplication of research by independent researchers (6, 7). Furthermore, risk of bias is minimized and potential academic misconduct might be prevented (4). Besides these benefits, prospective protocol registration also serves clinicians, patients, and policymakers. Clinical trial registration (i.e., ClinicalTrials.gov) informs patients and clinicians about novel clinical interventions in the investigation phase (8). Also, trial registration may help to prevent cover up of non-favorable clinical outcomes, thereby reducing publication bias and increasing patient safety (4, 6).

In the era of evidence-based medicine, a number of organizations, including Cochrane, the Campbell Collaboration, and the Joanna Briggs Institute have served the research and clinical community with high quality evidence syntheses. These organization were among the first to enable public access to protocols for systematic reviews (SRs). Although Cochrane reviews constitute a small fraction (7%) of all published SRs (9), reviews from this organization are considered the ‘gold standard’ (10). A recent study reported that on average 80 systematic reviews were published per day (9). Other research has found that 67% of meta-analysis are overlapping, with in one of the topics a total of 13 overlapping meta-analyses (11). This underscores the necessity of protocol registration for non-Cochrane reviews, that could potentially reduce research overlap and duplication and thereby research waste and waste of public finance (7, 12). For this the Prospective Register of Systematic Reviews (PROSPERO) was launched in 2011, allowing researchers from various disciplines to register protocols for SRs related to health care (10, 13).

Ten years after the launch of PROSPERO, the number of protocol registrations for systematic reviews is increasing (10, 13). A limited number of studies has assessed SR protocol registration (10, 13–15). Most of these studies, assessed whether a registered protocol resulted in a published SR (10, 13, 14). One interesting exception is a study from 2015 that revealed that only 20% of the published SRs had a registered protocol (15). Noteworthy is that most SRs with registered protocols were Cochrane reviews (15). Hence, the proportion of SRs that have a registered protocol, ten years after the launch of PROSPERO, is currently unknown.

Objectives

This study aims to assess the proportion SRs of intervention studies which have a registered or published protocol. In addition, to gain more insight into factors associated with the availability of an openly accessible protocol, we explored the association of SR characteristics with protocol registration or publication status.

The protocol of this study was registered on the Open Science Framework (OSF) on September 17th, 2021: osf.io/9kj7r/ (16). Reporting of this manuscript was informed by the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) (17, 18).

Information sources

PubMed was searched for systematic reviews of interventions using the search filter by health-evidence Canada (19). Details on the search strategy are provided in Supplementary table 1. The search was last carried out on July 2nd, 2021. From the records identified by the search strategy, two random samples of 10% were drawn using simple random stratified sampling in EPPI-Reviewer. (20)

Eligibility criteria

Systematic reviews of human intervention studies were eligible for inclusion in this study if “systematic review” or “meta-analysis” was explicitly stated in the title or abstract. To avoid selection bias of protocol registration due to the global COVID-19 pandemic, we included records published in January 2020 or January 2021. Either e-pub or journal publication date was used for selection, which did not match the indexed publication date in PubMed for some of the studies (21). We had no restriction on language. Non-interventional reviews, literature reviews, updates of reviews and umbrella reviews were excluded.

Study selection and data collection

The selection process and data collection were performed consecutively by a single author (either KvdB, MG or CO). First, title and abstract were screened. When potentially eligible, full text was obtained and further screened for inclusion. When all eligibility criteria were met, data extraction was performed by the same author. Difficulties in study selection or data extraction were discussed by the authors until agreement was reached.

Data were recorded in a piloted data extraction sheet in Microsoft Excel (version 16.54, 2021). We manually extracted data on the characteristics of the SR publications (publication year, number of authors, number of affiliations, funding, conflicts of interest statement, reporting of PRISMA and COVID-relatedness). Additionally, the following details regarding protocol registration or publication were extracted: database of protocol registration; if registration number was reported and correct; and where in the article the protocol was mentioned (abstract, methods or other location). We identified journals that endorsed PROSPERO by consulting the website of PROSPERO (22) and assessed the number of SRs that were published in these journals.

From Web of Science and the Journal Citation Reports, data were automatically retrieved for: topic of the publication (Web of Science categories), journal, journal impact factor and journal of published protocol (if applicable). As journals without indexation in the Science Citation Index Expanded (SCIE) or the Social Science Citation Index (SSCI), have no impact factor, it was considered to be equal to zero.

Data synthesis and analysis

The primary outcome for this study was the proportion of SR publications with a protocol registration or publication. Characteristics of the reviews were quantified using descriptive statistics. We used frequencies and percentages for categorical variables. Mean and standard deviation were used for normally distributed continuous variables and median and interquartile range if data were skewed.

For our secondary objective, multivariable logistic regression models were used to explore whether characteristics between SR publications with a protocol vs. those without a protocol differed. All statistical analyses were conducted in RStudio (23) and figures were created using the ggplot2 package (24).

A total of 6669 records were identified in our literature search (Fig. 1). A random sample of 1267 records was screened and assessed for eligibility. Reasons for exclusion were publication date (i.e., in months other than January), non-interventional studies (diagnostic or prognostic studies) or review types other than systematic review (e.g., updated review, scoping review) or non-human studies.

In total, 357 publications were included for analysis (n = 163 for January 2020 and n = 194 for January 2021). Characteristics of the included SRs are portrayed in Table 1. Overall, 90% of SRs reported a COI statement, 75% reported to use PRISMA guidelines and most SRs were either not funded (n = 132, 37%) or had non-profit funding (n = 115, 32%). The included SR publications showed great variety in origin of the corresponding author (Fig. 2, Supplementary Fig. 1), topics (n = 63, Fig. 3) and journals in which the article was published (n = 280, Supplementary table 2). We identified 19 (5% of total sample) journals in our sample that explicitly endorses PROSPERO. For 17 publications, no topic category could be assigned. The three most frequent topics were Surgery (n = 39); Pharmacology & Pharmacy and Medicine (n = 31); and General & Internal (n = 28).

Table 1. Characteristics of included articles

	All SRs	SR with protocol^*	SR without protocol^*	Cochrane Reviews
Total, n/N (%)	357	135/357 (38)	216/357 (61)	6/357 (1)
Year, n/N (%) 2020	163/357 (46)	54/163 (33)	106/163 (65)	3/163 (2)
2021	194/357 (54)	81/194 (42)	110/194 (57)	3/194 (1)
COVID related – Yes, N (%)	7 (2)	2 (1)	5 (2)	0 (0)
No. authors mean + SD	6 ± 3	6 ± 3	6 ± 3	5 ± 3
No. affiliations mean + SD	4 ± 3	4 ± 3	4 ± 3	4 ± 3
Funding source, N (%)
No funding	132 (37)	50 (37)	79 (37)	3 (50)
Nonprofit	115 (32)	45 (34)	67 (31)	3 (50)
Not reported	90 (25)	35 (26)	55 (25)	0 (0)
Not clear	13 (4)	3 (2)	10 (5)	0 (0)
For profit	7 (2)	2 (1)	5 (2)	0 (0)
COI statement – Yes, N (%)	324 (91)	122 (90)	196 (91)	6 (100)
PRISMA reported, N (%)
Yes	270 (75)	117 (87)	153 (71)	0 (0)
No	53 (15)	10 (7)	39 (18)	4 (67)
Only in flow-chart	34 (10)	8 (6)	24 (11)	2 (33)
Journal Impact Factor, mean + SD	3.8 (2.6)	4.3 (2.7)	3.5 (2.5)	9.3 (0)
Journal endorses PROSPERO, N (%)	19 (5)	6 (4)	7 (3)	6 (100)

Abbreviations: SR=Systematic reviews, COI=conflicts of interest, PRISMA= Preferred Reporting Items for Systematic Reviews and Meta-Analysis, SD=standard deviation

^*Both protocol registration or protocol publication was counted as ‘SR with protocol’. More detailed information on protocol registration or publication is provided in results section.

Proportion of SRs with a protocol & differences with SRs without a protocol

A total of 135 (38%) SRs had a protocol either registered, published or both and 6 (1%) were Cochrane reviews (Table 1). The number of authors, affilications, source of funding and COVID-19 relatedness were similar between the SRs with and without a protocol. The percentage of SRs with a protocol was 42% in 2021 compared to 33% in 2020. Between the continents, percentages of SRs with a protocol varied from 25% (Asia) up to 61% (Australia). Several topic categories had ³50% SR publications with a protocol, such as: Clinical Neurology; Psychiatry; Nutrition & Dietetics; Rehabilitation; Respiratory System; and Public, Environmental & Occupational Health. As there is a great variety in topics, only those with more than five publications are mentioned.

In SRs with a registered protocol, 87% reported to have applied the PRISMA guidelines. In contrast, in SR publications without protocol registration, the percentage that reported the PRISMA guidelines was 71%. The average journal impact factor was 4.6 (SD 2.6) for SRs with a protocol and 3.9 (SD 2.3) for those without a protocol. A total of 29 reviews were published in journals that were not indexed in the SCIE or SSCI. Of those reviews, 7 had a protocol and 22 did not have a protocol. Of the SRs published in PROSPERO-endorsed journals, 37% (7 of 19) did not have a registered or published protocol.

A multivariable logistic regression was fitted including year of publication, number of authors, number of affiliations, funding, COI, PRISMA, journal impact factor and continent (Supplementary table 3). Cochrane Reviews (n=6) were excluded from this analysis. Factor that were related to protocol registration or publication were; reporting of PRISMA (reported vs. not reported), journal impact factor and Asia as country of origin (versus Europe) (McFadden R²=0.09, p<0.001). Publications that reported PRISMA have higher odds of having a protocol than publications that did not report PRISMA (OR 2.6; 95% CI: 1.20 to 5.69). SRs with a higher journal impact factor, have a higher odds of having a protocol (OR 1.12; 95% CI 1.02 to 1.23). Publications from Asia had a lower odds of having a protocol (OR 0.37; 95% CI 0.20 to 0.69, reference category= Europe).

Protocol registration or publication

In Cochrane reviews, protocol registration is reported in a specific section of the publication, in which deviations from the protocol are clarified. In line with Cochrane instructions, in the six Cochrane reviews from our study sample, protocol registration was not reported in the abstract or methods section of the publication.

From the non-Cochrane review publications with a protocol (n=135), 96% (n=129) had protocol registration alone and 4% (n=5) had both registered and published their protocol. One SR had a published protocol. Protocols were published in four different journals and one protocol was posted on the National Institute for Health and Care Excellence (NICE) website (Supplementary table 4). Note that Cochrane reviews (n=6) and the one SR with protocol publication alone are not presented in Table 2.

Most SR publications reported protocol registration either in the methods section (76%) or both in the abstract and methods (19%) (Table 2). Three SR publications reported information on protocol registration solely in the abstract (2%). Other sections in which protocol information was reported were: at the end of the article (n=4), the introduction (n=1) or in the supporting information (n=1).

To date, several databases for protocol registration are available. PROSPERO was the first database established specifically for protocols of systematic reviews. In our study, PROSPERO was predominantly used for protocol registration (n=129; 96%). Other databases that were used are OSF (n=3; 2%), the International Platform of Registered Systematic Review and Meta-Analysis Protocols (INPLASY) (n=1; 1%) and the Research Registry (n=1; 1%). As a correct identifier or registration number is key for protocol retrieval, we assessed the number of publication with a correct registration number. Almost all publications reported a registration number (n=128; 96%), and most of those registration numbers were correct (n=119). Eleven publications (8%) provided a direct URL to the protocol.

Table 2. Characteristics of SRs with protocol registration

Items	With protocol registration (N=134) N (%)
Where is protocol registration or publication reported
Abstract	3 (2)
Methods	102 (76)*
Abstract & methods	25 (19)^
Other	4 (3)
Database of protocol registration
PROSPERO	129 (96)
OSF	3 (2)
Other	2 (2)
Registration number reported, Yes	128 (96)

Abbreviations: PROSPERO= Prospective Register of Systematic Reviews, CDSR= Cochrane Database of Systematic Reviews, OSF=Open Science Framework

* Besides reporting protocol registration in the methods, one SR reported it in supporting information as well

^ Besides reporting protocol registration in the abstract and methods, one SR reported it at the end of the article as well

We found that 38% of recently published SRs of interventions reported a registered or published protocol. Protocol registration was significantly associated with a higher impact factor of the journal publishing the SR and a more frequent reporting of use of PRISMA guidelines.

Compared to other studies, we observed a high percentage (38%) of interventional SRs with registered protocol. Studies dating in 2018 and 2021 reported 17%-20% of SRs with a registered protocol (15, 25). These two studies included different types of SRs, namely SRs of randomized controlled trials (15) and SRs of environmental studies with (non-)human intervention(25). The difference in observed proportions of protocol registration might therefore be explained by the earlier publication date of study, the research topic, or a combination of both.

In line with the studies of Ge et al. (2018) and Allers et al. (2018), we showed that SRs with a registered protocol were more likely to be published in journals with a higher impact factor and revealed higher quality of reporting (13, 15). In contrast with Ge et al. (2018), we did not find a associating between financial sponsorship and protocol registration. Interestingly, the number of SRs that reported the use of PRISMA in our sample was high for both SRs with a protocol (87%) and without a protocol (71%) compared with previous research (13).

It is remarkable that 71% of SRs without a protocol reported using PRISMA, but did not follow the directive of PRISMA that SRs should report on protocol registration or publication. A possible explanation is that PRISMA guidelines are more often consulted during the manuscript writing process than during the designing (and protocol writing) phase of the SR. This is supported by a recent survey study showed that almost 45% of researchers, who have written a SR between 2010-2016, have never registered a SR protocol (26). Most common reasons for not registering were: lack of knowledge on benefits or process of protocol registration, lack of time and non-mandatory requirement (26).

In 2005, the International Committee of Medical Journal Editors (ICMJE) has introduced mandatory trial registration guidelines for clinical trials on human subjects (27). No such requirement exists for SRs. Even though some journals and organisations have endorsed protocol registration in PROSPERO, our results show that one-third of SRs published in these journals did not have a protocol. Hence, recommendations in author guidelines are insufficient to effectively implement prospective SR protocol registration. High quality journals have the responsibility to uphold and continuously increase their quality standards to ensure high publication quality. Only after mandating registration of trials by the ICMJE, other journals followed and the uptake of trial registration increased. Journal and publishers could fulfill a pioneer role in raising quality standards by making protocol registration a mandatory requirement for publication. Furthermore, they could include protocol registration part of the peer-review process. Protocol registration awareness could be enhanced by introducing a mandatory protocol registration item in the abstract.

Preregistration of studies, which is similar to the Cochrane registration process, is adopted by some journals. During this process research articles undergo peer review at the study design or protocol stage (28). For Cochrane reviews, the scope of the proposed review and skills and experience of the proposed author team have typically been evaluated beforehand (29). This provides an opportunity for feedback and support in the designing phase of the review to increase SR quality.

In the context of the Open Science movement, the scientific community and journals need to navigate between recommendation versus obligation of protocol registration to strike a balance between transparency and quality on the one hand and researchers’ time investment on the other. From the perspective of patients and guideline developers, high quality, non-biased SRs are required for optimal clinical care and patient treatment for the reason that SRs and meta-analyses, more often than single clinical trials, directly inform medical guidelines and thereby medical treatment decisions.

Together, our results stress the need for further efforts to communicate the value of SR protocol registration more effectively, not only among researchers, but also among journal editors and peer reviewers. To guide further efforts to highlight the value of SR protocol registration, research is needed to gain understanding of the benefits and informativeness of SRs protocol registration amongst different stakeholders (researchers, journal editors, peer reviewers, policy makers, the public, clinicians, patients).

Strenghts & limitations

A strength of our study is that we have assessed protocol registration from a sample of published SRs. This in contrast with most previous studies, that have predominantly assessed SR protocol registration through PROSPERO records (13, 14, 30, 31). However, a limitation of this approach is that some publications might not have reported protocol registration in their manuscript. Aller et al. found that 12.5% of SRs with a protocol, did not report protocol registration or publication. We haven’t assessed whether this is the case in our sample and whether the proportion of intervention SRs with protocol registration we observed is an underestimation. When SR publications do not refer to the protocol, some of the benefits of protocol registration may be lost.

Another limitation is the single author selection and data extraction process. Through clear in- and exclusion criteria and well-defined items for data extraction, risk for errors and potential biases was minimized. Additionally, any difficulties in selection or data extraction were discussed amongst the researchers and a sample of the extracted data was checked.

Lastly, the generalizability of our study is limited, as we only included SRs of health care interventions. Protocol registration status might be different for other types of reviews (e.g., diagnostic, prognostic) and remains to be assessed in future research.

We found that only 135 out of 357 (38%) recently published interventional SRs had a protocol either registered or published. Our results did show that a higher journal impact factor and more frequent reporting of PRISMA guidelines was positively associated with protocol registration or publication. Noteworthy, is that the continent with the highest number of published SRs (Asia) has the lowest proportion (25%) of SRs with protocol registration.

COI: Conflict of Interest

INPLASY: International Platform of Registered Systematic Review and Meta-Analysis Protocols

JIF: Journal impact factor

NICE: National Institute for Health and Care Excellence

OSF: Open Science Framework

PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analysis

PROSPERO: Prospective Register of Systematic Reviews

SCIE: Science Citation Index Expanded

SR: Systematic review

SSCI: Social Science Citation Index

Funding: No funding was received for this study

Ethics approval and consent to participate: Not applicable as this study only includes already published systematic reviews.

Consent for publication: All authors were able to provide feedback on the manuscript and consent to publication of this version of the manuscript.

Availability of data and materials: The data supporting the conclusions of this article is available in the Open Science Framework (OSF) repository: https://doi.org/10.17605/OSF.IO/TSUF9

Competing interests: The authors declare no competing interests.

Authors' contributions:

Planning & designing, protocol development & registration: KB, MG, PH, JD, RS, KR, HL, LH

Feedback and polishing manuscript: KB, CO, PH, JD, RS, KR, HL, LH

Screening & data extraction: KB, MG, CO, PH, JD, HL

Statistical analysis: KB, JD

Writing manuscript & revisions: KB, CO

Jamali D, Barkemeyer R, Leigh J, Samara G. Open Access, Open Science, and Coronavirus: Mega trends with historical proportions. Bus Ethics. 2020.
Vicente-Saez R, Martinez-Fuentes C. Open Science now: A systematic literature review for an integrated definition. Journal of Business Research. 2018;88:428-36.
Tennant JP, Waldner F, Jacques DC, Masuzzo P, Collister LB, Hartgerink CH. The academic, economic and societal impacts of Open Access: an evidence-based review. F1000Res. 2016;5:632.
Munafo MR, Hollands GJ, Marteau TM. Open science prevents mindless science. BMJ. 2018;363:k4309.
Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1.
Krleza-Jeric K, Chan AW, Dickersin K, Sim I, Grimshaw J, Gluud C. Principles for international registration of protocol information and results from human trials of health related interventions: Ottawa statement (part 1). BMJ. 2005;330(7497):956-8.
Straus S, Moher D. Registering systematic reviews. CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne. 2010;182(1):13-4.
Zarin DA, Keselman A. Registering a clinical trial in ClinicalTrials.gov. Chest. 2007;131(3):909-12.
Hoffmann F, Allers K, Rombey T, Helbach J, Hoffmann A, Mathes T, et al. Nearly 80 systematic reviews were published each day: Observational study on trends in epidemiology and reporting over the years 2000-2019. J Clin Epidemiol. 2021;138:1-11.
Page MJ, Shamseer L, Tricco AC. Registration of systematic reviews in PROSPERO: 30,000 records and counting. Syst Rev. 2018;7(1):32.
Siontis KC, Hernandez-Boussard T, Ioannidis JPA. Overlapping meta-analyses on the same topic: survey of published studies. BMJ : British Medical Journal. 2013;347.
Stewart L, Moher D, Shekelle P. Why prospective registration of systematic reviews makes sense. Syst Rev. 2012;1:7.
Allers K, Hoffmann F, Mathes T, Pieper D. Systematic reviews with published protocols compared to those without: more effort, older search. J Clin Epidemiol. 2018;95:102-10.
Booth A, Mitchell AS, Mott A, James S, Cockayne S, Gascoyne S, et al. An assessment of the extent to which the contents of PROSPERO records meet the systematic review protocol reporting items in PRISMA-P. F1000Res. 2020;9:773.
Ge L, Tian JH, Li YN, Pan JX, Li G, Wei D, et al. Association between prospective registration and overall reporting and methodological quality of systematic reviews: a meta-epidemiological study. J Clin Epidemiol. 2018;93:45-55.
Assessing the proportion of prospectively available systematic review protocols and the databases or repositories used by authors for registering or posting their protocol [Internet]. 2021. Available from: https://osf.io/9kj7r/
https://doi.org/10.17605/OSF.IO/TSUF9.
Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. J Clin Epidemiol. 2021;134:178-89.
Murad MH, Wang Z. Guidelines for reporting meta-epidemiological methodology research. Evid Based Med. 2017;22(4):139-42.
Lee E, Dobbins M, DeCorby K, McRae L, Tirilis D, Husson H. An optimal search filter for retrieving systematic reviews and meta-analyses. BMC Med Res Methodol. 2012;12.
Thomas J, Graziosi, S., Brunton, J., Ghouze, Z., O'Driscoll, P., & Bond, M. EPPI-Reviewer: advanced software for systematic reviews, maps and evidence synthesis. London: UCL Social Research Institute: EPPI-Centre Software; 2020.
Stefanie Haustein TDB, Rodrigo Costas. When is an article actually published? An analysis of online availability, publication, and indexation dates. 2015.
Support for PROSPERO [Available from: https://www.crd.york.ac.uk/PROSPERO/#aboutpage.
R Core Team. R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2021.
Wickham H. ggplot2: Elegant Graphics for Data Analysis: Springer-Verlag New York; 2016.
The prevalence of recommended research practices in contemporary environmental health systematic reviews [Internet]. 2022 [cited 01-02-2020]. Available from: https://osf.io/j2d5v/
https://doi.org/10.17605/OSF.IO/J2D5V.
Tawfik GM, Giang HTN, Ghozy S, Altibi AM, Kandil H, Le HH, et al. Protocol registration issues of systematic review and meta-analysis studies: a survey of global researchers. BMC Med Res Methodol. 2020;20(1):213.
De Angelis C, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. Croat Med J. 2004;45(5):531-2.
Nosek BA, Beck ED, Campbell L, Flake JK, Hardwicke TE, Mellor DT, et al. Preregistration Is Hard, And Worthwhile. Trends Cogn Sci. 2019;23(10):815-8.
Cumpston M CJ. Chapter II: Planning a Cochrane Review. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.2 (updated February 2021). Cochrane. 2021.
Ruano J, Gomez-Garcia F, Gay-Mimbrera J, Aguilar-Luque M, Fernandez-Rueda JL, Fernandez-Chaichio J, et al. Evaluating characteristics of PROSPERO records as predictors of eventual publication of non-Cochrane systematic reviews: a meta-epidemiological study protocol. Syst Rev. 2018;7(1):43.
Tricco AC, Cogo E, Page MJ, Polisena J, Booth A, Dwan K, et al. A third of systematic reviews changed or did not specify the primary outcome: a PROSPERO register study. J Clin Epidemiol. 2016;79:46-54.

Supplementarymaterials.docx

The Score After Ten Years of Registration of Systematic Review Protocols

Status:

Version 1

Abstract

Figures

Introduction

Objectives

Methods

Information sources

Eligibility criteria

Study selection and data collection

Data synthesis and analysis

Results

Discussion

Conclusions

List of abbreviations

Declarations

References

Supplementary Files

Status:

Version 1