The International Platform of Registered Systematic Review and Meta-analysis Protocols (INPLASY) at three years: an analysis of 4658 registered protocols on inplasy.com, platform features, and website statistics

doi:10.21203/rs.3.rs-1651305/v2

Download PDF

Short Report

The International Platform of Registered Systematic Review and Meta-analysis Protocols (INPLASY) at three years: an analysis of 4658 registered protocols on inplasy.com, platform features, and website statistics

https://doi.org/10.21203/rs.3.rs-1651305/v2

This work is licensed under a CC BY 4.0 License

Journal Publication

published 30 Jul, 2023

Read the published version in Frontiers in Research Metrics and Analytics →

Version 2

posted

You are reading this latest preprint version

Background: INPLASY® is an international platform for registering systematic reviews and meta-analysis protocols that was launched in March 2020. INPLASY® provides an online database in which the protocols are maintained as permanent public records and can be accessed on its website (www.inplasy.com).

Methods: We described the database features and registered information of all records published since the launch of the registry on March 31, 2023. Additionally, the website statistics dataset was analyzed to explore the user experience and promote data transparency.

Results: Four thousand six hundred fifty-eight records were registered in INPLASY®, and more than 94% of the protocols were published within 24 h. Most of the submissions were from China, followed by Portugal, Taiwan, Malaysia, and Brazil. The INPLASY® website received 386,395 page views from 64,568 visitors during the first three years. The accesses were obtained from 170 countries. Most of the accesses were from China, followed by the US, the UK, and Portugal. The review status ‘‘completed and published’’ was observed in 898 protocols, and these studies were published in 372 different scientific peer-reviewed journals. The INPLASY® features include: (i) INPLASY® identifier, a unique protocol number; (ii) the digital object identifier (DOI) number, the URL of the protocol linked to a specific DOI; (iii) ORCID update, INPLASY® updates the authors' ORCID page including their protocol automatically; (iv) search tools, the protocols are freely accessible on www.inplasy.com.

Conclusions: INPLASY® has several practical and useful features that should be considered when planning the registration of a systematic review protocol. Furthermore, the sharp increase in the number of protocols registered in INPLASY® in the first three years and the database statistics demonstrate that INPLASY® has become an important source of systematic review protocols. Therefore, authors should access INPLASY® before planning a future review study to avoid unintended duplication of efforts or obtain timely registration.

INPLASY

systematic reviews

registration

protocols

registry

Registration of systematic review protocols is crucial to avoid duplication of systematic reviews and improve transparency.(1, 2) The protocol specifies the objectives and methods that will be applied to conduct the review, providing the ability for authors to track what studies are taking place.(3) Additionally, discrepancies between the methods described in the protocol and those in the published review can be identified, allowing readers to analyze the outcome reporting biases associated with the study. PROSPERO(4, 5) was the first prospective registry for systematic review protocols; however, recently, alternative platforms have become available.(6)

INPLASY® is an international publicly accessible platform for the registration of systematic reviews and meta-analysis protocols, officially launched in March 2020.(7) The INPLASY® registry provides an online platform to register systematic review protocols, which are maintained as permanent public records and free of access on inplasy.com. Although PROSPERO was the first available registry, it seems challenging for a single platform to register all systematic review protocols developed worldwide. Additionally, PROSPERO was supported by the National Institute for Health Research (NIHR), and submissions from the UK were prioritized during the registration process. In contrast, INPLASY® does not prioritize protocols according to the nationality of authors , providing the same registration time for all submissions. Additionally, INPLASY provides other relevant features, such as the digital object identifier (DOI) number, to easily cite each protocol and an automatic update system connecting the protocol with the final published article.

An important difference between PROSPERO and INPLASY is that PROSPERO does not accept systematic reviews without an outcome of clear relevance to human health. Conversely, INPLASY does not impose any outcome restrictions, and other review projects, such as scoping reviews, are accepted for registration.

We aim to describe the important features of the new international database and summarize the information from all records published in the database since the launch of the INPLASY® registry. Additionally, we collated a website statistics dataset to explore user experience and promote platform transparency.

We collected the following data from all protocols published on inplasy.com from inception until March 31, 2023: (i) type of review protocol described in the title, such as systematic review, scoping review, overview of reviews, meta-synthesis, mapping review, rapid review, meta-analysis, or network meta-analysis; (ii) study phase at which the protocol was registered (prospective or retrospective registration); (iii) country of the corresponding author; (iv) the number of versions of each protocol.

The INPLASY® website statistics were retrieved to quantify the following: the total number of protocols registered by month; number of user subscriptions; number of protocols registered by month; number of user subscriptions by month; number of website access; number of website access by month; number of protocol views; countries of visitors accessing INPLASY® website.

The review status of all INPLASY® protocols was checked on the website to describe the list of protocols completed and published in peer-reviewed journals. Electronic libraries, including MEDLINE via PubMed, Cochrane Library, and Embase, were examined to identify peer-reviewed protocols containing the INPLASY® unique registration number published in different indexed journals. The search strategy used to find these protocols was ((INPLASY*) OR ("International Platform of Registered Systematic Review and Meta-analysis Protocols")) AND (protocol[Title]). The total number of protocols published as a stand-alone peer-reviewed article registered in INPLASY® were collected.

Finally, we explored the INPLASY® website pages to list the main features of the platform, such as the types of studies accepted for registration, registration requirements, costs, version of the tracking system, funding model of the platform, processing time, and search structure, to find a record on the INPLASY® website.

3.1 Registration statistics and types of reviews

Four thousand six hundred fifty-eight records were identified in INPLASY® from inception to March 31st, 2023 (Fig. 1). The number of protocols registered per month is exhibited in Fig. 2. On average, approximately 129 protocols were registered per month. Most submissions were from China, followed by Portugal, Taiwan, Malaysia, and Brazil. There were 5782 subscribers in INPLASY® in March 2023. The number of registered users per month is shown in Fig. 2. These protocols have been developed in 63 countries. Of these, 86.6% were from Asia, 8.0% from Europe, 4.0% from America, 0.62% from Africa, and 0.60% from Australia/Oceania. The types of reviews registered in INPLASY® are presented in Table1. Approximately 80% of registered protocols were obtained from systematic reviews and/or meta-analyses.

3.2 Database statistics

The INPLASY® website received 386,395 page views from 64,568 visitors during the first three years. The accesses were from 170 different countries, most of which were from China, followed by the US, the UK, and Portugal.

3.3 Registration time and review status

Three thousand nine hundred fifty-seven protocols were registered prospectively (84.9%) and 701 protocols (15.1%) were registered retrospectively. The review status ‘‘completed and published’’ was observed in 898 protocols, published in 372 different scientific journals. Figure 3 shows the annual publication rate. Only 0.1% of the INPLASY® protocols presented the updated review status as ‘‘discontinued’’ by the authors. Additionally, we found 692 records registered in INPLASY® published as stand-alone peer-reviewed articles (14.85% of the sample). These peer-reviewed protocols were registered in INPLASY® and published in twelve scientific journals (PLoS ONE, Systematic review, Medicine - Baltimore, Trials, International Journal of Surgery Protocols, Annals of Palliative Medicine, BMJ Open, European Journal of Integrative Medicine, Evidence based complementary Alternative Medicine, Integrative Medicine Research, Journal of Pain Research Protocols).

3.4 Platform features

Registration number and DOI: INPLASY® provides a unique registration number for each protocol, which can be used to identify the review protocol in the final manuscript. Additionally, all protocols registered on inplasy.com have a DOI, and the URL in which each protocol is hosted is permanently linked to a specific DOI.

Eligible studies: INPLASY® accepts all types of systematic review protocols, including systematic reviews of interventions, diagnostic accuracy, prognostic factors, epidemiological characteristics, and preclinical studies. Systematic reviews assessing sports performance as outcomes are also accepted. The author can submit scoping review protocols using a standard systematic review form or a specific form developed exclusively for scoping reviews (https://inplasy.com/scoping-reviews/). The form was developed using the JBI manual for evidence synthesis.(8) One hundred and seventeen scoping review protocols were published in INPLASY® (2.52% of the sample).

Processing time: Four thousand six hundred fifty-eight published protocols were analyzed to determine the period between protocol submission and publication. Ninety percent of the records were published within 24 h, whereas less than 1% of the records took more than 48 h to be published because of technical issues in the platform or failure in the submission process.

Version tracking - updating a published protocol: Authors can update their protocols using the INPLASY® update form. We identified 4333 protocols with a single version, 288 protocols with two versions, 29 protocols with three versions, and eight protocols with four versions. All previous versions of the protocol were permanently maintained on the protocol page to allow a full audit trial for any modifications within the record.

Automatic update of the author’s ORCID: INPLASY® updates the author's ORCID page using the Crossref interface. Alternatively, authors can manually update their ORCID pages using the DOI number of the protocol. This feature was used in 589 (12,64%) protocols in which at least one of the authors of the registered protocol included ORCID details.

Funding model: INPLASY® was created by a for-profit organization to provide an online public platform on which researchers could register systematic review protocols. The publication fees in June 2023 were $20 to register a protocol and $9 for updating it. During the first three years, INPLASY® did not receive funding from government agencies, universities, or other institutions. Therefore, publication fees were the only source of financing during this period. Although a for-profit organization operates the platform, there is an ongoing process with the public ministry of Distrito Federal Brazil to create a nonprofit company that controls the INPLASY registry. With this transformation, a significant change in the origin of resources for the platform's maintenance is expected by 2023, eliminating any potential conflicts of interest in the for-profit model.

Search structure: INPLASY® platform offers a simple search tool where records can be found according to the unique identifier number or using free text terms. By June 2023, it was not possible to use the Boolean operators (AND, OR, and NOT) on a search page.

Review process: The INPLASY platform conducts a basic review to certify that a protocol is eligible for registration on the platform. The INPLASY records were not peer-reviewed or assessed for methodological quality. This is the responsibility of the authors. The methods or content registered on INPLASY do not constitute an endorsement of methods that are solely those of the authors. Additionally, INPLASY does not guarantee the accuracy of the English and is not responsible for errors arising from the text.

Since the launch of INPLASY® in March 2020, the number of records has increased progressively, reaching 4658 protocols in 63 countries by March 31, 2023. After three years of operation, INPLASY® has become the second-largest specific database for systematic reviews registration, behind PROSPERO in terms of the number of protocols.(6) INPLASY® protocol was developed based on PRISMA-P recommendations(8) and the PROSPERO registration form. The INPLASY® form has 33 fields, of which 24 are mandatory and 9 are optional. The number of records submitted to PROSPERO over the last ten years has increased considerably, resulting in an unprecedented number of registrations. Consequently, a significant delay in the registration process was reported before the onset of the COVID-19 pandemic. Puljak(10) reported waiting for more than six months to have a protocol published in PROSPERO, which is unacceptable from the author’s perspective. PROSPERO implemented a basic automated check system during the pandemic, reducing the waiting time to 30 days. Although the waiting time has been reduced, 30 days are still considered long waiting period when the pending registration is unaware and may result in duplicate efforts. During the first three years, more than 95% of the INPLASY® protocols were published in less than 24 h. The fast-track processing time of INPLASY® may reduce duplication of efforts and research waste because the longer the time interval between the submission and the registration, the greater the chances of duplicated protocols. As the platform does not perform a peer-reviewed process, a quick registration is possible.

Solla et al.(11) showed that PROSPERO registration does not prevent two registrations on the same topic. Therefore, the authors were responsible for searching for ongoing systematic reviews in the pipeline before submitting their review protocols. COVID-END, a time-limited network group formed by more than 50 of the world’s leading evidence-synthesis, indicated that before starting a new project, researchers should seek ongoing reviews not only in PROSPERO, but also in INPLASY® platform, National Collaborating Center, Center for Evidence-based Medicine, and VA Evidence Synthesis Program.(12)

Other available platforms for systematic review protocols registration include Cochrane Reviews, Joanna Briggs Institute, and Campbell Collaboration, which provide quality assurance and many other benefits for accepted review protocols. However, these platforms are very restricted, and only a small number of selected protocols can be published in the referred sources. Additionally, these organizations produce only a minority for all systematic reviews.(13) Banno et al.(14) identified the frequency with which systematic review protocols were registered outside the PROSPERO registry until 2019. They listed generic registries that accepted systematic review protocols. Pieper et al.(6) described five alternatives for registering systematic review protocols: PROSPERO, INPLASY, Research Registry, Open Science Framework Registries, and protocols.io. Among them, the first three are specific for systematic review registration. INPLASY® is the only specific systematic review registry that provides a DOI for each protocol. The unique INPLASY identifier and DOI number combination allows authors to identify, access, and cite their protocols easily and precisely.

Approximately 85% of the included studies were prospectively registered. Although it is possible to register protocols retrospectively using INPLASY, the registry does not recommend retrospective registration unless the authors explain the reasons that prevent prospective registration. The status of the study at the time of submission was registered in the protocol to permit editors, reviewers, and readers to identify which studies were retrospectively registered and evaluate whether retrospective registration was justifiable for each case. Many scientific journals accept retrospectively registered studies because the authors disclose this status in a brief statement in the manuscript. Two for-profit organizations operate INPLASY®, and on June 2023, all available sources of funds to support the platform were derived from publication fees. During its first three years, INPLASY concentrated its marketing activities in Asian countries. Consequently, the platform today has over 85% of records from this geographical area. A wider distribution is expected when the platform switches to a non-profit model because it will be more accessible to different researchers.

Andersen et al.(15) reported that few authors updated their review status in PROSPERO after publication. Similarly, less than 1% of all published reviews registered in INPLASY® were updated by the author after publication. INPLASY® automatically updates the review status of registered protocols and links them to the URL of the article. Thus, identifying the concluded projects is easier and straightforward. We identified 372 scientific journals containing reviews registered on the INPLASY® platform. A systematic review published by Li et al.(16) was an article registered in INPLASY® published in the journal with the highest impact factor (IF) during the first three years (The Lancet Gastroenterology & Hepatology IF = 45.042). The list of all peer-reviewed articles registered in INPLASY® is updated daily and available at https://inplasy.com/published-articles/.

High-quality protocols can be obtained by submitting records to peer-reviewed journals after registration. Several journals, such as PLoS ONE, BMJ Open, and Systematic Reviews, have published systematic review protocols as standalone peer-reviewed articles. The advantage is that the methods proposed in the protocol are critically appraised, thereby increasing the quality of the report and preventing potential flaws that may compromise the validity of the study. However, most systematic reviews do not refer to peer-reviewed protocols. Only 14.85% of all protocols registered in INPLASY® were published in peer-reviewed journals, confirming that most systematic review protocols have not yet been peer-reviewed.

Overall, the present findings indicate that the INPLASY platform has many desirable features and should be considered as a reliable and fast platform for registering systematic review protocols. Additionally, INPLASY® is an option for registering other evidence synthesis protocols, such as scoping, methodological, and rapid reviews. The sharp increase in the number of protocols registered in INPLASY® in the first three years and the database statistics demonstrate that INPLASY® has become an important source of systematic review protocols. Therefore, authors should access INPLASY® before planning a future review study to avoid unintended duplication of efforts or obtain timely registration.

Conflict of Interest

Joao Vitor dos Santos Canellas is the founder and CEO of the INPLASY registry. No other authors report any real or perceived conflict of interest.

Author Contributions

João Vitor dos Santos Canellas: Conceptualization; Data curation; Formal analysis; Investigation; Writing Original; Supervision; Draft Preparation; Final review the manuscript.

Fabio Gamboa Ritto: Data curation; Formal analysis; Validation; Visualization; Writing Original Draft Preparation.

Alessandro Rodolico: Visualization; Writing - Review & Editing the manuscript.

Eugenio Aguglia: Visualization; Writing - Review & Editing the manuscript.

Gustavo Vicentis de Oliveira Fernandes: Data curation; Investigation; Validation; Writing - Review & Editing the manuscript.

Carlos Marcelo da Silva Figueredo: Data curation; Validation; Writing - Review & Editing the manuscript.

Mario Vianna Vettore: Formal analysis; Supervision; Writing - Review & Editing the manuscript.

Funding

None.

Acknowledgments

None.

Data Availability Statement

The datasets generated/ analyzed for this study can be found in the INPLASY platform [https://www.inplasy.com].

1. Straus S, Moher D. Registering systematic reviews. CMAJ. 2010;182(1):13-4.

2. Dos Santos MBF, Agostini BA, Bassani R, Pereira GKR, Sarkis-Onofre R. Protocol registration improves reporting quality of systematic reviews in dentistry. BMC Med Res Methodol. 2020;20(1):57.

3. Chang SM, Slutsky J. Debunking myths of protocol registration. Syst Rev. 2012;1:4.

4. PROSPERO International prospective registry of systematic reviews. [Available: https://wwwcrdyorkacuk/prospero/].

5. Booth A, Clarke M, Ghersi D, Moher D, Petticrew M, Stewart L. An international registry of systematic-review protocols. Lancet. 2011;377(9760):108-9.

6. Pieper D, Rombey T. Where to prospectively register a systematic review. Syst Rev. 2022;11(1):8.

7. INPLASY International Platform of Registered Systematic Review and Meta-analysis Protocols [Available: https://www.inplasy.com].

8. Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;350:g7647.

9. Puljak L. Delays in publishing systematic review registrations in PROSPERO are hindering transparency and may lead to research waste. BMJ Evid Based Med. 2020.

10. Solla F, Bertoncelli CM, Rampal V. Does the PROSPERO registration prevent double review on the same topic? BMJ Evid Based Med. 2020.

11. COVID-19 Evidence Network to support Decision-making - Resources for researchers - Identify ongoing reviews/ avoid duplication effort; 2.2.2(Available: https://www.mcmasterforum.org/docs/default-source/covidend/covid-end_researchers.pdf?sfvrsn=437e56d5_4).

12. Page MJ, Shamseer L, Altman DG, Tetzlaff J, Sampson M, Tricco AC, et al. Epidemiology and Reporting Characteristics of Systematic Reviews of Biomedical Research: A Cross-Sectional Study. PLoS Med. 2016;13(5):e1002028.

13. Banno M, Tsujimoto Y, Kataoka Y. Proportion of Systematic Review Protocols Registered Outside of the International Prospective Register of Systematic Reviews (PROSPERO): A Short Report. Annals of Clinical Epidemiology, 2022; 4: 2; 49-52.

14. Andersen MZ, Fonnes S, Andresen K, Rosenberg J. Most published meta-analyses were made available within two years of protocol registration. European Journal of Integrative Medicine. 2021;44:101342.

15. Li Y, Choi H, Leung K, Jiang F, Graham DY, Leung WK. Global prevalence of <em>Helicobacter pylori</em> infection between 1980 and 2022: a systematic review and meta-analysis. The Lancet Gastroenterology & Hepatology. 2023; 8(6):553-564.

Download PDF

Journal Publication

published 30 Jul, 2023

Read the published version in Frontiers in Research Metrics and Analytics →

Version 2

posted

You are reading this latest preprint version

The International Platform of Registered Systematic Review and Meta-analysis Protocols (INPLASY) at three years: an analysis of 4658 registered protocols on inplasy.com, platform features, and website statistics

Status:

Journal Publication

Version 2

Abstract

Figures

Background

Methods

Results

3.1 Registration statistics and types of reviews

3.2 Database statistics

3.3 Registration time and review status

3.4 Platform features

Discussion

Conclusion

Declarations

References

Status:

Journal Publication

Version 2