We use the PRISMA-P (Preferred Reporting Items for Systematic review and Meta-Analysis Protocols) checklist to develop and report the methods for this protocol, where appropriate (15). Also, our systematic review will follow the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement, where appropriate (16). However, in this paper, we will focus on creating a database and a comprehensive bibliometric analysis of SR-HEs. We will make no effort to perform an evidence synthesis or risk of bias assessment in the present protocol, since the literature is likely to be vast and heterogeneous and any evidence synthesis and risk of bias assessment efforts may be decided in subsequent work. We will document any deviations from, or amendments to this protocol including details of the date, changes made, and the rationale for changes.
Eligibility criteria
We will include SRs with/without meta-analyses of full economic evaluations that included a search in at least one bibliographic database. By full economic evaluations we include cost-effectiveness, cost-minimization, cost-utility, cost-benefit, and cost-consequence analyses. We will not use any exclusion criterion based on the type of assessed interventions or medical conditions. We will include articles published in the English language. We will exclude (a) SRs of partial economic studies (e.g., cost-of-illness studies or program cost studies), (b) SRs of health economic assessments not focused on humans. We will also exclude clinical practice guidelines and secondary reports of health technology assessments, because our study will focus on scientific literature instead of documents stemming from regulatory and clinical guidance processes. We will also exclude conference abstracts, protocols, narrative reviews, commentaries, and editorials. We will exclude overviews of SRs with/without meta-analyses of health economic evaluations. Table 1 shows a detailed description of our eligibility criteria in terms of population, intervention, comparison, outcome, timing, and setting.
Table 1
| Population | Patients with any medical condition (for health economic evaluations of treatment interventions) or healthy individuals (for health economic evaluations of preventive interventions) |
| Intervention | Any interventions, such as pharmacological, psychological/behavioral, or surgical interventions, and vaccinations |
| Comparator | Any |
| Outcome | Average or incremental cost-effectiveness and net benefit outcomes. |
Literature search
We will systematically search Ovid MEDLINE for articles published from inception to April 19, 2022, using the following search algorithm: ((economic$.ti. or cost$.ti. or cost benefit analysis/ or (treatment outcome/ and ec.fs.)) not ((animals/ not humans/) or letter.pt.)) and (MEDLINE.tw. or systematic review.tw. or meta-analysis.pt. or intervention$.ti.). We created this search algorithm by combining a validated search filter for economic evaluations with a validated filter for the retrieval of systematic reviews (17, 18). The use of validated algorithms provides a good balance between sensitivity and precision.
Two raters (among MH, LMG, TM, LB) will independently assess the eligibility of each record identified in the literature search based on the above criteria. We will resolve conflicts through discussion and consensus between the reviewers. We will make the records of the search and selection process available at the time of publication of the review and document the study selection process in a PRISMA flowchart.
Data extraction
Due to the expected large number of eligible articles (expected to be several thousand eligible articles based on preliminary searches), we will focus our data extraction in this protocol on items we can retrieve and process automatically. We will collect all available publication metadata from Ovid MEDLINE. We will extract information about the name of scientific journal, year of publication, number of authors, and the affiliation and country of the first author and senior author. We will also examine whether the keyword “systematic review” or “meta-analysis” is reported in the title of the eligible articles, and whether a pre-registration number is reported in the abstract. Moreover, we will access Web of Science to enrich our dataset with journal subject areas, retrieve journal impact metrics, and to extract the number of citations received by the eligible articles.
Statistical analysis
We will present descriptive statistics (median and interquartile range for continuous variables, and frequencies for binary or categorical variables) for the extracted variables. We will also present these descriptive statistics by publication year, and by journal subject area.
Techniques for bibliometric analyses consist of performance analysis and science mapping. Performance analysis focuses on the contributions of research constituents to a research field (14). Specifically, we will identify the top 10 of (a) scientific journals, (b) countries, (c) academic or research institutions, and (d) first and senior authors publishing the highest number of SRs-HEs. We will examine whether there is an increasing trend in the number of SRs-HEs across the years, unadjusted and after adjusting for total number of MEDLINE items per year. Moreover, we will examine whether there is a change in the ranking of scientific journals, countries, and academic or research institutions publishing the highest number of SRs-HEs across the years (i.e., before 2000, 2000 to 2010, 2010 to 2020, and after 2020).
Science mapping refers to the study of the relationships between research constituents (14). Specifically, we will perform a citation analysis to identify the most highly cited SR-HEs. Also, we will inspect the collaborations among scholars and among different academic or research institutions, and we will identify the most productive collaborations.
We will also examine whether there are differences in publication patterns based on journal subject area, and geographic region. For this purpose, we will classify journals into three categories: general medical journals, medical subspecialty journals, and health economics journals. We will categorize countries according to World Bank country classifications (low, lower-middle, upper-middle, and high-income). We will use parametric tests (or exact tests, when necessary) to examine differences in publication patterns between the categories of scientific journals and geographic regions. We will use two-sided statistical tests and will present point estimates with 95% confidence intervals.
Software
We will use Abstrackr (19) for Title/Abstract screening, Zotero for literature management, and R statistical software for data management and statistical analysis.