The questions this review will address are: 1) What levels of evidence support signals of adverse drug reactions communicated by drug regulatory authorities or other pharmacovigilance stakeholders? 2) What are the similarities and differences between the levels of evidence supporting signals from different stakeholders? 3) Have the levels of evidence changed over time? 4) What is the interval between a signal and the first report of the corresponding suspected adverse drug reaction? 5) Has this interval shortened or increased over the years? 6) In the case of signals supported by reports of adverse drug reactions, what elements of the reports have led authors to signal suspected drug-related harms?
The aims of this scoping review are: (a) to provide an evidence synthesis for decision-making at a regulatory level; (b) to collate signals currently in different platforms; (c) to produce an overview of the types and levels of evidence of studies supporting signals; (d) to highlight any differences or similarities in assessment of the evidence used to support signals, with a particular focus on reports of ADRs; (e) to better understand the delays before signals are detected and to clarify whether they have changed over time.
Pharmacovigilance, as defined by the European Union directive 2001/83/EC [27]; regulation 2004/726/EC [28]; Good Vigilance Practices – Module IX [7]; International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use efficacy guidelines E2A to E2F [29], will serve as references to contextualize pharmacovigilance in a global landscape.
Searches
We shall search MEDLINE (via PubMed, 1946 - present), EMBASE (1974 - present), PsycINFO (1806 - present), Web of Science (indexes SCI-EXPANDED, CPCI-S, 1945 - present) for signals published by organizations involved in pharmacovigilance and/or independent research groups (e.g. regional pharmacovigilance centres or academia), and pharmaceutical companies. We shall use Web of Science to perform backward and forward citation screenings; if we are unable to use Web of Science, we shall use Google Scholar.
We shall search the English language websites of regulatory authorities or organizations involved in the WHO Programme for International Drug Monitoring through the websites’ search engines, to identify communicated signals and among minutes of meetings, expanding the list of countries compiled by Onakpoya et al. [21] where possible. If signals are not publicly available in a country, we shall forward requests to the competent authority. If the evidence used to support available signals is not publicly available, we shall issue freedom of information requests (where regulated) for Assessment Reports or equivalent documents.
We shall hand-search the Uppsala Monitoring Centre’s SIGNAL Document, the WHO Pharmaceuticals Newsletter, the Full List of WHO Medical Product Alerts, the WHO Drug Information, Australian Prescriber, Prescrire International, and other drug bulletins available in English via the International Society of Drug Bulletins’ index for relevant records.
We shall query Google Scholar in incognito browser tabs. We shall evaluate the results as long as two consecutive pages of results yield at least one useful article.
We shall also search OpenGrey and GreyNet International.
We shall use VigiBase [30], to obtain additional information on the year in which suspected ADRs were first reported.
The data lock point for the retrieval of all records is 6 August 2019.
Additional file 1 – Search strategy details the complete search strategy for electronic databases (adapted to PubMed) and for grey literature.
Inclusion Criteria
Peer-reviewed papers and government or agency reports and working papers concerning one or more medicinal products for human use [31] and one or more adverse events, effects, or reactions [32] that are statistically associated by disproportionality analysis [33-35], or
(a) with the primary aim of detecting, assessing, or reviewing one or more signals, as defined by [1];
(b) that explicitly mention the successful detection of a signal or whose results constitute a signal;
(c) that have been indexed or discussed as signals, e.g. in tables of contents or minutes of meetings.
Documents in languages other than English will be included if the review team can translate them into English. Requests for translations will be issued to the publishing stakeholder if the review team cannot translate.
Exclusion Criteria
(a) Abstracts or papers for which the full text is unavailable.
(b) Documents that claim that the evidence does not support a signal or that no signal was detected.
(c) Signals that are communicated in the same or similar form or substance more than once by the same stakeholder (only the first signal in chronological order will be considered, if no new information appears in subsequent reports).
(d) Non-English documents and websites for which a translation is unavailable.
Primary Outcome(s)
Evidence level assessment and comparison of evidence sources used to support signals.
Secondary Outcome(s)
Elements of reports of suspected ADRs that authors used to support putative causal relationships between drugs and adverse reactions in signals.
Data retrieval
All retrieved records will be imported into EndNote™ X8.2. We shall deduplicate records manually, based on list of authors, titles, abstracts, journal, paging.
Two reviewers will independently search the titles and abstracts to determine eligibility. Any disagreements will be resolved through discussion. If a consensus cannot be reached, a third reviewer will arbitrate.
Data extraction, selection, and coding
We shall extract data from eligible studies and documents with the aid of forms provided online by the Systematic Review Data Repository and adapted to scoping reviews. Studies will be screened by title and abstract, and then by full text, against the inclusion and exclusion criteria. Data to be extracted are: publication year, type of evidence used to support signals, definition of signal used in a study or signal communication, year of the case report of the suspected ADR that was first transmitted to VigiBase and that relates to the same medicinal product and ADR in a considered signal, country of origin/communication of a signal, stakeholder, design, eligibility criteria, population involved, setting, brandname of the medicinal product (where available), corresponding active pharmaceutical ingredient(s), formulation, considered ADR(s), and possible confounding factors. Medicinal products will be coded to substance level, using the WHO Drug Dictionaries (latest version available at the start of the review), and adverse events at the Low-Level Term level of the hierarchy in the Medical Dictionary for Regulatory Activities® (MedDRA) (latest version available at the start of the review), where possible (e.g. signals of increased mortality or serious adverse events cannot be coded to MedDRA).
One reviewer will independently extract and code the data, and a second will independently cross validate the findings. Any disagreements will be resolved through consensus. If no consensus is reached a third reviewer will arbitrate.
Stakeholder(s)
We consider “stakeholders” to be healthcare practitioners and patients, national and international regulatory agencies or authorities, national and regional pharmacovigilance centres, non-governmental organizations, research institutions, and academia.
Calculation of the time-to-signal
The date on which an ADR occurred, as stated in reports of ADRs entered in VigiBase will be used to calculate the time-to-signal, by accounting for the date on which the signal relating to the same combination of medicinal product and adverse effect was communicated. If the date of occurrence of the ADR is unavailable, we shall use the first date in which the report was entered in VigiBase instead.
Strategy for data synthesis and dissemination of results
We shall summarize findings narratively by level of evidence (documented using the Oxford Centre for Evidence-Based Medicine levels of evidence); stakeholder; signal subtypes (e.g. signals of disproportionate reporting).
Descriptive statistics, where appropriate, will accompany summaries of findings, and be applied to quantify the proportions of elements in spontaneous case reports associated with signals.
We shall also present the time-to-signal, and changes in evidence sources over time and across stakeholders, in timeline plots and pictures (if appropriate).
We shall report this scoping review according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension for Scoping Reviews [36].