The efficacy and safety of COVID-19 vaccine: A protocol of systematic review and meta-analysis

Abstract

Background: The coronavirus disease 2019 (COVID-19) is one of the most wide-spread and threatening infectious diseases in human history. Experts in the field of medicine and biology are working to develop methods to treat and prevent COVID-19. Currently, COVID-19 is predominantly treated with symptomatic therapy and there is still a lack of effective antiviral therapy. Therefore, the prevention and control of novel coronavirus is primarily focused on vaccine development. Several vaccines have been developed, but their relative efficacy and safety have not been proven. Therefore, the aim of this study is to investigate the efficacy and safety of COVID-19 vaccines.

Methods and analysis: The electronic databases we will use to retrieve information include PubMed, The Cochrane Library, Web of Science, China National Knowledge Infrastructure (CNKI), Wanfang Database and Weipu Electronics. The retrieval period is from the establishment of the database to March 2021. All randomized controlled trials in humans vaccinated with COVID-19 were collected, and data were independently selected and extracted according to predesigned inclusion/exclusion criteria. Full-text screening, data extraction and quality assessment were conducted independently by two reviewers. Two additional investigators will conduct report quality, risk of bias, sensitivity analysis and subgroup analysis to ensure the reliability of our study results. The software RevMan 5.3 was used for statistical analysis. Systematic review and meta-analyses will be conducted to evaluate the pooled evidence of efficacy and safety of the COVID-19 vaccines.

Result: This study will evaluate the efficacy and safety of the COVID-19 vaccines.

Conclusion: The conclusions of this study will provide an evidence-based analysis of the safety and efficacy of COVID-19 vaccines.

Systematic review registration: PROSPERO CRD42021242581

Background

The coronavirus disease 2019 (COVID-19) has become one of the most threatening infectious diseases in human history [1]. At present, more than 200 countries and regions in the world have reported confirmed cases of the disease: the main manifestations being fever, fatigue, and dry cough. Although, a small number of patients present with cough, sputum, chest tightness, dyspnea, muscle pain, diarrhea and other symptoms[2]. According to the World Health Organization (WHO), at 11:15 CET on 12 March 2021 the cumulative number of confirmed global COVID-19 was 118,058,503 and the cumulative death toll was 2,621,046[3]. Vaccination is one of the most effective and economical means of preventing and controlling the spread of viral infections as it protects vulnerable people; however, COVID-19 is still predominantly treated with symptomatic therapy because effective antiviral therapy is limited[4]. Therefore, the current methods for the prevention and control of COVID-19 is focused on the research and development of a vaccine[5, 6].

After Chinese scholars first published the complete genome sequence of novel coronavirus in Nature[7], scientists around the world worked hard to develop a vaccine against COVID-19, to curb the spread of COVID-19 epidemic. Vaccines can be divided into the following three categories based on their intended target and the vaccine technology. The first category is the classical technical route that uses a complete form of the virus and includes inactivated vaccines and attenuated vaccines. The second category is viral protein antigens that are expressed by means of genetic engineering. Examples are subunit vaccines and virus-like particle vaccines. The third category consists of viral vector vaccines (replicative, non-replicative) and nucleic acid (DNA and mRNA) vaccines based on viral legacies[8–13]. According to the WHO by late 2020 a total of 233 vaccines were in development and 61 clinical trials had been conducted using a variety of vaccine technologies[14, 15]. However, further development of a safe and effective COVID-19 vaccine is fraught with challenges and opportunities[16]. As of March 1, 2021, China, the United States, Israel, the United Kingdom, Russia, Germany, and other countries have approved mass vaccination plans of their own vaccines in their own populations.

However, there is not enough evidence to prove the efficacy and safety of COVID-19 vaccine for all populations, so this study conducted systematic review and meta-analysis of published clinical trial results of COVID-19 vaccine to further clarify the efficacy and safety of the vaccine.

Methods

The registration

The systematic review protocol was registered with the International Prospective Register of Systematic Reviews (PROSPERO) on March 15, 2020 (registration number CRD42021242581). The consent of this protocol report is based on the preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement guidelines [17]. (see Additional file 1).

Criteria for considering studies for this review

Studies that meet the following criteria will be included in this review.

Inclusion and criteria for study selection

Study designs. We will review all studies on the efficacy and safety of the preventative COVID-19 vaccine in humans. Because of language limitations, we will search for articles in both Chinese and English. To obtain a more objective and authentic evaluation, all articles must meet the following two conditions:

(1) Published documents with complete data

(2) The trial design is a randomized controlled trial (RCT)

Participants. We will include healthy men 18 years of age and older or non-pregnant women who have been vaccinated against COVID-19. Race, education and economic status will not be study factors. Those excluded from the study include pregnant women, participants with post-operative infections or mental illness, participants with severe pneumonia or other reasons for inability to exercise, and participants with severe cardiovascular and/or liver and/or kidney disease.

Interventions. The experimental group received a prophylactic COVID-19 vaccine, while the control group received a placebo.

Outcomes. Clinical trial results indicators include at least one or more of the following: local reactions (pain, itching, redness, swelling and induration, etc.), systemic adverse reactions (fever, diarrhea, fatigue, nausea, vomiting, lethargy, etc.), 14 days or 28 days after the last vaccination with live virus neutralization test of neutralizing antibody geometric average drop degree (GMT), serum conversion rate, mortality rates and other laboratory tests.

Data sources

Electronic data sources. The following electronic databases will be searched from inception to March 2021: PubMed, the Cochrane Library, Web of Science, CNKI, WanFang Data, Weipu Electronics. In addition, reference lists of the included studies were manually searched to identify additional relevant studies.

Other resources. Relevant references will be reviewed and screened. In addition, we will search the following registration website of the clinical trial: WHO ICTRP (www.chictr.org.cn, www.ClinicalTrial.gov) and the ISRCTN Register. Moreover, the relevant gray literature from the Health Management Information Database (HMIC), Open SIGLE Database, and the National Technical Information Service (NTIS) will be searched. Experts in the field will be consulted for relevant studies.

Search strategy

The search terms on PubMed are as follows: COVID-19 (e.g., “COVID-19 virus disease” or “novel corona virus” or “SARS-CoV-2”); randomized controlled trial (e.g., “randomized” or “randomly” or “clinical trial”); vaccines (e.g., “COVID 19 vaccines” or “COVID-19 virus vaccines” or “mRNA-1273 vaccine” or “ChAdOx1 COVID-19 vaccine”); randomized controlled trials (e.g., “randomized” or “randomized” or “clinical trials”). Combinations of medical subject headings (MeSH) and text words will be used. The same search term is used across the electronic databases. These search terms are shown in Table 1.

Data collection and analysis

Selection of studies. We chose the PRISMA flowchart to show the process of literature selection throughout the study (Figure 1.). All reviewers will discuss and determine the screening criteria before literature retrieval. Once the screening requirements are defined, two reviewers (XXL and DZY) will independently review and filter the titles and abstracts searched according to the inclusion criteria. To obtain eligible studies, full-text reports will be screened to determine if they meet the inclusion criteria, and then some duplicate studies or studies with incomplete information will be excluded. The obtained literature will be managed using Endnote software v.X8. Any inconsistencies should be discussed and resolved with a third investigator.

Data extraction and management. Two other researchers (LZ and FX) will independently extract the data and fill in a predesigned form. Information includes the first author, country, year, methods, quality and type of vaccine, vaccination dose, inoculation time interval, the number of participants, and baseline characteristics (race, sex ratio, age range or average age), research, design, results, specific data, conclusions, follow-up, adverse events, local and systemic adverse reactions, laboratory examination indexes, funds, sponsors and registration number, conflicts of interest, funding sources and ethical approval. The extracted data will be cross-checked by two researchers. If there is a disagreement, a third researcher (NL) will be involved. If necessary, we will contact the study authors for further information. All data will be transferred to the Review Manager software (RevMan v. 5.3) for analysis and synthesis.

Measures of treatment effect. In this protocol, efficacy data will be synthesized and statistically analyzed by two reviewers independently using RevMan 5.3. A risk ratio (RR) or odd ration with 95% confidence interval (CIs) will be adopted for dichotomous data, whereas a mean difference (MD) or standard mean difference (SMD) with 95% CIs will be utilized for continuous data. SMD will be employed if different assessment tools are used.

Management of missing data. We will try our best to ensure the integrity of the data. When there is missing data, we will try our best to contact the corresponding author of the article, including but not limited to, sending emails, or making a phone call. If the corresponding author cannot be contacted, we will use sensitivity analysis to assess the effect of the missing data on the outcome. If the effect is significant, we will remove the experiment with incomplete data. After data integrity is assured, intention analysis therapy and sensitivity analysis will be performed.

Assessment of heterogeneity. Statistical heterogeneity will be investigated using test and statistic. A fixed-effect model will be applied when heterogeneity is low (< 50%) and random-effects model will be used for moderate heterogeneity (50% < < 75%). When heterogeneity is considerably high, meta-analysis will not be performed.

Assessment of reporting biases. Funnel plots will be performed to assess potential reporting bias when more than ten studies are included. In additional, Egger regression and Begg correlation test will be conducted to identify the funnel plot asymmetry.

Data synthesis. In line with the Cochrane guideline, the fixed-effects model will be utilized for the pooled data if heterogeneity is deemed low and the random-effect model will be employed if heterogeneity is deemed moderate. Subgroup analysis or meta-regression will be performed to assess the potential sources with reasonable explanations if heterogeneity is considerably high. The statistical significance is defined as P < 0.05. If the meta-analysis is not feasible, a narrative description of the results will be provided. Data on vaccine effectiveness and safety will be summarized by country and specific geographic region (where applicable) (Table 2)

Assessment of reporting biases. In this analysis, if there are more than ten studies the funnel diagram will be used to determine whether there is a reporting biases.

Subgroup analysis. We will perform subgroup analysis according to the different details of interventions, study quality and outcome indicators.

Sensitivity analysis. Sensitivity analysis will be performed according to sample size, study design, heterogeneous quality, methodological quality and statistical model. Trials with quality defects will be excluded to ensure the stability of the analysis results.

Grading the quality of evidence. This paper will use the evidence quality rating method to evaluate the results obtained from this analysis. Grade will be assessed through the evaluation of domains of risk of bias, consistency, directness, precision, and publication bias[18]. In the context of the system review, quality reflects our confidence in the effectiveness of assessment. It has four evaluation levels, namely, high (further research is very unlikely to change our confidence in the estimate of effect), moderate (further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate), low (further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate), or very low (very uncertain about the estimate of effect).

Discussion

The COVID-19 pandemic has created a public health crisis, infecting millions of people and causing many deaths. The Government has implemented many public health measures to mitigate the spread of the virus. However, providing immunization services is crucial to ensure protection from vaccine-preventable diseases. There is an urgent need for COVID-19 vaccines and therapeutic agents in low and middle-income countries [19–21]. Mass vaccination sites will be an important innovation in the containment of the COVID-19 pandemic, but only if the effectiveness and safety of vaccination are ensured [22, 23]. Currently, there are still many unanswered questions in this area. The protection rate, protection time, safety, and immune response of COVID-19 vaccines requires further study[24]。Although few adverse reactions have been found in current clinical trials, But COVID-19 is evolving, therefore vaccines will evolve with it will be ongoing. So, extensive and long-term safety monitoring is needed to assess the efficacy and safety of COVID-19 vaccines.

This paper will systematically evaluate and meta-analyze the efficacy and safety of COVID-19 vaccines from four aspects: literature collection, literature screening, data extraction and data analysis. This study will provide evidence-based analysis of the efficacy and safety of COVID-19 vaccination. However, because of language limitations, we only searched Chinese and English literatures. Considering that there may be many studies that have not been published yet, it is necessary to update more high-quality clinical evidence to ensure the integrity of evidence in this study, which is also the deficiency of this study.

Abbreviations

COVID-19: Corona Virus Disease 2019; SARS-CoV-2: severe acute respiratory syndrome coronavirus 2; WHO: World Health Organization; RCTs: randomized controlled trials; RR: risk ratio; CIs: confidence interval; MD: mean difference; SMD: standard mean difference.

Declarations

Author contributions

Conceptualization: Xiali Xue.

Data curation: Ling Zhou, Fan Xu, Nan Wang, Xiaokun Wang.

Formal analysis: Xiali Xue, Zhongyi Deng

Investigation: Nan Wang, Xiaokun Wang, Fan Xu.

Methodology: Xiali Xue, Ning Li, Ling Zhou.

Software: Xiali Xue, Ning Li.

Writing – original draft: Xiali Xue.

Writing – review & editing: Zhongyi Deng, Ling Zhou, Ning Li.

All authors read and approved the final manuscript.

Funding

This systematic review and meta-analysis was funded by the Key Laboratory of Sports Medicine of Sichuan Province, Institute of Sports Medicine and Health, Chengdu Sport University (Approval Number: 2021-A003). The Institute of Sports Medicine and Health of Chengdu Sport University does not participate in any other aspects of the program, such as program design and analysis planning, collection and analysis, etc. Funders do not participate in the interpretation or publication of research results.

Availability of data and materials

Not applicable

Ethics approval and consent to participate

Not applicable

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests

References

1. Liu YC, Kuo RL, Shih SR. COVID-19: The first documented coronavirus pandemic in history. Biomedical journal. 2020;43(4):328–33. doi:10.1016/j.bj.2020.04.007.
2. Zhang X, Cao D, Liu J, Zhang Q, Liu M. Efficacy and safety of Lianhua Qingwen combined with conventional antiviral Western Medicine in the treatment of coronavirus disease (covid-19) in 2019: Protocol for a systematic review and meta-analysis. Medicine. 2020;99(30):e21404. doi:10.1097/md.0000000000021404.
3. WHO coronavirus disease (COVID-19) dashboard. World Health Organization. 2021. [cited 11 Mar 2021]. Available: https://covid19.who.int/.
4. Lv H, Wu NC, Mok CKP. COVID-19 vaccines: Knowing the unknown. Eur J Immunol. 2020;50(7):939–43. doi:10.1002/eji.202048663.
5. Silveira MM, Moreira G, Mendonça M. DNA vaccines against COVID-19: Perspectives and challenges. Life sciences. 2021;267:118919. doi:10.1016/j.lfs.2020.118919.
6. Lin DY, Zeng D, Gilbert PB. Evaluating the Long-Term Efficacy of COVID-19 Vaccines. Clinical infectious diseases: an official publication of the Infectious Diseases Society of America. 2021. doi:10.1093/cid/ciab226.
7. Wu F, Zhao S, Yu B, Chen YM, Wang W, Song ZG, et al. A new coronavirus associated with human respiratory disease in China. Nature. 2020;579(7798):265–9. doi:10.1038/s41586-020-2008-3.
8. Yadav T, Srivastava N, Mishra G, Dhama K, Kumar S, Puri B, et al. Recombinant vaccines for COVID-19. Human vaccines immunotherapeutics. 2020;16(12):2905–12. doi:10.1080/21645515.2020.1820808.
9. Polack FP, Thomas SJ, Kitchin N, Absalon J, Gurtman A, Lockhart S, et al. Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine. N Engl J Med. 2020;383(27):2603–15. doi:10.1056/NEJMoa2034577.
10. Zhu FC, Guan XH, Li YH, Huang JY, Jiang T, Hou LH, et al. Immunogenicity and safety of a recombinant adenovirus type-5-vectored COVID-19 vaccine in healthy adults aged 18 years or older: a randomised, double-blind, placebo-controlled, phase 2 trial. Lancet. 2020;396(10249):479–88. doi:10.1016/s0140-6736(20)31605-6.
11. Keech C, Albert G, Cho I, Robertson A, Reed P, Neal S, et al. Phase 1–2 Trial of a SARS-CoV-2 Recombinant Spike Protein Nanoparticle Vaccine. N Engl J Med. 2020;383(24):2320–32. doi:10.1056/NEJMoa2026920.
12. Abbasi J. COVID-19 and mRNA Vaccines-First Large Test for a New Approach. Jama. 2020;324(12):1125–7. doi:10.1001/jama.2020.16866.
13. Shi Y, Wang N, Zou QM. Progress and challenge of vaccine development against 2019-novel coronavirus (2019-nCoV). Zhonghua yu fang yi xue za zhi [Chinese journal of preventive medicine]. 2020;54(6):614–9. doi:10.3760/cma.j.cn112150-20200317-00366.
14. Draft landscape of covid-19 candidate vaccines. World Health Organization., 2021. [cited 8 Mar 2021]. Available: https://www.who.int/publications/m/item/draft-landscape-of-covid-19-candidate-vaccines.
15. Ahmad T, Murad MA, Baig M, Hui J. Research trends in COVID-19 vaccine: a bibliometric analysis. Human vaccines immunotherapeutics. 2021:1–6. doi:10.1080/21645515.2021.1886806.
16. ElBagoury M, Tolba MM, Nasser HA, Jabbar A, Elagouz AM, Aktham Y, et al. The find of COVID-19 vaccine: Challenges and opportunities. J Infect Public Health. 2020;14(3):389–416. doi:10.1016/j.jiph.2020.12.025.
17. Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;350:g7647. doi:10.1136/bmj.g7647.
18. Schünemann HJ, Santesso N, Vist GE, Cuello C, Lotfi T, Flottorp S, et al. Using GRADE in situations of emergencies and urgencies: certainty in evidence and recommendations matters during the COVID-19 pandemic, now more than ever and no matter what. J Clin Epidemiol. 2020;127:202–7. doi:10.1016/j.jclinepi.2020.05.030.
19. Rahman S, Montero MTV, Rowe K, Kirton R, Kunik F. Jr. Epidemiology, pathogenesis, clinical presentations, diagnosis and treatment of COVID-19: A review of current evidence. Expert review of clinical pharmacology. 2021. doi:10.1080/17512433.2021.1902303.
20. Bartovic J, Datta SS, Severoni S, D'Anna V. Ensuring equitable access to vaccines for refugees and migrants during the COVID-19 pandemic. Bull World Health Organ. 2021;99(1):3-a. doi:10.2471/blt.20.267690.
21. Urgent needs of low-income. and middle-income countries for COVID-19 vaccines and therapeutics. Lancet. 2021;397(10274):562–4. doi:10.1016/s0140-6736(21)00242-7.
22. Goralnick E, Kaufmann C, Gawande AA. Mass-Vaccination Sites - An Essential Innovation to Curb the Covid-19 Pandemic. The New England journal of medicine. 2021. doi:10.1056/NEJMp2102535.
23. Castells MC, Phillips EJ. Maintaining Safety with SARS-CoV-2 Vaccines. N Engl J Med. 2021;384(7):643–9. doi:10.1056/NEJMra2035343.
24. Whitaker DK. UK COVID-19 public inquiry and lessons from patient safety. Lancet. 2021;397(10277):878. doi:10.1016/s0140-6736(21)00381-0.

Tables

Table 1 Search strategy for the PubMed database.

Table 2. Characteristics of the published studies included in the meta-analysis.