The registration
The systematic review protocol was registered with the International Prospective Register of Systematic Reviews (PROSPERO) on March 15, 2020 (registration number CRD42021242581). The consent of this protocol report is based on the preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement guidelines [17]. (see Additional file 1).
Criteria for considering studies for this review
Studies that meet the following criteria will be included in this review.
Inclusion and criteria for study selection
Study designs. We will review all studies on the efficacy and safety of the preventative COVID-19 vaccine in humans. Because of language limitations, we will search for articles in both Chinese and English. To obtain a more objective and authentic evaluation, all articles must meet the following two conditions:
(1) Published documents with complete data
(2) The trial design is a randomized controlled trial (RCT)
Participants. We will include healthy men 18 years of age and older or non-pregnant women who have been vaccinated against COVID-19. Race, education and economic status will not be study factors. Those excluded from the study include pregnant women, participants with post-operative infections or mental illness, participants with severe pneumonia or other reasons for inability to exercise, and participants with severe cardiovascular and/or liver and/or kidney disease.
Interventions. The experimental group received a prophylactic COVID-19 vaccine, while the control group received a placebo.
Outcomes. Clinical trial results indicators include at least one or more of the following: local reactions (pain, itching, redness, swelling and induration, etc.), systemic adverse reactions (fever, diarrhea, fatigue, nausea, vomiting, lethargy, etc.), 14 days or 28 days after the last vaccination with live virus neutralization test of neutralizing antibody geometric average drop degree (GMT), serum conversion rate, mortality rates and other laboratory tests.
Data sources
Electronic data sources. The following electronic databases will be searched from inception to March 2021: PubMed, the Cochrane Library, Web of Science, CNKI, WanFang Data, Weipu Electronics. In addition, reference lists of the included studies were manually searched to identify additional relevant studies.
Other resources. Relevant references will be reviewed and screened. In addition, we will search the following registration website of the clinical trial: WHO ICTRP (www.chictr.org.cn, www.ClinicalTrial.gov) and the ISRCTN Register. Moreover, the relevant gray literature from the Health Management Information Database (HMIC), Open SIGLE Database, and the National Technical Information Service (NTIS) will be searched. Experts in the field will be consulted for relevant studies.
Search strategy
The search terms on PubMed are as follows: COVID-19 (e.g., “COVID-19 virus disease” or “novel corona virus” or “SARS-CoV-2”); randomized controlled trial (e.g., “randomized” or “randomly” or “clinical trial”); vaccines (e.g., “COVID 19 vaccines” or “COVID-19 virus vaccines” or “mRNA-1273 vaccine” or “ChAdOx1 COVID-19 vaccine”); randomized controlled trials (e.g., “randomized” or “randomized” or “clinical trials”). Combinations of medical subject headings (MeSH) and text words will be used. The same search term is used across the electronic databases. These search terms are shown in Table 1.
Data collection and analysis
Selection of studies. We chose the PRISMA flowchart to show the process of literature selection throughout the study (Figure 1.). All reviewers will discuss and determine the screening criteria before literature retrieval. Once the screening requirements are defined, two reviewers (XXL and DZY) will independently review and filter the titles and abstracts searched according to the inclusion criteria. To obtain eligible studies, full-text reports will be screened to determine if they meet the inclusion criteria, and then some duplicate studies or studies with incomplete information will be excluded. The obtained literature will be managed using Endnote software v.X8. Any inconsistencies should be discussed and resolved with a third investigator.
Data extraction and management. Two other researchers (LZ and FX) will independently extract the data and fill in a predesigned form. Information includes the first author, country, year, methods, quality and type of vaccine, vaccination dose, inoculation time interval, the number of participants, and baseline characteristics (race, sex ratio, age range or average age), research, design, results, specific data, conclusions, follow-up, adverse events, local and systemic adverse reactions, laboratory examination indexes, funds, sponsors and registration number, conflicts of interest, funding sources and ethical approval. The extracted data will be cross-checked by two researchers. If there is a disagreement, a third researcher (NL) will be involved. If necessary, we will contact the study authors for further information. All data will be transferred to the Review Manager software (RevMan v. 5.3) for analysis and synthesis.
Measures of treatment effect. In this protocol, efficacy data will be synthesized and statistically analyzed by two reviewers independently using RevMan 5.3. A risk ratio (RR) or odd ration with 95% confidence interval (CIs) will be adopted for dichotomous data, whereas a mean difference (MD) or standard mean difference (SMD) with 95% CIs will be utilized for continuous data. SMD will be employed if different assessment tools are used.
Management of missing data. We will try our best to ensure the integrity of the data. When there is missing data, we will try our best to contact the corresponding author of the article, including but not limited to, sending emails, or making a phone call. If the corresponding author cannot be contacted, we will use sensitivity analysis to assess the effect of the missing data on the outcome. If the effect is significant, we will remove the experiment with incomplete data. After data integrity is assured, intention analysis therapy and sensitivity analysis will be performed.
Assessment of heterogeneity. Statistical heterogeneity will be investigated using test and statistic. A fixed-effect model will be applied when heterogeneity is low (< 50%) and random-effects model will be used for moderate heterogeneity (50% < < 75%). When heterogeneity is considerably high, meta-analysis will not be performed.
Assessment of reporting biases. Funnel plots will be performed to assess potential reporting bias when more than ten studies are included. In additional, Egger regression and Begg correlation test will be conducted to identify the funnel plot asymmetry.
Data synthesis. In line with the Cochrane guideline, the fixed-effects model will be utilized for the pooled data if heterogeneity is deemed low and the random-effect model will be employed if heterogeneity is deemed moderate. Subgroup analysis or meta-regression will be performed to assess the potential sources with reasonable explanations if heterogeneity is considerably high. The statistical significance is defined as P < 0.05. If the meta-analysis is not feasible, a narrative description of the results will be provided. Data on vaccine effectiveness and safety will be summarized by country and specific geographic region (where applicable) (Table 2)
Assessment of reporting biases. In this analysis, if there are more than ten studies the funnel diagram will be used to determine whether there is a reporting biases.
Subgroup analysis. We will perform subgroup analysis according to the different details of interventions, study quality and outcome indicators.
Sensitivity analysis. Sensitivity analysis will be performed according to sample size, study design, heterogeneous quality, methodological quality and statistical model. Trials with quality defects will be excluded to ensure the stability of the analysis results.
Grading the quality of evidence. This paper will use the evidence quality rating method to evaluate the results obtained from this analysis. Grade will be assessed through the evaluation of domains of risk of bias, consistency, directness, precision, and publication bias[18]. In the context of the system review, quality reflects our confidence in the effectiveness of assessment. It has four evaluation levels, namely, high (further research is very unlikely to change our confidence in the estimate of effect), moderate (further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate), low (further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate), or very low (very uncertain about the estimate of effect).