The protocol is reported in accordance with the reporting guideline provided in the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols (PRISMA-P) statement (see Additional file 1) (48, 49), and is registered in the International Prospective Register of Systematic Reviews (PROSPERO CRD42020196492) database. The review will be carried out following recommendations outlined in The Cochrane Handbook of Systematic Review of Interventions (40).
Criteria for considering studies for this review
Types of studies
We will search for and include randomized clinical trials, irrespective of publication status, publication year, and language. We will not search for but will include quasi-randomized studies and observational studies identified during our search for trials for the assessment of harms.
Types of participants
Any participant not previously infected with SARS-CoV-2. Participants will be included irrespective of sex, comorbidities, immune status, and risk group.
Types of interventions
Experimental group
We will include any vaccine aiming to prevent COVID-19, i.e. all vaccines listed in Table 1 or any other vaccine irrespective of dose and duration of administration.
Control group
We will include randomized clinical trials with any control group, i.e. head-to-head comparisons versus placebo, ‘active placebo’ (a matching placebo that produces noticeable adverse effects that may convince the participant being vaccinated and blinded outcome assessors that the participants are receiving an active intervention), usual care (or similar terms), no intervention, another vaccine aiming at preventing COVID-19, or any other ‘active’ comparator. We will accept any of these control interventions irrespective of dose and duration of administration.
Co-interventions will be allowed provided they are administered equally to the comparison groups.
Primary outcomes
-
All-cause mortality.
-
Proportion of participants with confirmed COVID-19 (verified by RT-PCR or similar laboratory tests).
-
Proportion of participants with one or more serious adverse events. We will use the International Conference on Harmonization of technical requirements for registration of pharmaceuticals for human use—Good Clinical Practice (ICH-GCP) definition of a serious adverse event, which is any untoward medical occurrence that resulted in death, was life-threatening, required hospitalization or prolonging of existing hospitalization, and resulted in persistent or significant disability or jeopardized the participant (50). If the trialists do not use the ICH-GCP definition, we will include the data if the trialists use the term ‘serious adverse event’. If the trialists do not use the ICH-GCP definition nor use the term serious adverse event, then we will also include the data if the event clearly fulfills the ICH-GCP definition for a serious adverse event. We will exploratorily assess each type of serious adverse event separately (see below).
Secondary outcomes
-
Health-related quality of life (assessed on any valid continuous scale).
-
Proportion of participants with one or more adverse events not considered serious. We will exploratorily assess each type of adverse events not considered serious separately (see below).
Exploratory outcomes
-
Proportion of participants with positive antibody response against COVID-19 antigens.
-
Individual types of serious adverse events will be analyzed separately.
-
Individual types of adverse events not considered serious will be analyzed separately.
We will use the trial results reported at maximum follow-up.
Search methods for identification of studies
Electronic searches
An experienced information specialist will search Cochrane Central Register of Controlled Trials (CENTRAL), Medical Literature Analysis and Retrieval System Online (MEDLINE), Excerpta Medica database (EMBASE), Latin American and Caribbean Health Sciences Literature (LILACS), Science Citation Index Expanded (SCI-EXPANDED),
Conference Proceedings Citation Index—Science (CPCI-S), Chinese Biomedical Literature Database (CBM), China Network Knowledge Information (CNKI), Chinese Science Journal Database (VIP), and Wafang Database to identify relevant trials. We will
search all databases from their inception to the present. Trials will be included irrespective of language, publication status, publication year, and publication type. For a detailed search strategy for all electronic searches, see Additional file 2.
Searching other resources
We will identify additional references by manually searching the references of articles from the computerized databases. We will also search special COVID-19 trial sites, including a website with living mapping and living systematic review of Covid-19 studies (https://covid-nma.com/), a website developed by Vaccine Centre at the London School of Hygiene & Tropical Medicine (https://vac-lshtm.shinyapps.io/ncov_vaccine_landscape/), Lancet’s “A real-time dashboard of clinical trials for COVID-19 (https://www.thelancet.com/journals/landig/article/PIIS2589-7500(20)30086-8/fulltext), the preprint server for health sciences, www.medrxiv.org, and an open, accessible and frequently updated clinical trial registration for COVID-19 trials (https://doi.org/10.12688/wellcomeopenres.15821.1)
We will also search online trial registries such as ClinicalTrials.gov (clinicaltrials.gov), the Chinese Clinical Trial Registry (www.chictr.org.cn), the European Medicines Agency (EMA) (www.ema.europa.eu/), the World Health Organization (WHO) International Clinical Trials Registry Platform (www.who.int/ictrp/), and the Food and Drug Administration (FDA) (www.fda.gov/) for ongoing or unpublished trials. We will contact experts in the field and pharmaceutical companies to enquire about additional trials. We will search for grey literature in the System for Information on Grey Literature in Europe OpenGrey (www.opengrey.eu/).
Data extraction and management
Two review authors will independently extract data from included trials in a predefined form. Disagreements will be resolved by discussion, or if required, through consultation with a third author (JCJ or CG). The two review authors will assess duplicate publications and companion papers of a trial together to evaluate all available data simultaneously (maximize data extraction, correct bias assessment). Each trial will be named after the first author and year of the primary publication, and all secondary publications will be classified under that name. We will contact the trial authors by email to specify any missing data, which may not be reported sufficiently or not at all in the publication.
Living systematic review
A living systematic review is defined as a systematic review which is continually updated and incorporates relevant new evidence as it becomes available (51). This methodology may be particularly important in the COVID-19 pandemic, where research evidence is emerging rapidly, current evidence is uncertain, and new research may change policy or practice decisions (51).
There are four fundamental differences between conventional systematic reviews and living systematic reviews: publication format, work processes, author team management, and statistical methods (52). In this living systematic review, two independent investigators will receive an updated literature search file and include relevant newly published or unpublished trials once a week. The relevant meta-analyses, Trial Sequential Analyses, and network meta-analyses will continuously be updated, and if new evidence is available (judged by the steering committee of the LIVING VACCINE review), the results will be published. Every month, the steering committee will discuss whether searching once a week is necessary. The living systematic review process will be initiated June 27, 2020. For an illustration of the living systematic review workflow, see Fig. 1 (with permission from Juul et al.(19) and Systematics Reviews).
Trial characteristics
We will extract the following data: bias risk components (as defined below), trial design (parallel, factorial, crossover, cluster), number of intervention groups, length of follow-up, estimation of sample size, inclusion and exclusion criteria.
Participant characteristics
We will extract the following data: number of randomized participants, number of participants with comorbidities and types of comorbidities, number of analyzed participants, number of participants lost to follow-up/ withdrawals/crossover, age range (mean or median), and sex ratio.
Experimental intervention characteristics
We will extract the following data: type of vaccine, type of adjuvants, dose of intervention, duration of intervention.
Control intervention characteristics
We will extract the following data: type of control intervention, dose of intervention, duration of intervention.
Outcomes
All outcomes listed above will be extracted from each randomized clinical trial. For each outcome, we will identify if outcomes are missing, inappropriately measured, or selectively reported according to the criteria described later in the ‘missing outcome data’ bias domain, the ‘risk of bias in measurement of the outcome’ bias domain, and the ‘risk of bias in selection of the reported result’ bias domain.
Notes
We will search for information regarding industry funding of either personal or academic activities for each trial author. We will judge a publication at high risk of for-profit bias if a trial is sponsored by the industry or if just one author has affiliation to the industry. We will note in the “Characteristics of included studies” table if outcome data were not reported in a usable way. Two review authors will independently transfer data into the Stata file (53).
Assessment of risk of bias in the included studies
Our bias risk assessment will be based on the Cochrane Risk of Bias tool—version 2 (RoB 2) as recommended in The Cochrane Handbook of Systematic Reviews of Interventions (40). We will evaluate the methodology in respect of the following bias domains:
Bias arising from the randomization process
Low risk of bias Allocation was adequately concealed, AND there are no baseline imbalances across intervention groups at baseline appear to be compatible with chance, AND an adequate (random or otherwise unpredictable) method was used to generate allocation sequence, OR there is no information about the method used to generate the allocation sequence
Some concerns Allocation was adequately concealed, AND there is a problem with the method of sequence generation, OR baseline imbalances suggest a problem with the randomization process, OR no information is provided about concealment of allocation, AND baseline imbalances across intervention groups appear to be compatible with chance, OR no information to answer any of the signaling questions
High risk of bias Allocation sequence was not concealed, OR no information is provided about concealment of allocation sequence, AND baseline imbalances suggest a problem with the randomization process.
Bias due to deviation from intended interventions
Low risk of bias Participants, carers, and personnel were unaware of intervention groups during the trial, OR participants, carers, or personnel were aware of intervention groups during the trial but any deviations from intended intervention reflected usual practice, OR participants, carers, or personnel were aware of intervention groups during the trial but any deviations from intended intervention were unlikely to impact on the outcome, AND no participants were analyzed in the wrong intervention groups (that is, on the basis of intervention actually received rather than of randomized allocation).
Some concerns Participants, carers, or personnel were aware of intervention groups and there is no information on whether there were deviations from usual practice that were likely to impact on the outcome and were imbalanced between intervention groups, OR some participants were analyzed in the wrong intervention groups (on the basis of intervention actually received rather than of randomized allocation) but there was little potential for a substantial impact on the estimated effect of intervention.
High risk of bias Participants, carers, or personnel were aware of intervention groups, and there were deviations from intended interventions that were unbalanced between the intervention groups and likely to have affected the outcome, OR some participants were analyzed in the wrong intervention groups (on the basis of intervention actually received rather than of randomized allocation), and there was potential for a substantial impact on the estimated effect of intervention.
Bias due to missing outcome data
Low risk of bias No missing data OR non-differential missing data (similar proportion of and similar reasons for missing data in compared groups) OR evidence of robustness of effect estimate to missing data (based on adequate statistical methods for handling missing data and sensitivity analysis)
Some concerns An unclear degree of missing data or unclear information on proportion and reasons for missingness in compared groups AND there is no evidence that the effect estimate is robust to missing data.
High risk of bias A high degree of missing data AND differential missing data (different proportion of or different reasons for missing data in compared groups) AND there is no evidence that the effect estimate is robust to missing data.
Bias in measurement of outcomes
Low risk of bias The outcome assessors were unaware of the intervention received by study participants, OR the outcome assessors were aware of the intervention received by study participants, but the assessment of the outcome was unlikely to be influenced by knowledge of the intervention received.
Some concerns There is no information available to determine whether the assessment of the outcome is likely to be influenced by knowledge of the intervention received.
High risk of bias The assessment of the outcome was likely to be influenced by knowledge of the intervention received by study participants.
Bias arising from selective reporting of results
Low risk of bias Reported outcome data are unlikely to have been selected, on the basis of the results, from multiple outcome measurements (e.g., scales, definitions, time points) within the outcome domain, and reported outcome data are unlikely to have been selected, on the basis of the results, from multiple analyses of the data.
Some concerns There is insufficient information available to exclude the possibility that reported outcome data were selected, on the basis of the results, from multiple outcome measurements (e.g. scales, definitions, time points) within the outcome domain, or from multiple analyses of the data. Given that analysis intentions are often unavailable or not reported with sufficient detail, we anticipate that this will be the default judgment
for most trials.
High risk of bias Reported outcome data are likely to have been selected, on the basis of the results, from multiple outcome measurements (e.g., scales, definitions,
time points) within the outcome domain, or from multiple analyses of the data (or both).
Overall assessment of risk of bias
Low risk of bias The trial is judged to be at low risk of bias for all domains for this result.
High risk of bias The trial is judged to be at high risk of bias or to be at some concerns in at least one domain for this result. Our subgroup analysis will compare the intervention effect of trials at low risk of bias with trials at high risk of bias, that is one or more domains at some concern or high risk of bias.
We will assess the domains ‘missing outcome data’, ’risk of bias in measurement of the outcome’, and ‘risk of bias in selection of the reported result’ for each outcome result. Thus, we can assess the bias risk for each outcome assessed in addition to each trial. Our primary conclusions will be based on the results of our primary outcome results with overall low risk of bias. Both our primary and secondary conclusions will be presented in the Summary of Findings-tables.
Differences between the protocol and the review
We will conduct the review according to this published protocol and report any deviations from it in the ’Differences between the protocol and the review’ section of the systematic review.
Measurement of treatment effect
Dichotomous outcomes We will calculate risk ratios (RRs) with 95% confidence interval (CI) for dichotomous outcomes, as well as the trial sequential analysis adjusted CIs (see below).
Continuous outcomes
We will calculate the mean differences (MDs) and consider calculating the standardized
mean difference (SMD) with 95% CI for continuous outcomes. We will also calculate trial sequential analysis-adjusted CIs (see below).
Dealing with missing data
We will use intention-to-treat data if provided by the trialists (54). We will, as the first option, contact all trial authors to obtain any relevant missing data (i.e. for data extraction and for assessment of risk of bias, as specified above), when individual patient data is not available.
Dichotomous outcomes
We will not impute missing values for any outcomes in our primary analysis. In our sensitivity analyses (see paragraph below), we will impute data.
Continuous outcomes
We will primarily analyze scores assessed at single time points. We will analyze change
from baseline scores using a MD if the same scale is used across studies. For different measurement scales in the same analysis model, we will use the SMD effect size.
In case some studies do not report change scores, but provide follow-up values, we will combine them together in a single model using MD (40). If standard deviations
(SDs) are not reported, we will calculate the SDs using relevant trial data (e.g., P values), if available. We will not use intention-to-treat data if the original report did not contain such data, per protocol data will then be used. In our best-worst worst-best scenarios (see paragraph below) for continuous outcomes, we will impute data.
Assessment of heterogeneity
We will primarily investigate forest plots to visually assess any sign of heterogeneity. We will secondly assess the presence of statistical heterogeneity using the I2 statistic (40, 55, 56) and restricted maximum likelihood method (57, 58). We will investigate evident heterogeneity through subgroup analyses (see “Subgroup analyses and integration of heterogeneity” section below). We may ultimately decide that a meta-analysis should be avoided if heterogeneity is high (40).
Assessment of reporting biases
We will use a funnel plot to assess reporting bias if ten or more trials are included (40). We will visually inspect funnel plots to assess the risk of bias. We are aware of the limitations of a funnel plot (i.e. a funnel plot assesses bias due to small sample size) (40). From this information, we will assess possible reporting bias. For dichotomous outcomes, we will test asymmetry with the Harbord’s test (59) if τ2 is less than 0.1 and with the Rücker test if τ2 is more than 0.1 (40). For continuous outcomes, we will use the regression asymmetry test (60) and the adjusted rank correlation (61).
Unit of analysis issues
We will include randomized clinical trials for assessment of benefits and harms.
In case of trials with a cross-over design, we will include the data from the first trial period in order to avoid residual effects from the treatment (40). In order to avoid repeated observations on trial participants, we will use participant trial data at the longest follow‐up (40).
We will analyze cluster randomized trials using the procedures referenced in the Cochrane Handbook for Systematic Reviews of Interventions (40). Where results did not control for clustering, we will contact trial authors to request an estimate of the intracluster correlation coefficient (ICC). If the trial authors are unable to provide an ICC, we will calculate the ICC using design effects (62).
If we during our searches for trials identify observational studies reporting on harms, we will tabulate these harms and report them separately in the Results section. This is to cover rare and late occurring harms. We will tabulate the types of adverse events (serious and non-serious) that are reported in the non‐randomized studies retrieved with the searches for randomized clinical trials. This will limit the information on harms in our systematic review. If benefits of certain vaccines are found, then systematic reviews of harms, based on observational studies, should be conducted (63).
Data synthesis
All types of interventions will be included in the network meta-analysis. When analyzing individual patient data meta-analyses, aggregate data meta-analyses, and Trial Sequential Analyses, the results of each type of intervention will be analyzed separately. We will use intention-to-treat data in all analyses. We will consider using multiple imputation techniques as recommended by Jakobsen et al. (54). Please consult this publication for a detailed description of the handling of missing data. We will present best-worst and worst-best case scenarios if it is not valid to ignore missing data (41). Best-worst and worst-best case scenarios assess the potential range of impact of the missing data for the trial results.
In the ‘best-worst’ case scenario, it is assumed that all patients lost to follow-up in the intervention group have had a beneficial outcome, and all those with missing outcomes in the control group have had a harmful outcome (41). Conversely, in the ‘worst-best’ case scenario, it is assumed that all patients who were lost to follow-up in the experimental group have had a harmful outcome, and that all those lost to follow-up in the control group have had a beneficial outcome (41). When continuous outcomes are used, a “beneficial outcome” will be defined as the group mean plus two SDs of the group mean, and a “harmful outcome” will be defined as the group mean minus two SDs of the group mean (41).
Aggregate data meta-analysis
We will undertake the meta-analyses according to the Cochrane Handbook of Systematic Reviews of Interventions (40), Keus et al. (64), and our eight-step assessment suggested by Jakobsen et al. (41). We will use the statistical software Stata version 16 (53) to analyze data. We will assess our intervention effects with both a random-effects meta-analysis (45) and fixed-effect meta-analysis for each treatment comparison separately (65). We will report the more conservative point estimate of the two (41). The more conservative point estimate is the estimate with the highest P value or the widest confidence interval. If there is substantial deviation between the random-effects and fixed-effects meta-analyses, we will report and discuss the results. We will assess a total of three primary outcomes and two secondary outcomes, and we will therefore consider a P value of 0.0167 or less as the threshold for statistical significance (41). We will investigate possible heterogeneity through subgroup analyses. We will use the eight-step procedure to assess if the thresholds for significance are crossed (41). Where multiple trial arms are reported in a single trial, we will include only the relevant arms. If two comparisons are combined in the same meta-analysis, we will halve the control group to avoid double-counting (40). Trials with a factorial design will be included. In case of, e.g. a 2 × 2 factorial designed trial, the two groups receiving COVID-19 intervention will be considered experimental groups, while the two groups receiving a placebo, ‘active placebo’, standard care, no intervention, or ‘active’ comparator will be considered control groups.
Trial Sequential Analysis
Due to the continuous inclusion of new trials and hence repetitive testing of accumulating data when updating reviews, there is an increased risk of type I error. We wish to control the risks of both type I errors and type II errors. We will therefore perform Trial Sequential Analysis on all outcomes, in order to calculate the required information size (that is, the number of participants needed in a meta-analysis to detect or reject a certain intervention effect) and the cumulative Z-curve’s breach of relevant trial sequential monitoring boundaries (46, 47, 66–72). A more detailed description of trial sequential analysis can be found in the trial sequential analysis manual (66) and at http://www.ctu.dk/tsa/. For dichotomous outcomes, we will estimate the required information size based on the observed proportion of patients with an outcome in the control group (the cumulative proportion of patients with an event in the control groups relative to all patients in
the control groups), a relative risk reduction or a relative risk increase of 20% or 10%, an alpha of 1.67% for all our outcomes, a beta of 10%, and the observed diversity as suggested by the trials in the meta-analysis. For continuous outcomes, we will in the Trial Sequential Analysis use the observed standard deviation (SD), a mean difference equal
to the observed SD/2, an alpha of 1.67% for all outcomes, a beta of 10%, and the observed diversity as suggested by the trials in the meta-analysis.
Network meta-analysis
We will obtain information about the interventions of interest either from head-to-head trials, or from trials comparing a COVID-19 intervention with placebo, standard care, no intervention, or “active placebo.” Hence, the synthesis comparator set consists of all the interventions listed in the background section as well as a placebo, ‘active placebo’, standard care, no intervention, or ‘active’ comparator trials. Each specific vaccine will be analyzed separately and will also be clustered with similar vaccines. We will describe the characteristics of the eligible randomized clinical trials and their populations using frequencies and percentages for categorical data, and means and standard deviations for continuous data.
Descriptive statistics will be also generated for each treatment comparison describing important clinical and methodological characteristics (e.g. publication year, participant age). Each outcome dataset will be presented in a different network diagram, where the size of the nodes will be proportional to the total number of randomized
participants, and the width of each edge will be weighted according to the number of studies comparing the connected treatments. We will additionally plot the edges of each network according to the average risk of bias per treatment comparison, using green for low, yellow for moderate, and red for high risk of bias. We anticipate that any participant who meets inclusion criteria is, in principle, equally likely to be randomized to any of the interventions in the synthesis comparator set. Network meta-analysis will be performed using Stata16 under frequentist framework (53) (command: mvmeta) (73). The network meta-analysis synthesizes evidence for the comparative effectiveness of more than two alternative interventions for the same condition (74). We will only perform network meta-analysis if a connected network of trials can be conducted.
If network meta-analysis is performed, we will, before conducting the network meta-analysis, assess two major assumptions: transitivity and consistency. We will then perform the statistical analysis in five steps. First step is to draw a network geometry in order to overview the network relationship. Second step is to assess for the transitivity assumption across treatment comparisons in the network using boxplots, as well as evaluate the assumption of consistency using the design-by-treatment interaction model as a global test (56, 74). Third step is to make the network forest plot or interval plot in order to illustrate the summary effect size of the comparative effectiveness among interventions. Fourth step is to calculate cumulative rankings for identifying a superiority among interventions. Last step is to evaluate publication bias or effect modifiers for a valid inference from results. Potential effect modifiers will be the same as the subgroup analyses, i.e. for-profit bias, type of comparator, age, sex, and types of vaccine. We will also explore these through network subgroup meta-analyses (see section below). The synthesized evidence through five steps would be useful to evidence-based decision-making in healthcare. Thus, network meta-analyses should be activated to guarantee the quality of health care system. The estimation of each treatment comparison will be reported separately using the relevant effect size (RR), a 95% CI, and a 95% prediction interval. Along the estimated effect sizes, we will present the ranking probabilities for each treatment being at each possible rank, as well as the surface under the cumulative ranking curve (SUCRA) or relevant P scores (75, 76). A rank-heat plot will be used to depict the SUCRA values or P scores across all outcomes (77).
Individual patient data meta-analysis
Results of individual patient data meta-analysis will increase the possibility to identify subgroups of patients with specific effects of the assessed interventions (78–80). If we receive individual patient data for all eligible randomized clinical trials, we will analyze the data using a one-stage analysis model based on generalized linear mixed models. This analysis will be adjusted for the categoric baseline variables that the trials used as stratifications variables in their randomization (only the common variables that all of the trials adjust for). When analyzing continuous data, we will also adjust all analyses for the baseline value.
If we are unable to obtain sufficient individual patient data, we will secondly conduct a two-stage analysis, where at 1st stage, we will reduce available individual patient data to aggregate data for each study, and at 2nd stage, we will combine all available data in a meta-analysis.
Assessments of underlying statistical assumptions
We will systematically assess underlying statistical assumptions for all statistical analyses (53, 81, 82). In short, for all regression analyses, we will test for major interactions between each covariate and the intervention variable. We will, in turn, include each possible first order interaction between included covariates and the intervention variable. For each combination, we will test if the interaction term is significant and assess the effect size. We will only consider that there is evidence of an interaction if the interaction is statistically significant after Bonferroni adjusted thresholds (0.05 divided by number of possible interactions) and if the interaction shows a clinically significant effect. If it is concluded that the interaction is significant, we will consider both presenting an analysis separately for each (e.g. for each site if there is significant interaction between the trial intervention and ‘site’) and an overall analysis including the interaction term in the model (53, 81, 82). For detailed description of the planned assessments for underlying assumptions, please consult the recommendations of Nørskov et al. (53, 81, 82).
Subgroup analyses and integration of heterogeneity
Subgroup analyses
We will perform the following subgroup analyses when analyzing the primary outcomes (all-cause mortality, confirmed COVID-19, and serious adverse events).
-
Trials at high risk of bias compared to trials at low risk of bias. This is due to literature demonstrating overestimation of benefits and underestimation of harms in trials at risk of bias (83–85).
-
Trials without for-profit bias compared to trials at unknown or known risk of for-profit bias (37).
-
Type of vaccine (including but not limited to messenger RNA; DNA; non-replicating viral vector; replicating viral vector; inactivated virus; protein subunit; dendritic cell; other vaccines like live-attenuated, polysaccharide vaccine, conjugate vaccines).
-
Age (children and adolescents as defined by trialists; adults as defined by trialists; elderly as defined by trialists).
-
Type of antibody target (e.g., nonstructural replicase polyprotein, E protein, M protein, N-protein, S-protein or other targets)
-
Trials including participants who are immunocompromised compared to trials with participant that are not.
-
Trials including participants with history of chronic lung disease compared to trials without history of chronic lung disease.
-
Trials including unexposed participants at the time of vaccination compared to trials including participants exposed to SARS-CoV-2.
-
Trials including vaccines that target a T cell mediated response compared to trials that does not.
-
Trials at high or uncertain risks of vested interests compared to trials at low risk of vested interests. This subgroup will assess for-profit bias (37).
-
Trials using aluminium adjuvants compared to trials without aluminium adjuvants (86, 87).
-
Trials randomizing clusters compared to trials randomizing individual participants.
-
Trials using cross-over design compared to trials without cross-over design.
We will use the formal test for subgroup differences in Stata (53). We will perform any unanticipated subgroup analyses, if we identify these, as more information about this virus and its treatment becomes available.
Sensitivity analysis
To assess the potential impact of the missing data for dichotomous outcomes, we will perform the two following sensitivity analyses on all primary and secondary dichotomous outcomes.
‘Best-worst’ case scenario: We will assume that all participants lost to follow-up in the experimental group survived, did not have confirmed Covid-19, had no serious adverse events, did not have serious disease, had no non-serious adverse events, and had a positive antibody response, and that all those participants lost to follow-up in the control group did not survive, had confirmed Covid-19, had a serious adverse event, had serious disease, had a non-serious adverse event, and had no a positive antibody response.
‘Worst-best’ case scenario: We will assume that all participants lost to follow-up in the experimental group did not survive, had confirmed Covid-19, had a serious adverse event, had serious disease had a non-serious adverse event, and had no a positive antibody response and that all those participants lost to follow-up in the control group survived, did not have confirmed Covid-19, had no serious adverse events, did not have serious disease, had no non-serious adverse events, and had a positive antibody response.
We will present results of both scenarios in our review. We will present results of both scenarios in our review.
When analyzing quality of life, a “beneficial outcome” will be the group mean plus two SDs (we will secondly use one SD in another sensitivity analysis) of the group mean, and a “harmful outcome” will be the group mean minus two SDs (we will secondly use one
SD in another sensitivity analysis) of the group mean (41). To assess the potential impact of missing SDs for continuous outcomes, we will perform the following sensitivity
analysis:
-
Where SDs are missing and it is not possible to calculate them, we will impute SDs from trials with similar populations and low risk of bias. If we find no such trials, we will impute SDs from trials with a similar population. As the final option, we will impute the mean SD from all included trials.
We will present results of this scenario in our review. Other post hoc sensitivity analyses might be warranted if unexpected clinical or statistical heterogeneity is identified during the analysis of the review results (41).
Summary of Findings table
We will create a summary of findings table including each of the prespecified outcomes (all-cause mortality, confirmed COVID-19, serious adverse events, health-related quality of life, serious disease, non-serious adverse events, and positive antibody response against COVID-19 antigens). We will use the five GRADE considerations (bias risk of the trials, consistency of effect, imprecision, indirectness, and publication bias) to assess the quality of a body of evidence (41, 88–90). We will assess imprecision using Trial Sequential Analysis. We will justify all decisions to downgrade the quality of evidence using footnotes, and we will make comments to aid the reader’s understanding of the review where necessary. Firstly, we will present our results in the Summary of Findings table based on the results from the trials with overall low risk of bias, and secondly, we will present the results based on all trials.
Data sharing and availability
Full syntax of all statistical analyses will be published as supplementary material. All aggregate data will be published regularly. Anonymized individual patient data will also be published if possible (we will discuss this with the trialists).
Dissemination plan
Findings of this living systematic review will be published in international peer-reviewed scientific journals. Further, a dedicated webpage for the project will be developed, where iterative versions of the living systematic review will be accommodated with visual illustrations.