Quality assessment for the studies included in this meta-analysis was conducted using Cochran risk assessment tool (21), and NOS checklists (22). Quality assessment for RCTs is presented in the summary of the risk of bias graph Fig. 2 Quality assessment of observational studies is uploaded as supplementary material. LINK: https://docs.google.com/document/d/1199T2CxcP3z1X_L-5ICLY25E4xNIEsbg/edit?usp=sharing&ouid=100587500715661436749&rtpof=true&sd=true)
3.1 Mortality after vaccination
- Mortality after 7 days from dose 2
In total 6 studies assessed mortality related to the COVID-19 vaccine 7 days after vaccination. Four studies were randomized controlled trials, and two were observational. Only two studies reported a significant decline in the OR of mortality related to COVID-19 vaccination, however, the overall effect was statically significant. The odds ratio of mortality 7 days after full vaccination decreased significantly among vaccinated compared to the unvaccinated group OR=0.14(95CI, 0.05-0.41), I2=63%. The study of Logonov et al (29) was removed after conducting sensitivity analysis, this resulted in the reduction of heterogeneity to 54%, the OR became 0.10 (95CI%, 0.04-0.27).
- Two weeks after the second dose
Four studies addressed mortality after two weeks of the second dose, all these studies were observational. Except for the study of Sadoff et al (32), the odds ratio was significantly decreased. Pooled OR was 0.34 (95CI 0.26-0.44), however, the heterogeneity was statistically high 85%. After conducting a leave-one-out sensitivity analysis, the study of Khan et al (37), was omitted. The observed heterogeneity dropped to 0%, and the pooled effect was still significant OR= 0.46, (95% CI 0.35-0.61).
3.2 Severity after vaccination
- The severity of COVID 19 infection one week after the second dose
Seven research studied the severity of COVID-19 one week after the second dose. In total 213 of 4807683 vaccinated people developed severe COVID-19 one week after the second dose, compared to 3298 of unvaccinated 1915476 subjects. The OR of having severe COVID-19 decreased by 0.08 (0.03-0.25), I2 was 74%. After subgrouping the included studies into randomized and observational, the heterogeneity dropped to 0%. The difference between observational and interventional studies was not significant (P=0.46). The odds ratio of severe COVID-19 of RCT was 0.14 (0.03-0.75), while the OR in observational studies was 0.06 (0.02-0.24).
- The severity of COVID-19 infection one week after the first dose
With a heterogeneity of I2== 61%, OR of having severe COVID-19 one week after the first dose was 0.21 (95% CI, 0.11-0.40). After omitting the study of Khan et al (37), the OR of having severe COVID-19 after the first dose of vaccination among 79176 vaccinated and 78810 unvaccinated subjects in 6 studies was increased to 0.29 (0.19-0.46), I2 was 25%. Only this study of Sadoff et al (32), did not show any protective effect of vaccination.
3.3 Efficacy/effectiveness of the 1st dose
- Cases reported within the first week of vaccination after the 1st dose:
Total (Symptomatic and asymptomatic)
Two researchers evaluated the effectiveness of the COVID-19 vaccine in reducing numbers of cases within one week after the 1st dose; Dagan et al (33), reported that 1965 of 596618 vaccinated subjects get COVID-19 compared to 2362 of 596618 unvaccinated individuals, OR =0.83. Hall et al (41), highlighted that incidence of COVID-19 was 140 of 20641 unvaccinated subjects, still lower than cases reported among unvaccinated (977/2683), OR 0.01.
Bernal et al (38) reported that 346 of 864 vaccinated subjects versus 8988/24706 of unvaccinated subjects developed COVID-19, OR 1.17(95%CI 1.02-1.34) meaning that it has no protective effect. Dagan et al (33), mentioned that COVID-19 vaccine had a protective effect OR was 0.78 (95%CI, 0.72-0.84). Among vaccinated subjects (596618), about 1103 developed COVID-19, while among unvaccinated individuals 596618, about 1419 developed symptomatic COVID 19.
Due to significant heterogeneity, we could not pool the findings of these two outcomes.
- Cases reported within two weeks of vaccination after the 1st dose:
Total (Symptomatic and asymptomatic)
Four research studied effectiveness/efficacy of COVID-19 within two weeks after the first dose. In total 3909 of 637142 vaccinated people developed COVID-19 within 2 weeks after the first dose, compared to 5087 of unvaccinated 614989 subjects. The OR of having severe COVID-19 decreased by 0.17 (0.02- 1.72), I2 was 100%. After subgrouping included studies into randomized and observational, the heterogeneity dropped to 94%. The difference between observational and interventional studies was significant (P=0.001). The odds ratio of infection with COVID-19 of RCT was 0.79 (0.48-1.3), while the OR in observational studies was 0.17(0.02-0.172).
Symptomatic cases reported within 14 days of the first dose
Bernal et al (38) reported 958 symptomatic cases of 1154 vaccinated subjects compared to 89 of 8988 unvaccinated subjects the OR = 488 (95CI 337-633), meaning that vaccination increases the risk. The protective effect of the COVID-19 vaccine was addressed by Dagan et al (33), OR was 0.82 (1967/596618 vaccinated vs 2393/596618) and Fabiani et al (39), OR 0.1 (47/5333 vaccinated versus 89/1090 unvaccinated). Pooling of results was achieved due to significant heterogeneity.
Asymptomatic cases reported after the first dose.
Three studies addressed the effectiveness/efficacy of vaccination in reducing the risk of having asymptomatic COVID-19. The OR was 0.23 (0.06-1.63), meaning that it has no effect. The reported heterogeneity was 97%. After conducting leave one-out sensitivity analysis, the study of Fabiani et al (39) was omitted. Heterogeneity dropped to 80%, vaccination still have no protective effect, OR = 0.73 (95CI, 0.35-1.53)
- Cases reported after three weeks of vaccination with 1st dose
Total (Symptomatic and asymptomatic)
Five researchers studied the effectiveness/efficacy of COVID-19 within three weeks after the first dose. In total 4621 of 642572 vaccinated people developed COVID-19 within 3 weeks after the first dose, compared to 6318 of unvaccinated 620356 subjects. The OR of having COVID-19 was by 0.19 (0.03- 1.17), I2 was 100%. After subgrouping included studies into randomized and observational, the heterogeneity dropped to 0%. The difference between observational and interventional studies was insignificant (P=0.33). The odds ratio of infection with COVID-19 of RCT was 0.45 (0.31-0.65), while the OR in observational studies was 0.13 (0.01-1.49).
For Dagan et al (33); the OR of having symptomatic COVID-19 after vaccination was 0.73(95CI, 0.69-0.77), Of 596618 Vaccinated individuals 2250 developed COVID-19, compared to 3079 of 596618 unvaccinated subjects. Fabiani et al (39) reported that 51/53333 vaccinated subjects developed COVID-19 compared to 97/1090 unvaccinated subjects, OR =0.1(95%CI 0.07-0.14). results could not be pooled due to significant heterogeneity.
Only the study of Fabiani et al (39) reported this outcome. In total 15 of 5333 vaccinated subjects developed asymptomatic COVID-19 compared to 46 of 1090, OR = 0.06 (0.04-0.12)
- Cases reported within the first 4 weeks of vaccination with 1st dose
The total cases either symptomatic or asymptomatic were reported in two studies; Dagan et al (33) documented 4405 of 596618 vaccinated individuals compared to 5775 of 596618 subjects. Hall et al (41), diagnosed 427 of 20641 subjects, and 977 0f 2683 unvaccinated subjects the OR were 0.76 (95CI, 0.73-0.79) and (0.04 (95CI 0.03-0.4) respectively. The research team could not pool the OR ratio due to significant heterogeneity.
All cases reported after the first dose
Among 777171 vaccinated subjects with a single dose of COVID-19 vaccine, 8246 developed COVID-19, while 58261 of 1104745 unvaccinated subjects developed COVID, OR =0.14 (0.07-0.4) I2 =100%. After subgrouping based on the study design, the heterogeneity dropped to 0%, OR in RCT studies was 0.14 (95CI; 0.07-0.27), and in observational studies 0.15 (95CI; 0.06-0.4). The difference between subgroups was not significant, P=0.88
3.4. Efficacy/effectiveness of the second dose
- Cases reported within 7 days of the second dose
After the second dose, six studies reported COVID-19 cases after seven days of vaccination, two studies were RCT, and four observational studies, the odds ratio was significantly decreased by 0.06 (95%CI, 0.02-0.21) among vaccinated compared to the unvaccinated group. However, the heterogeneity was high 98%. The test for subgroup diﬀerences suggests that there is no statistically signiﬁcant subgroup eﬀect (P = 0.98), meaning that type of study did not statistically signiﬁcantly modify the effectiveness/efficacy of vaccination. Vaccination decreased the number of cases regardless of the study design, although the protective eﬀect was greater in RCT than in observational studies. There is no heterogeneity between results from the RCT, while the heterogeneity of observational studies was 99%. By conducting a meta-regression 100% of substantial heterogeneity can be explained, using type of vaccine and country as predictors, [ m RNA type, B= -3.33, P-value = 0.028; country UK, B= -3.7, P-value = 0.025; country US, B= -2.03, P-value = 0.037]
Symptomatic cases within 7 days of dose 2
Two observational studies reported symptomatic SARS-COV-2 cases, the odds ratio was decreased by 0.11 (95CI, 0.01-1.98) among the vaccinated group compared to the unvaccinated group with considerable heterogeneity of 77%.
- Cases within 14 days of second dose
Three studies highlighted the number of new cases reported within 14 days of dose 2, Baden et al (48), reported no cases among 14550 vaccinated subjects compared to 19 of 14598 unvaccinated individuals. Dagan et al (33), reported 332 confirmed cases of 187702 vaccinated subjects, while 949 cases were diagnosed among 186553 unvaccinated subjects. Hall et al (41), reported 10 cases of 1607 vaccinated subjects and 977 cases of 2683 unvaccinated individuals. The pooled OR was 0.05 (95% CI, 0.0-01.34), I2=99 meaning that the effect of the vaccine was insignificant. After conducting leave one out sensitivity analysis, the study of Dagan et al (33) was omitted, the OR became 0.01(95%CI, 0.01-0.02), I2= 0%, and still insignificant.
- Cases reported 7 days after 2nd dose
Among 4747653 vaccinated subjects, the number of confirmed covid-19 cases reported 7 days after the second was 6295, while 111252 confirmed cases were reported among 1854768 unvaccinated individuals. The OR was 0.03 (95%CI 0.02-0.05), meaning that vaccination is protective against COVID-19. The researchers explained 98.73% of this heterogeneity by the meta-regression of vaccine type and country predictors; where significant predictors are non m-RNA vaccine (B= 3.33, P-value =0.014), country Spain (B= 3.27, P-value =0.002), and country USA (B=1.53, P-value =0.039 ).
Symptomatic cases reported after 7 days of dose 2
With a heterogeneity of 0%, the effectiveness/efficacy of the COVID-19 vaccine in preventing symptomatic COVID-19 infection among 4721975 vaccinated and 1826910 unvaccinated subjects was 0.02 (95CI, 0.02-0.02). Three studies were included in the analysis one RCT and two observational studies.
Asymptomatic Cases reported after 7 days of the second dose
Two studies (recruited 4720118 vaccinated, 1825069 unvaccinated) assessed asymptomatic COVID-19 infection after 7 days of vaccination. Vaccination was protective against COVID-19.
- Cases reported within 14 days after the second dose
Total (Symptomatic and asymptomatic)
Regarding total cases either symptomatic or asymptomatic 14 days after second dose. There are nine studies included in this analysis. The overall effect was reported in 4080 cases from 4832289 vaccinated persons, while 119829 persons were diagnosed as COVID cases from 1940635 unvaccinated persons, OR = 0.08, 95% CI (0.02,0.34) with substantial heterogeneity I2 = 100%. Hence, the research team performed subgroup analysis in 2 steps, first, we categorized the cases as asymptomatic or symptomatic, then sub-grouped by study type either RCT or observational, but it didn’t affect the heterogeneity.
Therefore, a meta-regression was performed to understand the main predictors for this heterogeneity. Where we find that the type of vaccine and the country were responsible for 88.21% from this heterogeneity, [ non mRNA vaccine B=3.519, P-value= 0.004; Country Spain B=2.6256, P-value = 0.028).
- Cases reported 7-14 days after the second dose
The result couldn’t be pooled due to substantial heterogeneity I2 =99%. Dagan et al (33) reported 51 cases from 108529 vaccinated persons and 278 cases from 107209 unvaccinated one, OR = 0.18 (0.13-0.24), while Hall et al (41) reported 4 cases from 1607 vaccinated to 977 from 2683 unvaccinated, OR = 0 (0 - 0.01).
- All cases reported after the 2nd dose
In total 377 confirmed cases of 228715 subjects were reported after the second dose, while 2435 cases of 224569 unvaccinated subjects were reported, OR 0.179 (95CI, 0.15-0.19), I2 =98%. The test for subgroup diﬀerences suggests that there is no statistically signiﬁcant subgroup eﬀect (P = 0.98), meaning that type of study did not signiﬁcantly modify the effectiveness/efficacy of vaccination. Vaccination decreased the number of cases regardless of the study design, although the protective eﬀect was greater in RCT than in observational studies. There is no heterogeneity between results from the RCT, while the heterogeneity of observational studies was I2=99%.