The search identified 2,596 articles ranging from January 1st, 2020, to August 5th, 2022 (Fig. 1), of which 15 peer-reviewed articles were included in the final analysis (Table 2).
Table 2
Characteristics of the included randomized controlled trials (RCTs).
Author (Year) | Participants | Country | Arm | Interventions | Comparison | Time points | outcome |
Dai(2021) | N = 93354 (patients registered at UCLA health) | US | 5 | Basic reminder/Ownership reminder | No text message | 4 weeks | Vaccination uptake (+) |
Bartoš(2022) | N = 2101(general population adults) | Czekh | 2 | Consensus condition | No information | 2 weeks/4 months | Intention to vaccinate (+) and Vaccination uptake(+) |
Sasaki(2022) | N = 1595(25–34 young and 65–74 elderly) | Japan | 4 | social-comparison/gain-framed social influence/loss-framed social influence | Basic Information | 3 days | Intention to vaccinate(0) |
Tentori(2022 | N = 2000(50–59 adults) | Italy | 2 | opt-out group | OPT-IN group | 3 weeks | Vaccination uptake (+) |
Campos(2021) | N = 8286(18–49 adults) | Sweden | 5 | Incentives/Social impact/Information/Arguments | Basic reminder | After intervention /4 weeks | Intention to vaccinate (0) and Vaccination uptake (0) |
Santos(2021) † | N = 9723(Employees registered at Geisinger Health System) | US | 3 | social norms/reframing risks | Delayed control | 3 days | Vaccination registration (+) |
Ashworth(2021) | N = 3048(general population) | US | 9 | Private benefit/ Social benefit/ Economic benefit/Vaccine safety | No information | After intervention | Intention to vaccinate (+) |
Mehta(2022 | N = 16045(patients visit at a Penn Medicine primary care provider) | US | 9 | 2× 4 design; Call Back/ In-Bound Call + Standard Msg/ Clinician Endorsement/ Scarcity/ Opt-Out Framing | outbound call only | 1 month | Vaccination uptake (-) |
Patel(2022) | N = 2000(Ascension Associate Employee.) | US | 2 | Text message | Usual care | 2 weeks | Vaccination uptake (+) |
Jensen(2022) | N = 890(general population adults) | US | 5 | Video-based messages: safety/ social norm/ response efficacy/ self-efficacy | Basic Information | After intervention | Intention to vaccinate(+) |
Liu(2022) | N = 1926(general population) | China | 7 | Conventional Opt-out/ Improved Opt-out | OPT-IN group | After intervention | Intention to vaccinate(0) |
Motta(2021) | N = 7064(general population adults) | US | 12 | 3 × 2 × 2 design; personal health risks/collective public health consequences of not vaccinating/ economic costs + lay source/ expert sources + Highlighting the rigors of clinical trials/ Not highlighting the rigor of clinical trials | No information | After intervention | Intention to vaccinate(0) |
Freeman(2021) | N = 16455(general population adults) | UK | 10 | collective benefit (3 conditions)/personal benefit/seriousness/safety, direct/safety, indirect/collective and personal benefit/full combination | Safety and effectiveness statement | After intervention | Intention to vaccinate (0) |
Burger(2022) | N = 1324(general population) | Germany | 4 | Debunking/ Benefits/ Facilitation | No information | After intervention | Intention to vaccinate (-) |
Gong(2021) | N = 1316(general population adults) | China | 4 | Gain-Framed/ Loss-Framed/ Altruistic Messages | No information | After intervention | Intention to vaccinate (+) |
* The sign in parentheses indicates whether the results supported the effectiveness of the respective intervention (+), provided evidence of lack of effectiveness (−), or reported mixed results (0). |
† The primary outcome of the study was vaccination registration, but given that the study population was employees of a large health system and that there were employee-only vaccination clinics with approximate vaccination registration and actual vaccination numbers, we treated the outcome as vaccine uptake. |
A total of 167,127 people were included, predominately from the general adult population. Most of the included studies continued for 1 month after the intervention,[25, 33–45] except one study, which continued for 4 months.[46] All studies were conducted in high-income or upper middle-income countries (USA (n = 7),[25, 33, 35, 37, 39–41] China (n = 2),[38, 45] Czech Republic (n = 1),[46] Italy (n = 1),[43] Sweden (n = 1)[44], Japan (n = 1),[42] UK (n = 1),[34] and Germany (n = 1).[36] Seven studies incorporated control groups that received no intervention or usual care.[25, 33, 36, 37, 39, 45, 46]
The risk of bias for all included studies is summarized in Supplementary Fig. 1. Full details of the risk of bias assessment for each study are provided in Supplementary Table S2. Of the 15 randomized controlled trials included in the evaluation, 10 were rated as of high quality and none as of low quality. A lack of allocation concealment description and clear blinding were common factors that limited the quality of the studies.
Characteristics Of Nudge Interventions
After considering the operability of nudge strategies in the intervention studies, we identified a total of six types of nudging interventions employed across the selected studies (Table 3). Almost all of the included studies (12 of 15) were multinudge element interventions[25, 33–36, 38–42, 44, 45]; nine studies applied the nudge intervention of a salient reminder,[25, 34–37, 39–41, 44] seven studies applied an information framework,[33, 34, 36, 39, 42, 44, 45] three studies applied messaging,[33, 35, 46] three studies applied social norms,[40–42] two studies applied defaults,[38, 43] and one study applied financial incentives.[44]
Table 3
Summary of nudging interventions.
Type of nudging | Rationale | Nudging-based approaches employed in the included studies |
Salient reminder | A reminder can ensure that people can act immediately on the salient information that is provided in that particular moment.[31, 32] | 1. Use familiar and trusted sources (e.g., health centers, large survey companies, etc.) to provide vaccination reminders/recalls;[25, 35, 36] 2. Write down arguments that could best convince another person to get vaccinated;[44] 3. Highlighting the safety and efficacy of COVID-19 vaccine;[34, 44] 4. Juxtaposing the risk of the vaccine with the risk of COVID-19;[40] 5. Highlighting the safety and rigor of COVID-19 vaccine clinical trials;[39] 6. SMS highlights the limited availability of vaccines;[35] 7. Description of vaccine retention for participants, highlighting vaccine ownership;[25, 35, 37] 8. Video-based reminders and recalls to highlight vaccine safety, efficacy or vaccination retention information, convenience, etc;[25, 41] 9. Receiving information debunking vaccination myths increases the intention to get vaccinated.[36] |
Financial Incentives | Incentives, both tangible and intangible, can increase favoured behaviour.[31] | 1. Provide monetary incentives to increase vaccination intentions and actual uptake.[44] |
Messenger | Changing the source from which the information is delivered (eg, a trusted peer or a doctor) can alter the level of emphasis on the information.[31] | 1. Provide the true view held by physicians to correct misunderstandings about vaccine uptake);[46] 2. Clinicians provide vaccination advice;[35] 3. Expert (vs. lay) sources of vaccine support information.[33] |
Information Framework | The same outcome is perceived differently when the benefits gained (eg, lives saved) and losses avoided (eg, deaths avoided) inform the message.[31] | 1. Gain-framed messaging (describing benefits of receiving vaccines)[36, 45] versus loss-framed messaging (describing the cost of not receiving the vaccine);[45] 2. Make a list who would benefit from the participant getting vaccinated;[44] 3. Describe the private[34, 39], social[34, 39, 42], and economic benefit[39] of vaccination; 4. Describe the personal[33], Economic[33], and Collective[33, 42] Health Risk Frames); 5. Altruistic messages to motivate people to accept COVID-19 vaccination.[45] |
Social Norms | People tend to behave in ways that are consistent with their perceptions of the social environment and widely accepted behaviors in their peer groups.[31, 32] | 1. Based on emails, videos, etc. indicating that a large proportion of people around or colleagues have chosen to be vaccinated to improve vaccine uptake).[40–42] |
Defaults | Changing whether something is the default option versus requiring explicit opt-in can change behaviour.[31, 32] | 1. Vaccination appointment has been scheduled (with opting out possible);[43] 2. Opt-out policy and its improvements(different information was provided for the opt-out policy).[38] |
Effects Of Nudging Interventions
Various outcome measures were used to assess the effect of nudging interventions. The most frequent were vaccination uptake (n = 7)[25, 35, 37, 40, 43, 44, 46] and intention to vaccinate (n = 10).[33, 34, 36, 38, 39, 41, 42, 44–46] As can be seen from Table 2, half of the articles reported positive effects (n = 7),[25, 37, 39, 40, 43, 45, 46] while other articles reported partially positive effects with mixed or inconclusive overall results (n = 6).[33, 34, 38, 41, 42, 44] Significantly, two articles identified that nudging interventions has no effect.[35, 36] These negative findings involved nudges that were divided into different texts plus inbound/outbound call groups[35] and a message intervention debunking vaccination myths or highlighting benefits.[36] Detailed characteristics of all nudge interventions are described in Supplementary Table S2.
Impact Of Nudge Interventions On Vaccine Uptake
Seven studies reported vaccine uptake results, with a total of 131,719 people included.[25, 35, 37, 40, 43, 44, 46] Five studies found statistically significant increases in vaccination uptake compared with control groups,[25, 37, 40, 43, 46] one study[44] reported mixed results, and one study[35] identified no effects from the nudging intervention. The pooled results showed that vaccine uptake was significantly higher in the nudging intervention group compared to the control (RR: 1.19, 95% CI [1.07, 1.33], p < 0.01, Fig. 2). The heterogeneity was 93%, and a random-effects model was adopted. The funnel plot appeared to be asymmetric (Supplementary Figure S2), but Begg's test (p = 0.195) and Egger's test (p = 0.590) revealed no major publication bias in the included articles. Sensitivity analyses using the leave-one-out method did not significantly alter the overall outcome of vaccine uptake (Supplementary Figure S3).
The results of the subgroup analysis are shown in Table 4 and Supplementary Figures S4–S9. The nudging intervention appeared to have a greater effect on vaccine uptake in the group with follow-up < 4 weeks (RR: 1.59, 95% CI [1.16, 2.19]) than in the group with follow-up ≥ 4 weeks (RR: 1.12, 95% CI [1.05, 1.20]), although the difference was not significant. When we stratified, by comparison, the difference in vaccine uptake between the trial and control groups was significant only when the control was no intervention/usual care. Nudging appeared to have a more pronounced effect on vaccine uptake when the COVID-19 vaccine was available early and when general vaccination rates were low (i.e., literature published in 2021 or when the control group had vaccination rates less than 70%). Regardless of whether the study was conducted in the United States or Europe, the boost to vaccination uptake was significant (p < 0.05). There was no strong evidence that the information framework was effective at increasing vaccine uptake rates (p > 0.05). All types of nudge interventions except the information framework had a weak positive effect on vaccine uptake (p < 0.05). Among the six nudge types, a stronger effect on vaccine update was observed for social norms (RR: 2.04, 95% Cl [1.61, 2.57]), defaults (RR: 1.32, 95% Cl [1.03, 1.69]), and salient reminders (RR: 1.19, 95% Cl [1.04, 1.36]).
Table 4
Subgroup analysis of nudging intervention on vaccine uptake.
Intervention type | Outcome: received immunizations |
| Illustrative comparative risks* (95% CI) | Relative effect (95% CI) | No of participants (studies) |
Assumed risk | Corresponding risk |
| Without intervention | With intervention | | |
Nudge(summary) | 203 per 1000 | 242 per 1000 (217 to 270) | RR 1.19(1.07 to 1.33) | 131,719(7) |
Time points | | | | |
Nudge(≥ 4 weeks) | 215 per 1000 | 241 per 1000 (226 to 258) | RR 1.12(1.05 to 1.20) | 118,196(4) |
Nudge(< 4 weeks) | 74 per 1000 | 118 per 1000 (86 to 162) | RR 1.59(1.16 to 2.19) | 13,523(3) |
comparsion | | | | |
No intervention/usual care | 147 per 1000 | 179 per 1000 (168 to 191) | RR 1.22(1.14 to 1.30) | 96,974(3) |
Other intervention | 415 per 1000 | 486 per 1000 (398 to 593) | RR 1.17(0.96 to 1.43) | 34,745(4) |
Study location | | | | |
US | 131 per 1000 | 168 per 1000 (140 to 199) | RR 1.28(1.07 to 1.52) | 120,965(4) |
Europe | 668 per 1000 | 695 per 1000 (681 to 708) | RR 1.04(1.02 to 1.06) | 10754(3) |
Year | | | | |
2021 | 201 per 1000 | 255 per 1000 (219 to 297) | RR 1.27(1.09 to 1.48) | 90086(3) |
2022 | 239 per 1000 | 258 per 1000 (246 to 272) | RR 1.08(1.03 to 1.14) | 21622(4) |
Proportion of Vaccination |
≥ 70% | 715 per 1000 | 744 per 1000 (729 to 758) | RR 1.04(1.02 to 1.06) | 8797(2) |
༜70% | 131 per 1000 | 169 per 1000 (144 to 197) | RR 1.29(1.10 to 1.50) | 122922(5) |
Nudge types | | | | |
Salient reminder | 173 per 1000 | 206 per 1000 (180 to 235) | RR 1.19(1.04 to 1.36) | 118779(5) |
Messenger | 485 per 1000 | 519 per 1000 (485 to 548) | RR 1.07(1.00 to 1.13) | 5930(2) |
Defaults | 99 per 1000 | 131 per 1000 (102 to 167) | RR 1.32(1.03 to 1.69) | 1957(1) |
Financial incentives | 716 per 1000 | 759 per 1000 (723 to 788) | RR 1.06(1.01 to 1.10) | 3909(1) |
Information Framework | 716 per 1000 | 737 per 1000 (709 to 766) | RR 1.03(0.99 to 1.07) | 3892(1) |
Social Norms | 32 per 1000 | 65 per 1000 (52 to 82) | RR 2.04(1.61 to 2.57) | 6376(1) |
Meta-regression analysis (Fig. 3) indicated a nonsignificant association between vaccination and passage of time within 4 months (p = 0.095), but only one study exceeded 1 month; therefore, we excluded this study to explore a 1-month trend. The results after exclusion showed that within 1 month, the increase in vaccination was inversely associated with the passage of time (p < 0.0001).
Impact Of Nudge Interventions On Vaccine Intention
Among the 10 studies that considered intention to vaccinate as the primary outcome of interest,[33, 34, 36, 38, 39, 41, 42, 44–46] three studies found statistically significant increases in vaccination intention compared with control groups.[39, 45, 46] More than half of the studies (6 of 10) reported mixed results regarding vaccination intention among study participants,[33, 34, 38, 41, 42, 44] and one study identified no effects of the nudging intervention on vaccination intention (Table 2).[36] Table 5 summarizes the characteristics of the vaccine intentions for outcome studies. The supplementary table S2 provides a more detailed view of each study.
Table 5
Characteristics of nudge interventions on vaccination intentions.
Reference | Types of nudging | Implementation details | Findings |
Bartoš et al[46] | Messenger | Consensus condition, participants were provided with a summary of the survey among medical doctors, including three charts that displayed the distribution of doctors’ responses regarding their trust in the vaccines, willingness to get vaccinated themselves and intentions to recommend the vaccine to patients | Public misconceptions about what doctors think about COVID-19 vaccines are common, and correcting them will boost vaccination intentions. |
Sasaki et al[42] | Information Framework/ Social Norms | The three groups were randomly assigned peer information: comparison, influence-gain, and influence-loss. The effect of each piece of information on vaccination intentions was compared | The influence gain nudge was effective in increasing the number of older adults who newly decided to vaccinate, but had no boost for younger adults who were less willing to vaccinate at baseline. |
Campos et al[44] | Financial Incentives/ Salient reminder/Information Framework | The effect of a small cash reward, around US $24, was compared with the effect of several behavioral nudges. The nudge measures are social impact condition, arguments condition and information condition | Financial incentives significantly increased vaccination intentions and coverage rates, while other incentives increased vaccination intentions but had little effect on vaccination rates |
Ashworth et al[39] | Information Framework/ Salient reminder | "Private benefit", "social benefit", "economic benefit" and "vaccine safety" information individually and in combination to examine the impact on vaccine intent, Participants were randomly assigned to one of nine informational treatments | Several forms of public information can increase vaccine intent, but messages that emphasize individual health benefits have the greatest impact. |
Jensen et al[41] | Salient reminder/ Social Norms | Four different treatment videos were made based on planned behavior theory, (i) attitudes about vaccine safety, (ii) normative beliefs about subjective social norms of vaccination, (iii) attitudes about vaccine efficacy (response effect), and (iv) perceived behavioral control over vaccination (self-efficacy). | Short video encouraging messages that address specific issues with COVID-19 vaccines increase vaccination intentions |
Liu et al[38] | Defaults | There are seven vaccination policies, including an opt-in policy with no default, an opt-out policy with vaccination as the default, and five improved versions of the latter, including Social Norm, Transparency, Education, Feedback and Opportunity | Opt-out policies and their improvements (with the exception of opt-out transparency) increase the willingness to vaccinate. |
Motta et al[33] | Information Framework/Messenger | Respondents were randomly assigned to read a short pro-vaccine opinion article, depending on (i) the framework (individual health risks, collective public health consequences and economic costs of not vaccinating), (ii) the source of information (i.e. lay versus expert sources), And (iii)the presence or absence of pre-bunking information highlighting the rigors of clinical trials prior to reading the opinion piece. | Regardless of the source, information that emphasizes the individual health risks and collective health consequences of not vaccinating can significantly increase Americans' intention to get vaccinated. The economic cost framework had no significant effect on vaccine intent |
Freeman et al[34] | Information Framework/ Salient reminder | Participants were randomly assigned to ten information conditions stratified by their level of vaccine acceptance (willingness, skepticism, or strong hesitation). Condition 1 was the control group, whose text included safety and efficacy claims taken from the NHS website. Three conditions address only collective interests (conditions 2, 3 and 4), one individual interests only (condition 5), one combines collective and individual interests (condition 9), one concerns the severity of the pandemic (condition 6), two concerns security issues (conditions 7 and 8), and the final condition addresses the complete combination of interests, Seriousness and safety issues (Condition 10) | In populations that are very hesitant about COVID-19 vaccines, the provision of information of individual benefit reduced indecision to a greater extent than information of collective benefit. |
Burger et al[36] | Salient reminder/Information Framework | Assess whether debunking vaccination myths, highlighting the benefits of vaccination, or sending vaccination reminders will reduce hesitation | No increase in the willingness to vaccinate, regardless of the information provided |
Gong et al[45] | Information Framework | An online survey experiment consisting of a control group (exposed to non-framing information) and three experimental groups (exposed to gain framing, loss framing, or altruistic information) was designed to assess vaccination intentions | People exposed to the gain frame, loss frame, or altruistic messages were more likely to be vaccinated against COVID-19 than those exposed to non-frame messages. In addition, the loss of frame information had a more significant effect on vaccination intentions than the other two types of information. |
There was mixed evidence regarding the effectiveness of nudging for increasing vaccination intentions for different information frames. A randomized controlled trial in Japan[42] evaluated the effect of different peer messages on vaccination intentions; only gaining frame messages were effective at increasing the number of older adults who newly decided to get vaccinated. In contrast, a study in China showed[45] that people exposed to lose frames showed higher intentions of obtaining a COVID-19 vaccine compared to gain frame messages. Ashworth et al.[39] defined three messages emphasizing different benefits of the vaccine and one emphasizing vaccine safety, and evaluated the impact on vaccine intentions of transmitting the four messages individually and in combination. The results suggested that although safety concerns were the main reason for hesitation regarding the COVID-19 vaccine, the group exposed to the vaccine safety message did not exhibit significantly increased vaccination intentions.
The message emphasizing personal health benefits had the greatest impact. An RCT[33] conducted in the United States prior to the introduction of the COVID-19 vaccine found that intention to receive COVID-19 vaccination increased significantly when information was tailored to individual health risks, the consequences of not receiving the vaccine, and when these framing messages came from medical experts. In the United Kingdom, a randomized controlled trial[34] similarly showed that individuals who were strongly hesitant about the COVID-19 vaccine were influenced more effectively by personal benefit messages than were those who received collective benefit messages.
In addition to information frames, other types of nudges also led to different results. A study in the Czech Republic[46] focused of a facilitation intervention on changing the messengers assigned to convey vaccine-related health messages. A study in the US[41] adopted a short video of c. 30 seconds for the specific problem of COVID-19 vaccine, Campos et al.[44] used incentives as a mechanism for behavior change. In each of these studies, nudge measures all increased vaccination intention to varying degrees, but it is worth noting that in Campos et al., salient reminders and information framework nudge measures increased intention to vaccinate but had a small and statistically insignificant effect on vaccination rates. Mixed results were also seen in a randomized trial in China,[38] in which investigators considered the use of defaults and refined defaults in the unvaccinated population to increase intention to receive the COVID-19 vaccination. Opt-out policies increased the intention to receive vaccination, but none of the refined versions resulted in significant changes in participants' intention to vaccinate compared to the traditional opt-out condition. A German registry trial showed[36] that willingness to vaccinate did not increase regardless of whether the information provided debunked vaccination myths or emphasized the benefits of vaccination or sent vaccination reminders.