Pay-it-forward to improve in uenza vaccine uptake and public engagement among children and older adults in China: A three-arm quasi-experimental study


 China has low seasonal influenza vaccination rates among children and older adults. This quasi-experimental pragmatic trial examined the effectiveness of a pay-it-forward intervention on influenza vaccination compared to standard of care (user-paid vaccination) and free vaccination strategies among children and older people in China (ChiCTR2000040048). In pay-it-forward, people receive a free influenza vaccine from a local group and are offered an opportunity to donate financially to support vaccination among future individuals. The primary outcome was pre-specified as influenza vaccine uptake. Secondary outcomes included vaccine confidence and associated costs. Among 450 participants enrolled, 55/150 (36.7%) in the standard of care arm, 111/150 (74.0%) in the pay-it-forward arm, and 114/150 (76.0%) in the free vaccination arm received an influenza vaccine. The pay-it-forward arm had significantly higher vaccine confidence when compared to the standard of care arm. In the pay-it-forward arm, 107/111 (96.4%) of participants donated money for subsequent vaccinations and 19 of 60 invited (31.7%) created postcard messages.


Introduction
In mainland China, an average of 10 people die from in uenza-related illnesses each hour. 1 In uenza vaccination is the most effective way to prevent morbidity and mortality attributable to in uenza. 2 In uenza vaccine is increasingly important during COVID-19 because it might help reduce risks of acquiring SARS-CoV-2 that causes COVID-19. 3,4 The Chinese Center for Diseases Control and Prevention (China CDC) guidelines recommend in uenza vaccination for high-risk populations, including children aged below 5 years old and older adults (people older than 60 years old). However, in uenza immunization policies widely vary, 5 and most cities in China do not provide free in uenza vaccines to high-risk individuals. A meta-analysis reported less than one-fth of children and older adults in China received an in uenza vaccine in the past year. 6 There are several reasons for low in uenza vaccination uptake in China. 6,7 First, most people in China are unaware of in uenza vaccination and many people are hesitant about vaccine safety and effectiveness. 8 Second, there is minimal public engagement in vaccinations. 9 Despite a strong rationale for public engagement, few programs engage the public regarding in uenza vaccinations. Third, there are limited public resources to support in uenza vaccination among high-risk populations. The in uenza vaccine is largely not covered by mandatory health insurance schemes and, as a result, most people have to pay US$8.5-23.5 out-of-pocket to be vaccinated. 10 Innovative strategies are needed to improve in uenza vaccine uptake.
Pay-it-forward is a community-engaged social innovation, which has an individual receive a free in uenza vaccine and a hand-written postcard message co-created by previous participants informing them that someone else has paid for them to receive a free vaccine. 11 After they receive vaccination, they are asked if they would like to support the vaccination of a subsequent person (supplementary gures: Fig. 1). Our previous pay-it-forward studies focused on increasing testing for sexually transmitted infections among sexual minorities. The pay-it-forward arm had a chlamydia and gonorrhea dual test uptake of 56% compared to 18% in the standard of care arm, where participants had to pay out-of-pocket. 12 13 In addition, over 90% of participants donated to the rolling nance pool, 12,13 and qualitative data showed that trust in health services improved among participants in the pay-it-forward arm. 14 In this quasi-experimental pragmatic trial, we assessed the effectiveness of a pay-it-forward intervention to increase in uenza vaccination uptake at three study sites among children (aged between 6 months and 8 years) and older adults (aged 60 or above) in comparison to free vaccination and the current standard of care (user-paid vaccination) in China.

Results
In total, 254 children's caregivers and 262 older adults were approached at the three study sites (Fig 1).
Forty-six people declined to participate, and 30 had recently received the in uenza vaccine. In total, 450 participants were enrolled and completed the survey, including 150 in the pay-it-forward arm, 150 in the free vaccine arm, and 150 in the standard of care arm. All 450 responses were screened for completeness and included in the nal statistical analyses.
Characteristics of caregivers and older adult participants were similar across the three arms (Table 1). Overall, 111 (74.0%) of 150 pay-it-forward participants, 114 (76.0%) of 150 participants offered free vaccination, and 55 (36.7%) of 150 participants in the standard of care arm received in uenza vaccination (χ² test p<0.001; g 2, supplementary tables: table 1). Among children, the pay-it-forward arm had an uptake rate of 88.0% (66/75) compared to 53.3% (40/75) in the standard of care arm. Among older adults, the pay-it-forward arm had an uptake rate of 60.0% (45/75) compared to 20.0% (15/75) in the standard of care arm. Differences in uptake between the pay-it-forward and standard of care arms remained statistically signi cant for both groups after adjusting for study site and educational level (Supplementary tables: table 1). Uptake in the pay-it-forward and free vaccination arms were not signi cantly different for both age groups.  (Table 3). Similar trends were observed in con dence in vaccine effectiveness and importance.
Study participants contributed to the development of in uenza vaccination materials in several important ways. 19/60 (31.7%) people created handwritten postcards for subsequent participants. Six postcard designs were subsequently used to explain the pay-it-forward system to potential participants. Most handwritten messages expressed general good wishes. In addition, we co-created a video and one local older adult contributed to this video design, implementation, and evaluation (supplementary video link).
Among 111 participants in the pay-it-forward arm who received the in uenza vaccine, 107 (96.4%) donated money, with a total contribution of US$597.62. Donations covered 36.0% of vaccination costs in the pay-it-forward arm. The median donation was US$4.6. Only 30% of donors in the rural site contributed US$7.6 or more compared to 61.9% in the suburban and 40.0% in urban sites (supplementary Fig 7 and supplementary table 2).
The total nancial cost of implementing an in uenza vaccination intervention for children and older adults was US$2,725 for the standard of care arm, US$4,477 for the pay-it-forward arm, and US$4,665 for the free vaccination arm. The incremental cost for each treatment arm, the incremental number of people vaccinated, and the incremental cost-effectiveness ratios (ICERs) based on nancial and economic costs are shown in Table 4 and the supplementary costs le. Based on the ICER obtained from comparing payit-forward to the standard of care, the nancial cost required per additional person vaccinated was US$31.29. The nancial cost required per additional person vaccinated was US$62.67 when comparing free vaccination arm to pay-it-forward.
When economic costs are considered, the economic cost of implementing an in uenza vaccination intervention for children and older adults was US$3,557 for standard of care, US$5,062 for pay-it-forward, US$4,665 for the free vaccination arm.

Discussion
Low in uenza vaccination rates among high-risk populations are a major health problem in low and middle-income countries. Our quasi-experimental study assessed the effectiveness of a pay-it-forward in uenza vaccination intervention to improve uptake and engagement. Our study contributes to the literature by determining the effectiveness of a social innovation using a quasi-experimental study, developing new methods for in uenza vaccination public engagement, and identifying a new method to enhance in uenza vaccine uptake. Our data suggest that the pay-it-forward strategy may increase in uenza vaccine uptake among high-risk individuals compared to the current self-pay strategy for vaccination. This strategy substantially increased vaccine uptake compared to the standard of care, elicited nancial contributions, improved vaccine con dence, and co-created participatory messages.
We found that children and older adults who took part in pay-it-forward had higher in uenza vaccine uptake than they did if they needed to self-pay for vaccination. This nding is consistent with previous intervention studies using pay-it-forward to improve health services uptake. 12,13 The effect of pay-itforward might be related to the reduced costs associated with vaccination, enhanced public engagement, or both. The pay-it-forward arm had a similar vaccination rate to that of the free vaccine arm. This suggests that asking participants to make some nancial contribution to support the vaccination of subsequent participants did not deter them from participating.
We also observed that, among those enrolled in the pay-it-forward arm, nearly all voluntarily donated to support another person in receiving an in uenza vaccine, including those with a low annual income from a study site in a poor rural area. Compared to standard of care, the pay-it-forward arm had a higher nancial cost, but increased the number of people vaccinated. The incremental nancial cost per person vaccinated was lower than the median cost (US $50.78) per additional enrollee vaccinated from a systematic review published in 2018. 20 Donations collected using a pay-it-forward system can support more individuals in receiving in uenza vaccine services and can potentially reduce the nancial burden for local governments. Pay-it-forward could also potentially transition from out-of-pocket payments to government-funded in uenza vaccine programs.
Pay-it-forward has additional social bene ts; it generated many messages aimed at driving in uenza vaccine uptake. This is a rare example of public engagement in an in uenza vaccination program. 9 Public engagement is central to the success of public health programs; given that some engagement methods (in-person events) could facilitate in uenza transmission, 21 it is especially important to identify public engagement methods that are safe and effective. Engaging the community in vaccination services through cultivating kindness and reciprocity may also strengthen community solidarity, and increase con dence in vaccine services. 14,22 The study has several limitations. First, although our study was implemented after COVID-19 lockdowns were lifted, all sites were heavily focused on COVID-19 prevention, COVID-19 vaccination, and related COVID-19 activities. This caused some delays in recruitment despite the availability of in uenza vaccines. At the same time, this demonstrates the feasibility of pay-it-forward, even during an event as disruptive as COVID-19. Second, we examined people from only three sites. However, our sites all had a high in uenza prevalence, included different settings (rural, suburban, urban), and re ected common pathways for vaccination in China. Third, our study did not capture granular data on implementation. Future effectiveness research to examine different pay-it-forward implementation strategies is needed to differentiate effective components and determine optimal pay-it-forward practices. Finally, the study was mainly implemented by our project staff with assistance from local health workers. It remains unclear how feasible it is to decentralize implementation and integrate pay-it-forward into existing vaccine services.
Our study has implications for research, implementation, and policy. From a research perspective, this study expands the limited literature on public engagement in in uenza vaccine programs. It demonstrates how social innovation can engage key communities in the implementation process and build con dence in in uenza vaccination. This might help address vaccine hesitancy and anti-vaccine movements. The success of the pay-it-forward initiative and different donation levels across three study sites shows the potential to mobilize nancial resources between areas with different economic status (e.g., mobilize nancial resources from economically better-off areas to subsidize essential preventive services for people in more impoverished areas). Randomized controlled trials and qualitative research are needed to better understand the implementation of this system and integrate this intervention within health systems.
Pay-it-forward may be particularly relevant in the large number of countries that charge fees for in uenza vaccines, which contributes to disparities in in uenza vaccine uptake. Developing pay-it-forward programs could help nancially support expanded in uenza vaccination programs in these settings while also generating community-engaged messages. Further pay-it-forward in uenza vaccination research could help explore how to expand vaccine programs in a more equitable and pro-social manner.

Study design and participants
Guangdong is a subtropical province in southern China with a population of over 120 million. In southern China, in uenza is prevalent throughout the year. 15 In this study, we selected three research sites where in uenza vaccination was only available on a for-fee basis. These three study sites were: a rural site (Yangshan County, Qingyuan City), a suburban site (Zengcheng District, Guangzhou City), and an urban site (Tianhe District, Guangzhou, City). Study sites included community health centers (primary care facilities providing day-to-day healthcare in China) and vaccine centers. Clinics were selected because they had su cient in uenza vaccines in stock and health professionals (nurses, doctors) familiar with in uenza vaccination.
This study consisted of three stages 1) co-creation of the intervention with stakeholders and engagement strategies during a three-day hackathon; 2) a feasibility pilot to inform the recruitment process and sample size calculations; and 3) a pay-it-forward quasi-experimental pragmatic trial to evaluate the effectiveness of the intervention (supplementary study protocol).

Co-creation of intervention
Our team of three individuals joined a participatory hackathon from November 4-6 2019, to co-create the pay-it-forward intervention. Co-creation is an iterative, bidirectional partnership between researchers and the public to develop new ideas. 16 Participants included potential end users, public health practitioners, health innovators, communication experts and vaccine experts. We mapped out the following elements of the study: 1) key stakeholders of the study; 2) potential user journeys; 3) engagement strategies; 4) behavioral mechanisms; and 5) donation strategies. Community engagement strategies used in this study included the following: inviting community members to design postcards (Supplementary gures: Fig 2); working in partnership with a local older adult to develop a video to explain pay-it-forward; inviting study participants to create hand-written postcard messages during recruitment for future participants (Supplementary gures: Fig 3); and engaging local community staff in implementing the quasiexperimental study, including having one-to-two community staff members at each study site to help adjust recruitment and communication efforts to the local dialect.

Pilot
Before the quasi-experimental study, we carried out a feasibility pilot at the rural study site from January to April 2020, which occurred during the social distancing period due to COVID-19 in China. The primary outcome of the study was in uenza vaccination uptake. The purpose of the pilot was to nalize the payit-forward intervention process, assess feasibility, and estimate effect size to inform power calculations. This pilot demonstrated that, in the pay-it-forward arm, 90.9% (40/44) of participants received an in uenza vaccine and 93% (37/40) of participants donated funds. Thirteen of 57 participants (22.3%) in the standard of care arm received a vaccine.

Sample size calculation
We strati ed sample size calculations by age groups, given the differences between children and older adults. Based on our pilot data, we anticipated that the proportion of vaccine uptake in the standard-ofcare arm was 30%, and the proportion of vaccine uptake in the pay-it-forward arm was 80%, a signi cance level of 0.025; therefore, a sample size 100 (50 in the control arm and 50 in the intervention) would give us 90% power to test the proportion difference with a margin of 10%. We increased the sample size by 50% to allow for secondary analyses, resulting in a sample size of 75 for each age group in each arm. In addition, we included a free vaccine arm with the same sample size as the other two arms. This free vaccination arm was included because it has important implications for policy and global relevance to countries that already provide free in uenza vaccinations. In sum, we required the enrollment of 225 children and 225 older adults in order to have su cient power.

Quasi-experimental pragmatic trial
This trial evaluated the pay-it-forward intervention arm against both the standard of care and a free vaccination program implemented in rural, suburban, and urban study sites. Each study site implemented all study arms and recruited participants were chronologically allocated (non-random) into the speci ed study arms because of practical considerations. In uenza vaccine services are usually available in China from September to April. In uenza vaccine availability is idiosyncratic at speci c health facilities because of the periodical supply and procurement system in local settings. We allocated study arms to ensure a stable supply of vaccines (Supplementary gures : Fig 4: time-based recruitment). At each site, the standard of care arm was followed by the pay-it-forward arm. Despite discussions with health authorities and vaccine manufacturers, study sites encountered lapses in supply. The duration of time needed to recruit each study arm was related to the availability of vaccine and the number of people willing to participate.
The inclusion criteria for this study differed by age group and were determined according to China's national in uenza vaccine guidelines. 17 Childhood eligibility criteria included the following: aged between six months and eight years old; no acute moderate or severe illnesses; eligible to receive an in uenza vaccine based on clinical evaluation from a physician; has a legal guardian (e.g. a parent or grandparent) who lives in China and consents to participate in the study; and has not received an in uenza vaccine in the past year. Older adult eligibility criteria included the following: ≥ 60 years old; no acute moderate or severe illness; eligible to receive an in uenza vaccine based on clinical evaluation from a physician; capable of making informed decisions and consenting to participate in the study; and have not received an in uenza vaccine in the past year. If multiple people in a family were eligible to join the study, we only allowed one person to join. All eligible children and older adults presenting to these sites were invited to participate by local medical staff involved in the study during the recruitment periods.
Ethical approval for this study was obtained from the institutional review boards at the London School of Hygiene & Tropical Medicine (approval number 19100) and the Zhuhai Center for Disease Control (approval number 2020011). Online consent was obtained from guardians of children and older adults. The trial was registered in Chinese Clinical Trial Registry with the number of ChiCTR2000040048.

Procedures
Among all potential participants visiting the selected clinics, health care workers assessed eligibility for the study based on inclusion criteria and introduced eligible participants to project staff.
Standard of care Participants recruited in the standard of care arm were provided with a brief introduction to the in uenza vaccine by project staff using a pamphlet about in uenza and in uenza vaccination (Supplementary gures: Fig 5). They were then asked if they were willing to pay out of pocket at the standard market price (US$8.5-23.5 depending on the market price of vaccines provided at the clinic) to receive an in uenza vaccination. Those who agreed to pay were screened for vaccination eligibility, and those without any contraindications received the vaccine.
Pay-it-forward Participants recruited in the pay-it-forward arm were provided with the same introductory pamphlet about in uenza and in uenza vaccination. Project staff then explained the pay-it-forward program, including its purpose, the opportunity to receive one dose of in uenza vaccination for free, and the opportunity to donate money towards someone else's vaccine dose and write postcard messages (Supplementary gures: Fig 6). Participants were told that the normal price to receive an in uenza vaccine, including administration fees were RMB 56(US$ 8.5) for children and 153(US$ 23.5) for adults, and that previous participants had donated money to cover the costs and had also created handwritten postcards for them.
If the participants decided to receive vaccination, they were asked prior to receiving the vaccination whether they were willing to donate any amount of money into a pool of funds to support subsequent participants in receiving the same vaccine. They were assured that the donation was entirely voluntary, and any donation amount was acceptable and would not affect whether they received a vaccination or subsequent care. They were also invited to write anonymous postcard messages for future participants. A donation collection box was provided on-site for those who preferred to donate cash. A QR code using WeChat (a multifunctional social mobile app embedded with anonymous money transfer functions) was provided to those who chose to make online donations.
Donations were used to support the vaccination of subsequent participants and aggregated data on donation amounts were made publicly available on the website and WeChat newsletter of Social Entrepreneurship to Spur Health (a research hub in the UNICEF/UNDP/World Bank/WHO Special Programme for Research and Training in Tropical Diseases Social Innovation in Health Initiative). COVID-19 conditions at the rural site prevented participants from creating handwritten postcards during some periods of the trial.
Free vaccination Participants in the free vaccination arm were invited to participate using the same introductory pamphlet and were provided with free in uenza vaccination. They did not receive any community-created messages about the pay-it-forward program.
Participation in each arm was voluntary and anonymous. After introducing the intervention, all participants were asked to complete a short, self-administered online questionnaire to collect information about sociodemographic characteristics and attitudes towards in uenza vaccines (supplementary questionnaire). Vaccine con dence in importance, safety, and effectiveness were measured using survey items adapted to assess in uenza vaccine con dence in China. 18,19 Those who had di culty reading the questionnaire were assisted by the project and healthcare staff on-site. A small gift worth around RMB10 (US$1.5) was given to each participant after completing the questionnaire survey.

Data collection
Data collection was conducted from September 2020 to March 2021. The study collected the following information: administrative data recorded by research staff using a standard information tracking sheet including the number of invited and participating individuals, the number of participants who received the vaccine, the number of individuals who donated and the amount donated in the pay-it-forward arm, as well as survey data through a self-administered survey instrument. Administrative and survey data were linked using numerical IDs. We collected information about the number of participants in the pay-itforward arm who donated and corresponding donation amount, and those who created a postcard text for subsequent people. Costs associated with each arm were collected for an economic evaluation.

Data analysis
Descriptive analyses were conducted to summarize each arm's sociodemographic and behavioral characteristics, participation rate, and vaccination rate. We used a Chi-squared test to investigate differences in vaccination uptake between the standard of care, pay-it-forward, and free vaccination arms. We ran multivariable logistic regression to examine the association between vaccine uptake and interventions after adjusting for age, sex, study sites, education, occupation, income and marital status.
We also summarized the participants' donations in the pay-it-forward arm, and compared proportions of participants between rural, suburban and urban sites who contributed US$7.6 (close to a child vaccine cost) or more. All data were analyzed using SPSS Version 25 and STATA 17.

Cost Analysis
A decision tree was built to calculate and compare the costs and outcomes of the three in uenza vaccination arms examined in the quasi-experimental study. We evaluated the costs of all three arms using a micro-costing approach and reported this in 2020 USD. The costs of implementing each strategy were estimated using invoices, onsite staff's self-reporting the wages of healthcare workers, and estimated opportunity costs of community staff's time (supplementary costs le). The analysis was performed from the perspective of the healthcare provider, the Guangdong Department of Health. The time horizon considered was the duration of the seasonal in uenza vaccination program. We reported the total economic and nancial cost for each arm, the cost per person vaccinated, and the incremental costeffectiveness ratios. Incremental economic costs were greater for PIF compared to free vaccination because of additional costs related to volunteer time in the PIF design, as well as recruitment and donation process associated with the start-up, and recurrent costs. However, incremental nancial costs were greater for free vaccination compared to pay-it-forward because nancial costs were obtained from subtracting donation contributions from the total economic cost.

Data availability
Requests for data by researchers with proposed use of the data can be made to the corresponding author with speci c data needs, analysis plans and dissemination plans. Those requests will be reviewed by a study steering committee and the study sponsor for release upon publication. Original data and analysis codes used to generate primary ndings of the paper are attached as a zip le for the editor and reviewers.

Funding support
This work received nancial support from Bill & Melinda Gates Foundation (OPP1217240), and the National Institute for Health Research (NIHR200929).

22.
Konrath, S. & Brown, S. The effects of giving on givers. in Health and social relationships: The good, the bad, and the complicated. 39-64 (American Psychological Association, Washington, DC, US, 2013).   1 Out-of-pocket payment for the in uenza vaccine was the standard of care.

Tables
2 In addition to free in uenza vaccines, the pay-it-forward study arm received community engagement messages as well as the opportunity to make a donation to support the vaccination of other members of the community. 3 cOR = crude odds ratio 4 aOR = adjusted odds ratio 5 P-value obtained using Likelihood Ratio Tests 6 CI = con dence interval 7 Estimates were adjusted for age, gender, education level, income level.  Quasi-experimental study owchart In uenza vaccine uptake rates by intervention arms and age group in Guangdong Province, China, 2020-