Location, Location, Location: A Discrete Choice Experiment to Inform Vaccination Programme Delivery in the UK

Background: Large-scale vaccination is fundamental to combatting COVID-19. In March 2021, the UK’s vaccination programme had delivered vaccines to large proportions of older and more vulnerable population groups;however, there was concern that uptake would be lower among young people. This research was designed to elicit the preferences of 18-29-year-olds with respect to key delivery characteristics. Methods: From 25 March - 2 April 2021, an online sample of 2,021 UK adults aged 18-29 years participated in a Discrete Choice Experiment. Participants made six choices, each between two SMS invitations to get vaccinated;each choice also had an opt-out. Each invitation had four attributes (1 x 5 levels, 3 x 3 levels): delivery mode, appointment timing, proximity, and SMS sender. These were systematically varied according to a d-optimal fractional factorial design. Order of presentation was randomised for each participant. Responses were analysed using a mixed logit model. Results: The logit model revealed a large alternative-specific constant (β = 1.385, SE = 0.067, p <0.001), indicating a strong preference for ‘opting in’ to appointment invitations. Pharmacies were dispreferred to the local vaccination centre (β = -0.256, SE = 0.072, p <0.001), appointments in locations that were 30-45 minutes travel time from one’s premises were dispreferred to locations that were less than 15 minutes away (β = -0.408, SE = 0.054, p <0.001), and, compared to invitations sent by the NHS, SMSs forwarded by ‘a friend’ were dispreferred (β = -0.615, SE = 0.056, p <0.001) but invitations from the General Practitioner were preferred (β = 0.105, SE = 0.048, p = 0.028). Conclusions: The results indicated that the existing configuration of the UK’s mass vaccination programme was well-placed to deliver vaccines to 18-29-year-olds;however, some adjustments might enhance acceptance. Local pharmacies were not preferred;long travel times were a disincentive but close proximity (0-15 minutes from one’s premises) was not necessary;and either the ‘NHS’ or ‘Your GP’ would serve as adequate invitation sources. This research informed COVID-19 policy in the UK, and contributes to a wider body of Discrete Choice Experiment evidence on citizens’ preferences, requirements and predicted behaviours regarding COVID-19.

Therefore, a multi-disciplinary team -comprising members from Kantar Public UK's Behavioural Practice, Public Health England (PHE), NHS England (NHSE) and the Department of Health and Social Care (DHSC) -was assembled to design and implement research to ensure that the strategy was predicated on a robust evidence base. Speci cally, a discrete choice experiment (DCE) was used to elicit the preferences of 18-29-year-olds with respect to key delivery characteristics, and to provide evidence of which aspects of existing services might need to be adjusted and which new options might be required.
In addition to helping to inform the UK Government's strategy, this research contributes to the wider body of public health literature in three ways. It is the rst study that focuses exclusively on identifying COVID-19 vaccination programme attributes important to those aged 18-29; therefore, it may be useful for public health policymakers and practitioners globally who are formulating COVID-19 vaccination strategies aimed at younger people. Indeed, the issue of lower intended take-up is not limited to the UK: younger people in France [12], the United States [13] and Australia [14] all appear to share similar barriers; and other countries may encounter similar situations when their vaccination programmes reach the relevant stage.
Second, the study further demonstrates the value in applying discrete choice experiments to inform health policy in a live, rapidly changing setting. Third, its design involving a novel DCE presentation format, with choices operationalised in pictorial form, may be used in future to optimise a variety of interventions relating to public health programmes including -but not limited to -Short Message Service (SMS) invitations and communications materials.

Discrete Choice Experiment design
Discrete Choice Experiments (DCEs) present participants with a series of choices between options that have multiple attributes and then preferences over the attributes are extracted using a technique based on Thurstone's Random Utility Theory (RUT), which was extended by McFadden in the late 20 th century [15]. One of RUT's core assumptions is that choice is underpinned by a 'utility', which has a systematic and random component. Systematic components comprise latent values attached to attributes (and levels of each attribute) in choice alternatives, as well as covariates that are in uential in selection. In contrast, random components comprise unknown factors that may impact decisions [16].
DCEs have been increasingly applied in public health research to understand citizens' preferences with respect to interventions or programmes [17], and empirical research has demonstrated their robust external validity: for example, they have been used to accurately predict medical treatment [18] and vaccination behaviour [19]. During the pandemic, DCEs have been used to understand which vaccine characteristics are most in uential in decision-making [20]; health and economic trade-offs in lockdowns [21]; and exit strategies [22].
Given its application across domains in public health research and strong external validity, a DCE was chosen as the core component within this study.

Attributes, experimental design and operationalisation
In contrast to many other public health DCEs -which often rely on primary qualitative research to inform design (see [17]) -this study's DCE was envisaged and designed in collaboration with individuals responsible for the strategic planning of the UK's mass vaccination programme. The design of the DCE was informed by three sessions in the weeks commencing 8th and 15th March 2021; representatives from Kantar Public UK, PHE, NHSE and DHSC were present at each session.
The nal design comprised four attributes (1 x 5 levels, 3 x 3 levels), each of which can be seen in Table 1. The rationale for the de nition of attributes and levels was as follows: Mode of delivery. Mechanisms for vaccination through Primary Care Networks (PCNs) and Vaccination Centres (VCs) were already well established as part of the rst phase of the programme, and it was assumed that these would continue to play an important role in providing access for young people. There was also recognition of the likely need for more targeted, localised services: recruitment of local pharmacies had recently begun, and there was interest in the potential utility of mobile services (which could be deployed in convenient locations) as well as drive-through options. Therefore, the rationale for the inclusion of this attribute was to understand whether these options should be employed -or scaled up or down -according to young people's preferences.
Appointment time. Existing research shows that greater convenience plays a role in reducing hesitancy [23]; but evidence on whether extended opening hours are advantageous for young people speci cally is lacking. Additionally, as noted, COVID-19 presents a novel situation in which past experiences might not apply. Services had been commissioned to operate seven days a week with extended opening hours to heighten convenience. The rationale for the inclusion of this attribute was therefore to identify any strong preferences among young people for appointments outside of normal working hours, further to inform resource planning and allocation.
Proximity. Evidence for a distance decay effect, whereby people who live further away from healthcare facilities have lower levels of usage after adjustment for need [24], suggests that more proximate vaccination locations will result in higher levels of uptake. Vaccination Centres had been speci cally situated within 45 minutes of 99% of the population in England. As such, the rationale for the inclusion of this attribute was to identify any impacts of travel times ranging from 0 to 45 minutes on young people's propensity to attend a vaccination appointment. SMS invitation sender. Unpublished observations had indicated that SMS text messages from friends and family may be effective in motivating attendance at appointments among younger people; General Practice (GP) text messaging systems were already in operation, and a national SMS booking system was about to go live. Consequently, the rationale for the inclusion of this attribute was to provide evidence of any difference in motivational impact due to these SMS senders, to inform planning. A full factorial design with these attributes/levels would have included n=135 pro les (5 1 * 3 3 ), the presentation of which would have been infeasible in this study. Therefore, a d-optimal fractional factorial design was generated using the choiceDes package in R Statistical Software.
The nal design was unlabelled and comprised n=6 paired choice sets, each of which contained an opt-out to maximise external validity. Following the nalisation of the design, each of the choices was translated into an image of an invitation SMS message for presentation in the DCE, an example of which can be seen in Figure 1 below.
The use of pictorial choice options arguably delivers a more natural and relatable set of stimuli for participants than the more usual tabular format for DCEs. As Kahneman and Tversky have argued, "the method of hypothetical choices… relies on the assumption that people often know how they would behave in actual situations of choice, and on the further assumption that the subjects have no special reason to disguise their true preferences" [25]. The method used in this study brings participants closer to the "actual situation of choice" than is often the case with DCEs, enhancing its external validity.
Before starting the DCE, participants were provided with an overview of the scenario (detailing the vaccination programme) and their choice task, which involved selecting the vaccination appointment that they would be most likely to book based on its characteristics, or selecting 'neither appointment'. The introduction to the DCE and the complete choice set can be seen in the Appendix, and an example of a paired choice set can be seen in Figure 2.
In the experiment, the order in which the pairs were shown to participants was randomised to minimise the in uence of order effects [26].

Sample size requirement
There is no scienti c consensus on the sample size required for a su ciently powered DCE. However, rules of thumb have been proposed in the literature, the most common of which is that from Johnson and Orme [27]. According to the authors, the sample size required for a main effects DCE model can be calculated using the following equation: Where: a represents the number of alternatives (2); t represents the number of choice tasks (6), and c represents the number of levels in the largest attribute (5). According to this rule of thumb, we required a minimum of 209 participants, a total which we exceeded in our nal sample.

Sample pro le
This study was conducted online from 25 March to 2 April 2021, with sample sourced from LifePoints (Kantar's online access panel).
The sample for the experiment comprised n=2,012 adults aged 18-29 years who were currently living in the UK and had not been vaccinated at the time of interview. To ensure that the sample was nationally representative of this age group in terms of key demographic characteristics, we enforced exible parallel quotas on age and ethnicity. These quotas were based on mid-year population statistics from the ONS [28]. Quota targets and achieved sample can be seen in Table 1.  Table 2 legend: Age and ethnicity parallel quota targets, and the pro le of the nal achieved sample As an incentive for participation in the study, all participants were provided with LifePoints' reward points.

Statistical methods
Participants' choices were analysed using mixed logit models (alternatively termed a random parameters model), adjusted for their panel nature. Attributes were set as random parameters -each with a normal distribution -to allow for preference heterogeneity across participants [29]. A likelihood ratio test was conducted to test for a difference in t between a model which allowed for correlations between random parameters using Choleski decomposition; however, this test did not indicate signi cant improvement (χ 2 (55) = 31.149, p = 0.996), so the non-correlated model was selected for use. This model was estimated in R statistical software using the mlogit package [30], and can be written as: Where: α denotes the model alternative speci c constant (ASC; the systematic preference for 'opting in' to appointment options). β 1 -β 5 denote individual-speci c coe cients representing the effect of vaccination delivery modes (w 1 represents 'Vaccination centre', w 2 represents 'GP surgery', w 3 represents 'Nearby pharmacy', w 4 represents 'Drive-thru', w 5 represents 'Mobile/pop-up') on selection.
ε is the random error term, representing the non-systematic component in selection.
All attributes were dummy coded, such that 1 represented their presence in each choice card, while 0 represented their absence. Coe cients' signs re ect whether a level has a positive or a negative effect on utility compared to the reference category; further, their absolute values indicate their relative importance in selection, again compared to the reference category. To facilitate ease of interpretation, coe cients were exponentiated to generate odds ratios.

Opt-out analysis
Across the total sets, 83% of choices involved the selection of one of the two available appointments, while 17% of all choices involved 'opting out' of the two appointments presented. Only 5% of all participants (n=103) opted out in all six paired sets, with similar proportions observed across the three age subgroups within the sample (see Figure 3).

Mixed logit model
The preference weights for the DCE are contained in Table 3. In a result consistent with the opt-out gures above, the model's large ASC coe cient (β = 1.385, SE = 0.067, p < 0.001) indicated a strong systematic preference for 'opting in' to appointments in the DCE.
The model also highlighted variation in preferences with respect to appointment characteristics, particularly delivery mode and proximity. In terms of delivery mode, 'Nearby GP Surgery' was most preferred The nal two attributes -appointment time and invitation source -were, relatively speaking, less in uential in participants' decision making in the DCE. In terms of the former attribute, appointments scheduled afterhours throughout the week were least preferred (β =-0.234, SE = 0.056, p<0.001); in terms of the latter, invitations forwarded from one's best friend were the least favoured (β =-0.614, SE = 0.056, p<0.001). Akaike Information Criteria -12423 Table 3 legend: Coe cients, standard errors, odds ratios and p-values of the random parameters model. * Represents the mean of all individual-speci c coe cients within the total sample (n=2,012)

Preference heterogeneity
The distribution of individual-speci c coe cients in a mixed logit model provides information about the degree of preference heterogeneity across a given sample.
In this case, direction of preferences was relatively consistent across the sample, but strength of in uence often differed (see Table 4). The attribute levels that had the largest positive in uence upon appointment selection according to their coe cient -for example, Proximity 'Between 15 and 30m' -positively impacted selection for most in the sample; however, the extent of this impact differed (IQR = 0.257). Similarly, the levels that had the largest negative in uence upon appointment selection -for example, SMS invitation sender 'Best friend' -had a negative impact for most; however, again, the scale of this varied (IQR = 0.320).
There was, however, one attribute level whose presence polarised respondents more than others: Your GP as an invitation sender. For Your GP, the rst quartile was negative (Q 1 = -0.243) suggesting a preference for a message from the NHS; on the other hand, the third quartile was positive (Q 3 = 0.113), suggesting a preference for a message from Your GP. This result suggests that further research is required to determine the appropriate SMS messenger for vaccination invitations.

Discussion
This study aimed to identify the preferences of 18-29-year-olds with respect to key delivery characteristics in the UK's vaccination programme. The study used a novel pictorial DCE to understand the relative importance of delivery mode, proximity, timing and invitation source in this sub-population's consideration of vaccination appointments, thereby helping to inform programme planning and delivery.
Overall, there was a strong systematic preference for 'opting in' to the appointment invitations in the DCE, suggesting that most aged 18-29 will choose to receive a vaccine when invited given the presented programme characteristics. However, results did highlight preferences with respect to vaccine delivery, However, pharmacies -of which there were 524 in the vaccination programme, with further recruitment underway -were not preferred options for vaccine administration. As Table 4 indicates, there was considerable variation in this across the sample (more so, in fact, than for any other delivery characteristic), and while some participants displayed a strong preference for Vaccination Centres over pharmacies, for others the preference was slight.
The overall result is at odds with some of the empirical literature: previous research conducted in England demonstrated that community pharmacies can increase vaccination uptake [32]. However, as noted elsewhere, COVID-19 presents novel situations and challenges, and 18-29s are a speci c population subgroup, so there is no guarantee that evidence from other sources or regarding other sub-groups will apply equally in this context. Moreover, other research conducted in the UK has indicated that individuals may possess lower levels of trust in pharmacists, particularly when they are delivering 'unfamiliar' services considered to be high risk [33]. One may speculate that 18-29s are indeed less familiar with pharmacies than other groups in the population -although further research would be required to ascertain this.
Proximity was another attribute that strongly in uenced participants' choices in the DCE: there was a preference for administration locations less than half an hour from one's place of residence, particularly those 15-30 minutes away. The desire for closer locations for medical care is not unusual, as similar DCEs elsewhere have noted [34]. As discussed, Vaccination Centres are located to be within 45 minutes of 99% of the population, so while VCs should be expected to be preferred as modes of delivery in themselves, for some 18-29s they may be too far away to be viable options.
However, the overall preference for a location 15-30 minutes away -rather than less than 15 minutes away -is atypical and counter to expectations. First, it should be noted that this result was not consistent across the sample: as Table 4 indicates, preferences within the rst quartile were slight. Any explanation for this result would be speculation, but it may be that many were averse to the idea of large numbers of people coming to a location close to their home to receive a vaccine. Alternatively, it is possible that individuals were thinking of preferred vaccination locations near their actual place of residence, and assumed a travel time of 15-30 minutes (which is the approximate time it would take for a majority of the population to access vaccination locations: [35]). Further research would therefore be required to provide answers to this question.
The timing of appointments had a less pronounced in uence on participants' choices, but there was a consistent preference for those scheduled during between 9am and 5pm on Monday to Friday, and at weekends, as opposed to 'after hours' on Monday to Friday. This provides some evidence with which to address the question of whether extended opening hours would be required to encourage 18-29-year-olds to make appointments to receive a vaccine. It suggests that whilst extended hours may be appropriate for certain sub-groups, normal working hours should continue to be a key priority in resource allocation.
Finally, despite indications from unpublished work that SMS text messages from friends may be motivating, the DCE revealed a strong negative reaction to this approach among 18-29-year-olds, with a clear preference for one's GP and/or the NHS. The DCE did not test SMS text messages as a medium against other possible channels (all choice options were presented in as text messages, while the messenger varied). However, the fact that these choice options collectively generated a strong preference for 'opting in' to vaccination suggests that existing GP text messaging systems and the forthcoming national SMS booking system would both be effective channels for communication.

Strengths And Limitations
Some limitations in this study need to be acknowledged. Perhaps most obviously (given the discussion above), while this research can provide evidence of preferences between delivery characteristics, it cannot reveal the reasons underpinning those preferences. For example, the reasons for the deterrent effect of pharmacies, and of the shortest travel times, need to be inferred or hypothesised. Further research into these questions would be needed if explanations are required.
Second, to enable rapid collection of data and provision of evidence, this study's sample was drawn from an online access panel. Thus, people without access to the internet were necessarily excluded, and the sample may have been open to selection biases inherent to online panels. However, given that 99.5% of people in the age group in question had used the internet within the past three months in 2020 [28], impacts of the exclusion can be discounted; and steps were taken to minimise biases within the sample by setting and meeting quotas for age and ethnic sub-groups.
The study's strengths should equally be highlighted. First, the DCE was designed in close collaboration with vaccination policy and implementation specialists in Government. The DCE's attributes and levels were chosen to relate to existing provision and potential new delivery options based on practical capabilities and evidence of what has been effective in other contexts. As such, it was speci cally designed to answer live questions about the delivery of COVID-19 vaccines to 18-29-year-olds, and thus to inform decision-making regarding the continued roll-out of the vaccination programme in England. Second, the sample size was large compared with many DCEs relating to healthcare, providing sensitivity to detect small differences in preference. Third, eldwork was completed within one week, ensuring a comparable context across the sample despite the rapidly evolving environment. Fourth, the use of pictorial choice options (SMS text messages expressing the characteristics of each appointment type) arguably delivers a more engaging, natural and relatable set of stimuli for participants than the more usual tabular format for DCEs, thereby enhancing external validity.

Conclusions
Read as a whole, the results of this DCE suggest that the current con guration of the UK's mass vaccination programme is well-placed to deliver vaccines to 18-29-year-olds. Preferences for receiving a vaccine, and for existing delivery modes (GP surgeries and Vaccination Centres) were strong -provided distance from less proximate VCs is not a disincentive. This indicates that resources should continue to be deployed in their current form.
However, the DCE also provides evidence that suggests answers to some of the speci c questions regarding the needs of 18-29-year-olds. In particular, the assumption that convenience is a key driver of uptake needs to be examined further. Some new 'local' delivery options were preferred (e.g., mobile/pop-up services), but local pharmacies were not. Likewise, proximity was important, as travel time of 30-45 minutes was a deterrent; however, convenience was not an overriding factor as 15-30 minutes was preferred to 0-15 minutes in all sample quartiles, albeit with variable strength of preference. Finally, the fact that appointments between 9am and 5pm on Monday to Friday, and at weekends, were similarly appealing (the former slightly preferred) suggests that 18-29- year-olds may be prepared to be exible regarding the timing of appointments, provided a variety of times are offered throughout the day and seven days a week.
This study adds to a wider and growing body of DCE evidence of citizens' preferences, requirements and predicted behaviours regarding COVID-19.
Additionally, it provides further evidence of the ability of DCEs to provide targeted, robust evidence that informs planning in fast-moving public-health crises. Ethics approval for this study was granted following review by PHE's Research Support and Governance O ce (R&D 441). All participants provided consent for their participation in the study.

Consent for publication
All authors provide consent for publication.

Availability of data and material
The dataset used for the current study is available from the corresponding author on reasonable request.

Competing interests
None.

Funding
None.
Authors' contributions RM was responsible for conceptualisation, experimental and questionnaire design, ran the analysis and wrote the manuscript.
BT was responsible for conceptualisation, experimental and questionnaire design and wrote the manuscript.
NG was responsible for experimental design and contributed to the manuscript.
CB was responsible for conceptualisation, experimental design and contributed to the manuscript.
RR was responsible for experimental design and contributed to the manuscript.
TM was responsible for experimental design and contributed to the manuscript.
DW was responsible for experimental and questionnaire design.
RS was responsible for experimental and questionnaire design.
TC was responsible for experimental design.
RA was responsible for experimental design.
CKH was responsible for experimental design.
MM contributed to the manuscript.  Figure 1 Example SMS message presented in paired choice sets The DCE comprised six pairs of SMS invitations, the attributes of which differed systematically according to the d-optimal fractional factorial design.

Figure 2
Example paired choice set Participants completed six paired pictorial choice sets, the order of which was randomised. Each choice set also contained an optout: 'Neither appointment'.