A machine learning approach to identify psychological factors driving vaccine hesitancy in high income countries

Understanding factors driving vaccine hesitancy is crucial to vaccination success. We surveyed adults (N=2510) from February to March 2021 across ve sites (Australia=502, Germany=516, Hong Kong=445, UK=512, USA=535) using a cross-sectional design and stratied quota sampling for age, sex, and education. We assessed willingness to take a vaccine and a comprehensive set of putative predictors. Predictive power was analysed with regression analyses and machine learning algorithms. Only 57.4% of the participants indicated that they would denitely or probably get vaccinated. A parsimonious machine-learning model could identify vaccine hesitancy with high accuracy (i.e. 83% sensitivity and 82% specicity) using 10 variables only. The most relevant predictors were vaccination conspiracy beliefs, paranoid concerns related to the pandemic, COVID anxiety, high perceived risk of infection, low perceived social rank, lower age, and female gender. Campaigns seeking to increase vaccine uptake need to take mistrust as the main driver of vaccine hesitancy into account. a cross-sectional online-survey conducted in Hong Kong, Australia, USA, United Kingdom, and Germany. The survey was programmed using the online-survey platform Qualtrics. Participants were recruited using stratied quota sampling to ensure that each sample was representative of the respective general population based on sex, age, and educational attainment. No further eligibility criteria were applied. Data were collected between February and March 2021. We aimed for a sample size of 2500 taking into account the stratication and number of sites, the large number of predictors, and expected small effect sizes of some of the putative predictors. The survey took 25 minutes in total, beginning with informed consent, followed by socio-demographic assessment and the questionnaire battery, of which further details have been reported elsewhere 32 . To prevent missing data, participants were required to respond to all questions on each page before being able to continue leading to minimal missing data on isolated variables due to initial software errors (Missings for: “perceived infection risk”: 0.2%, n=7, “preferred sources of information”: 2.8%, n=72, and “social adversity”: 0.1%, n=3) or due to a “don’t know” answering option (“size of the current home city”: 9.3%, n=234). Participants who failed any of the attention checks, took shorter than half of the median completion time, or showed patterns of machine responses or duplicate patterns of response were excluded. The ow of participants across sites is shown in Table 1. All procedures were approved by each of the ethics committees of the institutions involved (i.e., University of London Research Ethics Committee,. Reference No. 2368, (2) New England - Butler Hospital Institutional Board, Reference (3) La University Human Research Ethics Committee, Application No. HEC21012, (4) Local Ethics Committee, Universität Hamburg, Application No. 2020_346, and (5) The Chinese University of Hong Kong Survey and Behavioural Research Ethics Committee Reference No. SBRE-20-233). a COVID paranoia global score and the three facets pandemic persecutory threat (15 items, e.g.: “People are deliberately trying to pass COVID-19 to me”), pandemic paranoid conspiracy (six items, e.g.: “COVID-19 is a conspiracy by powerful people”), and pandemic interpersonal mistrust regarding health measures (four items, e.g.: “I can’t trust others to stick to the social distancing rules”). Participants answer on a scale from 0=“not at all” to 4=“totally”. Based on the data used for this article, Kingston et al. 32 reported good reliability (internal consistency: α=0.90, test-retest reliability: 0.60 ≤ r ≤ 0.78), factorial validity, and criterion validity. For this study, the three subscales and the global score were calculated. Vaccine conspiracy beliefs were assessed by adapting the general 7-item Vaccine Conspiracy Beliefs Scale 35 , a valid one-dimensional scale with high internal consistency. The adaptation involved referring to COVID-19 vaccines specically and using present tense (full item-list in Supplement 1). Reliability in this study was α=0.97. General mistrust variables included paranoid ideation and general conspiracy mentality. Paranoid ideation was measured with the Revised Green Paranoid Thoughts Scale 36 . This 18-item questionnaire assesses ideas of reference and persecutory ideation over the past fortnight on two scales. Each item (e.g. “Certain individuals have had it in for me”) is rated on a scale from 0=“not at all” to 4=“totally”. Higher scores indicate higher levels of paranoia. Reliability in this study was α=0.94 for ideas of reference and α=0.96 for persecutory ideation. General conspiracy mentality was assessed with the Conspiracy Mentality Questionnaire 37 an instrument designed to eciently assess differences in the generic tendency to engage in conspiracist ideation within and across cultures. A one-dimensional and time-stable construct has been conrmed across several language versions. It consists of ve statements (e.g. “Many very important things happen in the world, which the public is never informed about”) that are rated in terms of their likeliness on from 0=“0% chance” to 11=“100% chance”. Reliability in this study was α=0.91. screened alongside socio-demographic variables with a four item self-report questionnaire used by and colleagues 20 . The items consisted of yes/no questions covering emotional neglect, psychological abuse, physical abuse, and sexual abuse (e.g., approached against in this α=0.95.


Introduction
As several COVID-19 vaccines are being rolled out, success of the vaccination will crucially depend on a su cient proportion of the population accepting a vaccine. Numerous studies have already investigated putative vaccine acceptance by asking people whether they would be willing to accept a COVID-19 vaccine if it were offered to them. Vaccine willingness rates vary around 65-75% of the population in most of the surveyed countries 1 . The few multi-national studies to date indicate considerable between country variance [2][3][4] . Even within the group of high income countries, which are now in the process of offering vaccines to all their citizens, the acceptance rates have been found to vary, with UK citizens showing particularly high vaccine willingness, Germans being more hesitant 3,4 and particularly low rates in Hong Kong 5 . Overall, however, it is clear that fewer people are willing to take a vaccine than required for su cient population immunity [6][7][8] .
Thus, some of the driving factors of COVID-19 vaccine hesitancy identi ed so far (i.e., sociodemographic factors, risk perception, trust in vaccine-safety) match those found for vaccine hesitancy in general 17,18 . Beyond those factors, the studies point to the relevance of factors indicative of a more fundamental mistrust, including mistrust of mainstream media and politics, conspiracy beliefs, and paranoid ideation.
However, we do not know how well these putative driving factors perform in predicting vaccine hesitancy, which factors are most relevant to an optimal prediction, or whether an optimal prediction in one country can be generalized to other countries. Identifying a globally stable algorithm to predict vaccine hesitancy based on a limited set of variables would provide an immensely helpful basis for targeted interventions to increase vaccine willingness. Thus, an important next step would be to probe for and optimize the prediction of vaccination willingness in a multi-national survey on the basis of variables identi ed as relevant so far. This could be done by using machine learning algorithms that are able to capture the complex relationships and interactions between variables 19 . Also, given the relevance of mistrust, it seems promising to place more focus on this construct in relation to vaccine willingness. This could be done by including a more ne-grained assessment of mistrust related variables along with predictors of mistrust that have been identi ed in clinical research on paranoia. These include social marginalization and adversity (e.g. having a minority status or interpersonal traumatization), and negative generalized beliefs about oneself, other people, and one's position in society 20 .
The present comprehensive multi-national survey included representative samples from ve high-income sites in the early phases of vaccine rollout, to 1) assess the prevalence of COVID-19 vaccine willingness across sites; 2) con rm that COVID-19 vaccination willingness is predicted by a) sociodemographic variables including those indicative of social marginalization, b) COVID-19 anxiety and risk perception, c) political views and types of information sources, d) speci c mistrust (i.e., vaccine conspiracy beliefs, pandemic-related paranoid ideation), e) general mistrust (i.e., conspiracy mentality, general paranoid ideation), f) social adversity, and g) generalized beliefs about the self, others, and ones' own social rank; and 3) use these variables to probe for the optimum prediction accuracy for vaccine hesitancy and explore the stability of the most promising predictive model across sites.

Results
Sample characteristics. Sample characteristics for the full sample and the individual sites are presented in Table 1.
[Insert Table 1 about here] Prevalence of vaccination willingness. Table 2 shows vaccine willingness across sites. Only 57.4% of all participants indicated that they would de nitely or probably get vaccinated. The distribution of the answers varied considerably between sites. In the USA and Germany, a bi-modal distribution of answers with peaks in de nite willingness and de nite rejection of the vaccine were found. In the UK and the Australian samples, by contrast, there were skewed distributions with most participants indicating de nite willingness. Finally, most participants in the Hong Kong sample answered in the mid-category indicating possible willingness for vaccination, with few participants responding with de nite acceptance or rejection. An ANOVA of vaccination willingness showed a signi cant effect of site [Insert Table 2 about here] Prediction of vaccination willingness using regression.
As can be seen in Table 3 (left column, correlation), most variables showed signi cant correlations with vaccine willingness. The strongest associations were found for COVID anxiety (positive association), vaccine conspiracy beliefs, COVID conspiracy beliefs, and general conspiracy mentality (all negative associations).
Among cluster-speci c linear regression models, the speci c mistrust model yielded the largest proportion of explained variance (R²=0.419), followed by the general mistrust model (R²=0.118) and the extended socio-demographic model (R²=0.086). The remaining clusters showed minimal levels of explained variance (see Table 2 mid columns). The combined regression model with all variables explained 47.2% of the variance in vaccine willingness (see Table 2 right columns).
[Insert Table 3 about here] Prediction of vaccination willingness using machine learning.
For both cross-validation methods, a machine learning model revealed a prediction model with high balanced accuracy (see Table 4; details on the hyperparameter tuning results are provided in Supplement 2). As can be seen, the model was able to correctly classify 82% and 83% of the participants who were willing to get vaccinated (i.e., sensitivity) that were left out for cross validation in the cross-site and leave-one-person-out validation respectively. Furthermore, the model was able to correctly identify most participants who indicated an unwillingness to get vaccinated (i.e., speci city). However, the full model predicting new data from a separate site showed somewhat lower speci city (78%) than the leave-one-person-out model (82%).
[Insert Table 4 about here] Feature importance analysis showed that vaccination conspiracy beliefs was the most informative variable with a decrease in accuracy of 23.5% when this variable was permuted. The remaining nine of the top ten informative variables included all speci c mistrust variables, social rank, COVID anxiety, perceived infection risk, age, and gender (see Table 4, left column). Calculating a new model without vaccine conspiracy beliefs, however, only resulted in a slight drop in accuracy with the speci city being more affected than the sensitivity. In this model, COVID speci c and general conspiracy beliefs increased in feature importance, and positive beliefs in oneself and others moved up into the list of the top ten most relevant variables (see Table 4, mid column). Leaving out all speci c and general mistrust variables led to a considerable drop in accuracy with the most informative variable in the model now being COVID anxiety followed by a mix of variables from all remaining clusters (see Table 4, right column). Subsequent models with further variable clusters removed performed only minimally less accurately (all ΔBAC≤0.02).
[Insert Table 5 about here] Calculation of a parsimonious models with only the ten or ve most important variables from the full model (Table 5, left column) yielded the same accuracy as the full model (ten variables) or a minimal decrease in accuracy ( ve variables; Table 4).

Discussion
Surprisingly, only 57.4% of the total sample indicated that they would de nitely or probably get vaccinated, which is a somewhat lower percentage than the 65-75% identi ed previously 3,4 . The lower rates might stem from the fact that vaccine side effects were receiving a lot of media attention during the assessment period 21 , and online misinformation on vaccination was rocketing 22 . However, differences between countries also need to be considered: Corresponding with the two previous multinational studies, we found the UK to show comparably high vaccine willingness and lower rates for the US and Germany 3,4 . Interestingly, the distributions of the willingness scale also differed between sites, with the US and Germany tending more to the extremes (i.e., clear refusal or willingness), whereas participants from Hong Kong showed more indecisiveness. Low vaccine willingness in Hong Kong was also reported in a previous survey 5 and may be partly explicable by safety and effectiveness concerns associated with speci c vaccines being offered in Hong Kong.
In terms of predicting vaccine willingness, we could con rm most of the included factors that were delineated either from previous research or were added based on clinical models of paranoia. The clearest nding was the strong predictive value of speci c mistrust, which explained 42% of the overall variance in vaccination willingness, exceeding previous associations between vaccine hesitancy and mistrust 11 . Within the variables indicative of mistrust, the strongest associations with vaccine willingness were found for vaccine conspiracy beliefs and pandemic paranoid conspiracy beliefs. Interestingly, one type of mistrust within this group, namely not trusting others to comply with the COVID measures, predicted higher willingness to get vaccinated, suggesting that pandemic mistrust is a multifaceted construct that leads to different behavioral responses. The next best predictors were variables indicative of more general mistrust, particularly general conspiracy mentality but also general paranoid ideation (about 12% of explained variance). In terms of demographics, we could con rm associations from prior studies (e.g. younger age, female gender, unemployment, living in larger city), except for migrant or minority status. In fact, the regression models found minority groups to have higher vaccination willingness. Together, the extended sociodemographic factors explained about 9% of the overall variance in vaccination willingness whereas each of the remaining categories of variables explained even less variance (2-6%).
Using machine learning, we were able to improve the prediction accuracy from the regression (47%) by nding a parsimonious model with a sensitivity of 82% and a speci city of 78-82%, depending on the type of cross-validation. This model con rmed the high predictive value of vaccine conspiracy beliefs and other indicators of speci c mistrust, but also used the perception of social rank, COVID anxiety and perceived infection risk, as well as age and gender to optimize its prediction (see Table 4, left column). Despite the high relevance of the vaccine conspiracy beliefs, they were not essential to good prediction and could be compensated for by putting more weight on COVID speci c and general conspiracy beliefs, resulting in almost as good prediction accuracy. In contrast, models that were not fed with any mistrust variables performed poorly.
There was no drastic drop in the models' prediction accuracy depending on the method of cross-validation. Accurate test predictions were found both when we trained the model on all participants but one and tested it on the remaining participant and when we trained it on the four sites before testing it on the fth site. This indicates that there is a large overlap in the crucial variables and their complex interactions in predicting willingness, irrespective of culture. Thus, the psychological mechanisms underlying willingness appear to be largely universal. Given the high sensitivity and speci city of our models, this implies that countries with lower vaccine willingness differ from those with higher willingness in the predictors identi ed as relevant. A straightforward conclusion would be that the most prominent predictor, vaccine conspiracy beliefs, must be higher in countries with lower vaccine willingness. Differences in vaccine and general conspiracy beliefs are indeed likely to explain a substantial part of the difference between the UK (highest willingness scores, lowest scores on all conspiracy scales, compare Supplement 4 for the mean values of all predictors by site) and Hong Kong (lowest willingness, high conspiracy beliefs) and the conclusion also ts in with the nding of extreme vaccine hesitancy in countries of the middle east with very high rates of conspiracy beliefs 1 . Looking merely at the extent of the most relevant predictors would nevertheless be oversimplifying things. For instance, the Australian sample in our study had the highest score on the vaccine conspiracy scale, but an average score on vaccine willingness, which is only explicable by taking the entire list of predictors and their complex interactions into account (e.g., interactions between conspiracy beliefs and the positive predictor pandemic interpersonal mistrust regarding adherence to social distancing). It also needs noting that there was a small gain in speci city when the model was generated using data from the same site/culture, which may indicate that there are also regional factors that affect rejection, albeit only to a very small extent.
A limitation of the study is that although respondents were broadly representative of the sites' adult general populations in terms of age, sex, and educational level, they are unlikely to be fully representative of the general populations. The percentages of those who declined participation varied across sites and the reasons for declining are unknown. Also, the sites do not re ect the global variability in cultures, thus the universality of predictors requires further validation in more heterogeneous samples of countries. Another limitation is the cross-sectional nature of the design. Although the causal interpretation that mistrust is driving vaccine refusal is intuitive, the opposite direction (vaccine conspiracy could be a post hoc rationalisation of not wanting a vaccine for other reasons) cannot be ruled out. Finally, it needs noting that vaccine willingness may not accurately predict actual vaccine uptake. However, the low willingness we found bears a striking resemblance to the (ongoing) vaccine uptake that is beginning to level off at rates below 60% 23 . The extent to which the machine learning model predicting vaccine willingness holds up for predicting actual vaccine intake is also an issue for future research.
In sum, we found that by using only ten variables we were able to achieve an 82% accuracy in predicting vaccination hesitancy, with the most crucial factors being mistrust in vaccines, governments, companies, and organizations. The reasons for pronounced societal mistrust are manifold 24 , but have been found to include both individual societal experiences, such as downward social mobility 25 and the perception of past and present institutional misperformance 26 . Institutions that do not perform well, be it by incompetence or elite misbehaviour and corruption, tend to generate distrust 27 . People are more likely to attend to and believe information that aligns with their expectations (con rmation bias) 28 . Conspiracy theories align well with negative expectations that have resulted from previous experiences, rendering them more likely to be believed. The high predictive value of vaccine conspiracy beliefs clearly corroborates the efforts towards strategic approaches to detect and mitigate the impact of anti-vaccine activities on social media, to support high acceptance of safe and effective vaccines 22,29,30 . However, given that our machine learning algorithm performed almost well in predicting vaccine hesitancy without vaccine conspiracy beliefs, based on more general mistrust, reducing or contradicting vaccine conspiracy information might not be su cient. Publically provided vaccine information needs to take mistrust into account, perhaps by counterbalancing negative emotions with positive emotional appeals, such as altruism and hope 31 , and by providing information on the safety and effectiveness of the vaccine in a way that enables the recipients to judge its validity for themselves. Also, information campaigns will probably need to be complemented by policies aimed at regaining trust in politicians, industry, science, and the medical profession.

Design & Procedure
The design was a cross-sectional online-survey conducted in Hong Kong, Australia, USA, United Kingdom, and Germany. The survey was programmed using the online-survey platform Qualtrics. Participants were recruited using strati ed quota sampling to ensure that each sample was representative of the respective general population based on sex, age, and educational attainment. No further eligibility criteria were applied. Data were collected between February and March 2021. We aimed for a sample size of 2500 taking into account the strati cation and number of sites, the large number of predictors, and expected small effect sizes of some of the putative predictors. The survey took 25 minutes in total, beginning with informed consent, followed by socio-demographic assessment and the questionnaire battery, of which further details have been reported elsewhere 32 . To prevent missing data, participants were required to respond to all questions on each page before being able to continue leading to minimal missing data on isolated variables due to initial software errors (Missings for: "perceived infection risk": 0.2%, n=7, "preferred sources of information": 2.8%, n=72, and "social adversity": 0.1%, n=3) or due to a "don't know" answering option ("size of the current home city": 9.3%, n=234). Participants who failed any of the attention checks, took shorter than half of the median completion time, or showed patterns of machine responses or duplicate patterns of response were excluded. The ow of participants across sites is shown in Table 1

Measures
Willingness to be vaccinated for COVID-19 was assessed with the following item: "If a COVID-19 vaccine was offered to you now, would you accept it?" The item was rated on a scale from 1= "De nitely not" to 5="Yes, de nitely'" adapted from Wong and colleagues 33 .
Sociodemographic data and related questions: Sociodemographic variables included age, sex assigned at birth, and current gender (options: "male", "female", "trans-male", "trans-female", "genderqueer", and "other"), size of the current home city (rated in six categories form ≤100.000 to ≥10.000.000), highest educational degree achieved (rated in nine categories from elementary school degree to PhD), annual income (seven categories from "under £18,500/US$24,999/18,000€" to "above £112,000/US$150,000/109,000€"), employment status over the last year (nine categories), migrant status, minority status ( ve categories, each rated as present or absent), and having a mental-health diagnosis.

Risk perception variables included (1) COVID-19 anxiety, (2) personal experiences with COVID-19 in family members or friends, (3)
perceived infection risk, and (4) perceived consequences of an infection. Following Shevlin et al. 34 COVID-19 anxiety was assessed using the question "How anxious are you about the coronavirus COVID-19 pandemic?" for which participants were provided with a 'slider' to indicate their degree of anxiety with 0="not at all worried" and 100="very worried". Personal experiences with COVID-19 in family members or friends were assessed by the following item: "Someone who is close to me has had a COVID-19 virus infection con rmed by a doctor" rated with 1="yes" 0="no". Perceived risk of a COVID-19 infection was assessed with the item: "What do you think is your personal percentage risk of being infected with the COVID-19 virus over the following time periods?" rated from 1="no risk" to 11="great risk" for each time period ("the next month", "the next three months", and "the next six months"). Similarly, perceived consequence of infection was assessed with "How bad do you think would be the consequences of you being infected with the COVID-19 virus over the following time periods?" rated from 1="not too bad" to 11="very bad". Mean scores of perceived risk and perceived consequences were calculated.
Political views were rated from 1="very left-wing" to 7="very right wing" and preferred sources of information ("How do you nd out about what is going on in the world?") were rated from 1="always from mainstream media" to 5="always from social media" 9 .
Speci c mistrust variables included (1) COVID-speci c paranoid ideation and (2) vaccine conspiracy beliefs. COVID-speci c paranoid ideation was assessed with the Pandemic Paranoia Scale 32 , a 25-item scale assessing paranoid thinking speci cally related to the COVID-19 pandemic. It comprises a COVID paranoia global score and the three facets pandemic persecutory threat (15 items, e.g.: "People are deliberately trying to pass COVID-19 to me"), pandemic paranoid conspiracy (six items, e.g.: "COVID-19 is a conspiracy by powerful people"), and pandemic interpersonal mistrust regarding health measures (four items, e.g.: "I can't trust others to stick to the social distancing rules"). Participants answer on a scale from 0="not at all" to 4="totally". Based on the data used for this article, Kingston et al. 32 reported good reliability (internal consistency: α=0.90, test-retest reliability: 0.60≤r≤0.78), factorial validity, and criterion validity. For this study, the three subscales and the global score were calculated. Vaccine conspiracy beliefs were assessed by adapting the general 7item Vaccine Conspiracy Beliefs Scale 35 , a valid one-dimensional scale with high internal consistency. The adaptation involved referring to COVID-19 vaccines speci cally and using present tense (full item-list in Supplement 1). Reliability in this study was α=0.97.
General mistrust variables included paranoid ideation and general conspiracy mentality. Paranoid ideation was measured with the Revised Green Paranoid Thoughts Scale 36 . This 18-item questionnaire assesses ideas of reference and persecutory ideation over the past fortnight on two scales. Each item (e.g. "Certain individuals have had it in for me") is rated on a scale from 0="not at all" to 4="totally".
Higher scores indicate higher levels of paranoia. Reliability in this study was α=0.94 for ideas of reference and α=0.96 for persecutory ideation. General conspiracy mentality was assessed with the Conspiracy Mentality Questionnaire 37 an instrument designed to e ciently assess differences in the generic tendency to engage in conspiracist ideation within and across cultures. A one-dimensional and timestable construct has been con rmed across several language versions. It consists of ve statements (e.g. "Many very important things happen in the world, which the public is never informed about") that are rated in terms of their likeliness on scale from 0="0% chance" to 11="100% chance". Reliability in this study was α=0.91.
Social adversity was screened alongside socio-demographic variables with a four item self-report questionnaire used by Jaya and colleagues 20 . The items consisted of yes/no questions covering emotional neglect, psychological abuse, physical abuse, and sexual abuse (e.g., "were you ever approached sexually against your will?").
Generalized beliefs about self, others, and one's own social rank were assessed with the Brief Core Schema Scales (BCSS) 38 and the Social Comparison Scale (SCS) 39 . The BCSS assesses negative and positive beliefs about oneself and others on four subscales of six items, respectively (e.g., "Other people are bad") that are rated as yes versus no. For each yes-response the degree of conviction is assessed on a scale from 1="no, do not believe it" to 5="yes, believe it totally". Reliability for the subscales in the current study ranged from α=0.85 to α=0.90. The SCS consists of 11 bipolar items that ranged from 0 to 10 (e.g., inferior-superior, left out-accepted) that are rated over the past four weeks. Lower scores indicate a more negative view of the self in comparison with others. Reliability in this study was α=0.95.

Analyses
Statistical analyses were conducted with SPSS 22 40 . First, we calculated Pearson correlations for all predictor variables. Next, we calculated multifactorial regression models for each of the variable clusters (1) extended socio-demographic data, (2) risk perception, (3) political view/news source, (4) speci c mistrust, (5) general mistrust (5) interpersonal trauma, and (6) beliefs about the self, others, and social rank in order to compare the explained variance in vaccination willingness for these different predictor types. In a nal regression model, all variables were entered to evaluate the overall explained variance. All signi cance tests for correlations and predictors in regression models were two-tailed tests based on available data without any further adjustments.
Next, to further test for optimization of prediction accuracy, we established a machine-learning algorithm to predict vaccination willingness (i.e. "de nitely" or "probably" getting vaccinated) versus unwillingness ("de nitely not" or "probably not" getting vaccinated) using all assessed variables (n=2116) and including missing values in the classi cation. The mid-category of ("possible") willingness was left out in this analysis. Calculation of machine learning models were carried out in Python 3.8.6 with the packages scikit-learn 0.23.2 41 , as well as Numpy, Pandas and imblearn. For all tested models we used random forest classi ers and conducted hyperparameter tuning on a class-balanced version of the dataset rst (see Supplement 2 for details). Next, we chose the hyperparameter con guration that had the best testing accuracy and evaluated model performance by cross-validating across the ve sites and by leave-one-person-out cross-validation 19 . Finally, we used the calculated machine learning model to evaluate the predictive value of the individual variables. We used permutation feature importance 42 (see Supplement 3 for details) to estimate the importance of each variable in a given model. This allowed for the selection of the highest ranking variables to test whether subsequent smaller machine learning models that use only a small selection of questionnaires retain accuracy. Furthermore, it allowed for the elimination of the highest ranking variables/variable cluster to further explore their absolute relevance (i.e., whether they could be compensated for by other predictors).

Declarations Data availability
The study protocol, statistical analysis plan, and the machine learning code for implementation of the models will be made available on OSF immediately following publication. The datasets including individual participant data generated during and/or analysed during the current study are not publicly available due to constraints in the ethical approvals in some of the sites but are available from the corresponding author on reasonable request and after receiving approval from the respective ethic boards. Aggregated data for metaanalyses is available from the second author.

Role of funding source
There was no funding source for this study.

Con ict of Interest
All authors declare to have no competing interests.  Tables   Table 1. Participant ow and sociodemographic details across samples.  Table 3 Correlation and multifactorial linear regression analyses predicting vaccination willingness Note: a) to avoid bias due to low cell counts the variables sex and gender were combined into a dichotomized variable to re ect the gender a participants most likely reads as at present (e.g. a person describing their sex as male and their gender as trans/female was labeled as female; a person describing their sex as female and their gender as other was labeled as female) leading to a recoding for 16 participants (0.63%); b) education level was dichotomized with GCSE or lower categorized as low education level and everything else as high education level; PPS = Pandemic Paranoia Scale; CMQ = Conspiracy Mentality Questionnaire; RGTPS = Revised Green Paranoid Thoughts Scale; Scale; BCSS = Brief Core Schema Scales; SCS = Social Comparison Scale. Table 4 Cross-validation results for the machine-learning models predicting answers for the left out countries and in a leave-one-person out crossvalidation