Policy support
None of the intervention messages increased support for the obesity prevention policies when compared to the control group (all ps > .0125, see Table 1 for full results).
[INSERT TABLE 1 HERE]
Causal beliefs (manipulation checks)
There was a statistically significant effect of the message presented in Group 2 pertaining to the environment’s role in obesity on beliefs about the environment’s influence on human behaviour. Participants who received this intervention message believed that the environment had a greater influence on human behaviour than those in the control group, B = 0.18, 95% CIs [0.04, 0.31], p = .009, d = .20, representing a small increase on the 1-7 rating scale (see Tables 1 for full results).
There was also a statistically significant effect of Group 3’s message about the environment’s influence on obesity on beliefs about the environment’s influence on obesity. Participants who received this intervention believed that the environment had less influence on obesity than those in the control group, B = -0.27, 95% CIs [-0.47, -0.07], p = .007, d = .21. This effect was in the opposite direction to that which was predicted.
There were no other statistically significant effects of any intervention, compared to the control group, on beliefs about the environment’s influence in obesity or in human behaviour in general (all ps > .0125).
Subjective Comprehension
There were no statistically significant differences between the two obesity messages in terms of subjective comprehension, B = -0.04, 95% CIs [-0.19, 0.11], p = .623, d = .04. However, the human behaviour message communicated in Group 5 was rated as significantly clearer and easier to understand when compared to the human behaviour message in Group 4, B = 0.21, 95% CIs [0.07, 0.36], p = .005, d = .22, a small increase on the 1-7 rating scale (see Table S2 for descriptive statistics).
Discussion
The primary aim of this study was to select messages that would be most effective at changing causal beliefs for use in Study 2. The results showed that none of the interventions changed the target belief in the hypothesised direction. In keeping with this, there were no changes in support for obesity-related policies. However, we found that participants who read one of the messages containing information about the environment’s influence on obesity (Group 2) were more likely to believe that the environment influenced human behaviour than those in the control group. Also, those who read the second obesity message (Group 3) were less likely to believe that the environment influenced obesity than those in the control group. Despite changes in causal beliefs from those who read these two messages, there was no subsequent change in support for policies among these participants. These findings suggest that messages designed to induce the belief about the environment’s influence on both obesity-related behaviours and human behaviour in general do not directly influence attitudes toward obesity policies.
The lack of evidence for an effect of any of the messages on policy support was unexpected as the message used in the Group 2 had previously been found to increase public support (Pearl & Lebowitz, 2014). Other studies using similar interventions also showed changes in attitudes toward obesity policies and beliefs about the causes of obesity (Ortiz et al., 2016). We offer two possible explanations about the conflicting results between our study and those of two others that reported statistically significant effects (Ortiz et al., 2016; Pearl & Lebowitz, 2014). The first explanation concerns cultural differences. Studies conducted by (Ortiz et al., 2016; Pearl & Lebowitz, 2014) were both conducted in the USA. The current study was conducted in Great Britain. There is some evidence that US populations are less supportive of regulation to change health-related behaviour than are those in Great Britain which may affect the sensitivity of the two populations to messages targeting these attitudes (Ipsos MORI, 2012; Petrescu, Hollands, Couturier, Ng, & Marteau, 2016). Second, there may be other differences in sample characteristics that affected responses to the interventions. The sample used in Pearl and Lebowitz (2014) consisted solely of people who were overweight or obese. It is possible that people who are already overweight or obese may be more likely to revise their beliefs about the causes of obesity when presented with information about the environmental influence on obesity-related behaviours. However, this hypothesis remains untested.
Study 2
The results of Study 1 were originally intended to inform which interventions to use in Study 2. However, as none of the four interventions changed the target belief in the hypothesised direction and none changed public support, we decided to re-examine the central claim of this research, which we assumed to be true prior to conducting Study 1. The revised aim of Study 2 was to determine if we could replicate previously published effects on the communication of obesity attribution messages and support for obesity policies. To do this we decided to use two interventions used in two previously published studies reporting effects of communicating information about the environment’s influence on support for obesity policies (Ortiz et al., 2016; Pearl & Lebowitz, 2014). We decided to reuse the intervention from Pearl and Lebowitz (2014) that we tested in Study 1, but we removed the images to ensure that the presentation of the message was identical to the original study. The second message that we selected to test also successfully changed beliefs and attitudes in its original study (Ortiz et al., 2016). A further goal was to test two explanations for the conflicting results between Study 1 and previous research: differences in nationality and BMI. To do this we tested for interactions between intervention, country (England vs USA), and BMI. Based on prior research, Study 2 was designed to test three pre-registered hypotheses:
I Communicating messages that attribute obesity to the environment will (a) increase support for obesity prevention policies and (b) strengthen the belief that the environment causes obesity
II These effects will be greater amongst:
- participants from the USA
- participants who are obese or overweight
III Participants from England will report greater levels of support for obesity prevention policies and will be more likely to believe that the environment causes obesity.
Method
This study was pre-registered with the Open Science Framework (DOI: https://osf.io/juemn/?view_only=6f26ee717e254c168172c60de5e912e0). There was one deviation from the registered protocol. The criteria for significance was changed as a principal components analysis (PCA) suggested a two-factor solution for our primary outcome. This change is described in the analyses section below. Supporting data can also be found on the OSF (https://osf.io/qkgdw/?view_only=322f2eab83a945bc99a487aeec1841ff).
Participants
Two nationally representative samples from England (n = 1397) and from the USA (n = 1315) were recruited via YouGov’s existing online panels. The recruitment method used quotas including age, gender, and education (for both countries); social grade, region, political attention, voting in the 2017 General Election and 2016 EU referendum race (for England only); and race, voter registration, and voting in the 2016 Presidential Election (for the USA only). This sample size ensured similar group size from a comparable study (Ortiz et al., 2016) and approximately 30 times greater group size from another (Pearl & Lebowitz, 2014). A power calculation suggested that the current sample size would provide 80% power to detect small effects between two groups (d = .14) after a Bonferroni adjustment (α = .025) and when combining the two samples. Data were collected between 10th and 13th December 2018. After applying weights, the mean age = 48.36 (SD = 17.01) and 51.5% were female for the English sample and mean age = 47.31 (SD = 17.69) and 51.4% were female for the USA sample. See Supplemental Table S3 and S4 for the full demographic characteristics of the sample.
Design
We conducted an online between-subjects experiment, in which participants were randomly allocated to one of three groups (see Box 1) differing in their exposure to messages about the environment’s influence on obesity.
Group 1: Control group: received no message.
Group 2: Obesity (a) Availability and cost: received a message that highlighted the role of food availability, cost, advertising, and portion size (Pearl & Lebowitz, 2014).
Group 3: Obesity (c) Advertising and placement: received a message that highlighted the role of food advertising and placement of unhealthy foods in supermarkets (Ortiz et al., 2016).
The randomisation was conducted using the research agency’s software. Participants completed a short questionnaire after receiving the interventions. The study was conducted simultaneously in England and the USA.
[PLACE BOX 1 HERE]
The interventions
Two messages were taken from previous studies. These messages below highlight several aspects of the environment that have been shown to influence obesity: cost, availability, portion size, placement, and marketing (Ejlerskov et al., 2018; Kirk et al., 2010; Swinburn et al., 2011). The only changes made were to the country name used (to match this to the two countries in which the current study was taking place).
Group 2: Obesity (a) (Pearl & Lebowitz, 2014): “Everyone makes choices about what they eat, but the food environment influences what choices are available. Currently, highly processed foods that are high in sugar and fat are easily available and much cheaper than healthier foods such as fruits and vegetables. There are also parts of [America/England] in which there is limited access to grocery stores and fresh foods, and high availability of fast food restaurants and convenience stores that sell less healthy food. Restaurant portion sizes have increased in recent years, leading people to eat more food overall, and research has indicated that food advertisements and marketing increases consumption of unhealthy foods. Therefore, aspects of the food environment play a role in causing obesity.”
Group 3: Obesity (c) (Ortiz et al., 2016): “Lately there has been a lot of talk about the factors that influence food choices in [America/England]. For example, food advertising can lead to the selection of unhealthy food and beverages. Certain food additives, such as extra salt, sugar, and caffeine, can also increase the desire for unhealthy food. And the placement of snack food and sugary beverages at checkout counters, especially in non-grocery retail stores, can often result in unintended food purchases and overeating. Consumers should be able to make their own dietary choices. But they also need to be free from the influence of heavy advertising, exposures to habit forming food ingredients, and invasive food product placement.”
Measures
Primary outcome(s). Acceptability of seven policies, randomly ordered, was assessed using one response item for each (Reynolds et al., 2018): “Do you support or oppose the new policy?” rated on a seven-point scale (1 = Strongly oppose; 7 = Strongly support). These seven policies were: a 20% tax on confectionary; reduction in the size of unhealthy snack foods; banning advertising for unhealthy foods during children’s television; a policy to increase the availability of healthy foods in worksites, schools, and hospitals; a limit on the maximum size of sugar-sweetened beverages in fast food restaurants; calorie labels on restaurant menus; and a ban on unhealthy snack foods in schools. We used a more comprehensive set of policies in Study 2 to match the policies assessed in the studies from which we sourced the interventions (Ortiz et al., 2016; Pearl & Lebowitz, 2014). These seven items were converted into two outcomes: support for encouraging policies and support for discouraging policies (see Analyses section).
Causal beliefs (manipulation checks). The belief that obesity is caused by the food environment, genetics, and a lack of willpower were each measured with two response items (r = .73; .70; .77, respectively) (Reynolds et al., 2018): “[Cause] is to blame for Obesity” and “People are obese because of [Cause]”. Each was rated on a seven-point scale (1 = Strongly disagree; 7 = Strongly agree). These items were presented in counter-balanced order.
Other variables. BMI was calculated from self-reported height and weight. The research agency provided demographic data including age, gender, socio-economic status (Lambert & Moy, 2013), education (Adapted from: Clarke et al., 2003), and region. For the English sample, educational achievement was recoded into three categories: low education (no education, GCSEs or similar); medium education (A-levels, non-degree teaching qualifications, or similar); and, high education (degree awards or higher). Socio-economic status was also recoded into three categories: low (DE), medium (C1C2), and high (AB). For these transformations see Methods section reported in Study 1.
For the USA sample, educational achievement was recoded into four categories: low education (no high school, high school graduates), medium-low education (some college, 2 year college), medium-high education (4 year college graduate), and high education (post-graduate degree).
Analyses
A PCA suggested a two-factor solution for the policy support items that explained 67% of the variance: (1) support for policies to discourage consumption of unhealthy foods and drinks (Discouraging policies); and (2) support for policies to encourage consumption of healthy foods (Encouraging policies). Support for Discouraging policies ranged from -2.07 (strongly oppose) to 2.13 (strongly support) and support for Encouraging policies ranged from -2.68 (strongly oppose) to 1.68 (strongly support). See Table S6 for factor loadings.
YouGov provide sampling weights to improve the representativeness of the sample, which were applied for all analyses in this study. The main analyses used hierarchical OLS regressions to test the main effects and interactions between country, intervention group, and BMI on support for policies to tackle obesity and beliefs about the causes of obesity. The pre-registered criteria for significance was set at α = .05 for the primary outcome (policy support), and α = .05/4 = .0125 for the three secondary outcomes, after applying a Bonferonni multiplicity adjustment. However, as the PCA suggested a two-factor solution for policy support, this was changed to α = .025 for the co-primary outcomes and α = .01 for the three secondary outcomes.
Potential confounding variables (SES, education, gender, age and region) were compared across groups using a percentage method to assess chance imbalances following randomisation (Moher et al., 2010). Several chance imbalances above 5% points were identified for all five of these variables across the groups and therefore gender and age were included as covariates in the models as a sensitivity analysis. It did not make sense to control for SES, region, and education as these were measured with different items across the English and USA samples. Sensitivity analyses were conducted in which covariates were not included into the models, to determine whether the main pattern of results would change (see supplement). Model diagnostics were examined and were acceptable.
Outliers (±3SDs from the mean) on continuous variables were removed. 47 outliers were removed (2%) from the Encouraging policies variable, and 50 were removed from the BMI variable (2%). There were no other outliers. Sensitivity analyses were conducted in which outliers were not excluded, to determine whether the main pattern of results would change. Data in Figure 1 and 2 were dichotomised (1-4 = 0, 4.01-7 =1) to indicate the proportions of participants that found each policy acceptable (i.e., those rating above the scale midpoint). These dichotomised data are provided to aid interpretation, and are not used in any inferential analyses. Cohen’s d statistics are covariate adjusted.
In an exploratory analysis, we used the two one-sided tests (TOST) procedure (Lakens, Scheel, & Isager, 2018), to evaluate whether our results were equivalent to those reported in the two studies that we were aiming to replicate. Equivalence bounds were set as ΔL= -.10 and ΔU= .10 given the size of effects in similar fields. This provided two p-values by using t-tests of both below the lower bound (Δ<ΔL, the lower tail p-value) and above the upper bound (ΔU>Δ, the upper tail p-value) with adjusted degrees of freedom using the Sattherwaite method (Lakens, 2017). Equivalence is shown if the largest p-value is significant (i.e. data is consistent with being within the two boundaries), and therefore only one p-value requires reporting. The inference criterion was set at α = .025 in line with the co-primary analyses.
Results
Policy support
There were no statistically significant effects of the interventions on the primary outcomes of support for policies. There was no effect of the Group 2 message when compared to the control group on support for encouraging policies, B = 0.05, 95% CIs [-0.03, 0.14], p = .215, d = .06, or for discouraging policies, B = -0.01, 95% CIs [-0.10, 0.08], p = .823, d = -.01 (see Table 2). The equivalence tests were non-significant for both encouraging policies, t(31.22) = 4.35, p = 1.00, and discouraging policies, t(30.91) = 3.94, p = 1.00. This suggests that neither of these two analyses was statistically equivalent to the results of the original study based on equivalence bounds of ΔL= -.10 and ΔU= .10 (Pearl & Lebowitz, 2014) where the effect size was larger d = .94.
There was also no effect of the Group 3 message when compared to the control group on support for encouraging policies, B = 0.06, 95% CIs [-0.03, 0.14], p = .171, d = .06, or for discouraging policies, B = -0.02, 95% CIs [-0.10, 0.07], p = .723, d = -.02 (see Table 2). The equivalence tests were non-significant both for encouraging policies, t(785.91) = 0.42, p = .663, and discouraging policies, t(737.47) = -0.42, p = .337. This suggests that neither of these two analyses was equivalent to the results of the original study based on equivalence bounds of ΔL= -.10 and ΔU= .10 (Ortiz et al., 2016) where the effect size was larger d = .14.
There were also no statistically significant interaction effects on either of these two policy support outcomes (see Table S7). This includes two-way and three-way interactions between intervention group, country, and/or BMI.
There was a statistically significant main effect of country on policy support. Participants from the USA reported less support for both sets of obesity prevention policies compared to English participants: Encouraging policies, B = -0.15, 05% CIs [-0.22, -0.08], p < .001, d = .17; Discouraging policies, B = -0.46, 95% CIs [-0.54, -0.39], p < .001, d = .48, on the 1-7 rating scale.
[INSERT TABLE 2 HERE]
[INSERT FIGURE 1 HERE]
Beliefs about the causes of obesity (manipulation checks)
There was no statistically significant effect of the interventions on the belief that the environment influences obesity, the belief that genetics influence obesity, or the belief that a lack of willpower influences obesity (see Table 3). There were also no statistically significant interaction effects on any of these three causal belief outcomes (see Table S8). This includes two-way and three-way interactions between intervention group, country, and/or BMI.
There was a statistically significant effect of country on two out of three causal beliefs. American participants were more likely than English participants to believe that genetics influences obesity, B = 0.61, 95% CIs [0.51, 0.71], p < .001, d = .46, whereas English participants were more likely than USA participants to believe that a lack of willpower influences obesity, B = -0.22, 95% CIs [-0.33, -0.11], p < .001, d = .15.