This study aimed to investigate the effectiveness of ProYouth OZ, an indicated Internet-based prevention program for young adults at risk of eating disorders. Unfortunately, the small sample size and thus a lack of power precluded the statistical examination of the primary research questions. However, recruitment challenges, low adherence, and high rates of attrition commonly compromise randomised controlled trials (67, 68) and in this respect our results are not unique. Thus, it has been argued that it is important to report “failed” trials as well as trials with negative or null results (46, 69). Accordingly, we first briefly discuss the trends in the limited available outcome data we obtained and then address recruitment, adherence and attrition challenges and lessons learned from the trial with the aim of informing future research protocols in the area of Internet-based programs for eating disorders.
Study Outcomes
As noted above, it was not possible to conduct group comparisons, nor to test for any pre- to post-intervention changes between the groups over time. We did, however, examine the individual outcome profiles of participants in each intervention for whom pre- and post-intervention data were available. Whereas most of the participants in the ProYouth OZ condition (i.e., 5 out of 6) failed to show any change in symptoms, both participants in the ProYouth OZ Peers condition reported decreased eating disorder symptoms immediately after the intervention; a pattern of findings that suggests that the role of peer support may be promising and warrants further investigation.
Challenges
In this section we focus on the challenges we encountered in the recruitment of eligible participants, program engagement, and attrition (i.e., failure to complete assessments), and consider our findings in the context of the extant literature on Internet-based indicated eating disorder prevention programs.
Recruitment of Eligible Participants
The current study sought to include individuals with elevated weight and shape concerns and/or subthreshold eating disorder symptoms and exclude those with a diagnosable disorder by employing a variety of recruitment strategies and screening interested individuals for their eligibility using a brief online screening survey. While our recruitment efforts attracted a large number of individuals (almost 800), of whom more than 50% completed the relevant screening questions, less than 18% of those interested in the study were eligible to participate in ProYouth OZ. The majority of interested individuals (51%) were excluded due to their severe eating disorder pathology (e.g., binge eating, vomiting, use of laxatives, excessive exercise, and dieting) while only a small percentage were excluded due to an absence of symptoms (15%). Based on these results, it would have been necessary to screen almost 3,000 individuals in order to achieve the target sample size of 200 with the described recruitment strategies.
The low level of eligibility identified in the current study is consistent with two other indicated trials of StudentBodies (29, 32) which reported that less than 8% of screened participants were eligible. Higher eligibility rates were reported by Stice et al. (33) who included 42% of screened participants in their trial of The Body Project. Importantly, other trials have failed to report the number of initially screened participants (30, 31). While findings suggest that the proportion of eligible participants varies greatly between studies, it is notable that, in contrast to the current study, those trials providing relevant data all reported a relatively low level of exclusion due to an eating disorder. This may reflect a difference in the nature of the population recruited using the present methodology (see below). In addition, in contrast to the current study, the other trials employed either face-to-face or telephone diagnostic interviews (29, 32) or an online validated structured self-report measure (33) to exclude participants with an eating disorder. Thus, it is possible that the present study yielded a high number of false positives (i.e., overdiagnosed eating disorders). Nevertheless, based on the current and previous research, it is likely that indicated Internet-based intervention outcome studies require the implementation of a large scale screening strategy (i.e., 1,500-4,500 participants) (29, 32, 33). The following section considers in more detail the effects of different recruitment strategies on reaching this population and identifying eligible participants.
Recruitment Strategies. In the current trial, more than half of the screened participants were recruited online from the community (i.e., social media, Google, and websites of mental health organisations), whereas only one quarter were recruited in a university setting. In contrast, other indicated Internet-based prevention trials recruited mainly from universities, resulting in a larger proportion of university students (e.g., 29, 32; 84%-95% students compared to 66% in the current study). The online recruitment strategy in the current study attracted the majority of eligible participants (60%). Importantly, despite the use of the same advertisement text in the online and offline recruitment strategies, individuals who were recruited online versus in face-to-face reported significantly higher weight and shape concerns and rates of eating disorder diagnosis than those recruited through offline strategies. Indeed, Bauer et al. (70) reported a similar trend. They argue that participants recruited online are subject to greater self-selection; thus, it may be that individuals who showed interest in ProYouth OZ online were more actively seeking eating disorder related information and support.
As noted above, in the current trial more than 50% of screened participants were excluded due to severe eating disorder symptoms, in marked contrast to less than 15% in other indicated prevention trials (29, 31, 33). It is likely that the larger proportion of interested individuals with severe eating disorder symptoms is attributable at least in part to extending recruitment for the current study to websites of eating disorder organisations (e.g., National Eating Disorder Collaboration, Butterfly Foundation) and mental health services (e.g., University Counselling Centre, Headspace). Future studies should report details on recruitment pathways and advertising materials, screening processes, and associations between recruitment strategies and eligibility to identify the recruitment strategies that most successfully attract eligible participants for indicated prevention programs.
Screening and Eligibility Criteria. Studies vary greatly with respect to how participants are screened (e.g., online, face-to-face) and the inclusion and exclusion criteria that are used. While most indicated Internet-based eating disorder prevention trials have used an online screening survey to identify those at risk (29, 30, 33), some also employed an additional telephone or face-to-face assessment process (29-31). Such interview-based, person-to-person assessments have the advantage of reliably excluding those with full syndrome eating disorders and may increase participant commitment; however, the use of face-to-face contact also mitigates some of the key benefits of Internet-based interventions including accessibility, anonymity, and relatively low intervention cost. More importantly, person-to-person assessments are less feasible in real-world settings when the aim is to broadly disseminate programs to a wider population.
During the screening procedure, eligibility criteria were used to identify participants at risk for an eating disorder and exclude both individuals with no and overly severe eating disorder symptoms. Consistent with other studies (29, 31-33), the current trial excluded participants who met criteria for a diagnosable eating disorder. Selecting inclusion and exclusion criteria to identify participants at risk is complex and has been applied differently across studies. While the current study used a combination of moderate to high weight and shape concerns (i.e., WCS > 57) as well as dysfunctional eating behaviours to include participants, other studies have broadened their criteria to include individuals with lower levels of weight concerns (e.g., WCS > 47; 29, 30) or merely body dissatisfaction without any additional behavioural criteria (33). It has been suggested that highly specific inclusion criteria may result in difficulties identifying suitable participants, especially for low prevalence conditions and hard-to-reach populations (67). There is an urgent need for further research to determine which eligibility criteria result in an at-risk sample that is most likely to access and benefit from these programs.
Program Engagement
In the field of Internet-based interventions, adherence is often described as the extent to which users actively engage with a program as intended or “prescribed” (71, 72). However, the intended or optimal amount of use for a program to be effective is not always clearly defined or operationalised. Only if the intended use of the program is specified a priori and measured accordingly throughout the intervention can the extent of adherence be established (73). In contrast to other indicated prevention programs including StudentBodies and The Body Project, ProYouth OZ uses a flexible approach and did not explicitly instruct participants to engage with psychoeducational information. Participants were encouraged to complete weekly self-monitoring and in the ProYouth OZ Peers condition, weekly chat sessions. We therefore defined engagement with respect to these functions. In fact, one fifth of participants (20.6%) randomised to one of the two intervention conditions never accessed the program (i.e., did not log in or complete a monitoring assessment). This finding is similar to that reported in other indicated trials of the more structured programs StudentBodies and The Body Project where 5%-19% of participants never accessed the intervention (30-33). Over 73% of participants from the two intervention groups completed at least one self-monitoring and on average these participants completed an average of 42% of all six scheduled monitoring assessments. This is not surprising considering the unstructured nature of ProYouth OZ. However, in the four indicated trials of StudentBodies, participants completed between 12%-68% of required monitoring assessments (29-32), suggesting that more structured interventions are not invariably associated with superior adherence. Unfortunately, the direct benefits of regular self-monitoring on program outcomes remains unclear due to the lack of available data in the current study. It may be that participants benefited sufficiently from the feedback they received after completing, for example, the first monitoring assessment; or it may be that participants did not perceive self-monitoring to be useful, beneficial or engaging, and accordingly ceased using this component of the program. Suggestive of the latter interpretation, results from the ProYouth program in Germany suggest that participants prefer to use more interactive components of the intervention (52, 70), such as the forum or chat sessions (74).
Given that the study was designed to examine the impact of online peer support, engagement with the chat sessions was of particular interest. Almost 60% of participants from the ProYouth OZ Peers group attended at least one chat session. However, engagement with this component of the program decreased over time with only 20% of the participants attending more than two of the six scheduled sessions. It is noteworthy that participants who attended chat sessions were generally satisfied with this program component, with qualitative feedback indicating that they valued the safe and supportive environment, the interesting and relatable content, and the benefits gained from the discussion with others who were experiencing similar problems. Some participants particularly highlighted the positive interaction with the peer facilitator and the moderator.
In summary, engagement with ProYouth OZ was low which is not uncommon in Internet-based mental health preventive intervention trials. Further, it is likely that engagement with such programs will be even lower when disseminated and used in a more naturalistic setting (24, 48).
An understanding of the reasons for low adherence has the potential to increase the effectiveness of preventive interventions. In a study of the Healthy Body Image Internet-based program, the strongest reported barrier for program engagement across at-risk and symptomatic groups was low perceived need for help (75). It may be that participants in the current study did not consider their problems to be severe enough, which is a common barrier in this group (12). Interestingly, however, in the current study there were no significant differences in perceived need between those who did and those who did not access the intervention.
Although Internet-based programs can overcome some of the practical barriers of face-to-face preventive approaches (e.g., flexible delivery independent of time and place), the time required to participate in these programs can constitute a substantial barrier (76), especially if the perceived need for help is low and positive outcomes are not immediate. For example, in the current trial, some participants reported that they were unable to attend chat sessions due to other commitments. On the other hand, feedback from ProYouth OZ Peers positively highlighted the helpfulness of peer videos in recovery. These findings suggest that young adults may respond better to brief, engaging, visual (as opposed to text) content that they can use independent of specifically scheduled sessions.
It has been suggested in the broader Internet-based prevention field that those with higher levels of eating disorder symptoms are more likely to participate in online programs (70, 74, 75, 77, 78). However, the current study did not find an association between eating disorder symptoms and access to the program. Other studies investigating indicated prevention programs have reported mixed findings. Whereas participants with higher eating disorder symptoms were more likely to drop out of the eBody Project intervention (33), trials of the StudentBodies program either did not find associations between eating disorder symptoms and program usage (30) or demonstrated that higher eating pathology was associated with higher adherence (32). These inconsistent findings raise the possibility that different programs are suitable for different individuals along the continuum of eating disorder symptoms, and is a question that requires further investigation.
Assessment Completion
Failure to complete assessments (i.e., attrition) has been reported as a common issue in Internet-based interventions (79). Assessment completion in the current trial was low, with only 30% of participants completing post-intervention measures (i.e., 24% from the intervention groups vs. 44% from the control group). Thus, attrition rates were higher compared to indicated trials of StudentBodies and The Body Project, where assessment completion ranged from 63-91% at post-intervention (29-31, 33). Notably, most of the latter trials assessed participants face-to-face or on the telephone at all time points which may have attracted more motivated participants than online assessments. The resulting lack of personal contact with the research team may also have negatively impacted on assessment completion.
Further, in other studies with higher completion rates, participants were often provided with incentives for completing study assessments (31, 33). Monetary incentives have been found to increase enrolment and assessment completion in online interventions (80, 81). In fact, a previous meta-analysis of the findings from StudentBodies has highlighted that the provision of incentives may be essential for assessment completion (82). In contrast, in order to model real-world conditions, the current study did not offer monetary incentives for completing assessments. The absence of such payments in combination with a lack of face-to-face assessments may account for the low completion of assessments (83). Importantly, however, these expensive strategies raise questions about the external validity of findings.
It may be that other factors including the timing or length of assessments (duration of 30 minutes) further impacted completion rates. A less ambitious set of aims and a corresponding reduction in outcome measures might have increased completion rates and the ecological validity of the study given that such outcomes would not normally be incorporated into a real-world intervention.
Future Research and Implications
Although no conclusions can be drawn about the effectiveness of ProYouth OZ and the additional effect of online peer support, findings from the current study have important implications for future developments and research. It has been noted previously that successful eating disorder prevention requires scalable interventions. However, consistent with existing evidence, our findings demonstrate that young adults at risk for eating disorders are an extremely hard to reach and engage population (32, 70). Given that prevention programs have maximal potential to reduce disease burden (84), it is essential to identify the most effective strategies to reach and engage these individuals. Thus, future studies should extend emerging research to identify optimal strategies for reaching at-risk populations (e.g., online vs offline) as this will inform important future dissemination efforts (32, 70). Efficacious prevention programs will not benefit individuals in real-world settings if they fail to reach the broader target populations for whom the programs were designed. As argued previously (24), there is also an urgent need to understand who and under what conditions young people engage or fail to engage in Internet-based prevention programs for eating disorders. For example, future research should systematically measure baseline perceived need and motivation to change to better understand if and how these factors impact uptake and engagement. Relatedly, future research should also investigate if, as proposed by Wilksch et al. (78), brief motivation enhancement techniques pre-intervention increase the likelihood of participants engaging with eating disorder prevention programs.
There is also a need to examine other potential interventions for increasing engagement such as the use of co-design (Thabrew et al., 2018) and involving peers in the promotion of such programs in order to enhance a sense of belonging and increase perceived need for the intervention. Since videos of young adults with a lived experience were positively regarded by participants in the current trial, future studies could examine the potential of peer support to increase engagement with the intervention. The use of gamification in such programs might also impact on engagement in young people at risk of eating disorder (85, 86), especially as individuals with these disorders are often highly perfectionistic and achievement focussed (87, 88). In the broader field of behavioural economics, financial incentives have been found to be a useful strategy to promote healthy and discourage unhealthy behaviours. A recent conceptual framework of engagement with digital behaviour change interventions hypothesised an association between rewards, motivation, and engagement (89, 90). Therefore, it may be worthwhile to implement incentives in real-world settings since reducing unhealthy disordered eating behaviour is highly desirable from a public policy perspective. However, carefully designed cost-benefit studies would need to establish the sustainability of such approaches.
A final consideration concerns the nature of the unmet need in the community (as demonstrated by interested individuals) as it relates to the programs offered. While online recruitment in the current study appeared to attract more participants with severe eating disorder symptoms, ProYouth OZ was not designed for those with a diagnosable disorder. This finding is not unique (70, 78, 91) and researchers have argued that that in real-world settings, such interventions might be mainly accessed by participants who experience greater eating disorder symptomology as they may be more motivated to change their behaviour (78, 92-94). However, the majority of these programs were developed as prevention programs (universal, selective, or indicated) rather than treatment programs. While participation in such prevention programs may be helpful for some individuals with an eating disorder (78), rigorous research trials to investigate the benefits and risks for such individuals are currently lacking. It might be more appropriate to develop new and innovative interventions designed specifically for individuals with eating disorders who choose to access help online, but may not be ready or able to seek professional face-to-face treatment due to a variety of barriers. Based on a number of studies in different countries it would appear that this population experiences the greatest level of unmet need (70, 77, 78). Such programs could provide initial self-help strategies and information designed to facilitate help-seeking, to reduce the significant treatment gap in the field of eating disorders and consequently reduce the disease burden (84).
