The Climate Schools Combined Study
The Climate Schools Combined (CSC) study was a multisite, cluster-randomised controlled trial of an online universal prevention program for mental health and substance use, delivered between 2014 to 2016 in 71 secondary schools to 6,386 students across three Australian states; New South Wales (NSW), Western Australia (WA), and Queensland (QLD). Schools were randomly assigned to one of four intervention groups; (1) Climate Schools Mental Health (mental health prevention modules), (2) Climate Schools Substance Use (substance use prevention modules), (3) Climate Schools Combined (both mental health and substance use prevention modules) and (4) control (health education as usual). The study aimed to evaluate the effectiveness of a combined approach to mental health and substance use prevention, specifically, depression and anxiety, and alcohol and cannabis use. Participants were recruited in Year 8 (NSW and WA) and Year 9 (QLD) when they were between the ages of 13–14 years at baseline and with 6 initial follow-up surveys planned over 2.5 years. At the final wave of data collection in 2016, participants were asked to provide consent to be contacted for a long-term follow up study.
The Climate Schools Combined Longitudinal Study
The Climate Schools Combined Long term follow-up (CSCL) study was a follow up of the CSC study conducted between August 2018 and July 2021. The follow up study involved two additional online survey occasions when participants were 18–19 years old, and 20–21 years old, approximately 5- and 6-years post baseline. All participants who had not declined interest in the long-term follow up study at the final wave of data collection, or who had not withdrawn consent since, were eligible to be contacted (n = 5,417). The first longitudinal wave of data collection occurred between August 2018 and December 2019 in the participants’ first year after graduating high school. The second longitudinal wave of data collection occurred between July 2020 and July 2021. Further details of the CSC study and CSCL study have been described elsewhere (25).
CSCL Study Participant Retention Methods
Retention methods for the long-term follow-up period are shown in (Fig. 1). The research team used participant locator information collected during the final wave of the CSC study to contact participants for the CSCL. For all participants, mandatory survey fields in the final CSC wave included: first name; last name; and high school. Optional survey fields meant that further locator information was only given by some participants: home address; email; secondary email; mobile; home phone; and Facebook (FB) username.
Participants were first contacted via automated emails and text messages sent from the CSC study website. Emails and texts were sent simultaneously on three separate occasions at 1, 3, and 7 days after the survey opened. The research team then attempted to contact participants via FB Messenger, through study-based FB accounts that included the team members name and some indication of the CSC study, for example, ‘Jennifer CSC Study’ or ‘Jennifer Climate’. Other identifiable factors including the research organisation, role, and a cover photo including the CSC logo were added to the FB profiles to increase legitimacy. Participants’ first and last names were searched on FB for any identifying information, such as location, date of birth, and school. Once a potential participant’s FB page was identified, the research team would investigate profiles for further verifying factors including photos, mutual friends, likes, and comments. Once a participant had been verified, they would be contacted via FB Messenger regarding their previous participation in the CSC study. If they recalled participation in the study, they would be invited to complete the survey. Non-respondents on FB were sent two more messages one week apart.
Participants who did not respond or failed to complete the survey after three FB message attempts were then contacted by telephone and text message. Three contact attempts were made via phone calls, at weekly intervals, calling both a participant’s home phone and mobile at each contact attempt. Participants who did not answer a call were left a voice message and a text message. On the occasion that phone calls were answered by family members such as parents or siblings, participant contact details would be requested. If they were not willing to do so, the research team would request that the study information be passed on to the participant. For both FB messages and phone calls, the research team followed a preapproved script. Once a participant indicated interest in the survey, or if they had partially completed the survey, with ethics approval the research team were permitted to contact the participant until the survey was completed or the participant declined.
Alumni letters were then sent to schools for those participants who had no valid contact information for the above methods. Schools would then send the prepaid letters to the last documented postal address on file for the participant inviting them to take part in the study.
Finally, participants with valid contact information, who had not yet completed the survey and had not declined participation, were sent three final reminder emails at 7, 3 and 1 day before survey closure. Once a participant had completed the survey or declined participation, contact attempts would cease.
The number of completions per contact method can be found in Fig. 2.
The second longitudinal wave of data collection followed the same recruitment methods, differing slightly with initial emails sent through the research team’s organisational email accounts and final reminders sent via both text messages and email to increase participant retention. Alumni letters were deemed too resource intensive to prepare and distribute while only yielding small participant re-engagement numbers, so the decision was made to remove this from the follow-up procedure. Participants were reimbursed for their time with an online ‘Prezzee’ prepaid gift card worth $A20 for the first data collection wave, increasing to $A30 for the second. On completion of the survey, participants were also given the opportunity to ‘refer a friend’ to complete the survey for a chance to go into a draw to win a ‘Prezzee’ gift card worth $250, with conditions that the referred friend was a CSC participant who hadn’t already completed the survey.
Measures
Demographics
Participants’ age at baseline was measured in years and months at the time of the survey. Baseline gender was measured as a binary variable coded as; Male = 0 and Female = 1. Country of birth was measured with a free text box and recoded as the binary variable born in Australia (yes/no). Truancy was measured as how many days the participant reported having off school in the last 12 months without parental permission. This was measured as a categorical variable with 5 levels: 0 days; 1–2 days; 3–5 days; 6–10 days; and, 10 + days. Grades were measured by asking participants “what grades do you usually get in school”. This was measured as a categorical variable with 6 levels: 49% and below; 50–59%; 60–69%; 70–79%; 80–89%; and, 90–100%.
Mental health
Mental health knowledge was assessed through a 13-item questionnaire measuring participants’ knowledge about anxiety and depression with scores ranging from 0–13. Higher scores indicate greater mental health knowledge at baseline.
Depression symptoms were measured over the previous two weeks using the 8-item Patient Health Questionnaire (PHQ-8) with scores ranging from 0–24 (26). Scores of 10 or more were considered to indicate probable depressive disorder.
Anxiety symptoms were measured using the 7-item Generalised Anxiety Disorder (GAD-7) screener with scores ranging from 0–21 (27). This variable was measured as continuous with scores 10 or greater indicating a probable anxiety disorder.
The Substance Use Risk Profile Scale (SURPS) is a 23-item questionnaire used to assess personality risk factors for substance use across four dimensions; anxiety sensitivity (score 0–20), impulsivity (0–20), sensation seeking (score 0–24) and hopelessness (score 0–28) (28).
The Strengths and Difficulty Questionnaire (SDQ) is a 25-item brief behavioural screening questionnaire for 3–16-year-olds. The survey asks about 25 negative and positive attributes which are assessed on a 3-point Likert scale indicating how each attribute applies to the participant. The screener is divided into 5 subscales consisting of 5 items each; emotional symptoms, conduct problems, hyperactivity/inattention, peer relationship problems, and prosocial behaviour (29).
Alcohol use
Alcohol use variables were measured using an adapted version of the Patterns of Alcohol index (30). Frequency of alcohol use was assessed by asking participants how often they consumed a full standard alcoholic drink in the past 6 months. Heavy episodic drinking was assessed by asking participants how often they consumed at least five or more standard drinks on a single occasion in the previous 6 months. Both alcohol variables were rated on a 5-point scale from “never” to “daily or almost daily”. Participants were provided a standard drinks chart to assist reporting. Responses for heavy episodic drinking were converted to a dichotomous outcome: any heavy episodic (binge) drinking at least monthly versus less than monthly in the past 6 months.
Statistical Analysis
All data were analysed using R statistical software package version 4.2.2. To understand which baseline participant characteristics predicted retention in long term follow-up, a series of logistic regression models were used. Predictor variables were selected from previous literature where possible as well as including primary outcomes from the CSC trial as registered on the Australian and New Zealand Clinical Trials Registry (ANZCTRN: 12613000723785). Baseline demographic variables (age, gender, country of birth, truancy, grades), intervention group, alcohol use (binge drinking and alcohol use frequency) as well as sum scores from the PHQ-8, GAD-7, mental health knowledge questionnaire, the SURPS and SDQ were entered as predictors in a series of univariate-level logistic models, with a single observation per participant. The dependent or outcome variable was long term retention with the survey and was coded as 1 if participants were retained at either the 5-year or 6-year follow-up surveys. All variables which passed the threshold in the univariable analysis (p < 0.05) were entered into a multivariable logistic regression model. Clustering within schools was modelled using cluster robust standard errors in the multivariable model. Variance Inflation Factors (VIFs) were used to assess collinearity between variables in the multivariate model, with a VIF > 10 indicating high collinearity. There was no collinearity between variables.