Study Design
A cluster randomised controlled trial [RCT] among public mental health and/or alcohol and other drug use services within 15 LHDs across New South Wales [NSW], Australia will be conducted (See Fig. 2). Cluster randomisation at the LHD level will be used to avoid contamination through communication between clinicians across services. One mental health and one alcohol and other drug use service, or one combined service, will be recruited in each LHD. Each LHD will be randomised to receive one of two interventions for a period of 6 months: (1) a Direct-to-Consumer [DtC] Marketing Strategy, or (2) the DtC strategy in addition to an Integrated Translation and Engagement Model [ITEM] of implementation designed to engage ‘users’ (i.e., healthcare services and consumers) with eCliPSE. While the DtC marketing strategy and support offered as a part of ITEM will cease after the trial, participating services and individuals will have ongoing access to the eCliPSE website after participation and the trial has ceased.
Study Setting and Participants
The study will aim to recruit one mental health service and one alcohol and other drug use service, or one combined service, across all 15 NSW LHDs (i.e., maximum of 30 services in total). Expressions of interest will be sought from MHAOD use services for involvement in the trial. A minimum of 50 consumers per service catchment area (n = 1500 consumers in total) will be recruited to the eCliPSE website.
The eligibility of services to participate will depend on the service manager and clinicians’ willingness to participate in the proposed research project. Individual consumers will be eligible for inclusion if they are willing to participate in the research as outlined in the participant information statement [PIS] and are residing in a NSW LHD catchment area at the time of the study.
Prior to registration, individual participants will be explicitly advised via an online PIS prominently displayed in the eCliPSE website registration page that any data collected through eCliPSE will be used as part of a research project. Registration will imply consent to participate. The PIS will be able to be downloaded and contains instructions on how to withdraw from the study. Participants can withdraw their consent to participate in the study by contacting the research team. Participants who choose to withdraw can indicate whether they would like to have all of their data deleted, or whether data already provided can be retained and analysed for research purposes.
Five clinicians (and/or service managers) and five consumers per service will be recruited (n = 150 clinicians; n = 150 consumers in total) to participate in the service audits at baseline and 6-months later, to assess the capacity of the service to offer treatment for comorbid MHAOD use pre- and post-eCliPSE access. Clinicians and service managers will be invited to participate in semi-structured interviews. Clinicians and service managers who agree to participate will be provided with a PIS and asked to provide written consent. Clients attending the services will be invited to complete a brief online survey. These clients will be recruited by an appropriate service manager or clinician, via an email inviting them to complete a short survey about their experience of using the service. The email will include a PIS and a link to the survey. Completion of the survey will imply consent to participate. For more information, see Additional file 1 for copies of PIS and consent forms [CF].
In addition, 50 people with a lived experience of mental health problems and/or alcohol and other drug use (based on self-identification) will be recruited via social media networks (i.e., Instagram, Facebook, and LinkedIn) to participate in online co-design focus groups (5 sessions, n = 10 participants per session). These co-design focus groups will aim to explore the perceptions and interaction that participants have with digital and social media health content. Participants will be required to provide written and oral consent before and at the commencement of the online focus group. Following the planned focus groups, 500 people who have registered with the eCliPSE website will be recruited to participate in an online survey to investigate the factors that influence active engagement with, and sustained usage of, eCliPSE. Participants will be recruited from the trial’s social media platforms (i.e., eCliPSE Instagram and Facebook accounts) and registered users of eCliPSE website via a social media and email campaign. Members of the research team will email potential participants a PIS and a survey link. Consent to participating in the survey will be inferred when the participant opens the survey link.
[Insert Fig. 2 about here]
Randomisation
The unit of randomisation in this research will take place at the LHD level. Stratification by geographical area will be used to ensure the two conditions are evenly distributed across regional/remote and metropolitan LHDs. The allocation sequence will be generated and implemented by a statistician (MS) not involved in the recruitment of LHDs. Researchers, health service staff, and consumers will not be blinded to conditions. However, once assigned, researchers will avoid discussing other study arms to decrease awareness of the second study condition among participating MHAOD service staff and consumers. MHAOD health services allocated to either DtC or ITEM + DtC will receive separate versions of the PIS and CF that are relevant to their study condition.
Interventions
Direct-to-Consumer [DtC] Marketing Strategy
The DtC strategy aims to communicate the benefit, availability, and functionality of the eCliPSE website via a targeted social media campaign. This four-phase strategy aims to guide prospective users through the eCliPSE social media marketing funnel by leading them through the initial stages of awareness and recognition through to conversion and uptake of eCliPSE (i.e., sign up) and its resources (i.e., watch mindfulness self-help videos).
Phase 1 focuses on attracting prospective users to eCliPSE through the creation of high quality, targeted, and valuable social media content that functions to build awareness and knowledge of what eCliPSE has to offer people experiencing co-occurring MHAOD use. Initially, this activity will include the repurposing of existing eCliPSE website content pieces (i.e., factsheets and eBooks) as well as the development of new long-form (i.e., blog) content and social media marketing communication. Consequently, this phase will also include the development of social media eCliPSE brand pages on Facebook, Instagram, and Linktree (social media reference page). To ensure effective marketing content development and dissemination in this phase, focus-groups and an online survey conducted with individuals experiencing co-occurring MHAOD use will be undertaken in order to understand their social media characteristics and engagement motivations (19). These discussions are pivotal to assisting the marketing team in designing compelling and valuable social media content that functions to encourage prospective users to visit the eCliPSE website for further details. They will also assist in informing the search engine optimisation of the eCliPSE website and the inbound marketing strategies used in Phases 2 and 4. Planned focus-groups and online survey will occur concurrently with the RCT described here, with full details of methods to be used provided in Additional file 2.
In Phase 2, prospective users exhibit greater information search behaviours (20) and, as such, the emphasis of social media and the DtC strategy more broadly is to provide more detailed information that communicates eCliPSE as a solution to the challenges faced by individuals experiencing MHAOD use problems. Specifically, content shared in this phase works to build deeper trust and confidence in the capabilities and use of eCliPSE amongst prospective users, and in turn, facilitate users to engage with, and remember, eCliPSE. Content developed in Phase 1 will be used and repurposed in this phase to provide personalised communication to prospective users, by using: (i) strategically created keywords to improve the search engine ranking of eCliPSE, (ii) blogs and other short form content to educate and inform the target audience about eCliPSE, and (iii) Facebook and Instagram advertising to disseminate the content.
Through the development of relationships with prospective users in social media in Phase 1 and 2, engaging with users in Phase 3 will involve continued nurturing of relationships until the prospective users are ready to convert (i.e., sign up) with eCliPSE. Prospective users will be exposed to content developed in Phase 1 including testimonials, videos, interviews, and infographics. This content will provide education on the value of using eCliPSE, and calls-to-action will be integrated on social media to encourage prospective users to access gated information and programs that cannot otherwise be accessed on the website. Phase 3 will also develop lookalike, consumer audience groups on social media (matched on demographics, interests, and co-occurring MHAOD use, but outside the study LHDs) to generate prospective user leads, target and test the content for face validity and effectiveness.
In Phase 4, the focus will be on the use of remarketing campaigns with custom and organic user groups who have shown interest in eCliPSE through engaging with the previous parts of the funnel (i.e., website and resources) but are yet to convert and sign up. Monitoring of social media and Google analytics throughout Phases 1–3 will facilitate this process, ensuring that the social media advertising has received sufficient engagement and that the goals set out in the earlier stages of the social media marketing funnel have been met.
Integrated Translation and Engagement Model [ITEM]. Based on the latest evidence for effective implementation, the ITEM is designed to reflect the multidimensional, dynamic, and relational nature of digital health implementation. This four-phase model synthesises evidence on facilitators and barriers to translation and focuses on the various disorderly interactions between researchers and users at different stages of knowledge production, dissemination, and utilisation. Phase 1 develops partnerships with key stakeholders (e.g., persons with lived experience, service providers, policy makers) as a means of converting basic knowledge into practice to drive the implementation model (21). Early engagement and consultation are crucial for building strong relationships and enable joint identification of practice-relevant research questions (22). A translation team will be convened, consisting of researchers and health service clinicians, to drive implementation throughout the study. This will include service leaders who have the autonomy and authority to influence and enact change within their organisations (23–25).
In Phase 2, knowledge exchange and co-design will assess: i) gaps and opportunities for service provision, ii) capacity for digital integration, and iii) facilitators and barriers to implementation. Healthcare environments are complex and assessing the setting prior to engaging in implementation is critical (26). Assessment will also consider the less visible aspects of health service organisations, such as norms, values, and basic assumptions. Most change efforts are targeted as visible, objective aspects of an organisation, such as tasks, structures, and behaviours, neglecting the less tangible workplace cultures and ways of thinking that can hinder implementation efforts (27, 28).
Phase 3 implementation and support serves to bridge identified cultural barriers to uptake and encourages seamless integration of digital therapeutics across health services (29). Translation will be enhanced through ‘trigger encounters’ that are informal, personal, and engender an emotional response (22). It is hypothesised that uptake is enhanced when information provision and training occurs in a clear, user-friendly way (24), emphasising implications and relevance for practice (30), and directly engaging with push back and defensiveness (22). Organisational commitment and managerial patience will be encouraged, as acceptance and uptake of new practices is not rapid and must consider the different processes experienced by individuals such as pre-contemplation, contemplation, preparation, action, and maintenance (31).
Phase 4 identifies ‘champions’ to support implementation and advocate for organisational recognition of activities to establish a strong foundation for sustained implementation (32, 33). In collaboration with champions, the research team will establish organisational systems that support ongoing implementation, such as: (i) individual/team debriefing or structured supervision about clinical progress and experiences/challenges using eCliPSE; (ii) clear communication pathways and feedback mechanisms within the organisation, and with the research team, to ensure eCliPSE remains flexible and responsive to changing clinical/service needs; and (iii) formal recognition and incentives for implementation activities, such as rewards or key performance indicators.
Outcomes
Primary outcome
The primary outcome of this research is the uptake of eCliPSE within LHDs in NSW, Australia. as assessed by the number of users who register for an account with the eCliPSE website in each LHD.
Secondary outcomes
In line with our research hypotheses, we will be examining four additional outcomes
-
User engagement with the eCliPSE website, assessed via automatically collected engagement metrics including number of logins to eCliPSE, the number of eCliPSE pages accessed, and the average session duration per login.
-
Mental health symptoms and alcohol and other drug use of eCliPSE users, assessed via the Depression, Anxiety and Stress Scale [DASS-21], alcohol consumption on the Opioid Treatment Index [OTI], Alcohol Use Disorder Identification Test [AUDIT], Severity of Dependence Scale [SDS], and the Assessment of Quality-of-Life questionnaire [AQoL-8D completed at first login (baseline), and at 12-weekly intervals throughout the study period.
-
Cost-effectiveness from a health sector perspective assessed as the difference in costs required to implement DtC + ITEM compared to DtC only, relative to the difference in QALYs calculated from utility values derived from participant responses on the AQoL-8D.
-
Healthcare service capacity to deliver evidence-based interventions for service users presenting with MHAOD comorbidity will be assessed via the Dual Diagnosis Capability in Addiction Treatment (DDCAT) or the Dual Diagnosis Capability in Mental Health Treatment (DDCMHT) audit tools at baseline and 6-months later. Data collected via interviews and surveys with staff members and services users in each LHD. Participating services will also be asked to provide copies of any program manuals, policies, procedures, intake forms and other relevant documents. Following each audit, each MHAOD service will be given an overall capability rating on a 5-point scale.
Data Collection
eCliPSE registration and use data
Registrations to eCliPSE will be assessed for the duration of the study and will be recorded electronically via the eCliPSE content management system. eCliPSE registration data will be geotagged to record the location of users. In addition, users will be asked to enter their postcode upon registration, allowing the research team to determine in which LHD users are residing. Similarly, user engagement with the eCliPSE website will be automatically recorded using the eCliPSE content management system and Google analytics, in the form of number of logins to eCliPSE, the number of eCliPSE pages accessed, and session duration per login.
Self-Report Secondary Outcome Measures
The eCliPSE website will prompt users to complete a brief self-report questionnaire at first login (baseline), and at 12-weekly intervals throughout the study period. The self-report questionnaire includes the Depression, Anxiety and Stress Scale [DASS-21], alcohol consumption on the Opioid Treatment Index [OTI], Alcohol Use Disorder Identification Test [AUDIT], Severity of Dependence Scale [SDS], and the Assessment of Quality-of-Life questionnaire [AQoL-8D].
Service Audit
In order to assess participating services’ capacity to deliver evidence-based interventions for service users presenting with MHAOD comorbidity the Dual Diagnosis Capability in Addiction Treatment [DDCAT] and Dual Diagnosis Capability in Mental Health Treatment [DDCMHT] indexes will be used to generate an overall capability score (1 – Focused on providing services to persons with Addiction or Mental Health disorders only to 5 – Capable of providing services to any individual with co-occurring substance use and mental health disorders) for each service at each timepoint. These audit tools help organisations evaluate their capacity to provide treatment for people diagnosed with MHAOD use comorbidity. Audits will be completed at two timepoints (T1: baseline; T2: 6-months post-baseline). First, researchers will identify an appropriate contact person (i.e., service manager or clinical lead), and pre-define the scope of the audit, including the time and resource requirements.
The following methods will be used, as appropriate for each service, to collect data to inform DDCAT/DDCMHT scoring:
Ethnographic observations. Researchers mat conduct site visits at participating services to facilitate ethnographic observations. Ethnographic observation of the milieu and physical settings is intended to be brief and facilitate a broad impression of the social environment and physical setting. Observation will only be conducted in common spaces of participating services via a tour of the site. No observation will occur with clinicians while they are engaging with a client either in-person or remotely.
Document review. Document review may include program policy and procedure manuals, brochures, daily patient schedules, telephone intake screening forms, and other materials that may seem relevant. Data collection will follow a predetermined data management plan and be recorded using a data collection sheet.
Qualitative interviews and surveys. Semi-structured interviews with clinicians and service managers at participating services will be conducted using the recommended questions in the DDCAT/DDCMHT indexes, to identify different perspectives on practices and procedures of the service. Based on feedback from potentially participating services that site visits and provision of service documents may be too burdensome, additional questions have been added to the interview schedule to capture information about the milieu, physical setting, policies, and procedures of participating services. Interviews will be conducted with staff of the service (i.e., directors, supervisors, clinicians, and other personnel). Typically, the DDCAT/DDCMHT indexes also recommend that interviews are conducted with consumers. However, due to the scale and complexity of this study, consumers will be engaged via an online survey. The online survey, anticipated to take 10 minutes to complete, asks service users about their experiences interacting with the service, such as the type of support they have received and if information and support received has addressed both mental health and substance use issues. Following the methodology of Matthews and colleagues (34), a minimum of 5 staff members from each service will take part in the DDCAT/DDCHMT at each audit timepoint, along with a minimum of 5 consumers of each service.
Data Analysis
To address the primary outcome of this study, the number of registrations to eCliPSE completed within each LHD in NSW will be calculated. Total number of registrations will be compared between LHDs receiving ITEM + DtC versus DtC only.
To examine engagement with eCliPSE, eCliPSE website analytics will be converted into an overall index of engagement with the website, and this index will be compared between LHDs receiving ITEM + DtC versus DtC only.
To examine mental health symptoms and alcohol and other drug use of eCliPSE users, overall change in DASS21, OTI, and AUDIT scores between baseline (first login), 12- and 24- weeks will be modelled using multilevel latent growth models with time nested in consumers, and consumers nested in LHDs. As standard practice, the functional form of change in the model will be examined by fitting preliminary curves with different polynomials of time to represent linear and non-linear change. Analyses will also explore symptom change across different users with different levels of eCliPSE use.
A within trial economic evaluation will be undertaken to assess the value for money of the DtC + ITEM strategy compared to DtC only. Costing of the DtC and ITEM interventions will be undertaken using micro-costing methods which incorporate staff time required for providing the interventions, the cost of implementing the DtC campaign, and financial data provided by the research team. The preferred outcome measure in health economic evaluations is the QALY due to their inherent value for money connotations. The utility values from the AQoL-8D for each participant at each time point will be used to calculate QALYs using the area under the curve method. The incremental costs of DtC + ITEM compared to DtC only will be calculated and compared to the incremental QALYs between the groups, referred to as a cost-utility analysis. Additional cost-effectiveness analyses comparing incremental costs to incremental differences in secondary self-reported outcomes (e.g. DASS21, OTI, AUDIT and SDS) will also be undertaken. The budget impact of implementing the cost-effective intervention will be estimated based on population-level scale-up costing utilising epidemiological data.
To assess participating services’ capacity to deliver evidence-based interventions for service users presenting with MHAOD comorbidity change in DDCAT/DDCMHT scores between T1 (baseline) and T2 (6-month follow-up) will be compared between services within DtC only LHDs and ITEM + DtC LHDs. Specifically changes in DDCAT scores will compared between AOD services (and combined MHAOD services) within DtC only LHDs and ITEM + DtC LHDs and changes in DDCMHT scores will be compared between MH services (and combined MHAOD services) within DtC only LHDs and ITEM + DtC LHDs. This analysis will be accompanied by a non-parametric test, and two-tailed analyses to reduce type-1 error rates and for non-normal distributions resulting from the small sample size.
As this is an exploratory study, no interim analysis for efficacy will be performed. There is no intention for auditing the conduct of this trial. All planned analysis will compare LHDs allocated to the DtC only and ITEM + DtC conditions as randomised.
Data Management and Safety
All data collected by the eCliPSE website is stored on a secure database using industry standard encryption techniques and accessible only to the research team. Data collected as a part of the DDCAT/DDCMHT audits will be de-identified, linked over time via unique identifier codes and stored on a secure server at the University of Newcastle, accessible only to the research team. At the end of the study all data will be stored for a period of 20 years. After this time all data will be securely disposed of. Only aggregate data will be presented in publications and individual participants will not be identifiable. Additional analyses will examine whether there are any systematic differences between groups and/or baseline variables in relation to drop out.
To mitigate potential risks of psychological distress when completing questionnaires or accessing resources through the eCliPSE website, participants are provided with links to relevant support services through a prominent emergency tab on the website.
All study procedures, data and safety monitoring will be under the supervision of Dr Thornton and Prof Frances Kay-Lambkin including monitoring website use and reviewing any adverse events that may arise. Any adverse events, serious adverse events, violations of or changes to the approved protocol will be reported directly to the appropriate HREC and study sponsor, and actions to mitigate further violations will be undertaken.
Publication policy / Dissemination of results
Deidentified and aggregate results will be disseminated through peer-reviewed publications in reputable scientific journals, presentations at conferences, and clinically focused workshops, guidelines, and webinars. If participating services and individuals will be able to request a copy of the study results by contacting the research team via the eCliPSE website or email. They will be contacted via email with a summary of the research findings and copies of the associated publications. To be eligible for authorship on papers describing this study, authors will be required to make a significant contribution to the study design, the conduct of the trial, and/or analysis.
Trial Status
Recruitment for the trial has begun and baseline data collection among initial sites started. Study recruitment is ongoing and no cleaning, or analysis of data has been executed prior to submission of this manuscript. Further administrative information about the study protocol is provided in Additional file 3.