Phase 1: Development of the Weathering Well App
Participants
WQPHN initiated the project in response to identified priorities of supporting low intensity wellbeing interventions within a stepped care approach for Australian drought affected farming communities. The project team included senior Menzies project coordinator based in Adelaide (MS), two senior Menzies researchers with clinical mental health expertise based in the Northern Territory (TN and KD). The Coordinator, Service Provider Commissioning services at WQPHN (AS) brought clinical, education and policy development expertise to the team.
Using purposive sampling, service providers including WQPHN representatives were recruited to form an Expert Advisory Group (EAG) to inform the adaptation process. The following inclusion criteria guided recruitment: farmers affected by drought who support others, service providers (e.g. financial advisors, remote area counsellors, Primary Health Network representatives) working with farming communities affected by drought, and information technology (IT) consultants. The EAG included membership from New South Wales (NSW) and Queensland (Qld) and comprised graziers, growers, rural financial counsellors, rural and remote mental health clinicians, IT experts, PHN representatives, peak body representatives (NSW rice growers) and drought impacted Shire Council representatives. In total, 13 participants formed the final EAG and provided their informed consent to participate in the study. All methods were performed in accordance with the relevant guidelines and regulations by including a statement. Ethical approval was granted by the Human Research Ethics Committee for the NT Department of Health and Menzies School of Health Research (reference number HREC 2018–3280).
Procedure
The adaptation process followed an iterative and collaborative process of discussion, review, analysis, interpretation, and verification. The co-design process brought those with technical expertise and lived experience together, allowing users to actively contribute to the creative process and leveraged their expertise to inform resource development.
A three-day workshop was held in December 2018 in Adelaide, facilitated by Menzies project team member (MS) and Patj Patj Janama Robert Mills (a senior Cultural Advisor who assists in guidance of Stay Strong app development and ongoing revision), and attended by the 13 members of the EAG. Notes were taken by the facilitators, and participant worksheets were distributed and collected at the end of each workshop day. Thematic analysis was conducted concurrently allowing findings from each day to inform the content of the next day of the workshop. The first day introduced Terms of Reference, background to Stay Strong app and its current use, and relevance of the underlying approach to the farming context; the second day covered potential adaptations to content including wording, images and functionality; the third day condensed the discussions into a list of recommendations for app adaptations. During the workshop, members were invited to present alternate views for discussion and consideration within the group. Group facilitation techniques were used to achieve consensus.
The finalized draft list of agreed changes was member-checked by the EAG and then forwarded to the app developer. An EAG subcommittee and the research team liaised with the app developer to complete the first adapted version for user-testing. EAG members and the research team user-tested the adapted app and provided feedback to the app developer. The app was modified in response to identified functional and aesthetic issues. The final ‘Weathering Well’ app was reviewed by the EAG and gained approval for public release in September 2019.
The Weathering Well App
The Weathering Well app, incorporates the same therapeutic principles as the Stay Strong app. It uses similar visual design techniques to assist a person to identify family support, strengths and worries prior to setting lifestyle goals for change. It includes wellbeing tips for change, outcome measures (Kessler 5 and 10; validated short questionnaires assessing psychological distress), an appointment calendar, session rating questionnaire, and a care plan summary with email and print function, reflecting the Stay Strong app design. A number of page headings and texts were changed to reflect preferred language in farming communities. For example, the page entitled “Things that keep me strong’ with subheadings: spiritual and cultural, physical, family, social and work, and mental and emotional wellbeing, in the Stay Strong app was changed to ‘Things that keep me well’ with new subheadings: surviving to thriving, physical, family, social and work, and emotional and mental wellbeing. The app design supports primary healthcare service providers working with the farming community who do not necessarily have mental health training, such as financial advisors and remote area counsellors, to deliver a structured and evidence based wellbeing intervention as a component of usual care. Please see supplementary materials for example images of some sections of the Stay Strong app and the revised versions in the Weathering Well app.
Phase 2: Adaptation and Delivery of the Weathering Well Training
Training adaptation procedure
As mentioned, Menzies provides training in digital mental health approaches, including the Stay Strong app, to service providers, under the eMHPrac project (20). The training workshop was adapted by the research team to support service providers who work with farming communities to use the Weathering Well app within their practice. Key training topics include: introduction to digital mental health, development and the underpinning therapeutic principles of the Weathering Well app, app navigation, goal setting, using the app while counselling (demonstrated through role play), and integration of the app into practice. The training resources include slide presentation, user guide, brief training manual, and facilitator manual (see supplementary information). The training was delivered by TN and MS, who have been delivering training in digital mental health approaches to primary care service providers since 2013.
Participants
Between September 2019 and March 2020, 39 participants completed the Weathering Well training. Six 1-day training workshops were held in Queensland and NSW (Brisbane, Wagga Wagga, Young, Griffiths, and Deniliquin). Participants either self-selected or were selected by their organization to attend the training. Before and after training, participants completed a paper-based survey which included adapted visual analogue scale (ratings from 1 to 10) to assess knowledge, competency, confidence, and perception of digital mental health approaches and the Weathering Well app. The survey is adapted from the questionnaire used to assess the Stay Strong app training workshop (20).
Participant ages ranged from 23–76 years old (M = 43.60, SD = 14.29, 4 missing), most were female (74.4%,1 missing), and non-Indigenous Australians (71.8%). Other identified cultural backgrounds represented were: Aboriginal (2.6%), Polish (2.6%), English (2.6%), New Zealander European (2.6%); 4 participants did not report their cultural backgrounds. Participant professions and service type are reported in Table 1. ‘Other’ professions reported include rural financial counsellor, farming community counsellor/agricultural health specialist, mental health advocate, suicide prevention project officer, and volunteer community member. ‘Other’ service types include, ‘not for profit’, ‘community service pharmacy’, ‘farming financial counselling’, and ‘volunteer’. Most participants reported having access to a laptop (82.1%), the internet (86.8%), and/or a desktop computer (69.2%).
Table 1
Participant employment characteristics
Characteristics
|
Number
|
Percentage
|
Profession
|
|
|
Manager/Coordinator/CEO
|
7
|
17.9
|
Social Worker
|
7
|
17.9
|
Psychologist
|
2
|
5.1
|
Peer support worker
|
2
|
5.1
|
Aboriginal Mental Health Worker
|
1
|
2.6
|
General Nurse
|
1
|
2.6
|
Mental Health Nurse
|
1
|
2.6
|
Other
|
18
|
46.2
|
Service type
|
|
|
Primary health care network
|
11
|
28.2
|
Non-government community
|
10
|
25.6
|
Other
|
9
|
23.1
|
Aboriginal Community Controlled Health Service/Aboriginal Medical Service
|
2
|
5.1
|
Government community
|
5
|
12.8
|
General practice
|
1
|
2.6
|
Missing
|
1
|
2.6
|
Phase 3: Evaluation of the Weathering Well training and app usage
Training data analytic strategy
Participants who completed the Weathering Well training were each given a non-identifying code to link their pre-and post-training data. Differences between pre and post-training scores were analysed using paired t-tests. Descriptive statistics were used to analyse demographic data and ratings on five different aspects of the training presentation, including clarity, appropriateness, structure, speed, and learning materials. Thematic analysis was conducted on qualitative data (comments and suggestions) embedded at the end of the questionnaire.
App analytic strategy
Un-identifiable Weathering Well app data is collected within a linked database. A check box at commencement of the app allows an individual client to provide consent for data collection or to opt out. Both the app and database are password protected. Names are not collected in the on-line database which collects gender and year of birth (not date of birth). The present analysis presents subsets of app data deemed to be useful, including demographic data, identified support network, strengths and worries, number of participants who engaged in goal setting, selected wellbeing and substance use tips, and K5 and K10 scores.