The Weathering Well App, A New Low Intensity Digital Mental Health Intervention for Australian Farmers: Formative Research

Australian farming communities demonstrate signicant resilience in the face of many challenges, including drought, bushres, and more recently COVID-19. Nevertheless, worries related to nancial hardship, farm viability and productivity, and remoteness render the population vulnerable to mental health problems. Despite this vulnerability, people in these communities are unlikely to seek help from mental health services due to attitudes, service access and geographic barriers. A tailored, low intensity digital mental health intervention may provide an acceptable and appropriate early wellbeing intervention for the population and help bridge the mental health service gap. This study has three objectives: 1) to adapt an existing tablet-based, therapist-guided, low intensity digital mental health intervention to address the wellbeing needs of Australian farming communities (development of the Weathering Well app), 2) to adapt, deliver, and evaluate training in utilisation of the new app, and 3) to conduct preliminary analyses on app usage data during the 12-months post-app release.

ranging from mental health promotion, psychoeducation, prevention, early intervention, treatment, to relapse intervention. The approach also allows exibility in mode of delivery, with resources available for self-guided use and to complement traditional therapy (6, 7).
Emerging evidence demonstrates effectiveness of digital mental health approaches in improving depression, anxiety, eating disorders and other common mental health issues (13)(14)(15). These approaches can also provide effective early intervention. Demonstrated by a recent systematic review, digital interventions were found to provide screening and signposting for treatment and self-management strategies for people with subthreshold common mental health symptoms (10).
Within Australia, digital mental health approaches have been developed, utilised, and shown to be effective for First Nations and culturally and linguistically diverse communities (16)(17)(18)(19). Therefore, collaborative tailoring of such approaches may help to address the challenges to helpseeking within Australian farming communities. However, low technology literacy, limited infrastructure such as connectivity, preference for face to face contact, and relevance to the farming life are important challenges to implementation of digital mental health interventions in the farming context (12).

The Stay Strong app
The Stay Strong app is a therapist-guided, tablet-based app which delivers a low intensity cognitive behavioral wellbeing intervention. Developed by Menzies School of Health Research (Menzies), in partnership with Queensland University of Technology (QUT), it is one the few co-designed digital mental health interventions for Aboriginal and Torres Strait Islander people. The therapy adopts an empowering, person-centered, holistic and strengths-based perspective which incorporates structured problem solving, motivational interviewing, and brief behavioural and cognitive approaches (19). The Stay Strong app has been shown to be effective in improving wellbeing outcomes and is well-utilised within primary care services (20)(21)(22)(23). It has been adapted for use in a number of settings such as renal services, pharmacy settings, custodial settings, and youth populations.
Under the e-Mental Health in Practice project (eMHPrac) initiated by the Australian Government in 2013, Menzies, in collaboration with multiple organizations across Australia, has been providing training to service providers to support uptake of digital mental health approaches including the Stay Strong app. Through this program of work, Menzies has networked with Primary Health Networks (PHNs) across Australia, including Western Queensland Primary Health Network (WQPHN).

Western Queensland Primary Health Network (WQPHN)
WQPHN is one of 31 PHNS established by the Australian Commonwealth Government. It is a commissioning body which works with primary healthcare services in the Western Queensland region of Australia. As part of the Australian government's stepped care approach to mental health, WQPHN executive identi ed a need for appropriate low-intensity interventions for the farming population residing in the region. Provision of lowintensity interventions for people with low to moderate mental health concerns within a stepped care framework streamlines service delivery and helps reserve intensive care for people most in need (8).
The Stay Strong app provides an evidence-based low-intensity intervention, designed to be delivered opportunistically in primary care settings by non-mental health professionals. It is thus well-placed within the stepped care approach. WQPHN has supported implementation of the Menzies Stay Strong app within Indigenous Social and Emotional Wellbeing programs. This experience led to recognition of the potential for the app to be adapted to address the needs of Australian farming community members by maintaining the essential therapeutic principles but changing the text and images. The app is designed as a supported face to face intervention which does not require internet connection, factors which render it appropriate to the rural and remote farming context (12).

This study
This three-phase research project was funded by WQPHN in collaboration with Menzies. Phase one involved collaborative co-design to adapt the Stay Strong app and nalize the new Weathering Well app. Phase 2 included adaption of and delivery of the Weathering Well app training to 39 service providers across seven workshops ( ve training and two Train the Trainer). Phase three consisted of evaluation of the Weathering Well training and app usage data at 12 months post-release.
The objectives of the study were: 1) to describe the adaptation process of the Stay Strong app, 2) to evaluate the Weathering Well training, and 3) to provide preliminary ndings regarding feasibility, acceptability, and appropriateness of the Weathering Well app.

Phase 1: Development of the Weathering Well App
Participants WQPHN initiated the project in response to identi ed priorities of supporting low intensity wellbeing interventions within a stepped care approach for Australian drought affected farming communities. The project team included senior Menzies project coordinator based in Adelaide (MS), two senior Menzies researchers with clinical mental health expertise based in the Northern Territory (TN and KD). The Coordinator, Service Provider Commissioning services at WQPHN (AS) brought clinical, education and policy development expertise to the team.
Using purposive sampling, service providers including WQPHN representatives were recruited to form an Expert Advisory Group (EAG) to inform the adaptation process. The following inclusion criteria guided recruitment: farmers affected by drought who support others, service providers (e.g. nancial advisors, remote area counsellors, Primary Health Network representatives) working with farming communities affected by drought, and information technology (IT) consultants. The EAG included membership from New South Wales (NSW) and Queensland (Qld) and comprised graziers, growers, rural nancial counsellors, rural and remote mental health clinicians, IT experts, PHN representatives, peak body representatives (NSW rice growers) and drought impacted Shire Council representatives. In total, 13 participants formed the nal EAG and provided their informed consent to participate in the study. All methods were performed in accordance with the relevant guidelines and regulations by including a statement. Ethical approval was granted by the Human Research Ethics Committee for the NT Department of Health and Menzies School of Health Research (reference number HREC 2018-3280).

Procedure
The adaptation process followed an iterative and collaborative process of discussion, review, analysis, interpretation, and veri cation. The co-design process brought those with technical expertise and lived experience together, allowing users to actively contribute to the creative process and leveraged their expertise to inform resource development.
A three-day workshop was held in December 2018 in Adelaide, facilitated by Menzies project team member (MS) and Patj Patj Janama Robert Mills (a senior Cultural Advisor who assists in guidance of Stay Strong app development and ongoing revision), and attended by the 13 members of the EAG. Notes were taken by the facilitators, and participant worksheets were distributed and collected at the end of each workshop day. Thematic analysis was conducted concurrently allowing ndings from each day to inform the content of the next day of the workshop. The rst day introduced Terms of Reference, background to Stay Strong app and its current use, and relevance of the underlying approach to the farming context; the second day covered potential adaptations to content including wording, images and functionality; the third day condensed the discussions into a list of recommendations for app adaptations. During the workshop, members were invited to present alternate views for discussion and consideration within the group. Group facilitation techniques were used to achieve consensus.
The nalized draft list of agreed changes was member-checked by the EAG and then forwarded to the app developer. An EAG subcommittee and the research team liaised with the app developer to complete the rst adapted version for user-testing. EAG members and the research team user-tested the adapted app and provided feedback to the app developer. The app was modi ed in response to identi ed functional and aesthetic issues. The nal 'Weathering Well' app was reviewed by the EAG and gained approval for public release in September 2019.

The Weathering Well App
The Weathering Well app, incorporates the same therapeutic principles as the Stay Strong app. It uses similar visual design techniques to assist a person to identify family support, strengths and worries prior to setting lifestyle goals for change. It includes wellbeing tips for change, outcome measures (Kessler 5 and 10; validated short questionnaires assessing psychological distress), an appointment calendar, session rating questionnaire, and a care plan summary with email and print function, re ecting the Stay Strong app design. A number of page headings and texts were changed to re ect preferred language in farming communities. For example, the page entitled "Things that keep me strong' with subheadings: spiritual and cultural, physical, family, social and work, and mental and emotional wellbeing, in the Stay Strong app was changed to 'Things that keep me well' with new subheadings: surviving to thriving, physical, family, social and work, and emotional and mental wellbeing. The app design supports primary healthcare service providers working with the farming community who do not necessarily have mental health training, such as nancial advisors and remote area counsellors, to deliver a structured and evidence based wellbeing intervention as a component of usual care. Please see supplementary materials for example images of some sections of the Stay Strong app and the revised versions in the Weathering Well app.

Phase 2: Adaptation and Delivery of the Weathering Well Training
Training adaptation procedure As mentioned, Menzies provides training in digital mental health approaches, including the Stay Strong app, to service providers, under the eMHPrac project (20). The training workshop was adapted by the research team to support service providers who work with farming communities to use the Weathering Well app within their practice. Key training topics include: introduction to digital mental health, development and the underpinning therapeutic principles of the Weathering Well app, app navigation, goal setting, using the app while counselling (demonstrated through role play), and integration of the app into practice. The training resources include slide presentation, user guide, brief training manual, and facilitator manual (see supplementary information). The training was delivered by TN and MS, who have been delivering training in digital mental health approaches to primary care service providers since 2013.

Participants
Between September 2019 and March 2020, 39 participants completed the Weathering Well training. Six 1-day training workshops were held in Queensland and NSW (Brisbane, Wagga Wagga, Young, Gri ths, and Deniliquin). Participants either self-selected or were selected by their organization to attend the training. Before and after training, participants completed a paper-based survey which included adapted visual analogue scale (ratings from 1 to 10) to assess knowledge, competency, con dence, and perception of digital mental health approaches and the Weathering Well app. The survey is adapted from the questionnaire used to assess the Stay Strong app training workshop (20).
Participant ages ranged from 23-76 years old (M = 43.60, SD = 14.29, 4 missing), most were female (74.4%,1 missing), and non-Indigenous Australians (71.8%). Other identi ed cultural backgrounds represented were: Aboriginal (2.6%), Polish (2.6%), English (2.6%), New Zealander European (2.6%); 4 participants did not report their cultural backgrounds. Participant professions and service type are reported in Table 1. 'Other' professions reported include rural nancial counsellor, farming community counsellor/agricultural health specialist, mental health advocate, suicide prevention project o cer, and volunteer community member. 'Other' service types include, 'not for pro t', 'community service pharmacy', 'farming nancial counselling', and 'volunteer'. Most participants reported having access to a laptop (82.1%), the internet (86.8%), and/or a desktop computer (69.2%). Participants who completed the Weathering Well training were each given a non-identifying code to link their pre-and post-training data. Differences between pre and post-training scores were analysed using paired t-tests. Descriptive statistics were used to analyse demographic data and ratings on ve different aspects of the training presentation, including clarity, appropriateness, structure, speed, and learning materials. Thematic analysis was conducted on qualitative data (comments and suggestions) embedded at the end of the questionnaire. App analytic strategy Un-identi able Weathering Well app data is collected within a linked database. A check box at commencement of the app allows an individual client to provide consent for data collection or to opt out. Both the app and database are password protected. Names are not collected in the on-line database which collects gender and year of birth (not date of birth). The present analysis presents subsets of app data deemed to be useful, including demographic data, identi ed support network, strengths and worries, number of participants who engaged in goal setting, selected wellbeing and substance use tips, and K5 and K10 scores.

Results
Weathering Well training Paired t-tests were conducted to compare pre-to post-training scores across all participants. Results showed signi cant improvements across all items, apart from 'appropriateness of Weathering Well app' which did not change. Mean (M) and Standard Deviation (SD) are reported in Table 2.
Additionally, items 8-14 have an option to select "don't know". Across the 7 items, this option was selected 63 times prior to training, compared to 13 times in the post-training evaluation, suggesting that participants gained knowledge in response to the training workshop. In the post-training evaluation, 49% of the participants indicated that they would use the Weathering Well app in the future, 31% indicated 'maybe', and 3% said 'no'. Participants also indicated the kinds of support required to help implement the Weathering Well app. The most frequently selected item was the development of organizational protocols. This is reported in Table 3.
Participants were invited to rate the training session from 1 (not at all) to 4 (very) across ve different aspects of the presentation: 1) clarity of the presenter's explanation ( Participants were also invited to write additional comments and suggestions for improvement at the end of the post-training evaluation survey. Twenty-eight participants provided comments which were analysed using thematic analysis. Two major themes were identi ed: comments regarding the training workshop and comments relating to the Weathering Well app. Comments regarding the training workshop were mostly demonstrating participants' satisfaction with the training, e.g., "great training session", "wonderful training session. Thank you!", and one participant commented that "everything was explained clearly and easily understood." Some participants also commented on the usefulness of the training: "I took a lot of skills out of it", "Thank you. Very informative". These suggest that the training is appropriate and acceptable for the target audience. One participant commented that the training could be improved by allocating more time for practicing the Weather Well app.
Two subthemes were identi ed for comments regarding the Weathering Well app: acceptability and app improvements. Participants reported the app to be acceptable: "I liked the app overall. Easy to use. Comfortable to use and interactive", and appropriate: "really well structured app. It would be very useful in our organisation." They also perceived the app to have the "ability to integrate with other programs", as well as it being useful for farmers: "make this accessible to farmers. It is a great resource. Thank you!" In terms of app improvements, participants provided ve different areas in which the app could be further improved: 1) inclusion of live links of support services: "links to mental health support site alongside phone numbers", "provide links to psycho-education resources on 'Worries' page"; 2) include other functions: "Ability to put pictures/photos of support people", and "bring up, 'Where to get help page' if someone indicates selfharm/suicide"; 3) expand to other digital platforms, i.e., "make available on android"; 4) extend to other age groups such as young people: "make a version for primary aged kids"; and 5) suggestions for word changes such as "replace 'self-harm' with 'suicide'", and "Surviving and Thriving' heading needs to be improved. More confusing than the others."

Support Network
On average, clients reported 2-3 supportive relationships, the vast majority of which were immediate family, i.e., son/partner/spouse/mother/brother/daughter/sister/father. Others included friends, counsellors, work mates, a neighbour, and a pet dog.

Strengths & Worries
Clients reported 3-4 strengths on average. Hobbies, work, and social connection were the most common. Similarly, 3-4 worries were also reported on average, with nance, sleep, alcohol tobacco and other, and stress and anxiety, being the most common.

Goals
The app provides the option to set two goals and enter 3 steps per goal. Most clients set at least one goal (64%), and most reported the underlying reasoning for their goals. Among those who engaged in goal setting, most set only one goal (81%) and 1-2 steps for the goal. More than half of the clients (69%) identi ed when they would take their rst step, and 62% indicated when they would take their second step.
Overall, these ndings demonstrate that most clients were engaged with the goal setting function of the app and were able to follow effective goal setting process by indicating their speci c goal, and how and when it could be achieved. Goals set by the clients t primarily into 3 categories: The majority of clients (56%) selected 1-4 wellbeing tips. 'Spend more time with kids'; 'maintain healthy lifestyle'; and 'explore your optionssuccession planning' were the most common tips selected. In regard to substance use tips, 60% selected 1-5 tips, and most common tips selected were 'drink water rst -so that you aren't thirsty'; 'avoid reminders and triggers -like friends who use, places or things that remind you of using'; and 'delay starting -start later in the day'.

Discussion
The Stay Strong app was successfully adapted through collaborative co-design to potentially address the wellbeing needs of Australian farming communities. The result is the Weathering Well app. Existing training for the original Stay Strong app was then adapted to the Weathering Well app context. The training workshop was found to be effective in improving participants' knowledge and skills in using the Weathering Well app.
Participants' feedback also showed reasonable acceptability of the training with minor suggested improvements. Real-life implementation outcomes of the Weathering Well app collected through app analytics, analysed at 12-months post app release suggested that the majority of clients who completed the app were engaged with the process and showed high levels of distress.
Despite major changes to imagery and text, the nal Weathering Well app retained the key four steps of the therapeutic approach underpinning the Stay Strong app: identi cation of family support, strengths and worries, and setting lifestyle goals for change. This suggests that the personcentered, holistic, and strengths-based approach to mental health was perceived by the EAG members to be acceptable and appropriate for non-Indigenous farming communities. This view is also supported by the training evaluation data and to some extent by the level of engagement demonstrated through app usage data. Findings showed that all clients engaged with the core aspects of the intervention, i.e., identi cation of 2-3 supportive relationships, reported 3-4 strengths, setting at least one goal and 1-2 steps for each goal. This provides early tentative evidence for appropriateness and acceptability of the intervention within the Australian farming communities.
Given that both the Stay Strong app and the Weathering Well app embody similar principles, person-centered, holistic, and strengths-based, they align with overall trends toward client-centred mental health care (22). By considering a person's agency within their context as integral to wellbeing interventions; such approaches are empowering, meaningful, and can be tailored to the person within their social and cultural settings (23).
Emerging evidence demonstrates effectiveness of such approaches in improving wellbeing for both Indigenous and non-Indigenous populations across different countries (24)(25)(26). The app is designed to be used as part of face-to-face care which was reported to be the preference of the farming communities (12). This may be another contributor to acceptability. In addition, the integration of commonly used wellbeing outcome measures (K5 and K10) creates opportunity for assessment and routine screening. This integration is identi ed as important in addressing the mental health service gap, along with bush knowledge to facilitate rapport (12). App graphics and content are tailored to the farming context potentially facilitating conversations relevant to farming life. Farmers have reported preference for a practical approach to wellbeing concerns and the app allows selection of useful wellbeing tips, meaningful goals and practical steps to achieve them (12). These added app elements may also support successful app implementation.
Although low technological literacy and poor internet connection within the farming communities are reported as potential barriers to utilisation of digital mental health approaches (12), the provision of a supported intervention which does not require internet connection may overcome these challenges. Our ndings suggest that the Weathering Well app may be acceptable to people of varied ages, as shown by the age range of app users from 28-72 years old, as well as for those who experience high levels of distress. The Weathering Well app was also perceived by the service providers who attended training to be useful, easy to use, and easy to integrate into their organisation. These ndings provide important early evidence for service providers' perspectives regarding the app's feasibility, acceptability, and appropriateness, which are essential to successful implementation (27). However, some changes to the app were recommended across ve different aspects: 1) inclusion of live links of support services, 2) inclusion of other functions, e.g., appearance of the 'help page' in the different sections of the app and options for uploading pictures of supportive people, 3) compatibility with other digital platforms, e.g., android, 4) extension to other age groups such as young people, and 5) suggestions for some word changes.
The Stay Strong training was also successfully adapted to facilitate training in the Weathering Well app. This is evidenced by the improvement in the participants' perceived knowledge, competence, con dence, and perception regarding digital mental health approaches and the Weathering Well app post-training. Participants also demonstrated good levels of satisfaction with the training, indicated by ratings across the different aspects of the training, i.e., clarity of presentation, appropriateness of the level and structure of the course, speed of information presented, and helpfulness of the learning materials. Participants' feedback comments were also positive and supportive of the training. These ndings are similar to that of the original Stay Strong training, indicating that the adaptation maintained effective knowledge translation after tailoring to the new farming context (20).
These ndings provide preliminary evidence for feasibility, acceptability and appropriateness of a new digital mental health resource for the farming community. However, there are some important limitations worth noting. Over the 12-month post-app release period, only 25 sessions were recorded. This may be explained by a number of reasons. Firstly, the 2020 COVID-19 pandemic signi cantly impacted face-to-face service delivery in Australia through the targeted services following limited release of the app in late 2019. The pandemic also impacted training which was scheduled in the rst half of 2020. As a result only 39 service providers attended face to face training in the Weathering Well app. Secondly, clients have the option to opt out of data collection; thus, it is unknown how many clients completed the app but did not have their data collected. Future app revisions will allow identi cation of the number of clients who choose to opt out of data collection.
Thirdly, organisational factors such as IT resources and infrastructure, leadership and support, and policies and protocols around digital mental health utilisation, may impact app usage. These factors, particularly development of organizational protocols, were highlighted in the post-training evaluation as essential support for implementing the Weathering Well app. These same factors were found to signi cantly in uence service provider use of the Stay Strong app (28, 29). Lastly, this study was conducted as a pilot to establish feasibility. The app was therefore not released to the public but was hosted through the WQPHN website with speci c permission for download granted to participating organisations. It was not accompanied by a promotional and marketing campaign. The positive ndings, however, and the increased mental health concerns in the context of the COVID-19 pandemic, support plans to raise awareness of the app within primary healthcare organisations and the broader mental health care community and to release the app publicly.
Another important limitation of the study is the inability to discern individual sessions due to the anonymity of data collection. It is likely that some clients completed the app more than once, i.e., participated in follow-up sessions with their service providers. However, the app usage data collection did not include identi cation of follow-up sessions and progress toward their goals. Additionally, consumer perspectives addressed by session rating questions at the end of app completion were not collected in the database. Therefore, it was not possible to gain insight into participants' experience of service delivery. These database shortcomings will be addressed in future revisions. Lastly, as this is formative research, it only provides preliminary ndings for the Weathering Well app's feasibility, acceptability, and appropriateness. Further testing through a more rigorous research design, e.g., qualitative and experimental, is required to determine the app's effectiveness and implementation outcomes.
Overall, this study suggests successful adaptation, resulting in the Weathering Well app release and good outcomes from service provider training. Early app usage data demonstrates good app engagement despite low overall rates of use during 2020. The study highlights the need for app revisions secondary to suggested changes from service providers, as well as database changes allowing more in-depth evaluation of usage and impact. The study provides a good foundation for further research including rigorous testing of effectiveness and evaluation of implementationrelated outcomes.