Assessment of feasibility
Recruitment
Objective 1- Willingness of employers to participate in the trial.
REST was one of three pilot trials under the INWORK umbrella, conducted as part of the Mental Health and Productivity Pilot Programme (MHPP). The MHPP programme is a collaboration of experts working together to provide mental health support and resources to Midlands’ business communities and their employees, which is evidence-based, affordable and sustainable. A total of 301 businesses from the Midlands region were contacted as part of the wider programme, of which 104 businesses agreed to an initial introductory meeting during which the trials, confidentiality and recruitment through standard business communications (e.g emails and newsletters) was explained. This led to 48 businesses requesting more information about recruitment and the different categories of data collected through the study. Of the 48 businesses, 33 formally agreed to become a partner in the trial.
Objective 2 – Willingness of employees to participate in the trial/Data completeness. Participant recruitment took place over 9 months, from April 2021 until January 2022 through two channels: (i) the MHPP businesses (assessed as per objective 1) and (ii) through direct recruitment via online social media (e.g., Twitter, LinkedIn and paid ads on Facebook) and print (e.g., leaflets and flyers in public and retail settings) advertisements. Individuals who expressed interest through the latter pathway were from the wider working community in the Midlands region. 902 workers across both pathways expressed an initial interest to take part in one of the INWORK trials. Around 60% proceeded to complete the eligibility screening questionnaires, and of those, 14% were eligible and invited to enrol in REST. 71% of those eligible consented to the trial and provided baseline measures. Table 1 shows the participant numbers per sector recruited at expression of interest and enrolled in trial (i.e. baseline responses). Figure 2 shows the numbers of participants at each stage in the process, from initial expression of interest until final follow-up timepoint at 12 months (T5) post intervention completion.
Engagement
Objective 3: Adherence/usage of participants of the treatment platform
Of the 52 enrolled participants, 7 (only one of which was initially in the treatment arm) did not engage at all with the digital platform, despite weekly email reminders to log in. Of the remaining 45, 17 had an adherence/engagement above 87% completing nearly all required topics over the 8 weeks period. Of the remaining 28 participants, 11 had an adherence ranging 50-78%, 10 with an adherence ranging 20-48%, and 7 with the lowest engagement of less than 12%. The sample initially allocated to the intervention group completed on average Mean=54.40% (SD=34.84) of the content, compared to a lower rate seen in those initially allocated to the control groups Mean=50.18% (SD=41.62). However, the groups did not significantly differ in percentage of content completion t(49.52) = -0.40, p=0.693.
Objective 4: Impact of the intervention on quantitative outcome measures
Baseline Mean and standard deviation for demographic, secondary outcomes and summary statistics (t-tests and chi-square tests for continuous and categorical variables respectively) are summarised in Table 2 for control and intervention groups. There were no significant differences between the groups at baseline for any variables (p > 0.05).
Outcome assessments
Whilst estimates of significance are presented below, summary statistics of the data are of more importance as the aim of the study is to assess the feasibility of the study and look for variability in the outcome measures to power a future larger RCT.
Linear mixed effect models were run on the study’s secondary outcomes taking a case completeness approach (i.e. excluding those who did not provide end-of-intervention questionnaires measures at week 8) assessing change from baseline to 8 weeks between the waitlist control and interventions groups. Linear mixed effects models showed no significant differences in depression (F(1,40)= 2.82, p = 0.101) and anxiety (F(1, 40)= 2.14, p=0.151), measured on the PHQ-9 and GAD-7 respectively, between treatment allocation, timepoint (i.e., baseline and end of treatment), and ‘timepoint x allocation’ interaction. There was a significant interaction observed for the insomnia symptoms measured by the ISI (F(1,40)= 5.18, p = 0.028), however, post-hoc test findings for differences between groups were not significant.
GLMM was used with data from all timepoints (baseline, end of intervention (week 8), two, six- and twelve-months follow-up) for all outcome measures. The best fit to the data was a non-linear logarithmic function of time, shown by Logn time, having the lowest -2 log likelihood score for all outcome variables (see Table 1 in supplementary appendix 2 for different time distributions used). Main effect terms are used to demonstrate change over time for each variable (Table 3; Figure 3). Negative beta values indicate a reduction in scores over the course of the of each timepoint. Significant changes over time were only found for PHQ-9 scores and WPAI activity impairment (see Figure 1 in Appendix 3 in Supplementary for GLMM plots for logN for non-significant outcomes). There were moderate effect sizes on the depression and anxiety measures (PHQ-9 and GAD-7 respectively), and small to null effect sizes on the insomnia and all other work-related measures.
Assessment of acceptability
Objective 5: Qualitative Assessment
Recruitment of participants for process evaluation interviews commenced from 15th September 2021 (first invites) until 22nd March 2022. Overall, 29 participants were invited to participate, where 16 did not respond, 2 formally declined with the following reasons- “Just to let you know that I’d prefer not to take part in the interview stage”, “I had initially agreed to take part in the interview, but I'm afraid I'm no longer available”, and 1 interview did not record. This resulted in ten participants (five for the initially randomised to the intervention arm, and the other half from the control group who then received the intervention as well) being interviewed, representing 23.8% of intervention completers (n=42). Demographic characteristics are presented in Table 4.
Interviews lasted for a mean length of 27.99 minutes (range: 19.57 to 41.08 minutes). Participants were given an opportunity to ask any questions once the interview had finished and the recording had been stopped. Field notes were not taken.
Thematic analysis of interview data led to the identification of four interlinking themes (see Figure 4), summarised with representative quotes below.
Theme: Individual-level impacts from the practical skills and techniques
Overall, participants reported that the CBT approach was very beneficial and resulted in positive changes in multiple areas of their life. Compared to other psychoeducational programmes, practical approaches were more dominant in REST, which was received positively.
“I do think they were probably the sort of the difference between sort of other wellbeing things that I’ve done before, which was sort of more information that’s sort of given to you but you don’t always know what to sort of do with it. Whereas this course was very like practical-based, giving you tools and things to actually overcome little stresses.” – Participant I
Whilst it was recognised not every technique works for every person, participants mentioned there were several new techniques and takeaways they learned throughout the programme, which aligned with their personal preferences and circumstances. Of particular note, the REST diary, the cognitive reframing exercises and the stepping away strategy often used in Acceptance and Commitment Therapy (ACT), were all highlighted as particularly useful.
“it did actually give me some tools that when my brain was racing at night and particularly when kind of anxiety is high, I could use some of the things that I learned to try and quell some of that and to … stop it from spiralling into something a bit more stronger and a bit more serious. So, I think it did that well and certainly these tasks then and sort of lessons that I've learned that I can then put into practical sense as well.” – Participant F
Several participants reported how the programme led to improvements in self-awareness and reflection. More specifically, it helped them to identify not only their external sources of stress, but also to become more aware of their own internal stress responses and behaviours. In turn, this helped them to normalise and accept their emotional responses to stressful situations.
“It made me realise more than anything that where my main cause of stress was coming from. And, really, what the reason for it was... But I could then see the behaviours that I was … showing were … coming out as stress... So I was able to identify those as well.” – Participant D
“Because a lot of the time…it’s normal to have different emotions, positive and negative, so it’s just sort of realising it’s not a terrible thing to be annoyed or whatever. But it’s just sort of recognising it at the time, taking a bit of a breather and realising … it’s fine to just settle back down again.” – Participant I
Although participants acknowledged some stress in life is inevitable, the programme provided them with the skills and techniques to manage their emotions. As touched upon by participant F, these techniques helped to prevent the vicious cycle of stress from spiralling into worsening mental health problems.
“My stress levels are high, which I mean let’s face it in this world stress levels are high, but the, shall we say, the peaks are less high than they were because I’m making an attempt to stop them getting that high” – Participant A
In addition to the improvements in awareness and emotion regulation management, participants also noticed positive changes in other areas of their life. For example, some participants experienced improvements in their sleep, whilst others developed an improved sense of self-perception and agency over their life.
“I would say I'm better at falling asleep which is a positive for me… and it allowed me to kind of switch up my routine and to think about what sleep routine was before going to bed. And often, I would do things on my phone or scroll through things just to try and get myself to kind of switch off and fall asleep to get overtired almost. But, actually, it kind of gave me different things to think about, to challenge my brain in a different way. So, I would say it improved my sleep to some extent and certainly kind of pushed the worry bunnies away a little bit” – Participant F
“... and things you just like, you know, spending a bit of time to put some moisturiser on your legs… I just, I hadn’t done that for months … and I did go out and buy some clothes, actually… It did make me think about taking, yeah, just more active agency … to your own life rather than sorting out the life of others.” – Participant G
Furthermore, some participants discussed how the programme initiated even broader impacts, at both an individual and wider community level. Participants mentioned that the programme acted as gateway to speak about mental health more openly and seek support. Moreover, Participant D also noted that their increased awareness and understanding of stress and mental health was particularly beneficial in their managerial position at work, to help them to identify potential signs of stress within their team.
“… there are so many people that can benefit from this even as a starting point to be like this is a really good opportunity and actually identifies other things that you could then tap into and you could identify maybe, kind of it might give some of the confidence to speak out a bit more and maybe ask for help in other areas as well.” – Participant F
“So, for my own stress and dealing with that which is why I signed up, it was useful, but … going through it, I actually thought .. if I can see, or pick up those signs of stress in others … it’s something that I could like… reach out to them and just try and you know, start to either make sure they’re okay, or try and assist them, by … pointing them to the right place for help” – Participant D
Most participants discussed utilising the techniques they learnt throughout the programme in the future. Some mentioned they had already integrated them into their daily routines, whereas others referred to their toolbox of techniques they can turn to when needed. Related to this, they also mentioned that participation encouraged future uptake of similar digital psychological intervention programmes.
“… a couple of tools that I took out and started putting into practise in my general day-to-day with my own sort of personal journal. And I’ve been using that pretty steady for a few months now. It does help.” – Participant B
“And I probably would do something similar again actually. I'm probably more likely to sign up to something that’s digital than I was before.” – Participant F
Theme – ‘Anyone could pick that up and find it very useful’
As highlighted above in the previous subtheme, several participants expressed that they would like for the programme to be rolled out more widely, due to the benefits it could have for so many individuals. Participants discussed how it could be useful for anyone along a broad spectrum of emotion regulation difficulties, from those with general stress and worry to those with diagnosed generalised anxiety or depression. Particularly given the limited capacity and resources existing mental health services have, participants reported how the REST programme could be a good alternative.
“So, I feel like it was for anyone who’s suffering with generalised anxiety or depression or even just worry in general, I think anyone could pick that up and use it and find it very useful… I think for me, I could be anyone going into that as long as I’m stressed and I need help.” – Participant B
“I would love to recommend this to people. I kept signing people up. When I've done the first week, I was like, please join this. I think it's amazing. So, I'm sending it around to people. Because I think with counselling and services being so overstretched, something like this is a really good self-guided method and easy to use. I think it's fantastic and I think it should be available longer term to a lot more people.” - Participant F
More specifically, it was noted the programme was particularly useful when the need was higher. Around half of the participants reported how the programme “couldn’t have been more timely,” due to the duration of the programme coinciding with a stressful period in their lives. The timing of the programme not only gave participants the opportunity to directly apply their learning, but also helped to prevent what could otherwise have been stress-related sickness absence.
“I valued the time, just to come out and reflect, actually, on how to manage the stress that I was experiencing. It couldn’t have been more timely, I think if I hadn’t been on it, I probably would have gone off sick.” – Participant G
The programme provided real life examples to showcase how participants can apply the techniques and skills to their own lives. Several mentioned these to be so relatable and how they resonated with them and could relate them back to situations in their own lives.
“There was one bit particularly about a woman who'd gone to do a presentation, and someone hadn't turned up and she was getting really angry and riled. And then, it kind of showed you how to maybe reframe that and think about it differently. That was really applicable. That's like a genuine thing that's happened in my life. So, that was great.” – Participant J
“I could apply it immediately to the situations that I was experiencing… It wasn’t like you had to work through a hypothetical crappy example, I could put it immediately into practise and it very much helped.” – Participant G
However as mentioned previously, not every technique works for every person and some elements were less relevant than others, due to an incompatibility with an individual’s specific situation.
“So I’d say there were some weeks where it’s a little bit less kind of relevant, but then there were some weeks that it was more relevant, but I would say overall, it was quite interesting. Like even if you found it less relevant, there was still something in there that you could sort of relate to as well, and it was kind of an interesting sort of side to it.” – Participant I
Some participants also noted that if they were already familiar with a technique, they did not get quite as much value from those elements of the programme.
“I don’t think there’s anything I especially disliked. I’m quite aware of some of that, because I’d seen so like, I’m already using some of the little strategies for it… I wouldn’t say I disliked it, but I probably didn’t get as much from that, because I already am aware of those, but obviously that’s just an individual thing.” – Participant E
Theme - Programme structure and design facilitated engagement
Participants reported multiple elements of the programme design, online structure and delivery facilitated their engagement with it. One of the key components included the staggered release each week, such that there was not an overwhelming amount of content to complete at any one time.
“So, it’s just much more engaging than like a whole chunk of text to just read through. You just break it down nicely into little chunks.” - Participant C
“I think if it was shorter, condensing down the material would almost dilute it a little bit and taking too much in all at once would probably make it a bit harder to deal with. So, I liked the amount of content you got each week, it didn’t seem too much at any point.” – Participant B
“I found the format really easy to use. I liked that it was all broken down into sections. It kind of made it quite manageable to go through. So, I didn't find it too time consuming or anything like that”- Participant H
Some participants related this staggered release of content to how it aligned with and complemented their learning style.
“Having it split up like that so that this week, this is what we’re going to look at, next week we’ll look at something else… I found quite useful because it did kind of focus the mind on one particular way of looking at things or one particular thing to look at. So that was definitely useful for me, … it matches the way that I like to look at things, to split things up into smaller chunks and consider them like that.”- Participant A
“I liked the fact that it started off, ‘just write this bit down’ and then later on… ‘now add this bit, now add this bit’, and it built up… and throughout the programme it was a case of at the end of it, you’re writing down—they can just be like five bullet points a day but you’re writing down a decent amount, that helps get things out of your head.”- Participant B
Participants also reported on how engaging the online platform interface was, relating to the multi-media components and general ease of use and navigation. Participants similarly reported the variation in content as beneficial for individuals with different learning styles.
“there’s obviously a visual and a text there at the same time, which is always handy. I’m quite of visual person sometimes and… I like to read and see when I’m listening, I’m not great at just listening.”- Participant E
“And I really liked the video versus the kind of tasks versus the visual. I think you really captured the different sorts of learning that people can do as well it worked well from the visual learner so it was nice and easy to be able to kind of spot”- Participant F
“But in terms of navigating and you could really clearly see on the left-hand side where you got to, which week you were up to, what was incomplete. So, for me, it was really nice to be able to see the progress as well because you can almost feel, “I've done that.” That’s quite a nice positive thing to see.” – Participant F
However, some participants noted the programme may be better suited to more technologically minded individuals.
“I can see that if it was somebody who was less technological… somebody like my mother who was not at all technological, she might struggle with it a little because she’d only be able to do it when she was sitting in front of the computer and she’d probably struggle to get into it and.. so, I think it definitely was something that suited somebody who is kind of computer minded but might not suit everybody.” – Participant A
“I loved the fact that everything was online, but I know that probably wouldn’t work for someone who’s not computer-based.” – Participant B
Nevertheless, certain aspects of the programme helped increase accessibility such as the availability of transcripts for all multimedia content, handouts participants could download and print to use outside of the platform as well as the use of lay vocabulary used to convey complex psychological concepts.
“I also really liked the transcripts because I'm not great…I know this is more accessibility, but I find sitting through and watching videos more of a task than just reading a transcript. So, I found that really useful that I could just read it at my own pace and kind of move on, than have to sit through the video.” – Participant H
“The resources were good in terms of like they were all like downloadable and stuff. So, I have … downloaded and have like now a little catalogue. So, I really like that…”– Participant C
“It was really simple in terms of the information. It wasn’t overwhelming or too scientific.”- Participant F
The online nature of the programme also facilitated engagement, due to the flexibility it provided to be completed whenever was best suited. This allowed participants to flex the programme around both their personal and work commitments, particularly given the hybrid working arrangements many were following at the time of their participation.
“I liked the fact that I could just sit with what I was worried about and what my concerns were, and work through the programme in my time, where I wanted—whether it was at my computer or on my phone or at Costa on my iPad or laptop …”- Participant B
“…it wasn’t sort of like a live session, you’re more flexible in how you could approach it, which I quite like that flexibility. And especially since I was working from home at the time… I think that flexibility was like really appreciated.”- Participant I
“… I’ve also got a now 7-month-old baby, so, again, like it’s quite hard to just be out of the house or book an absolute fixed slot… it meant I was getting what I was needed but I wasn’t feeling panicked…had it have been you must be online at, you know, 6:00 every Monday, I might not been able to commit to that at the moment.”- Participant E
The online nature of the programme also aided privacy and minimised any concerns participants had regarding stigma related to mental health.
“But I also think it's a good thing that.. it's online because then you can do it anywhere in private space. Whereas I guess if you have to go and see people or go into a particular building or go into an area, you might then feel worse because you might think someone might see me as I'm going. Whereas, there was no one to really know what you're doing. I just incorporated it into my normal working day so no one would know really unless I told them.” – Participant F
Relating to privacy and stigma, the overall delivery structure of the programme facilitated engagement as well. Participants noted that the delivery by independent providers but within the workplace setting where employers acted as gatekeepers and allowed employees take time out-work helped them overcome feelings of guilt looking after their mental health during working hours.
“… But because our employer made it quite clear, this is something that we’re encouraging you to do, it’s something we’d like you to do, so, this is protected time, that you can take the time to do this. And I found that quite helpful.”- Participant I
“…But I think because it's not within my team. It's not like my team reviewing it or knowing anything about me, therefore, I trust that the university is confidential in the way that it needs to be in that I haven't worried too much, but I can see that would put some people off, but yeah, it's not within my team, so I've not worried too much about.”- Participant H
Theme- ‘This isn’t a quick fix’
Participants reported that their journey to lowering their stress and improving emotion regulation difficulties was not quick nor straightforward. Participants acknowledged how further effort, repetition and practice of skills and techniques learnt in the programme were required beyond the scope of the programme to achieve optimal results and benefit fully from the programme. Particularly given the self-guided nature of the programme, this meant participants relied on self-motivation to maintain engagement with the programme, which was challenging at times for some.
“… a part of realising when I was going through the programme, this isn’t a quick fix. However, many techniques are there, it’s not just a case of, oh well, I’ve tried that technique, ooh ka-ching it’s worked… And I wasn’t expecting that. But, this will take a considerable amount of effort on my part, and I, as much as I say, I know why the reasons are, because of what’s going on at work, part of it is down to me to help fix as well though, it’s not just down to them to do.”- Participant D
“I think a barrier was probably myself at times because it wasn’t necessarily out there as a priority… because it's digital so we don’t have anyone being like you need to be here at a certain time so it is all the way down to self-motivation.”- Participant F
“if you don’t use it you lose it… So that kind of keeping, making time to keep on top of things with this, and to keep my mind kind of like engaged with it, it’s something that I need to do. And that’s the other thing that I’ve not had time to kind of like do really.”- Participant D
Time pressures were frequently reported as barriers to both short- and long-term engagement with the programme. Participants mentioned finding it difficult to fill in the REST diary, as they struggled to find time to complete it between their work and personal commitments. Sometimes these time constraints meant participants experienced less emotional or behavioural change than they had initially expected, as they had not had as much time to implement and practice the techniques as they would have liked.
“I think, the time is just the big thing, and obviously you have to make the time for what’s important, and it’s really difficult when obviously there’s a lot going on… It’s one of those things where I know I need to try and do something to help with this, but actually time pressure is massive. So finding that time and getting round to doing the diary regularly, when you’re sort of flying around and finally sit down at 9:00 at night, that’s the unavoidable barrier I think.”- Participant E
“But mentally, I was hoping it would help more than it would, and I’m not saying it didn’t, what I haven’t had time to do, and.. is actually make time to practice. That’s what I was saying about, like look at the techniques, use them to my advantage, find out which ones work for me… And, that’s what I haven’t got to, and that’s, I wouldn’t really say that’s a reflection on the programme, that’s reflection on the fact that I haven’t got time or, I haven’t made time”- Participant D
In addition, some participants mentioned that further support and guidance whilst completing the programme would have enhanced their experience and increased engagement. This related to participants’ desire for more personal support, an opportunity to ask questions on any elements they were unsure of or for further guidance on how to put some of the techniques into practice. Some suggestions included running group sessions, being offered a couple of “face-to-face” sessions with someone, or requesting on-demand guided support on the platform through a chat box to ask a question while completing the content for that week without having to contact the research team by email and wait for a reply, which was an available option already.
“But, I sometimes felt slightly on my own doing it, rather than, I’m not sure whether like, people would have time to do things in a group, or whatever, but I think, like during the course of it, to have a couple of maybe touch- sessions where you kind of catch up with somebody, a bit face-to-face.” “Just like sometimes a little bit of reassurance that you are going down the right routes… But, yeah, just like, a kind of a couple of contacts would have been useful”- Participant D
“I feel like on the site as you’re going through the materials, if there was an opportunity to have maybe an open chat with someone or the opportunity to communicate with someone, just in case you get a little lost or you’re not quite grasping something or even if you just need someone to talk to about something that you’ve read—that would be really useful.”- Participant B
Further guidance and signposting on how to maintain their progress and continue utilising the skills and techniques learnt beyond the programme was also suggested by participants.
“but yeah to go to the next step, I guess I'd need somebody to…whether they're reading the rest diary or whatever, but somebody to kind of guide me through how I actually implement the practical stuff and use it day to day” – Participant H