DOI: https://doi.org/10.21203/rs.3.rs-988972/v1
Background: Mobile health interventions are being widely tried because of their attractive advantages. However, there is not enough evidence for its effectiveness. This study aimed to evaluate the effects of mobile app-based stress management intervention (mSMI) on highly stressed workers.
Methods: A sample of 82 white-collar workers with elevated symptoms of perceived stress (Perceived Stress Scale-10≥22) were randomly assigned to the intervention or control group. The mSMI consisted of three modules: self-management including a work diary, counseling based on cognitive behavioral therapy and interventions focused on music, meditation, relaxation and image healings. Self-report data were collected the baseline and post-intervention. Study outcomes were perceived stress, anxiety, depression, and work engagement. Data were assessed using analysis of covariance with covariates.
Results: There was significantly reduced perceived stress from baseline to 6 weeks in mSMI (t=5.788, p<.001) and control group (t=3.184, p=.003). After adjusting for covariates, the between-group difference in the perceived stress was significantly different (F=4.051, p=.048); however, the effect size was small. There was no significant intervention effect on anxiety, depression, and work engagement. The process evaluation indicated that most participants (85.3%) were satisfied with the intervention and their mental health benefited.
Conclusion: This study found that mobile health intervention facilitated stress management for highly stressed workers. Further studies should address job-related outcomes and mental health symptoms in workers by applying the latest information technology and addressing the limitations of mobile interventions.
Trial Registration: Not applicable.
Workers suffer from various mental health problems, including stress, depression, anxiety, posttraumatic stress disorder (PTSD), self-stigma [1], and insomnia [2]. These mental health problems may manifest as physical diseases, including cancer and heart disease [3], and extended work hours [4], workplace bullying [5], workplace violence [6], and disaster [7]. Therefore, workers should engage in mental health-promoting activities that facilitate inspection and management of their mental health.
To keep up with the advances in information technology, mental health interventions are being recently offered as internet-based programs [8, 9], with utilization of mobile communication and social media [10]. Mobile health (mHealth) interventions that utilize smartphone apps help overcome geographical distance and allow limited face-to-face access [11]. Further, mHealth self-care interventions can be simultaneously administered to many people, which makes them an inexpensive strategy [3].
A systematic review by Wang et al [11] on studies regarding mHealth interventions targeting stress, anxiety, depression, alcohol disorder, sleep disorder, suicidal behaviors, and PTSD found that 14 out of 16 apps significantly reduced mental health symptoms. Further, an analysis of 27 studies regarding mobile app-based interventions and short message service (SMS) demonstrated that mobile apps are highly effective (effect size 0.05–3.37) in reducing stress, anxiety, and depression immediately after the intervention. Moreover, SMS was found to be useful in delivering psychoeducation [12].
Recently, studies on mindfulness meditation mobile-app intervention for university students to reduce stress decreased perceived stress [13, 14]. However, there have been inconsistent reports by studies on workers. One study [15] found that an in-person intervention for workers with work-related stress was more useful than a smartphone-delivered intervention. Specifically, the in-person intervention reduced stress and emotional burnout as well as promoted well-being; contrastingly, the smartphone-delivered intervention only promoted well-being. A mindfulness-based mHealth intervention for healthcare workers during the coronavirus disease 2019 (COVID-19) pandemic significantly improved depression, anxiety, stress, and insomnia in the group receiving psychotropic medications or psychotherapy compared with the control group [16]. This shows that mHealth interventions are effective when coupled with other treatment modalities. Therefore, there remains limited evidence supporting the effectiveness of mHealth interventions in workers.
We aimed to develop and evaluate the effects of a mobile app-based stress management intervention (mSMI) for highly stressed workers familiar with mobile devices who require mental health management.
We used an equivalent comparison group pretest-posttest design to evaluate the effectiveness of mSMI in workers with high perceived stress.
We enrolled workers aged ≥20 years with high perceived stress (Perceived Stress Scale-10 [PSS-10] score ≥22 at the screening test [9]) who owned an Android-based mobile device and could use mobile apps. The exclusion criteria were a PSS-10 score<22, a self-reported diagnosis of mental illnesses, and current use of psychotherapy or psychotropic medications. This study was approved by the Institutional Review Board before data collection (CUIRB-2020-0041).
The sample size was calculated using the G* Power 3.1.7 software [17]. As a reference, we used the effect sizes reported by a study on an internet-based stress intervention for workers (Cohen d = 0.75) [8]. For an effect size of 0.70, significance level of 0.05, and power of 0.80 for two-tailed tests, the minimum sample size was calculated to be 68. Considering a 20% drop-out rate, 82 participants were required.
We performed convenience sampling of office workers at public administration, information technology (IT), education, and health industries through health managers and a recruitment announcement. Among 634 individuals who participated in the screening test, we enrolled 82 eligible participants (selection rate 12.9%). The enrolled participants were randomly allocated to the experimental or control group. At the 6-week posttest survey, there were 64 participants in all groups, indicating a dropout rate of 22.0% (Figure 1).
To determine the contents of the mSMI, we searched for studies in databases using keywords, including “mental health,” “stress,” “workers,” “application,” “intervention,” and “self-management.” The following contents and features for the mobile app were identified through collaboration with a mental health care nurse. First, there was a “self-management” feature. Participants were instructed to log in the times they start and finish work to monitor their daily work hours as well as check their daily stress levels and work intensity. Additionally, the participants could record their self-stress management efforts and methods, including taking walks and exercising. Second, a “counseling” feature was added to help workers requiring counseling and assistance. Participants were asked to describe their feelings and thoughts during a specific problem situation, their current thoughts and feelings, and how they would deal with similar situations in the future. Here, the contents were devised based on cognitive behavioral therapy; additionally, two assigned mental health care professionals checked these notes and replied within 24 hours. Third, to facilitate stress relief, features and techniques such as music [18], meditation [14], relaxation [19], and image healing [20] were utilized. We included a poem recital feature with music and a breathing exercise [20] (Figure 2).
The contents of the developed app were evaluated by an expert panel comprising college professors, nurses, and clinical psychologists. Furthermore, all items were confirmed to have an item content validity index (I-CVI) ≥0.80. Additionally, the quality of the developed app was rated using the Mobile Application Rating Scale (out of 5) [21]. The app was rated 4.16 and 3.72 by five experts and five non-experts, respectively. After correcting errors and modifying the app based on the feedback, the mSMI (called MindCafe) for workers was finalized.
The mSMI was provided as a self-guided intervention. The certification process, how-to guides for each feature, and information regarding privacy policies were explained on the Frequently Asked Questions (FAQ) page. Participants were asked to use the app for 10 minutes twice a week for 6 weeks. During the intervention period, we sent out an SMS thrice a week to encourage participation [12]. The SMS contained messages for motivating the participants to utilize the counseling, music, meditation, and relaxation features. Further, we opened up a chat channel (Kakao channel) for participants to ask questions regarding the app during the intervention period, including app downloading problems, instructions for each feature, and data entry, which we responded to around the clock. Using self-report questionnaires, data were collected at the baseline (before intervention) and 6 weeks after completing the intervention. The control group did not undergo any intervention.
We measured perceived stress using the PSS-10, which was developed by Cohen [22] and translated into Korean and validated by Lee and colleagues [23]. This scale comprises 10 items, with a higher score indicating higher perceived stress.
We measured anxiety and depression using the Hospital Anxiety and Depression Scale. This scale was developed for patients with physical illness; however, it has been validated for use in the general population and has been used for a large worker population in Korea [4]. This scale comprises 14 items, with seven items each for depression and anxiety. The total score for each subscale is 21, with a higher score indicating more severe anxiety or depression symptoms.
Work engagement was measured using the Utrecht Work Engagement Scale–Korean Version [24], which is the Korean version of the 9-item short form of the UWES developed by Schaufeli and Bakker [25]. This scale comprises three subscales, vigor (three items), dedication (three items), and absorption (three items), with a higher score indicating greater work engagement.
Statistical analyses were performed following the intention-to-treat principle (ITT) using the SPSS/WIN 24.0 software (IBM, Armonk, New York, USA). We used the multiple imputation procedure to impute missing sum scores for participants who didn’t complete the post intervention evaluation. Between-group differences in general characteristics were analyzed using the chi-square test, independent t-test, and analysis of variance. Within- and between-group differences in post-intervention changes were analyzed using a paired t-test and analysis of covariance (ANCOVA) with the baseline scores and age as covariates.
Table 1 shows the results of the between-group homogeneity test. The mean age was 38.0 ± 8.96 and 32.7 ± 7.23 years in the experimental and control groups, respectively (t = 3.000, p = .004). There were no significant between-group differences in gender, marital status, education level, income, and workplace size.
Variables |
Categories |
Total (N=82) |
mSMI (N=39) |
Control (n=43) |
χ2/t/F |
p |
---|---|---|---|---|---|---|
N (%) or Mean±SD |
N (%) or Mean±SD |
N (%) or Mean±SD |
||||
Gender |
Male Female |
18(22.0) 64(78.0) |
9(23.1) 30(76.9) |
9(20.9) 34(79.1) |
0.055 |
.815 |
Age(years) |
<30 30~39 ≥40 |
35.2±8.49 28(34.2) 32(39.0) 22(26.8) |
38.0±8.96 8(20.5) 16(41.0) 15(38.5) |
32.7±7.23 20(46.5) 16(37.2) 7(16.3) |
3.000 7.876 |
.004 .019 |
Marital status |
Unmarried Married |
44(53.7) 38(46.3) |
17(43.6) 22(56.4) |
27(62.8) 16(37.2) |
3.032 |
.082 |
Education |
≤University ≥Master |
65(79.3) 17(20.7) |
30(76.9) 9(23.1) |
35(81.4) 8(18.6) |
0.249 |
.618 |
Income (won millions) |
<400 400~499 ≥500 |
49(59.8) 16(19.5) 17(20.7) |
19(48.7) 8(20.5) 12(30.8) |
30(69.8) 8(18.6) 5(11.6) |
5.169 |
.075 |
Workplace size (number of employees) |
<50 50~299 300~1,999 ≥2,000 |
20(24.4) 17(20.7) 18(22.0) 27(32.9) |
8(20.5) 9(23.1) 9(23.1) 13(33.3) |
12(27.9) 8(18.6) 9(20.9) 14(32.6) |
0.702 |
.873 |
Table 2 shows post-intervention changes within each group. There was a significant post-intervention reduction in perceived stress in the experimental group (t = 5.788, p < .001) and control group (t = 3.184, p = .003). However, there were no significant post-intervention changes in anxiety, depression, and work engagement.
Variables |
Categories |
Baseline (Mean±SD) |
6 weeks (Mean±SD) |
Difference* (Mean±SD) |
t-test (p) |
---|---|---|---|---|---|
PSS-10 |
mSMI |
24.79±2.34 |
21.20±4.15 |
3.60±3.88 |
5.788(<.001) |
Control |
24.44±2.56 |
22.23±4.87 |
2.21±4.56 |
3.184(.003) |
|
Anxiety |
mSMI |
9.44±3.60 |
8.59±2.73 |
0.85±3.40 |
1.554(.129) |
Control |
8.67±3.29 |
8.89±3.06 |
-0.21±2.77 |
-0.506(.616) |
|
Depression |
mSMI |
10.79±3.15 |
10.70±3.37 |
0.09±3.46 |
0.165(.870) |
Control |
9.81±2.66 |
9.93±2.70 |
-0.12±2.35 |
-0.330(.743) |
|
UWES |
mSMI |
25.15±8.63 |
26.54±10.14 |
-1.38±5.83 |
-1.479(.147) |
Control |
24.26±8.23 |
25.33±8.24 |
-1.07±5.85 |
-1.203(.236) |
|
*Scores at baseline–Scores at 6 weeks; PSS=Perceived Stress Scales; UWES=Utrecht Work Engagement Scale |
Table 3 shows the between-group differences in the post-intervention scores. There were between-group differences in the age distribution. Moreover, since the baseline score affects the post-intervention score, we adjusted for age and baseline score as covariates. There was a significant between-group difference in the post-intervention perceived stress score (F = 4.051, p = .048); however, the effect size was small (0.23). There were no significant between-group differences in anxiety, depression, and work engagement scores.
Variables |
Difference in Means* (95% CI) |
ANCOVA† F(p) |
Adjusted R2 |
---|---|---|---|
PSS-10 |
-1.03 (-3.03 to 0.97) |
4.051(.048) |
.176 |
Anxiety |
-0.30 (-1.58 to 0.98) |
1.960(.165) |
.272 |
Depression |
0.77 (-0.57 to 2.11) |
0.064(.801) |
.251 |
UWES |
1.21 (-2.84 to 5.25) |
0.082(.776) |
.601 |
*Post-intervention scores in the experimental group–Post-intervention scores in the control group; †Controlling for baseline scores and age; 95% CI=95% Confidence Interval; ANCOVA=Analysis of covariance; PSS=Perceived Stress Scales; UWES=Utrecht Work Engagement Scale |
Table 4 shows the users’ comments regarding their experiences and satisfaction with the mobile app. Most users (85.3%) were satisfied with the app. Two-thirds of the users commented that the mobile app increased their stress management awareness. Furthermore, more than half of the users stated that the mobile app motivated them to change their attitude and behavior towards stress management. Other comments included requests for alert settings, stress measurement using a sensor, music repeat, and periodic content updates.
Items |
Categories |
N (%) |
Other comments |
---|---|---|---|
Overall satisfaction with mobile application |
Satisfied |
29 (85.3) |
Alarm settings Stress measurement using sensor Music repeat Periodic contents updates |
Dissatisfied |
5 (14.7) |
||
Does this mobile application increase your stress management awareness? |
Agree |
23 (67.6) |
|
No change |
9 (26.5) |
||
Disagree |
2 (5.9) |
||
Does this mobile application improve your attitude toward stress management? |
Agree |
18 (52.9) |
|
No change |
11 (32.4) |
||
Disagree |
5 (14.7) |
||
Does this mobile application bring behavioral change in stress management? |
Agree |
20 (58.8) |
|
No change |
9 (26.5) |
||
Disagree |
5 (14.7) |
||
This study developed an mSMI for highly stressed workers and evaluated its effects. We found that mSMI effectively reduced perceived stress. This is consistent with previous findings that a mobile app-based stress management program (called Mind Healer) for nurses [18] and a mindfulness-based mHealth intervention for health care workers during the COVID-19 pandemic reduced perceived stress [16].
However, we found a small effect size of mSMI on perceived stress. A recent meta-review of seven meta-analyses on mobile app mental health interventions reported a small to medium effect size of mobile apps for stress management [26]. Contrastingly, a mobile application administered for 4 weeks to nurses had a high effect size on perceived stress [18]. Our observed small effect size could be attributed to the inadequate treatment dose. We instructed participants to use the mSMI for at least 10 minutes per session twice a week for 6 weeks. However, our logs showed that the participants accessed the app for an average of 10.4 minutes per week. Since we used a self-guided intervention, we delivered information regarding how-to instructions for each feature and personal information processing through the FAQ page and SMS. Additionally, we answered participants’ questions through a chat channel. Nevertheless, the participants did not adequately use the interventions. In the post-intervention satisfaction survey, participants mentioned that the periodic reminder text messages were stressful for them. There didn’t reinforce program engagement. Facilitating program engagement requires an approach that includes enhancing participants’ initial motivation, frequent and effective reminders, education on the importance of active participation, and financial and verbal incentives [27].
Also, as in the method of this study, the limitation of non-face-to-face interventions could have contributed to the failure to induce marked stress relief. A study [15] that administered a 6-week resilience training to health care workers with work-related stress reported that a smartphone-delivered intervention only improved well-being. Contrastingly, an in-person intervention reduced stress and emotional burnout as well as promoted well-being. Therefore, one or two face-to-face coaching sessions should be included to boost the effectiveness of mHealth interventions. If in-person interventions are not possible due to factors, including the recent COVID-19 pandemic, video conferences (e.g., Zoom, Meet, Teams, and Webex) should be conducted at the participants’ convenience to emphasize the importance of self-management and encourage steady participation.
In our study, mSMI was not effective for anxiety, depression, and work engagement. This is consistent with a previous report that a 4-week stress-management intervention for nurses using a mobile app did not significantly mitigate depression, anxiety, and emotional labor [18]. However, meta analyses presented [26, 28] that mobile-app interventions significantly improved on depressive or anxiety symptoms with medium effect size. In particular, a 3-week intervention to reduce stress for university students using a cognitive behavioral therapy-based mobile app (called Feel Stress Free) changed anxiety and depression symptoms [29]. The eHealth stress management for employee has already improved anxiety [9], insomnia [8, 9], burnout [8, 9], work engagement [9], and work experience [8]. Digital interventions improved work and social functions as well as reduced depression and anxiety [30]. Future studies on mHealth intervention should examine changes in mental health symptoms or job-related outcomes among workers. Anxiety and depression increase by 2.15 and 1.87 times, respectively, among manufacturing workers working >60 hours a week [4]. Therefore, there is a need for intervention studies to examine mental health symptoms with respect to extended work hours.
In our study, participants were generally satisfied with the app; however, they suggested several improvements, including incorporating the latest IT such as automatic stress measurement using a sensor or voice control features. User acceptability and adherence rates are especially critical for mobile interventions [31]. Specifically, emotionally relaxed, warm, and appealing designs should be applied to boost users’ adherence rates to ensure effective prevention and treatment of mental health conditions. Therefore, there is a need for customized mobile apps tailored to specific target populations by incorporating the latest technology to apps promoting mental health management.
This study has some limitations. First, the treatment dose was lower than planned, which could have compromised the internal validity. Second, the study was conducted during the COVID-19 pandemic; therefore, external factors, including COVID-19-related depression, may have influenced the intervention outcomes. However, this study is significant since we administered an individualized mHealth intervention on highly stressed workers for preventing mental health problems.
This study developed an mHealth intervention and evaluated its stress management effects on workers highly vulnerable to mental health problems. We found that a self-guided, non-face-to-face mSMI for 6 weeks reduced perceived stress; however, it did not improve anxiety, depression, and work engagement in office workers. Future studies should develop mHealth interventions addressing the shortcomings of mobile-based interventions.