Impact of an Automated Internet-Based Cognitive Behavioral Therapy Program on Suicide Thinking and Risk Among US Rural Adults

Cognitive Behavior Therapy (CBT) is an evidence-based psychotherapy for mental disorders, including depression. Internet-based CBT (iCBT) programs increasingly are showing similar impact to clinician-delivered CBT. We assessed the impact of Thrive, a fully-automated iCBT depression treatment program on suicidal thinking. Participants were randomly assigned to Thrive (n=218) or a waitlist control group (WLC, n= 230). Intent-to-treat analyses tested for group differences at 8-weeks in suicidal thinking (CHRT-SR 3 subscale, primary outcome), and secondary outcomes including depression symptoms (PHQ-9), anxiety symptoms (GADS-7), work and social adjustment (WSAS), and resilience (CD-RISC-10). Using self-reports, participants were evaluated at baseline, 4 and 8 weeks for each outcome. Thrive program adherence (n = 218) was assessed by number of lessons completed. The Thrive group was 38.7% less likely than the control group to present with suicidal thinking at 8 weeks (odds ratio 0.61, p = 0.10). Comparison of 8-week depression symptom slopes showed statistically signicant effects favoring the Thrive group (WLC = -3.04 vs. Thrive = -4.32, p = 0.007) (d = 0.08); no other signicant group differences were observed. Lessons completed were signicantly related to lower PHQ-9 (p = 0.026) and GAD-7 scores (p < 0.01). Study results are consistent with a previous study showing nonsignicant effect of an automated iCBT program for reducing suicidal thinking, but a signicant positive impact on depression symptoms among rural US adults. Future studies should test whether strategies for boosting lesson completion are successful in enhancing the ecacy of Thrive to reduce suicide risk.


Introduction
Epidemiologic studies have consistently found greater suicide prevalence and trends of increasing suicide rates among US rural populations compared to urban populations [1][2][3][4][5][6][7][8][9]. Data from 2008-2018 show ageadjusted suicide rates growing faster in rural areas compared to urban areas with an increase of 48% compared to 34% [8]. Hence, there is a greater need to address risk factors for rural US adults.
Research has identi ed a number of factors placing rural communities at greater suicide risk, including greater geographic or social isolation, economic strain, access to lethal means such as guns, and long distances to and/or limited availability of mental health treatment resources compared to urban or suburban communities [10]. In the US, Western mountain states have been labeled the "suicide belt", with both higher suicide rates compared to other US states [11]. The vast majority of communities in these states are rural and have signi cantly greater barriers to accessing mental health care, including stigma, cost, and inability to schedule appointments. For example, in Montana, more than 50% of counties do not have a psychologist; over 65% do not have a psychiatrist [12]. Other universal barriers include stigma, cost, and inability to schedule appointments are also present [13]. Yet, the need for care is comparably greater in Montana, given the state's higher depression prevalence than the national rate (24.1% versus 19.7%) [14] and its long-term consistent top three ranking of states with the highest suicide rates in the nation [15]. Given greater barriers to care, novel solutions are needed to address suicide risk in rural communities. One recommendation is to disseminate Internet-based psychoeducation programs [16]. Internet-based programs that teach Cognitive Behavior Therapy (iCBT) skills have emerged as evidence-based solutions to address depression and anxiety symptoms [17]. Fully-automated versions of iCBT programs (those not requiring human support) have the additional advantage of partially addressing the enduring shortage of mental health care providers [18].
Currently, most iCBT studies have been conducted outside of the US [19]. This study is one of two randomized clinical trials (RCT) evaluating a culturally adapted iCBT program called "Thrive" with rural US adults [20]. The rst Thrive trial with 343 adults demonstrated signi cant 8-week and clinically meaningful reductions in depression and anxiety symptoms (effect sizes d=0.63 and 0.47, respectively) in the Thrive group compared to the waitlist control (WLC) group [21]. Furthermore, longitudinal analyses showed that these improvements were sustained for 12 months [22]. The present study builds upon this earlier work with the primary aim to assess the impact of the Thrive intervention on suicidal thinking as the primary outcome measure. A second aim was to assess the relationship between program adherence and outcomes.

Trial Design
We used a randomized controlled trial (RCT) design, randomly assigning participants to immediate program access (THRIVE group) versus 8-week delayed access to Thrive (WLC group). Potential participants were provided with a link to the study website, consented electronically, and presented with the same assessment battery at study entry (baseline), 4-and 8-weeks. Study eligibility criteria included being at least age 18 years, Montana residency, a score of ≥ 5 on the Patient Health Questionnaire (PHQ)-9 [23], and having access to a device (computer, tablet, or smart phone) with broadband Internet. Compared to the rst trial, the current study methodology provided more modest incentives for research participation ($10 versus $25 with the rst trial) for each completed follow-up assessments. The study protocol was approved by the Institutional Review Board of Montana State University. The trial was registered at nihtrials.gov: NCT03595254.

Participants and Procedure
Participants for the study were recruited between November 2018 and April 2020 using presentations to health care providers, yers, posters, social media advertisements, and state Extension agents. Of 1,102 individuals assessed for eligibility, 729 were deemed eligible and consented to participate. A simple computer auto-generated program was used to randomize 366 THRIVE group participants and 363 WLC group participants. The analytic sample was further reduced by eliminating those who either: 1) were deemed fake participants (n = 249; we refer readers to the rst Thrive RCT publication [21] for details concerning identi cation of fake participants], 2) entered invalid email addresses (n = 7), 3) re ected duplicate participation (n = 27) or entered an invalid date of birth (1), or 4) did not complete the suicide risk baseline assessment (n = 27). The nal analytic sample included 218 THRIVE group and 230 WLC group participants (see Figure). All participants were told to keep receiving the care that they were currently using.

Intervention Description
Thrive is a fully automated intervention for depression based on CBT principles and techniques, including cognitive restructuring, behavioral activation and social skills training [24]. These CBT themes are captured in each module (Constructive Thinking, Rewarding Activities, and Assertive Communication) with each module containing 10 lessons and suggested exercises for users to practice off-line as homework pertinent to their own self-identi ed goals. Thrive uses videos, interactive tools, and sophisticated algorithms that dynamically adjust the individual's use of the intervention, including a suite of 320 videos that explain CBT concepts, demonstrate skills, and present case histories of individuals who used CBT skills to improve depression symptoms. The program also features periodic PHQ-9 self-assessments, followed by tailored feedback based on the scores. Over a third of the demonstration and case history videos were replaced with new videos featuring rural characters, story lines, and settings; all other features of the Thrive program (i.e., didactic videos, interactive tools, and algorithms) were unchanged.

Study Instruments
Demographic Variables: Participant demographic information included age (years), gender (female vs male), race (White vs other), marital status (married/domestic relationship vs all other statuses), employment status (employed full-time/part-time/student vs unemployed/retired), educational attainment (some college without a degree, bachelor's degree, master's degree or higher), and rural classi cation (urban vs large rural/small rural/isolated). Participants' addresses (zip codes) were used to determine 10 ruralurban commuting area (RUCA) codes which classify United States census tracts based on population density, urbanization, and daily community (USDA Economic Research Service, 2020). These codes were further collapsed Thriveo 4 codes: urban, large rural, small rural, and isolated. Participants also selfreported whether they were receiving any care or taking medication(s) for mental health treatment.
Primary Outcome Measure: The primary outcome measure was the 3-item Suicidal Thinking subscale (CHRT 3 ) of the 7-item Concise Health Risk Tracking Self-Report (3-15 higher scores indicate greater suicidal thinking in the past two weeks; Cronbach Alpha = 0.80) [25,26]. The measure's three items were combined and dichotomized to form a binary outcome measure of suicidal thinking (agreement/disagreement). The three items are: "I have been having thoughts of killing myself", "I have thoughts about how I might kill myself," and "I have a plan to kill myself. An "agree" or "strongly agree" response on any of the three items indicated presence of suicidal thinking (coded as 1). Participants who self-reported neither agree nor disagree, disagree, or strongly disagree on all 3 items indicated an absence of suicidal thinking (coded as 0). Secondary Outcome Measures: Pre-speci ed secondary outcomes included: depression symptom severity, measured using the 9-item Patient Health Questionnaire (PHQ-9; score range: 0-27; higher scores indicate greater frequency of symptoms in the past two weeks) [27]; anxiety symptom severity, measured with the Generalized Anxiety Disorder Scale (GADS-7; score range: 0-21; higher scores indicate greater frequency of symptoms in the past two weeks) [28,29]; daily functioning, using the Work and Social Adjustment Scale (WSAS; score range: 0-40; higher scores indicate a greater adverse impact of depression on daily functioning) [30]; and resilience, as measured by the Conner-Davidson Resilience Scale (CD-RISC-10; score range: 0-40; higher scores indicate greater resilience) [31].

Data Analyses
The dichotomized primary outcome variable was modeled using a logistic regression model within a Generalized Estimating Equation (GEE) framework with repeated measures to compare the THRIVE and WLC groups suicidal risk over time [32]. This model contained xed-effects terms for treatment, time (baseline, week 4, week 8), treatment × time interaction, and the baseline mean suicidal thinking score as a covariate. For easier interpretation of the treatment effect, odds ratios were estimated as part of the binary logistic model. The change over 8 weeks in the for each continuous secondary outcome measures was compared between the THRIVE and WLC groups using a linear mixed model analysis of repeated measures [21]. All models contained xed-effects terms for treatment, time (baseline, week 4, week 8), treatment × time interaction, and the respective baseline mean score for each outcome (prior to the intervention) as a covariate. For better interpretation of the treatment effect, the least squares mean (adjusted treatment mean scores) were estimated as part of each mixed model. Rates of remission and relapse were assessed longitudinally for the THRIVE group using the PHQ-9. Remission was de ned as a treatment response in which an individual with mild, moderate, moderately severe, or severe depression at baseline (PHQ-9 scores ≥5) achieved a subsequent PHQ-9 score lower than 5 at 4 weeks and/or 8 weeks. Relapse was de ned as a PHQ-9 score ≥10 for those who had achieved remission at 4 weeks.
Analyses of Thrive program adherence (Thrive group only) were completed by assessing the longitudinal change over time in the outcomes to assess the relationship with lessons completed in the program. Using separate logistic regression models for the primary outcome and linear mixed models with repeated measures for secondary outcomes, the adherence measure (lessons completed) was assessed at baseline, 4 weeks and 8 weeks for each outcome [22]. All models contained xed-effects terms for the lessons completed, time (baseline, week 4, week 8), and respective baseline outcome measure as a covariate.
Statistical analyses were performed using SAS software (Version 9.4, SAS Institute, Inc, Cary, North Carolina). Maximum likelihood estimators allow e cient parameter estimation using only available data under an assumption of missing at random [33][34][35]. The level of signi cance was set at α = 0.05 (twotailed). Bonferroni adjustments were assessed to ensure the 95% CIs for the point estimates of treatment group effects matched the signi cance levels in the corresponding test. A priori evaluable sample size for a statistical power of 80% was estimated (enrollment of n=99 per group).

Sample Characteristics
In Table 1, we present descriptive statistics of study participants. The mean age of participants was 42.7 (± 13.7) years. The sample was comprised predominantly of women (83.1%) and White participants (89.5%) who were employed and/or a student (80.3%) and residing in rural areas (73.2%). Over 50% of participants reported having a Bachelor's or higher degree; nearly 60% reported receiving some of mental health treatment at baseline.  Table 2 presents the longitudinal mean 8-week trends for the primary outcome. Suicidal ideation was reported by 28.1%, 17.6%, and 12.3% of participants at baseline, week 4, and week 8, respectively (Figure 1). The predicted odds of elevated suicidal risk (CHRT-SR 3 ) for the intervention group showed a steeper decline from baseline to week 8 than that of the control group but this difference failed to reach statistical signi cance (odds ratio 0.62, p = 0.10); the Thrive group was 38% less likely than the control group to present with suicidal thinking following the entire 8-week follow-up period. Table 3 presents the longitudinal mean 8-week trends for each secondary outcome. We found a statistically signi cant (p = .007) depression severity slope (THRIVE = -4.32 vs. WLC = -3.04) showing contrasts favoring the THRIVE group (d = 0.08). No signi cant between-group slopes were observed for the remaining secondary outcomes.

Discussion
This second RCT of Thrive, a fully automated iCBT program, demonstrated three ndings. First, the iCBT program did not signi cantly reduce suicidal thinking among rural US adult participants when comparing between group differences. As to why the suicidal thinking outcome did not achieve statistical signi cance as did depression symptom severity, it could be that for many of those experiencing suicidal thinking more intensive interventions are needed. A second nding demonstrating signi cant improvements in depression symptoms, a risk factor for suicide risk, is heartening given that US rural residents face substantial barriers to mental health care [13]. For rural residents, iCBT programs may help to address accessibility issues such as cost of care as such programs have been shown to be cost-effective compared to traditional forms of care [36]. These programs may also be useful as complementary care to traditional forms of mental health care. A third nding from this study shows that program adherence (number of lessons completed) is associated with improvements in depression and anxiety.
Besides our earlier trial of Thrive, we know of only six other RCT studies examining an iCBT interventions on suicidal thinking [37]. In all six studies, the intervention group showed signi cantly reduced suicidal ideation compared to the control group. Four of the studies used the Beck Scale for Suicidal Ideation [38], one the Suicidal Ideation Questionnaire, and one the Intensity of Suicidal Ideation subscale [39] of the Columbia Suicide Severity Rating Scale [40]. Given the diversity of suicidal ideation outcome measures used, it is worth an inquiry comparing these measures across studies. Also, more research in needed to understand the impact of iCBT interventions, alone and in conjunction with other suicide prevention strategies [41].
Unlike our rst 8-week RCT of Thrive, we did not nd statistically signi cant group differences for anxiety symptom severity, social functioning, or resilience. In our rst trial, we reported small to moderate betweengroup effect sizes favoring the THRIVE group for each of those secondary outcomes. Of the iCBT trials that have examined anxiety, the results are also mixed with four studies showing no signi cant group differences [42][43][44][45] and three reporting small group differences [46][47][48]. Four trials assessing daily functioning/disability found either no signi cant group differences [45,49] or a moderate effect size [46,50]. We did not identify any studies examining iCBT impacts on resilience.
In the rst Thrive trial [22], the number of lessons completed predicted reduced depression and anxiety symptom severity and improved social functioning. Yet, in this second trial, our program adherence analyses produced less consistent ndings of lessons completed with each outcome measure. compared to the rst trial. Speci cally, in this second trial the number of lessons completed only predicted depression and anxiety symptom severity. In interpreting our overall results, it is important to note that participants completed an average of 4.6 lessons, compared to 7.7 lessons completed in the rst trial. At present, while experts agree that adherence is an important potential predictor of treatment outcome in digital interventions [18], there is no agreed-upon measure or set of measures of program adherence in iCBT or other digital mental health interventions. To increase adherence, experts suggest that Design Thinking should be applied in the development of future digital mental health interventions [51]. Design Thinking addresses existing barriers to user engagement, which is a primary path to program adherence and promises to overcome the "empathy gap" that may exist with digital mental health programs by focusing on the emotional and motivational nuances of users [51].
This second Thrive trial shared implementation features (exception noted below) and eligibility criteria with the rst Thrive trial, yet ndings did not fully replicate. Among the recruitment strategies we described, only the second trial employed added social media advertisements (on Facebook and Google). More participants in the second trial indicated that they learned about the study via social media advertisements or from friends. Whether this change in recruitment impacted the sample composition in terms of demographic characteristics or the program adherence or effectiveness cannot be determined retrospectively (e.g., we did not randomize potential participants to be recruited via different strategies). It is possible, however, that people who tried Thrive on the recommendation of a friend or because they saw an advertisement may approach the program differently (e.g., may have invested less effort or completed fewer lessons) from people who enrolled in the trial on the recommendation of a professional such as their doctor or an Extension agent.
Regarding demographic differences, participants in the second trial did not differ signi cantly from rst trial participants by gender, race/ethnicity, or relationship status, but they were signi cantly more likely to report less educational attainment, to be unemployed or retired, and to reside in an urban area. We cannot rule out that these differences in participants' pro le may have contributed to the differences in study results between the two trials.
The two study samples did not differ signi cantly on baseline measures of all outcome measures (CHRT-7 was not used in the rst trial). Finally, participants' cumulative program usage was substantially less than was usage in the rst trial. Given that in both trials, lesson completion was associate positively with decrease in depression, it is possible that the stronger effects observed in the rst trial may be a function of better program adherence among participants in the rst versus the second trial. Importantly, however, neither trial was powered to test for moderators of Thrive outcomes; hence we can only speculate as to how these differences in sample composition and adherence may have affected each outcome. Future studies are needed to test whether iCBT programs needs to be adapted based on users' demographic pro les. Research also should explore strategies for reaching those who may bene t by using iCBT program and enhancing user engagement to ensure adequate exposure to the various CBT components.
Our ndings of an iCBT intervention's favorable impact on reducing depression symptoms found here and in a previous study [21] should encourage policies that support dissemination of these intervention types for enhancing the work of mental health professionals. With growing evidence of clinical effectiveness and cost-effectiveness [36], policies aimed at increasing access to mental health care should consider supporting the availability of iCBT and other evidence-based programs for rural residents. Rural mental health practitioners and other community leaders should be aware of these programs as complementary to traditional forms of care.

Limitations
Several limitations should be considered when interpreting our ndings. Our 8-week attrition rates were disparate between study groups (nearly 62% among the THRIVE group and 37% among the WLC group), which may have skewed our results. The study used self-assessments for each outcome measure, which may underestimate intervention effects compared to clinician-rated assessments [52]. Our adherence analyses were exploratory, as the study design did not systematically vary dosing of the lessons. Last, our ndings should not be generalized to clinical populations; this was a pragmatic trial of adults who may or may not have received a mental health diagnosis.

Conclusions
In the context of US rural communities' higher suicide rates and signi cant barriers to mental health care [8,13], iCBT interventions offer a scalable strategy to deliver CBT skills for reducing depression, a risk factor of suicide ideation and behavior. Increasing evidence exists on the effectiveness of iCBT interventions for reducing depression and anxiety [53]. Digital psychiatry interventions in general are showing promise [54].
We expect that with converging trends in technology and digital health science, we will be able to capitalize on innovative ways to address the existing mental healthcare access issues through digital mental health tools. Given the great tragedy of suicide in all communities, further study of the role iCBT programs for reducing depression, anxiety and other suicidal risk factors is clearly warranted.

Declarations Data Availability
The data that support the ndings of this study are available from the corresponding author upon request.

Figure 1
Study Consort Flow Chart Note: "Improper group allocation" occurred via an automated randomization glitch.