Leveraging Natural Language Processing to Evaluate Young Adults’ User Experiences with a Digital Sleep Intervention for Alcohol Use

Evaluating user experiences with digital interventions is critical to increase uptake and adherence, but traditional methods have limitations. We incorporated natural language processing (NLP) with convergent mixed methods to evaluate a personalized feedback and coaching digital sleep intervention for alcohol risk reduction: ‘Call it a Night’ (CIAN; N = 120). In this randomized clinical trial with young adults with heavy drinking, control conditions were A + SM: web-based advice + active and passive monitoring; and A: advice + passive monitoring. Findings converged to show that the CIAN treatment condition group found feedback and coaching most helpful, whereas participants across conditions generally found advice helpful. Further, most participants across groups were interested in varied whole-health sleep-related factors besides alcohol use (e.g., physical activity), and many appreciated increased awareness through monitoring with digital tools. All groups had high adherence, satisfaction, and reported feasibility, but participants in CIAN and A + SM reported significantly higher effectiveness than those in A. NLP corroborated positive sentiments across groups and added critical insight that sleep, not alcohol use, was a main participant motivator. Digital sleep interventions are an acceptable, novel alcohol treatment strategy, and improving sleep and overall wellness may be important motivations for young adults. Further, NLP provides an efficient convergent method for evaluating experiences with digital interventions.


Introduction
Most mental health disorders have a peak age of onset during or before young adulthood (18-25 years) 1 .
However, less than half of young adults with any mental health disorder receive treatment, with the proportion dropping to around one-third among young adults of color 2 .Clinical services are often unavailable or unaffordable for young adults, and the gap between treatment demand and availability has increased since the COVID-19-pandemic 3 .Further, young adults may not seek treatment for some highly prevalent but normalized mental health concerns, like alcohol use disorders (AUDs) and heavy ("binge") drinking 2,[4][5] .In contrast, young adults show concern for other common wellness issues, like sleep quality, that are related to alcohol use [6][7] .In a sample of young adults with similar levels of sleep complaints and heavy drinking, most (80%) were concerned about sleep whereas very few (5%) were concerned about drinking 8 .Therefore, sleep interventions could offer a more appealing alternative for young adults facing sensitive issues like alcohol risk 9 .
Digital interventions can also increase appeal, access, and convenience of mental health treatment for young adults 10 .This population makes extensive use of digital health tools compared to other adults 11 , with generally positive experiences 12 .Despite a saturated market of mobile mental health apps and other digital tools, user ratings may be discrepant and uninformative to app designers and potential users alike 13 .Over 95% of users stop using most mobile mental health apps after 30 days 14 and adherence and engagement are generally low 15 .In the case of sleep and alcohol apps, user engagement may increase with features like personalized feedback, interaction and support from providers, self-monitoring, and user-friendliness [16][17][18][19] .Further, gami cation may signi cantly reduce attrition in mobile mental health app use in general 20 .With the incorporation of machine learning, apps are also becoming increasingly more personalized.Assessing both users' perspectives and intervention quality is critical to promote higher treatment uptake and adherence in a rapidly growing, global digital health market 21 .Therefore, implementing precise, consistent evaluations of user experiences represents a signi cant gap in digital intervention research.
Incorporating natural language processing (NLP) with convergent mixed methods can support usercentered design for digital health interventions 22 .Natural language processing is an umbrella term for a growing suite of machine learning methods that uses arti cial intelligence (AI) to understand (e.g., summarize, retrieve information) and/or generate language content.Traditional evaluation methods, such as qualitative thematic analysis of exit interviews, can reveal nuance and detail, but can be impractical, especially with large sample sizes 23 .Further, exit surveys using Likert-type satisfaction scales may be subject to response bias.A hybrid approach combining NLP, quantitative survey analysis, and targeted qualitative interview analysis, may reveal broad and rich user experiences 23 while maximizing researcher time and effort.
The current study used convergent mixed methods to evaluate user experiences in a randomized clinical trial of "Call it a Night" (CIAN) and two control conditions (see Fig. 1).CIAN is a personalized feedback and coaching digital sleep intervention for young adults that addresses heavy alcohol and other substance use (NCT #03658954).In the parent trial 6 , CIAN was tested against control conditions comprising either web-based advice only (A) or advice + active diary self-monitoring (A + SM).All participants wore sleep and alcohol biosensors.NLP methods, particularly topic modeling analysis (Latent Dirichlet Allocation) and sentiment analysis, were selected to assess convergence with qualitative thematic analysis of exit interviews and descriptive and predictive analysis with exit surveys.The aims of the study were to leverage these NLP evaluation methods to: 1. characterize young adults' user experiences during the digital sleep interventions for heavy drinking and 2. determine whether user experiences varied with their demography (e.g., age, race, gender, student status, psychiatric diagnoses) or trial condition.

Sample
We recruited 120 trial participants, and 118 completed the exit survey.Half (51%) were female and 80% were white with a mean age of 21.14 years (see Table 1).Three-fourths (74%) were college students.
Most (81%) met lifetime criteria for any MHD or substance use disorder (SUD) based on a diagnostic interview, and most also met lifetime criteria for an AUD (72%).a There were no signi cant differences between trial groups on any demographic variable.

Thematic Analysis of Exit Interviews
Qualitative thematic analysis of exit interviews resulted in overarching thematic categories related to helpfulness of different intervention components and suggestions from participants (see Table 2 for themes and salience; see Supplemental Materials for all thematic analysis de nitions and exemplar quotes).Participants across trial groups provided comments on whether wearing the biosensors (transdermal alcohol ankle monitor and sleep watch) was helpful on its own without consideration of the data collected.Most described Neutral aspects of biosensors and did not nd them helpful or unhelpful without data.Among those who found the biosensors helpful on their own, the most mentioned aspect was that the Ankle biosensor increased awareness of their alcohol use: "The physical monitors all over you…they de nitely make you…think like, 'Wow…should I really be drinking this much?'"Likewise, others noted the General helpfulness of the biosensor(s) or that the Watch biosensor increased awareness of their sleep patterns irrespective of data obtained: "If I saw my sleep watch before I was going to bed, I was like, 'You know what, I should probably try and go to bed early tonight.'"Among those who found one or both biosensors unhelpful, the most common complaint was Burdensomeness of ankle biosensor, including itchiness, discomfort, and di culty completing some activities: "I just really didn't like the [ankle] device…couldn't run as much."Other unhelpful aspects mentioned were Unhelpful without data feedback, Lack of behavior change, Burdensomeness of watch biosensor (e.g., forgetting to push buttons), Stigma of ankle biosensor (e.g., connection to court-ordered monitors), and Ankle biosensor increased drinking.
Active Self-Monitoring (Diaries): Helpful, Unhelpful, and Neutral Aspects (n = 78) CIAN and A + SM participants commented on the helpfulness of completing diaries, a component of their study conditions.Most described one or more helpful aspects, most commonly that diaries Increased mindfulness of sleep: "I found it…helpful to…realize it [my sleep schedule] and…just re ect on…my lifestyle and…how much sleep I'm actually getting."Others noted the General helpfulness of diaries and Increased mindfulness of alcohol use: "I…never really…used to like keeping track [of drinking].So, I'd… drink a lot, and then I'd feel too drunk."Some noted the helpfulness of the New experience of keeping a diary, the Ease of answering questions in the diaries, or the Motivation to change behaviors prompted by diaries.Fewer participants noted unhelpful aspects of diaries, such as Challenges answering questions in the diaries, Lack of behavior change, and Lack of new information (i.e., self-monitoring did not increase awareness of patterns nor motivate them to change behaviors.Some also noted a Neutral aspect of diaries, they were neither helpful nor unhelpful. Personalized, Tailored Coaching (Feedback): Helpful and Unhelpful Aspects (n = 50) Participants in the CIAN condition with personalized feedback and coaching largely described helpful aspects of the feedback and coaching intervention component.Almost all CIAN group participants described the Helpfulness of personalization, the individualized nature of reports and suggestions based on monitoring data: "Having that sort of feedback, both the data and just the explanation behind it, I think that's a really…unique insight into…a part of yourself that…most people wouldn't ever really get to actually see mapped out."Most also noted aspects, such as the General helpfulness of feedback and Helpful data presentations: "She [the coach] made it very easy for me to understand the charts and what everything meant."Only a few described one unhelpful aspect of feedback: Lack of new information, i.e., reports did not contribute new insights into their sleep or drinking.

Most In uential Interventions
Each participant was asked to select which received intervention component(s) were most in uential for behavior change.In order from most to least helpful, participants selected personalized feedback/tailored coaching (i.e., three-fourths of CIAN participants), the website (i.e., over one-third of all participants), diaries (i.e., almost one-third of CIAN and A + SM participants), and biosensors (i.e., less than one-fth of all participants).
Suggestions for Future Interventions

Nonmetric Multidimensional Scaling of Interview Themes
We used multivariate analysis, nonmetric multidimensional scaling (NMDS) 24 , to visualize the interrelationships among qualitative themes and participants.NMDS is an ordination method that condenses variation in matrices, such as participant-by-theme frequency, to a small number of orthogonal dimensions 25 .We used ve orthogonal dimensions based on stress-level testing, and the rst two dimensions are plotted in vector space in Fig. 2. Also, multivariate correlational analysis allowed us to t participant characteristics as factors to the NMDS ordination of themes.Trial group (R 2 = .29,p = .001)and lifetime history of any MHD/SUD (R 2 = .03,p = .04)were signi cantly correlated with NMDS scores.That is, participants' statements about study interventions in their exit interviews were associated with trial group and diagnostic history.Other participant characteristics (e.g., age, gender, race, ethnicity, student status) were not associated with NMDS scores.In the NMDS shown in Fig. 3, CIAN and A + SM participants were closer to themes of feedback and diary helpfulness than website or biosensor helpfulness in vector space, which suggests their preference for feedback and diaries.Conversely, control A and A + SM participants were closer to themes of helpful website aspects in vector space than CIAN participants, and control participants were also signi cantly more likely to state these themes during interviews (X 2 = 27.34,p < .001).

Natural Language Processing of Exit Interviews ( n = 112)
We also used quantitative Latent Dirichlet Allocation (LDA), a topic modeling analysis that identi es the likelihood of terms occurring in topics and topics occurring in documents 26 , with an expanded dataset of 112 participants' exit interviews and coaching transcripts and identi ed nine topics across participants' statements (see Fig. 3; see Supplemental Material for all topic de nitions and example quotes).The goals of NLP analyses were to help qualitative thematic analysis be more targeted and assess convergence of ndings with thematic analysis and exit survey analyses.These topics did not vary signi cantly by trial condition (X 2 = 16.72,p = .40).We found that Awareness with monitoring was the most likely topic in the largest number of interviews (n = 18, 16%).This included participants' perceptions that both active diary self-monitoring and passive biosensor monitoring increased their awareness and mindfulness of their behaviors.The two least frequent topics included the burden of wearable biosensor devices.The topic Strategies, not devices (n = 8, 7%) focused on helpful website aspects, attempts to implement sleep strategies, and challenges with wearable devices, especially the transdermal ankle monitor.Similarly, the topic Feedback, not devices (n = 6, 5%) focused on the bene ts of personalized feedback and coaching while also discussing di culties with wearable devices.
Exit Survey ( n = 118) On the exit survey, participants across conditions (CIAN, A + SM, and A) reported generally positive user experiences (n = 118; see Table 3).On a 5-point scale, participants reported high overall program satisfaction, website helpfulness, diary helpfulness, and feedback helpfulness.Mean feasibility ratings were generally high for the overall program, website, watch biosensor, and diaries.However, mean feasibility ratings for the ankle biosensor were lower based on visual inspection, especially due to interference with clothing, feeling uncomfortable, and being noticeable.Perceived effectiveness was higher among CIAN (Δ = 0.48, p = .008)and A + SM participants (Δ = 0.55, p < .001)than A participants (F(2, 115) = 8.45, p < .001).Further, young adults with a lifetime history of any MHD/SUD rated their intervention as more effective on average than those without diagnoses (F(1, 116) = 4.64, p < .001;Δ = 0.32, p = .03).Adherence to interventions was also high across study phases.Almost all participants (98%) completed the two-week intervention phase, and 96% completed the 12-week follow-up appointment.Regarding monitoring activities, 98% of participants wore the sleep watch biosensor for 14 days, and 95% wore the alcohol ankle biosensor for 14 days.Further, 95% of participants in the CIAN and A + SM groups completed their assigned 14 days of active diary self-monitoring of sleep and alcohol use.

Discussion
The current study used an innovative hybrid approach in which NLP ndings converged with and corroborated more traditional mixed methods to e ciently evaluate user experiences with a digital sleep intervention to reduce drinking.Major ndings showed that participants across conditions generally found all intervention components helpful, satisfying, and feasible, but the CIAN group markedly preferred the personalized feedback and tailored coaching with a health professional they received to other components.Convergent results also underscored young adults' strong interest in gaining a more holistic picture of their wellness in the future and increasing their awareness using monitoring through diaries and biosensors.Our hybrid approach using NLP enabled qualitative thematic analysis to target speci c interview questions more e ciently, optimizing researcher's time.Further, assessing the convergence of ndings between NLP and other methods helped address potential researcher bias in thematic analysis and potential user response bias in exit surveys.

The Importance of Personalized and Interactive Feedback and Coaching
Mixed methods ndings on user experiences, including NLP, emphasized greater effectiveness and satisfaction with personalized feedback and coaching than other intervention components despite generally favorable experiences with web-based advice, active diary self-monitoring, and passive biosensor monitoring.CIAN and A + SM group participants who received feedback and/or actively selfmonitored (diaries) rated their intervention as more effective than A control participants in the exit survey.Topic modeling (LDA) and thematic analysis showed that some participants found web-based advice strategies di cult to implement, and negative themes about web-based advice were associated more with CIAN participants.
Our study adds insight to extant literature on young adults' preferences for interactive digital interventions, including alcohol and sleep treatment.An umbrella overview of meta-analyses and systematic reviews 21 found that digital interventions offering interaction with a health professional or other social support had higher adherence and effectiveness and lower attrition.Mobile alcohol treatment apps providing personalized feedback may be especially engaging 17 .Further, recent research on mobile sleep apps 16,[18][19] highlights increased engagement and satisfaction with personalized feedback and communication with health professionals.Our study adds ndings that individually tailored coaching from a health professional signi cantly enhanced participants' experience interpreting digital health data, which likely supported their understanding.Current research also shows that user-friendliness 17 and evidence-based information 18 increase engagement, which was consistent with our nding that webbased advice was considered helpful and user-friendly, especially by control groups.Therefore, whereas web-based advice and other automated interventions are likely to be helpful to many young adults, tailored feedback and coaching with individualized data are likely to be preferable in the current digital health market.
Empowering Young Adults with Digital Health Tools for General Wellness Consistent with prior ndings 11 , our results showed that young adults particularly want to use digital tools to pursue holistic health and wellness goals.Thematic analysis and LDA with interviews converged to show that participants are interested in understanding varied other lifestyle factors that impact sleep beyond alcohol (e.g., caffeine, diet, exercise, stress, environment).Most participants expressed a willingness to use additional digital tools (e.g., more biosensors, additional diary self-monitoring) to attain whole-health, personalized feedback.LDA results highlighted greater perceived awareness and mindfulness of sleep and drinking through diary self-monitoring and/or wearing biosensors.Thus, while young adults are interested in and capable of using varied digital health tools, incorporating appealing treatment foci could be important to increase young adults' generally low engagement and adherence 15 .
Our study suggests that digital interventions improving overall health may represent a more appealing treatment approach to young adults for potentially stigmatizing diagnoses, such as AUDs.Most available digital SUD interventions for young adults involve web-based advice explicitly targeting alcohol use to the exclusion of other substance use or wellness factors 27 .Whereas young adults may not be concerned about alcohol use or seek traditional alcohol treatment 5 , they are concerned about improving sleep quality 6,8 and other aspects of their general wellness.In this study, LDA with exit interviews showed that participants commonly joined the study to improve their sleep quality rather than address their alcohol use.Further, survey and interview thematic analysis results, respectively, showed that participants with a lifetime history of an MHD/SUD found the current intervention to be especially effective and described intervention helpfulness differently than those with no diagnoses.Therefore, digital interventions, like the current study which focused more explicitly on improving wellness (e.g., enhancing sleep quality), may be especially engaging to young adults with AUDs or other diagnoses who have not presented to treatment.

Implications and Recommendations
Our study has implications for clinical researchers and mobile app designers making user-centered app designs.User experience evaluations of digital health tools should incorporate a hybrid, convergent approach to optimize depth, breadth, and e ciency.Each user experience evaluation method has distinct strengths and potential weaknesses 23 .Thematic analysis of exit interviews can provide detailed information across complex subjective experiences.In the current study, along with post hoc analyses, thematic analysis revealed more distinctions between perceived helpfulness of different intervention components than any other method.However, as sample sizes increase, qualitative thematic analysis can become impractical and unfeasible.Our use of NLP topic modeling (LDA) to derive overarching topics across interview questions enabled us to target and abbreviate our thematic analysis to interview questions focusing on intervention component helpfulness.Exit surveys are also time-e cient like NLP but may be subject to user response bias.NLP, speci cally sentiment analysis, helped con rm the generally positive perceptions derived from exit survey analyses.
Our convergent evaluation ndings also have speci c implications for digital intervention design.Young adults are enthusiastic about receiving a broad, holistic picture of factors impacting their wellness, including different aspects of their lifestyle, in addition to appreciating individualized and speci c coaching.Therefore, mobile mental health apps and linked biosensors should have a variety of choices for active and passive monitoring that can be tailored to user feasibility and interest.Digital tools should integrate seamlessly into young adults' schedules that often include exercise, which may make ankle transdermal monitors challenging and potentially stigmatizing 28 compared to watch biosensors.Also, digital interventions, including for sleep and alcohol use, should empower young adults to actively explore their wellness with support from a health professional.While participants in the current study valued the mindfulness derived from digital health monitoring, they preferred to also receive personalized data reports with explanations and suggested health tips from a coach.Some individuals may lack the expertise to interpret their health data or to devise action plans for improvement.Young adults commonly experience AUDs, binge drinking 2 , and sleep problems 7 .Further, sleep problems in adolescence lead to heavier drinking and AUDs in adulthood [29][30][31] .Notably, young adults who drink heavily may be less concerned about their drinking than their sleep 8 .In general, digital tools that explicitly focus on less sensitive behavioral goals, like improved sleep, with implications for improving potentially stigmatized goals, like reduced drinking, are likely to be more appealing and engaging to young adults.

Strengths and Limitations
Distinct strengths of the current study included a relatively large sample size, especially for a qualitative analysis of exit interviews; generally high adherence across intervention phases and groups; and minor amounts of missing data.Each of these attributes increased the validity of ndings.Further, our hybrid, convergent approach incorporating NLP represents an important strength as these methods are promising and relatively new in digital medicine [22][23] .Our research also had notable limitations.Although missing data were minor, it is possible that additional themes would have emerged in a complete set of exit interviews.NLP helped address response bias in exit survey items, but survey responses were not normally distributed and generally skewed towards higher responses.Our sample was primarily college students, so it may not be representative of young adults in general.Further, study participants were paid, so implementation and uptake may differ in clinical or other naturalistic settings.

Conclusion
The current evaluation demonstrates the value of NLP in convergent mixed methods to e ciently capture broad and nuanced user experiences with digital health interventions.Speci cally, our results show that digital sleep interventions for heavy drinking may increase appeal and access to alcohol treatment for young adults, especially when they include tailored coaching.Broadly, the current ndings emphasize the importance of digital tools for young adults that provide a holistic, dynamic view of health coupled with interactive, individualized feedback.Consistent and precise evaluations that leverage user feedback are critical to support higher uptake and adherence to effective digital health interventions, which can begin to address the mental health treatment demand gap for young adults.

Study design
In the current study, we evaluated user experiences during a randomized clinical trial of CIAN, a novel, two-session personalized feedback and coaching digital sleep intervention developed for young adults to reduce drinking (N = 120).Participants were randomly assigned to one of three conditions using a 2:1:1 ratio for two weeks:

Recruitment
To be eligible for the CIAN clinical trial, participants needed to a) be 18-25 years of age, b) be uent in English, c) self-report three or more heavy drinking occasions in the last two weeks (i.e., 5 or more drinks on one occasion for men or 4 or more for women), d) score at risk of harm from drinking on the AUDIT-C [32][33] , and d) self-report sleep concerns.Exclusion criteria are described further in the study protocol paper 6 .Participants were recruited using online (e.g., Facebook, Instagram, Snapchat) and in-person advertisements placed around the local community.Online advertisements and yers directed individuals to a web-based pre-screening survey, and those who met pre-screening eligibility were invited to an intake visit for nal eligibility determination.

Exit Survey and Interview
Upon completion of the two-week intervention phase, we asked participants to complete a self-report exit survey (see Supplemental Material for exit survey items).The exit survey included Likert-type and yes/no questions to assess user experiences, including overall satisfaction, intervention helpfulness and effectiveness, and intervention feasibility.Participants were also asked to detail areas for improvement in open-ended response items.
Participants across trial groups also completed semi-structured exit interviews to provide information about their subjective user experiences (see Supplemental Material for exit interview protocol).Exit interviews were administered by three research staff members (two men, one woman), who were trained in the protocol and familiar with study interventions.The exit interview protocol included questions about the following topics: participants' sleep and alcohol use before and after the study, perceptions of these behaviors relative to their peers, use of sleep hygiene strategies from the intervention, helpfulness of intervention components, the impact of payment on study participation, what drew them to the study, and suggestions for intervention improvement or future studies targeting sleep or alcohol use.

Data Analysis
We conducted both qualitative thematic analysis and quantitative NLP with exit interviews.For thematic analysis, we used Braun and Clarke's six-step process 34 , during which we used a recursive and iterative team coding process to ensure rigor and trustworthiness in the process of identifying, naming, and categorizing recurrent themes.Two members of the research team completed an initial open coding of the entire text of a large subset of exit interviews (n = 80) to derive initial themes.Then, guided by ndings from NLP topic modeling analysis, the current thematic analysis targeted key portions of the exit interviews, including questions related to helpfulness of intervention components and participants' suggestions.Three research team members engaged in a rigorous team coding process of the entire dataset of completed two-week exit interviews (n = 107).Using an initial codebook based on previous rst-round, open thematic coding, these three researchers independently coded a random selection of 20% of the entire interview dataset (n = 21) to evaluate interrater reliability.All themes had percent agreement of 95% or greater, but an extensive reconciliation meeting was still undertaken to audit the initial codebook and resolve any disagreements until an overall kappa exceeding 0.7 was reached between each pair of coders.The remaining exit interviews were divided among the three researchers to code using the revised codebook.Auditing was used throughout the qualitative process to ensure transparency, including eld notes maintained by each coder and reviewed by other coders and ongoing check-ins with the entire research team.
To visualize the interrelationship between participants and themes, we conducted NMDS with theme frequency counts using the vegan package 24 in the programming language R 35 .NMDS is a multivariate ordination method that condenses variation in matrices to a small number of orthogonal dimensions 25 .
We selected NMDS for post-hoc analysis of themes because it does not place assumptions of normality on frequency count data compared to other multidimensional scaling methods 25 .To assess the goodness of t when selecting the number of orthogonal dimensions, we used stress level cutoffs and a stress plot (Shepard diagram) 24 .We selected ve orthogonal dimensions, resulting in a good to fair stress score of .13,which is likely to result in minimal distortion 36 .As shown in Fig. 2, we used a multivariate correlation analysis with vegan in R to t factors (trial condition and participant gender, race, ethnicity, student status, and diagnosis history) and a vector (participant age) to the NMDS ordination of participants and themes.This enabled us to assess whether participants' user experiences (as interview themes) varied signi cantly with their assigned trial condition or personal characteristics.
For NLP of exit interviews, we conducted topic modeling 26 and sentiment analyses 37 with an expanded dataset of all of participants' statements during completed exit interviews and feedback session transcripts (n = 112).LDA with the topicmodels package 26 in R was used for topic modeling analysis.To boost rigor and ensure the most parsimonious number of topics (k) were used for LDA, we undertook a preliminary analysis with three model selection methods to test different k values [38][39][40] in the ldatuning package 41 in R and ultimately selected nine topics.Text in participants' statements was preprocessed before NLP, including removing punctuation, numbers, capitalization, and English 'stopwords' as well as stemming all remaining words.The LDA determined which of the nine topics was most likely to occur in each document and which terms were most likely to occur in each topic.Topics were systematically named by closely reading ve or more interview transcripts determined most likely to contain the topic by LDA.Chi-squared tests determined whether topics varied with participants' assigned trial condition.
For sentiment analysis, we used the AFINN lexicon 42 and the syuzhet package 37 in R. The AFINN lexicon is a well-established and commonly used sentiment lexicon 37 that assigns a valence score from − 5 to 5 to selected terms, and documents are scored based on an aggregate of their terms' scores.An aggregate AFINN sentiment score of "0" for a document indicates neutral term valence.Bivariate regression models, including analysis of variance (ANOVA), were used to assess whether participants' trial condition or personal characteristics predicted the sentiment of their statements during exit interviews.
Descriptive and predictive analyses were conducted with exit survey results.These included summary statistics of each item and linear modeling to determine if participants' trial condition or personal

Figures
Figures

Table 1 CIAN
33it Survey Sample Characteristics (n = 118) Night, the full digital sleep intervention, including personalized feedback and coaching, active diary self-monitoring, passive biosensor monitoring, and web-based advice."A+SM" stands for advice plus self-monitoring control condition, which also includes passive monitoring."A"stands for advice control condition, which also includes passive monitoring."AUDIT"stands for Alcohol Use Disorders Identi cation Test33.

Table 2
Note.Theme salience (% of those who stated the theme) is based on the number of participants who answered the interview question that resulted in the theme.General questions and questions about the website and biosensors were asked of all 107 participants across trial groups.Questions about diaries were asked of the 78 CIAN and A + SM group participants who completed the diaries and the exit interviews.Questions about feedback were asked of the 50 CIAN participants who received feedback and completed the exit interview.
During exit interviews, participants also offered suggestions for current and future intervention improvement.Many participants were interested in measuring additional sleep-related factors, most commonly that they Want diet and exercise-related feedback: "[I'd like to know] what time when you exercise, how it affects your sleep…what that does to…your whole body chemistry, everything."Othersmentionedthey Want caffeine-related feedback, Want environment-related feedback (e.g., impacts of light, noise, and temperature on sleep), or Want other potential feedback (e.g., cognitive alertness, tobacco, and stress): "Maybe you can see how stressed you are during… the week.Just get those numbers, and then I think that would be pretty interesting to gure[out]."To receive additional feedback, most responded that they would be Willing to wear more devices and Willing to complete new diary questions.Fewer stated they were Unwilling to add a question and/or device, usually disinterest in another ankle biosensor.Several volunteered suggestions for received interventions in the current study.
These included Improvements for feedback (e.g., combined, summary report for both intervention weeks, Improvements for diaries (e.g., higher pay per diary), Improvements for biosensors (e.g., eliminating ankle biosensors, measuring GPS location), and Improvements for website (e.g., wider distribution, alternative summaries).
As one participant stated, "Knowing that I had to do the sleep diary the next day and…people were watching what I was doing…I was just more aware of my habits."Othercommon topics focused on web-based advice, especially sleep strategies.The topic, Website improves sleep (n = 16, 14%), included experiences of beginning to improve sleep using sleep strategies, such as consistent sleep schedules.A participant described, "My sleeping habits were de nitely awful before now, and I think they've improved a lot, since starting this [study]…I'm starting to go to bed earlier and…reduce my activities before bed."Almost as many interviews were most likely to include the topic, Strategy barriers (n = 15, 13%), or descriptions of challenges implementing sleep strategies, such as situational factors (e.g., work and school schedules, dormitory environment) or personal factors (e.g., memory, motivation).The topic Changing poor sleep (n = 15, 13%), included descriptions of poor sleep quality or nonrestorative sleep, how this motivated study participation, and a desire to learn about different sleep factors.Other topics focused on participants' interest in gaining more wellness-related knowledge to change sleep or alcohol use.Thirteen interviews (12%) were most likely to include the topic Learning and reduced drinking, which focused on gaining new information from digital interventions and reducing drinking given its impacts on sleep.The topic Using feedback and website (n = 11, 10%) included participants' accounts of how they integrated personalized feedback and coaching with web-based advice, including information and strategies, to alter their sleep and alcohol use.The topic Multiple strategies and factors (n = 10, 9%) focused on participants' curiosity and interest in the impacts of varied sleep-related factors, such as situations, environments, substances other than alcohol, and diet and exercise.

Table 3
Note. "CIAN" stands for Call it a Night, the full digital sleep intervention, including personalized feedback and coaching, active diary self-monitoring, passive biosensor monitoring, and web-based advice."A+SM"standsfor advice plus self-monitoring control condition, which also includes passive monitoring."A"stands for advice control condition, which also includes passive monitoring.Total means and standard deviations are based on 118 participants who completed the exit interview, including 60 CIAN, 29 A + SM, and 29 A participants.Some items had minor levels of missingness (< 5 participants).Note."CIAN"standsfor Call it a Night, the full digital sleep intervention, including personalized feedback and coaching, active diary self-monitoring, passive biosensor monitoring, and web-based advice."A+SM" stands for advice plus self-monitoring control condition, which also includes passive monitoring."A"stands for advice control condition, which also includes passive monitoring.Total means and standard deviations are based on 118 participants who completed the exit interview, including 60 CIAN, 29 A + SM, and 29 A participants.Some items had minor levels of missingness (< 5 participants).feedback and coaching, active diary self-monitoring, passive biosensor monitoring, and web-based advice."A+ SM" stands for advice plus self-monitoring control condition, which also includes passive monitoring."A"stands for advice control condition, which also includes passive monitoring.Total means and standard deviations are based on 118 participants who completed the exit interview, including 60 CIAN, 29 A + SM, and 29 A participants.Some items had minor levels of missingness (< 5 participants).
Thus, digital interventions should include mechanisms for interactive feedback, such as an on-call health provider, prescheduled video or text check-ins with a health provider, or a chatbot that provides a spectrum of suggestions and individual data descriptions.Digital interventions for speci c, sensitive clinical issues, like heavy drinking, could engage more young adults via a focus on related general health and wellness behaviors, like sleep.Interventions that increase accessibility and engagement for young adults are important given that only half of young adults with psychiatric disorders currently receive any mental health treatment 2 .Digital sleep interventions for reduced drinking can target two risky, prevalent, and modi able issues: drinking and sleep problems.