The study was conducted with a telemedicine platform (Talkspace) used by independently practicing, licensed therapists in the United States. The platform is accessible through internet search, through Employee Assistance Programs, and as a behavioral health benefit through some individual insurances. Patients first meet with an intake clinician through a live messaging system to conduct a brief, standardized intake to identify the presenting complaint, patient treatment history, and the patient’s provider preferences. This information informs a matching algorithm that prioritizes and presents three providers with the desired characteristics for the patient to choose among. Once a clinician is chosen, the provider is alerted, and the patient is immediately introduced to the messaging “room” where treatment takes place. Patients complete a self-report baseline assessment and the provider walks them through the informed consent and emergency contact process after which treatment can begin. Observations in this study include data collected as part of organizational quality assurance and program management processes between January 1, 2016 and February 1, 2018. All patients and clinicians give written consent to the use of their data in a de-identified, aggregate format as part of the user agreement before they begin using the platform. Study procedures were approved as exempt by the institutional review board at Teachers College, Columbia University (15-426).
Participants were individuals who presented with a chief complaint of anxiety or depression, were seeking treatment through the service, and who completed at least one PHQ-9 and/or GAD-7. Inclusion criteria consisted of: (1) being English speakers in the United States, (2) between the ages of 18 and 65, (3) having regular internet or cellphone access, (4) receiving a depression or anxiety diagnosis from their assigned licensed mental health provider based on a clinical intake and live messaging or video-based interview, as recorded in the electronic medical record with ICD-10 codes, (5) scoring 10 or higher on the PHQ-9 and/or GAD-7. Exclusion criteria consisted of current or past diagnoses of: (1) bipolar disorder, (2) any schizophrenia spectrum and psychotic disorder, or psychotic features, (3) any medical or neurological condition that would better account for the symptoms, (4) substance or alcohol use disorder (5) any condition requiring hospitalization; or (6) suicidal thoughts and/or behavior sufficient to be marked a “Yes” on any of questions three through six (at least thoughts about a potential suicide method), on the Columbia Suicide Severity Rating Scale Lifetime-Recent Screen,27 requiring a more intensive level of care that interrupted treatment on the platform. 23,901 patient records were reviewed with these criteria; the final sample consisted of 10,718 patients.
Clinicians in the provider network were currently licensed in at least one state, were required to have a Masters degree or above, and had at least three years of post-licensure experience delivering mental health care. Clinicians were matched only to patients where licensure included the patient’s residence. There were a total of 1,599 clinicians – 43.7% of whom reported five to nine years of post-licensure experience, and 36.5% reporting ten or more years of experience. Eighty-eight percent (88.0%) were female. Providers had a mean age of 40 (SD = 10.04) years, and they reported a mix of orientations as part of their provider profile: 62.5% cognitive-behavioral treatment, 22.5% an integrative or eclectic approach, 8% primarily psychodynamic, and 7% third-wave cognitive behavioral interventions.
Methods and Procedures
Clinicians and patients asynchronously exchanged text-, audio-, and video-based messages using a secure, HIPAA-compliant platform accessible on mobile devices and on desktop computers. Patients could freely send messages at any time without limit, and all messages were stored for the clinician when they returned to review the message history. Therapists responded to messages from their patients at least once a day, five days a week. Clinicians were expected to adhere to all reporting, professional, and ethical standards for their respective fields, and appropriate referrals were provided for patients judged to need a higher level of care.
The number of words exchanged between therapists and patients is automatically counted as meta-data by the platform, and was used as a proxy to quantify the extent of therapeutic interaction through the asynchronous messaging medium. Raw counts of words sent by clinicians and patients were used in supplementary analyses. Raw counts of the number of audio and video messages sent by each party were also analyzed.
Patients were assessed for depression and anxiety symptoms at baseline and then every three weeks for the duration of treatment, or until the patient opted to stop receiving assessments. Assessments are introduced to patients as an important aspect of their care that facilitates goal setting and to track progress. In this study, five assessments from baseline to week 12 were analyzed, including: Baseline, Week 3, Week 6, Week 9, and Week 12.
The 9-item Patient Health Questionnaire28 was used to identify the clinical severity of depression. Responses on all items were given on a 4-point Likert scale (0 =Not at all to 3 = Nearly every day) with a total maximum score of 24. Scores greater or equal than 10 have been shown to have high sensitivity and specificity as a threshold for clinical depression, or at least moderate depression.29,30
Anxiety symptoms were assessed with the 7-item Generalized Anxiety Disorder questionnaire.31 Responses on all items were given on a 4-point Likert scale (0 =Not at all to 3 = Nearly every day) with a total maximum score of 21. Scores of 10 or above have been shown having high sensitivity and specificity as a clinically significant threshold for at least moderate anxiety.32
Patients opting to leave the platform were asked to indicate the reason for leaving. Reasons included feeling better or meeting their goals, having money concerns, not liking the therapy medium, having frustrating technical issues, not liking their therapist, deciding to continue treatment face-to-face, or longer having the time necessary to engage in treatment.
Data Analytic Strategy
Latent Growth Modeling was performed using Mplus 8,34 to tease patterns of change in anxiety and depression symptoms over the examined 12 weeks of messaging treatment. LGM identifies heterogeneous treatment subpopulations or classes (i.e., such as responders and non-responders), and examine their differences. Prior to the analyses, missing values were iteratively imputed by random forests (500 trees, 10 iterations) for variables with ~40% or less missingness35 using the R package missForest.36 Examined predictors were imputed while masking clinical and outcome variables, to prevent information leakage. Categorical data was subsequently converted into binary variables from modal values. All LGM models were estimated under missing at random assumptions using maximum likelihood estimation.
To concurrently capture changes in both anxiety and depression outcomes, the latent growth model included parallel processing34 of PHQ-9 and GAD-7 scores. Specifically, two sets of distinct intercept, slope, and quadratic growth parameters were assigned to each measure, estimating separate trajectories of anxiety and depression symptoms over five assessments (weeks: 0, 3, 6, 9, and 12). The classes were then determined based on joint patterns of PHQ-9 and GAD-7 scores growth. The optimal number of classes was determined comparing nested unconditional LGM with increasing numbers of classes. Variance of the growth parameters was fixed to zero, to increase delineation of classes. Examined model fit indices included Bayesian Information Criterion (BIC), sample-size adjusted Bayesian Information Criterion (SSBIC), Akaike Information Criterion (AIC), relative Entropy, Lo–Mendell–Rubin–adjusted likelihood ratio test (L-M-R LRT), and bootstrapped likelihood ratio test (BLRT). The best fitting solution was estimated based on model fit indices, as well as explanatory properties of the solution.37,38
After determining the solution with the best relative fit, demographic variables and weeks before treatment dropout (or completion) were nested as covariates in a conditional LGM, to analyze class membership predictions. Auxiliary 3-step method multinomial logistic analyses for latent class predictors39 were then performed on the conditional model. This approach to latent class logistic regression analyses takes into account measurement error in the most likely class attributions, to estimate the predictive role of quantitative treatment delivery characteristics (i.e., the average number of words per week used by therapists and clients over the course of treatment) in determining group membership. Word counts were log-transformed to improve odds ratio dose-response interpretability.