Automated mood disorder symptoms monitoring from multivariate time-series sensory data: Getting the full picture beyond a single number.

doi:10.21203/rs.3.rs-3149234/v1

Download PDF

Article

Automated mood disorder symptoms monitoring from multivariate time-series sensory data: Getting the full picture beyond a single number.

https://doi.org/10.21203/rs.3.rs-3149234/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 26 Mar, 2024

Read the published version in Translational Psychiatry →

You are reading this latest preprint version

Mood disorders (MDs) are among the leading causes of disease burden worldwide. Limited specialized care availability remains a major bottleneck thus hindering pre-emptive interventions. MDs manifest with changes in mood, sleep, and motor activity, observable in ecological physiological recordings thanks to recent advances in wearable technology. Therefore, near-continuous and passive collection of physiological data from wearables in daily life, analyzable with machine learning (ML), could mitigate this problem, bringing mood disorders monitoring outside the clinician’s office. Previous works predict a single label, either the disease state or a psychometric scale total score. However, clinical practice suggests that the same label may underlie different symptom profiles, requiring specific treatments. Here we bridge this gap by proposing a new task: inferring all items in HDRS and YMRS, the two most widely used standardized scales for assessing MDs symptoms, using physiological data from wearables. To that end, we develop a deep learning pipeline to score the symptoms of a large cohort of MD patients and show that agreement between predictions and assessments by an expert clinician is clinically significant (quadratic Cohen’s κ and macro-average F1 score both of 0.609). While doing so, we investigate several solutions to the ML challenges associated with this task, including multi-task learning, class imbalance, ordinal target variables, and cross-subject invariant representations. Lastly, we illustrate the importance of testing on out-of-distribution samples.

Health sciences/Biomarkers

Health sciences/Biomarkers/Diagnostic markers

Mood disorders (MDs) are a group of diagnoses in the Diagnostic and Statistical Manual 5th edition¹ classification system. They are a leading cause of disability worldwide² with an estimated total economic cost greater than USD 326.2 billion in the United States alone³. They encompass a variety of symptom combinations affecting mood, motor activity, sleep, and cognition and manifest in episodes categorized as major depressive episodes (MDEs), featuring feelings of sadness and loss of interest, or, at the opposite extreme, (hypo)manic episodes (MEs), with increased activity and self-esteem, reduced need for sleep, expansive mood and behavior. As per the DSM-5 nosography, MDEs straddle two nosographic constructs, i.e. Major Depressive Disorder (MDD) and Bipolar Disorder (BD), whereas MEs are the earmark of BD only⁴.

Clinical trials in psychiatry to this day entirely rely on clinician-administered standardized questionnaires for assessing symptoms severity and, accordingly, setting outcome criteria. With reference to MDs, Hamilton Depression Rating Scale-17⁵ (HDRS) and Young Mania Rating Scale⁶ (YMRS) are among the most widely used scales to assess depressive and manic symptoms⁷, quantifying behavioral patterns such as disturbances in mood, sleep, and anomalous motor activity. The low availability of specialized care for MDs, with rising demand straining current capacity⁸, is a major barrier to this classical symptom monitoring pipeline. This results in long waits for appointments and reduced scope for pre-emptive interventions. Current advances in machine learning (ML)⁹ and the widespread adoption of increasingly miniaturized and powerful wearable devices offer the opportunity of personal sensing, which could help mitigate the above problems¹⁰. This can involve near-continuous and passive collection of data from sensors, with the aim of identifying digital biomarkers associated with mental health symptoms at the individual level, therefore backing up clinical evaluation with objective and measurable physiological data. Personal sensing holds great potential of being translated into clinical decision support systems¹¹ the for the detection and monitoring MDs. Specifically, it could be particularly appealing to automate the prediction of the items of the HDRS and YMRS scales as they correlate with changes in physiological parameters, conveniently measurable with wearable sensors^12–14.

However, so far, the typical approach has been to collapse MDs detection to the prediction of a single label, either the disease state or a psychometric scale total score^15,16, which risks oversimplifying a much more complex clinical picture. Figure 1 illustrates this issue: patients with different symptoms and thus (potentially very) different scores on individual HDRS and YMRS items are “binned together” in the same category, leading to a loss of actionable clinical information. Predicting all items in these scales can instead align with everyday psychiatric practice where the specialist, when recommending a given intervention, considers the specific features of a patient, including their symptom patterns, beyond a reductionist disease label^17,18. Figure 1 illustrates a case in point where knowledge of the full symptoms profile might enable to tailor treatment: on the face of it, patient (a) and (b) (top row) share the same diagnosis, i.e. MDE in the context of MDD; however, considering their specific symptom profile patient (a) might benefit from a molecule with stronger anxiolytic properties whereas patient (b) might require a compound with hypnotic properties. Furthermore, an item-wise analysis can lead to the identification of drugs symptom specificity in clinical trials^19,20.

Table 1 summarizes previous works in personal sensing for MDs and showing that all previous tasks collapsed the complexity of MDs to a single number. Côté-Allard et al.²¹ explored a binary classification task, that is distinguishing subjects with BD on a ME from different subjects with BD recruited outside of a disease episode, when stable. The study experimented with different subsets of pre-designed features from wristband data and proposed a pipeline leveraging features extracted from both short and long segments taken within 20-hour sequences. Pedrelli et al.²², expanding on Ghandeharioun et al.²³, used pre-designed features from a wristband and a smartphone to infer HDRS residualized total score (that is total score at time \(t\) minus baseline total score) with traditional ML models. Tawaza et al.²⁴ employed gradient boosting with pre-designed features from wristband data and pursued case-control detection in MDD and, secondarily, HDRS total score prediction. Similarly, Jacobson et al.²⁵ predicted case-control status in MDD from actigraphy features with gradient boosting. Nguyen et al.²⁶ used a sample including patients with either schizophrenia (SCZ) or MDD wearing an actigraphic device and explored case-control detection where SCZ and MDD were either considered jointly (binary classification) or as separate classes (multi-class classification). Of notice, this was the first work to apply artificial neural networks (ANNs) directly on minimally processed data, showing that they outperformed traditional machine learning models. Lastly, the multi-centre study of Lee et al.²⁷ investigated mood episodes prediction with a random forest and pre-designed features from wearable and smartphone data. Further to proposing a new task, our work stands out for a sample size larger than all previous works by over 2 dozen patients, with the exception of a multi-centre study by Lee et al.²⁷, where, however, clinical evaluation was carried out retrospectively, thereby inflating chances of recall bias²⁸ and missing out on the real-time clinical characterization of the acute phase. Indeed, collecting data from patients on an acute episode, using specialist assessments and research-grade wearables, is a challenging and expensive enterprise.

Relatively to previous endeavors, the contribution of this work is two-fold: (1) Taking one step beyond the prediction of a single label, which misses on actionable clinical information, we propose a new task in the context of MDs monitoring with physiological data from wearables: inferring all items in HDRS (17 items) and YMRS (11 items), as scored by a clinician, which enables a fine-grained appreciation of patients’ psychopathology therefore creating opportunity for bespoke treatment (Fig. 1). (2) We investigate some of the methodological challenges associated with the task at hand and explore possible ML solutions. c1: inferring multiple (28) target variables, i.e. multi-task learning (MTL, see Section 3.4.1). c2: modeling ordinal data, such are HDRS and YMRS items (see Section 3.4.1). c3: learning subject invariant representations, since, especially with noisy data and a sample size in the order of dozens, models tend to focus on subject specific features to master the task at hand leading to poor generalization²⁹ (see Section 3.4.2). c4: learning from imbalanced data, as patients on an acute episode usually receive intensive treatment and acute states therefore tend to be relatively short periods in the overall disease course^30,31 thereby tilting scores towards low values.

2.1 Data collection and cohort statistics

The following analyses are based on an original dataset being collected as part of a prospective, exploratory, observational, single-center, longitudinal study with a fully pragmatic design embedded into current real-world clinical practice. A fuller description of the cohort is provided in Anmella et al.³². In brief, subjects with a DSM-5 MD diagnosis (either MDD or BD) were eligible for enrollment. Those recruited on an acute episode had up to four assessments: T0 acute phase (upon hospital admission or at the home treatment unit), T1 response onset (50% reduction in total HDRS/YMRS), T2 remission (total HDRS/YMRS ≤ 7), and T3 recovery (total HDRS/YMRS continuously ≤ 7 for a period of ≥ 8 weeks)³³. On the other hand, subjects with a historical diagnosis but clinically stable at the moment of study admittance (euthymia, Eu) were interviewed only once. At the start of each assessment, a clinician collected clinical demographics, including HDRS and YMRS, and provided an Empatica E4 wristband³⁴ which participants were required to wear on their non-dominant wrist until the battery ran out (∼48 hours). A total of 75 subjects, amounting to a total of 149 recording sessions (i.e., over 7000 hours), were available for the present study. An overview of the cohort clinical-demographic characteristics is given in Table 2. The E4 records the following sensor modalities (we report their acronyms and sampling rates in parenthesis): 3D acceleration (ACC, 32Hz), blood volume pressure (BVP, 64Hz), electrodermal activity (EDA, 4Hz), heart rate (HR, 1Hz), inter-beat intervals (IBI, i.e., the time between two consecutive heart ventricular contractions) and skin temperature (TEMP, 1Hz). IBI was not considered due to extensive sequences of missing values across all recordings. This is likely due to high sensitivity to motion and motion artifacts, as observed previously³⁵.

2.2 Data pre-processing

An E4 recording session comes as a collection of 1D arrays of sensory modalities. We quality-controlled data to remove physiologically implausible values with the rules by Kleckner et al.³⁶ and the addition of a rule to remove HR values that exceeded the physiologically plausible range (25–250 bpm). Each quality-controlled recording session was then segmented using a sliding window, whose length (τ, in wall-time seconds) is a hyperparameter, enforcing no overlap between bordering segments (to prevent models from exploiting overlapping motifs between segments). These segments (\({x}_{i}\)) and the corresponding 28 clinician-scored HDRS-YMRS items (\({y}_{i}\)) from the subjects wearing the E4 formed our dataset, \({\left\{\left({x}_{i},{y}_{i}\right)\right\}}_{i=1}^{N}\),HDRS-YMRS items map symptoms spanning mood, sleep, psycho-motor activity. Some likely fluctuate over a 48-hour session, especially in an ecological setting where treatments can be administered (e.g., Y9 disruptive-aggressive behavior may be sensitive to sedative drugs). To limit this, we isolated segments from the first five hours (close-to-interview samples) and used them for the main analysis, splitting them into train, validation, and test sets with a ratio of 70-15-15. Then, to study the effect of distribution shift, we tested the trained model on samples from each 30-minute interval following the first five hours of each recording (far-from-interview samples). It should be noted that further to a shift in the target variables, a shift in the distribution of physiological data collected with the wearable device is to be expected³⁷, owing to different patterns of activity during the day, circadian cycles, and administered drugs. Details on number of recording segments in train, validation, and test splits are given in Supplementary Table (ST) 1.

2.3 Evaluation Metrics

HDRS and YMRS items are ordinal variables: H11 anxiety somatic, for example, have ranks 0-Absent, 1-Mild, 2-Moderate, 3-Severe, or 4-Incapacitating. The item distribution (see Supplementary Figure (SF) 1) was imbalanced towards low scores due to patients on an acute episode usually receive intensive treatment and acute states, therefore, tend to be relatively short- lived periods in the overall disease course^30,31. This can be quantified with the ratio between the cardinality of the majority rank and that of the minority rank (ρ). Metrics accounting for class imbalance should be used when evaluating a classification system in such a setting to penalize trivial solutions. We used Cohen’s κ, in particular its quadratic version (QCK), since, further to its suitability to imbalanced ordinal data, it is familiar and easily interpretable to clinicians^38–41. It expresses the degree to which the ANN learned to score segments in agreement with the clinician’s assessments. This is similar to psychiatric clinical trials where prospective raters are trained to align with assessments made by an established specialist⁴². Cohen’s κ takes values in [-1, 1], where 1 (-1) means perfect (dis)agreement and a value of 0 indicates random agreement. QCK penalizes disagreements proportionally to their squared distance. As individual HDRS-YMRS items have different distributions we checked whether item level performance was affected by sample Shannon entropy (H) or ρ by computing a simple Pearson correlation coefficient (R) between item QCK and H/ρ.

2.4 Model Design

The task at hand is supervised, specifically, we sought to learn a mapping from recording segment to HDRS and YMRS scores: \(f:{x}_{i}\mapsto \widehat{{y}_{i}}\). The model we developed comprised two independent sub-models (Fig. 2): (a) a neural classifier (\(CF\)), responsible for learning the above mapping, and (b) a patient critic (\(\mathsf{C}\mathsf{R}\)), which has the objective to help \(\mathsf{C}\mathsf{F}\) learn representations that do not memorize information specific to single patients which can act as shortcuts to predict the label items. The \(\mathsf{C}\mathsf{F}\) module itself consisted of three sequential modules: (a.1) a channel encoder (\(\mathsf{E}\mathsf{N}\)) for projecting sensory modalities onto the same dimensionality, (a.2) a representation module (\(\mathsf{R}\mathsf{M}\)) for extracting features, and lastly (a.3) 28 parallel item predictors (\(\mathsf{I}\mathsf{P}\)), indexed by \(j\in \left\{1,\dots ,28\right\}\) and each learning \(p\left({y}_{i,j}|\mathsf{I}{\mathsf{P}}_{\mathsf{j}}\left({h}_{i}\right)\right)\), that is the conditional distribution over ranks for the \({j}^{th}\) item given \({h}_{i}=\mathsf{R}\mathsf{M}\left(\mathsf{E}\mathsf{N}\left({x}_{i}\right)\right)\). The critic module \(\mathsf{C}\mathsf{R}\), instead, uses the representation from \(\mathsf{R}\mathsf{M}\) for telling subjects apart. \(\mathsf{C}\mathsf{R}\) competes in an adversarial game against \(\mathsf{E}\mathsf{N}\) and \(\mathsf{R}\mathsf{M}\), designed to encourage cross-subjects invariant representations. Details on the model’s architecture and the model’s loss are given in Supplementary Methods.

2.5 Learning from imbalanced

We adapted to our use case the following three popular imbalance learning approaches. (i) Focal loss⁴³: the categorical cross-entropy (CCE) loss from each item predictor \(\mathsf{I}{\mathsf{P}}_{\mathsf{j}}\) was multiplied during training by \({\alpha }_{j,k}{\left(1-{p}_{j,k}\right)}^{\gamma }\) where: \({\alpha }_{j,k}\) and \({p}_{j,k}\) were respectively the inverse frequency (as estimated from the training set) and predicted probability of the ground truth \({k}^{th}\) rank for the \({j}^{th}\) item, \(\gamma\) was a hyperparameter (usually set to 2). \({\alpha }_{j,k}\in {R}^{+}\) was a rank-wise weight that was used to increase the importance of the minority class. (ii) Probability thresholding⁴⁴: during inference, probabilistic predictions for each rank under the \({j}^{th}\) item were divided by the corresponding rank frequency (computed on the training set), plus a small term to avoid division by zero in case of zero frequency ranks. The new values were then normalized by the total sum. (iii) Re-sampling and loss re-weighting: segments were assigned a label based on HDRS/YMRS severity bins (defined in ³³) which was then used to either random under-sampling (RUS) or random over-sampling (ROS) segments with, respectively, over-represented and under-represented labels. The loss of the \({x}_{i}\) was then re-scaled proportionally to the re-sampling ration of its class.

2.6 Model training and hyperparameter tuning

Given the explorative nature of this work, with various model configurations and optimization objectives, we performed an exhaustive search using the Hyperband Bayesian optimization⁴⁵ to find the hyperparameters that yield the best QCK in the validation set. All models were trained with AdamW optimizer⁴⁶ for a maximum of 400 epochs. Moreover, to speed up the training and search procedure, we employed an early stopping learning rate scheduler: we reduce the learning rate \({\alpha }_{\text{L}\text{R}}=0.3{\alpha }_{\text{L}\text{R}}\) if the model has not improved in its validation performance after 10 consecutive epochs; we terminate the training procedure if the model has not improved after 2 learning rate reductions. Dropout⁴⁷ and weight decay were added to prevent overfitting. ST2 shows the hyperparameters search space and the configuration of the best model after 300 iterations. We also computed which hyperparameters were the best predictors of the validation QCK. This was obtained by training a random forest with the hyperparameters as inputs and the metric as the target output and deriving the feature importance values for the random forest.

2.7 Non-deep-learning baseline model

Most previous works into personal sensing for MDs (as discussed in the Introduction) did not use deep-learning for automatically learning features from minimally processed data but deployed classical ML models on hand-crafted features. Thus, we developed a baseline in the same spirit, in order to better contextualize our deep-learning pipeline performance on close-to-interview samples. Namely, from the same recording segments inputted to the ANN we extracted features with a commonly used feature extractor for Empatica E4, named FLIRT⁴⁸, and developed random forest classifiers (28 in total, as many as there are HDRS and YMRS items), using random oversampling to handle class. We opted for random forest since it was a popular choice in previous relevant works^22,27. The hyperparameter space was explored with a random search of 300 iterations for each classifier. Details are given in ST3.

2.8 Prediction Error Examination

Using the best performing setting among those explored in the experiments detailed above we computed residuals on close-to-interview samples and illustrated their distribution across items. For the sake of better comparability, items with a rank step of two (e.g. Y5 irritability) were re-scaled to have a rank step of one like other items. Furthermore, towards investigating correlations between residuals, checking for any remarkable pattern in view of the natural correlation structure of HDRS and YMRS, we estimated a regularized partial correlation network, in particular a Gaussian graphical lasso (glasso⁴⁹), over item residuals. Network edge sparsity, to avoid false positives, is enforced with the the least absolute shrinkage and selection operator (LASSO⁵⁰), which indeed shrinks all edge-weights towards zero and sets small weights to exactly zero. The strength of the regularization is traded-off by a hyperparameter \({\lambda }\), selected with the Extended Bayesian Information Criterion (EBIC⁵¹). The EBIC itself has a tuning parameter \({\gamma }\) controlling the trade-off between sensitivity and precision, which we set to 0.25 as in ⁵². We also estimated node predictability, measuring how well a node can be predicted by nodes it shares an edge with, which can be interpreted similarly to \({\text{R}}^{2}\)⁵³. Lastly, bootstrapping routines were used to gain insight into the stability of the estimated parameters.

2.9 Channels Importance

We took a simple, model-agnostic approach to assessing each individual channel contribution to the task at hand. That is to say, we selected the system performing best on the task and re-trained it including all channels (tri-axial ACC, EDA, BVP, HR, and TEMP) but one. For each left-out channel we measured the difference in performance across items relatively to the baseline model (the one trained on all channels).

Best model details - ANN The loss type is the hyperparameter most predictive of validation QCK (ST4). The selected model employs the Cohen's \({\kappa }\) loss with quadratic weightage³⁹ (c2). The best model uses a (small) critic penalty (\({\lambda }=0.07\)) added to the main objective, i.e. scoring HDRS/YMRS (c3). However, the training curve shows that the reduction in the MLT loss across epochs is paralleled by the reduction in the (cross-entropy) loss paid by \(\mathsf{C}\mathsf{R}\), tasked with telling subjects apart. Resampling and loss re-weighting (c4) is the preferred strategy for class imbalance. A segment length of 16 seconds is selected as best value for segmentation. The difference in QCK (\({{\Delta }}_{\text{Q}\text{C}\text{K}}\)) for other choices of \(\tau\) (in seconds) relative to the best configuration is -0.092 (8s), -0.100 (32s), -0.191 (64s), -0.246 (128s), 0.355 (256s), -0.4431 (512s), -0.577 (1024s). Note that \(\tau\) was explored among powers of 2 for computational convenience and that, when segmenting the first 5 hours of each recording, different \(\tau\) values produced different samples number and length (the lower the \(\tau\) values, the higher the number of samples, the shorter the sample). The predictive value of hyperparameter \(\tau\) towards validation QCK is fairly low relatively to other hyperparameters.

Main results Our best ANN model achieves an average QCK across HDRS and YMRS items of 0.609 in close-to-interview samples, a value that can be semi-qualitatively interpreted as moderate agreement⁵⁴, confidently outperforming our baseline random forest model that only reached an average QCK of 0.214. Item level QCK correlates weakly (\(\text{R}=0.08\)) with item class imbalance (\({\rho }\)) but fairly (\(\text{R}=0.42\)) with item \(\mathcal{H}\). Table 3 shows QCK for each item in HDRS and YMRS. Briefly, QCK is highest for H12 somatic symptoms gastrointestinal (0.775) and lowest for H10 anxiety psychic (0.492). H10 has also the highest \(\mathcal{H}\) (1.370), however, H7 work and activities, despite having the second highest \(\mathcal{H}\) (1.213), has a QCK of 0.629, ranking as 9th best predicted item.

Shift over time When tested on far-from-interview samples, our system overall has a drop in performance (SF2). The average QCK is 0.498, 0.303, 0.182 on segments taken respectively from the first, second, and third thirty-minute interval. Thereafter it fluctuates through the following thirty-minute intervals with 0.061 as the lowest value 15 hours into the recording. The items with the biggest drop in QCK relatively to their baseline value across the first three 30-minute intervals are H9 agitation, H10 anxiety somatic, Y4 sleep, and Y9 disruptive-aggressive behavior. On the other hand, items that retain their original QCK value the most in the first three 30-minute intervals are: H1 depressed mood, Y11 insight, H2 feelings of guilt, and H17 insight. This pattern matches clinical intuition as items in the former group may be more volatile and reactive to environmental factors (including medications), whereas items in the latter group tend to change more slowly.

4.1 Post-hoc diagnostics

In order to gain further insights into the errors that our system made on close-to-interview holdout samples, we studied distribution of residuals. SF3 illustrates that the model is correct most of the times, residuals are in general evenly distributed around zero, and when wrong the model is most often off by just one. Furthermore, we investigated the correlation structure among item residuals to check whether any meaningful pattern emerged. SF4 shows the undirected graphical model for the estimated probability distribution over HDRS and YMRS item residuals. The graph only has positive edges, that is only positive partial correlation between item residuals and co-variates. HDRS and YMRS nodes tend to have weak interactions across the two scales, with the exception of nodes that map the same symptom, e.g. Y11 and H17 both query insight. Within each scale, partial correlations are stronger among nodes underlying a common symptom domain, e.g. H1 and H2 constitute “core symptoms of depression”⁵⁵, and speech (Y6) is highly related to mood (Y1) and thought (Y7, Y8)⁵⁶. Average node predictability for HDRS and YMRS items, a measure of how well a node can be predicted by nodes it shares an edge with, akin to \({\text{R}}^{2}\), is 48.43%. Stability analyses showed that some edges are estimated reliably (i.e., they were included in all or nearly 500 bootstrapped samples), but there also is considerable variability in the edge parameters across the bootstrapped models.

4.2 Channels contribution

We were interested in whether physiological modalities contributed differently towards performance across items. This question, further to clinical interest, has also practical implications since other devices may not implement the same sensors as Empatica E4. Figure 3 shows that while all modalities seem to positively contribute to test performance across items, this is markedly the case with ACC as the model records the biggest drop in performance upon removal of this channel from input features. Specifically, upon zeroing out the contribution of ACC, the biggest deterioration in performance was observed for items mapping anxiety (e.g. H11 anxiety somatic\({{\Delta }}_{\text{Q}\text{C}\text{K}}=-0.321\)), YMRS4 sleep and YMRS9 disruptive-aggressive behavior (with a \({{\Delta }}_{\text{Q}\text{C}\text{K}}\) of -0.371 and − 0.281 respectively), and core depression items (e.g., H1 depressed mood\({{\Delta }}_{\text{Q}\text{C}\text{K}}=-0.276\)). On the other hand, the contribution of BVP was relatively modest since upon dropping this channel items generally had only a marginal reduction in QCK, most pronounced for H16 loss of weight (\({{\Delta }}_{\text{Q}\text{C}\text{K}}=-0.095\)) and H11 anxiety somatic (\({{\Delta }}_{\text{Q}\text{C}\text{K}}=-0.078\)).

In this work, we proposed a new treatment of MDs monitoring with personal sensing: inferring all 28 items from HDRS and YMRS, the most widely used clinician-administered scales for depression and mania respectively. Casting this problem as a single label prediction, e.g. disease status or the total score on a psychometric scale, as done previously in the literature, dismisses the clinical complexity of MDs, thereby losing actionable clinical information, which is conversely preserved in the task we introduced here.

We developed and tested our framework with samples taken over five hours since the start of the clinical interview (close-to-interview samples), achieving moderate agreement⁵⁴ with expert clinician (average QCK of 0.609) on a holdout set and showing that our deep-learning pipeline vastly exceeded the performance (average QCK of 0.214) of traditional ML baseline relying on hand-crafted features. Item level performance showed fair correlation with item \(\mathcal{H}\), indicating that items with a higher “uncertainty” in their sample distribution tend to be harder to predict. Difference in \(\mathcal{H}\)is partly inherent to the scale design as different items admit a different number of ranks. When used on samples collected from thirty-minute sequences following the first five hours of the recordings (far-from-interview samples), our model had a significantly lower performance with average QCK declining down to 0.182 in the third half-hour and then oscillating but never recovering to the original level. Consistently with clinical intuition, items suffering the sharpest decline relative to their baseline performance were those mapping symptoms that naturally have a higher degree of volatility (e.g. H9 agitation) while items corresponding to more stable symptoms (e.g. H17 insight) had a more gentle drop in performance. Besides (some) symptoms plausibly changing over time, a shift in the physiological data distribution is very likely in a naturalist setting.

Residuals on holdout close-to-interview samples showed a symmetric distribution, centered around zero, thus the model was not systematically predicting either over- or under-predicting. The network of item residuals illustrated that our model erred along the correlation structure of the two symptom scales, as stronger connections were observed among items mapping the same symptom or a common domain. Re-training our best model including all input channels, but one showed that ACC was the most important modality, landing further support to the discriminative role of actigraphy with respect to different mood states¹⁴. Coherently, items whose QCK deteriorated the most upon removing this channel were those mapping symptom domains clinically observable through patterns of motor behavior.

Limitations We would like to highlight several limitations in our study. (a) All patients were scored on HDRS and YMRS by the same clinician. However, having scores from multiple (independent) clinicians on the same patients would help appreciate model performance in view of inter-rater agreement. (b) The lack of follow-up HDRS and YMRS scores within the same session did not allow to estimate to what degree a shift in target variables might be at play. Relatedly, we acknowledge that the choice of five hours for our main analyses may be disputable and other choices may have been valid too. Five hours was an informed attempt to trade off a reasonably high number of samples with a minimization in the distribution shift over both target variables and physiological data; studying the effect of different cut-offs was not within the scope of this work. (c) Given the naturalist setting, medications were allowed, and their interference could not be ruled out. (d) The study is single-center, and thus we could not investigate generalization to different centers.

Future work (i) The decline in performance over future time points stands out as the main challenge towards real-world implementations and suggests that the model struggles to adapt to changes in background (latent) variables, e.g. changes in activity patterns. Research into domain-adaptation should therefore be prioritized. We also speculate that MD symptoms and relevant physiological signals have slow- as well as fast-changing component. A segment length of 16 seconds would seem unsuitable for representing the latter and an attempt should be made at capturing both. (ii) Generalization to patients unseen during model development is a desirable property in real-world applications and something we consider exploring in the future. Another approach to tackle this point is to develop (or fine-tune) a model for each individual patient, as done in related fields⁵⁷. (iii) As supervised learning performance has been shown to grow roughly logarithmically in the size of annotated datasets and annotation remains a major bottleneck in biomedical settings⁵⁸, self-supervised learning⁵⁹ should be considered for applications using the E4 device. (iv) For a ML system to be trustworthy and actionable in a clinical setting, further research into model explainability and uncertainty quantification is warranted⁶⁰.

Code & Data availability

The software codebase used in this work will be made publicly available upon publication. Python 3.10 programming language was used for the symptoms scoring system, where deep learning models were implemented in PyTorch⁶¹, hyperparameter tuning and visualization model performance were performed in Weights and Biases⁶², and random forest classifiers were developed in sklearn⁶³. Graphical modeling of the residuals and related analyses were performed in R 4.2.2 using packages qgraph⁶⁴ for network estimation and visualization, and bootnet⁶⁵ for bootstrapping. Data in de-identified form may be made available from the corresponding author upon reasonable request.

Ethics and confidentiality

This study was conducted in accordance with the ethical principles of the Declaration of Helsinki and Good Clinical Practice and the Hospital Clinic Ethics and Research Board (HCB/2021/104). All participants provided written informed consent prior to their inclusion in the study.

Acknowledgements

The authors acknowledge the contribution of all the participants and collaborators of this study.

This project was funded by the ISCIII (FIS PI21/00340, TIMEBASE Study), co-funded by the European Union, as well as a Baszucki Brain Research Fund grant (PI046998) from the Milken Foundation. The ISCIII or the Milken Foundation had no further role in study design, collection, analysis, or interpretation of data, writing of the report, decision to submit the paper for publication.

FC and BML are supported by the United Kingdom Research and Innovation (grant EP/S02431X/1), UKRI Centre for Doctoral Training in Biomedical AI at the University of Edinburgh, School of Informatics. GA is supported by a Rio Hortega 2021 grant (CM21/00017) from the Spanish Ministry of Health financed by the Instituto de Salud Carlos III (ISCIII) and cofinanced by Fondo Social Europeo Plus (FSE+). AM is supported by a contract funded by MCIN/AEI/TED2021-131999BI00 Strategic Projects Oriented to the Ecological Transition and the Digital Transition 2021 and by the “European Union NextGenerationEU/PRTR”. IG thanks the support of the Spanish Ministry of Science and Innovation (PI19/00954) integrated into the Plan Nacional de I+D+I and cofinanced by the ISCIII-Subdirección General de Evaluación y el Fondos Europeos de la Unión Europea (FEDER, FSE, Next Generation EU/Plan de Recuperación Transformación y Resiliencia_PRTR); the ISCIII; the CIBER of Mental Health (CIBERSAM); and the Secretaria d’Universitats i Recerca del Departament d’Economia i Coneixement (2017 SGR 1365), Centres de Recerca de Catalunya (CERCA) Programme or Generalitat de Catalunya as well as the Fundació Clínic per la Recerca Biomèdica (Pons Bartran 2022-FRCB_PB1_2022). EV has received grants and served as consultant, advisor, or CME speaker for the following entities: AB-Biotics, AbbVie, Angelini, Biogen, Biohaven, Boehringer-Ingelheim, Celon Pharma, Compass, Dainippon Sumitomo Pharma, Ethypharm, Ferrer, Gedeon Richter, GH Research, Glaxo-Smith Kline, Idorsia, Janssen, Lundbeck, Medincell, Novartis, Orion Corporation, Organon, Otsuka, Rovi, Sage, Sanofi-Aventis, Sunovion, Takeda, and Viatris, outside the submitted work. DHM has received CME-related honoraria and served as consultant for Abbott, Angelini, Ethypharm Digital Therapy and Janssen-Cilag.

Conflicts of Interest

All authors report no financial or other relationship relevant to the subject of this article.

American Psychiatric Association D, Association AP, others. Diagnostic and statistical manual of mental disorders: DSM-5. American psychiatric association Washington, DC, 2013.
Santomauro DF, Herrera AMM, Shadid J, Zheng P, Ashbaugh C, Pigott DM et al. Global prevalence and burden of depressive and anxiety disorders in 204 countries and territories in 2020 due to the COVID-19 pandemic. The Lancet 2021; 398: 1700–1712.
Greenberg PE, Fournier A-A, Sisitsky T, Simes M, Berman R, Koenigsberg SH et al. The economic burden of adults with major depressive disorder in the United States (2010 and 2018). Pharmacoeconomics 2021; 39: 653–665.
Vieta E, Berk M, Schulze TG, Carvalho AF, Suppes T, Calabrese JR et al. Bipolar disorders. Nat Rev Dis Primer 2018; 4: 1–16.
Hamilton M. A rating scale for depression. J Neurol Neurosurg Psychiatry 1960; 23: 56.
Young RC, Biggs JT, Ziegler VE, Meyer DA. A rating scale for mania: reliability, validity and sensitivity. Br J Psychiatry 1978; 133: 429–435.
Tohen M, Bowden C, Nierenberg AA, Geddes J. Clinical trial design challenges in mood disorders. Academic Press, 2015.
Satiani A, Niedermier J, Satiani B, Svendsen DP. Projected workforce of psychiatrists in the United States: a population analysis. Psychiatr Serv 2018; 69: 710–713.
Su C, Xu Z, Pathak J, Wang F. Deep learning in mental health outcome research: a scoping review. Transl Psychiatry 2020; 10: 1–26.
Mohr DC, Zhang M, Schueller SM. Personal sensing: understanding mental health using ubiquitous sensors and machine learning. Annu Rev Clin Psychol 2017; 13: 23–47.
Jacobson NC, Feng B. Digital phenotyping of generalized anxiety disorder: using artificial intelligence to accurately predict symptom severity using wearable sensors in daily life. Transl Psychiatry 2022; 12: 1–7.
Faurholt-Jepsen M, Brage S, Kessing LV, Munkholm K. State-related differences in heart rate variability in bipolar disorder. J Psychiatr Res 2017; 84: 169–173.
Sarchiapone M, Gramaglia C, Iosue M, Carli V, Mandelli L, Serretti A et al. The association between electrodermal activity (EDA), depression and suicidal behaviour: A systematic review and narrative synthesis. BMC Psychiatry 2018; 18: 1–27.
Tazawa Y, Wada M, Mitsukura Y, Takamiya A, Kitazawa M, Yoshimura M et al. Actigraphy for evaluation of mood disorders: A systematic review and meta-analysis. J Affect Disord 2019; 253: 257–269.
Culpepper L, Muskin PR, Stahl SM. Major depressive disorder: understanding the significance of residual symptoms and balancing efficacy with tolerability. Am J Med 2015; 128: S1–S15.
Earley W, Durgam S, Lu K, Ruth A, Németh G, Laszlovszky I et al. Clinically relevant response and remission outcomes in cariprazine-treated patients with bipolar I disorder. J Affect Disord 2018; 226: 239–244.
Salagre E, Vieta E. Precision psychiatry: complex problems require complex solutions. Eur Neuropsychopharmacol J Eur Coll Neuropsychopharmacol 2021; 52: 94–95.
Serretti A. Precision medicine in mood disorders. Psychiatry Clin Neurosci Rep 2022; 1: e1.
Vieta E, Durgam S, Lu K, Ruth A, Debelle M, Zukin S. Effect of cariprazine across the symptoms of mania in bipolar I disorder: analyses of pooled data from phase II/III trials. Eur Neuropsychopharmacol 2015; 25: 1882–1891.
Lisinski A, Hieronymus F, Näslund J, Nilsson S, Eriksson E. Item-based analysis of the effects of duloxetine in depression: a patient-level post hoc study. Neuropsychopharmacology 2020; 45: 553–560.
Côté-Allard U, Jakobsen P, Stautland A, Nordgreen T, Fasmer OB, Oedegaard KJ et al. Long–Short Ensemble Network for Bipolar Manic-Euthymic State Recognition Based on Wrist-Worn Sensors. IEEE Pervasive Comput 2022.
Pedrelli P, Fedor S, Ghandeharioun A, Howe E, Ionescu DF, Bhathena D et al. Monitoring changes in depression severity using wearable and mobile sensors. Front Psychiatry 2020; 11: 584711.
Ghandeharioun A, Fedor S, Sangermano L, Ionescu D, Alpert J, Dale C et al. Objective assessment of depressive symptoms with machine learning and wearable sensors data. In: 2017 seventh international conference on affective computing and intelligent interaction (ACII). IEEE, 2017, pp 325–332.
Tazawa Y, Liang K, Yoshimura M, Kitazawa M, Kaise Y, Takamiya A et al. Evaluating depression with multimodal wristband-type wearable device: screening and assessing patient severity utilizing machine-learning. Heliyon 2020; 6: e03274.
Jacobson NC, Weingarden H, Wilhelm S. Digital biomarkers of mood disorders and symptom change. NPJ Digit Med 2019; 2: 3.
Nguyen D-K, Chan C-L, Li A-HA, Phan D-V, Lan C-H. Decision support system for the differentiation of schizophrenia and mood disorders using multiple deep learning models on wearable devices data. Health Informatics J 2022; 28: 14604582221137536.
Lee H-J, Cho C-H, Lee T, Jeong J, Yeom JW, Kim S et al. Prediction of impending mood episode recurrence using real-time digital phenotypes in major depression and bipolar disorders in South Korea: a prospective nationwide cohort study. Psychol Med 2022;: 1–9.
Hidalgo-Mazzei D, Young AH, Vieta E, Colom F. Behavioural biomarkers and mobile mental health: a new paradigm. Int J Bipolar Disord 2018; 6: 1–4.
Özdenizci O, Wang Y, Koike-Akino T, Erdoğmuş D. Adversarial deep learning in EEG biometrics. IEEE Signal Process Lett 2019; 26: 710–714.
De Dios C, Ezquiaga E, Garcia A, Soler B, Vieta E. Time spent with symptoms in a cohort of bipolar disorder outpatients in Spain: a prospective, 18-month follow-up study. J Affect Disord 2010; 125: 74–81.
Verduijn J, Verhoeven JE, Milaneschi Y, Schoevers RA, van Hemert AM, Beekman AT et al. Reconsidering the prognosis of major depressive disorder across diagnostic boundaries: full recovery is the exception rather than the rule. BMC Med 2017; 15: 1–9.
Anmella G, Corponi F, Li BM, Mas A, Sanabra M, Pacchiarotti I et al. Exploring digital biomarkers of illness activity in mood episodes: hypotheses generating and model development study. JMIR Mhealth Uhealth 2023.
Tohen M, Frank E, Bowden CL, Colom F, Ghaemi SN, Yatham LN et al. The International Society for Bipolar Disorders (ISBD) Task Force report on the nomenclature of course and outcome in bipolar disorders. Bipolar Disord 2009; 11: 453–473.
Empatica. E4 wristband technical specifications – Empatica Support. E4 Wristband Tech. Specif. 2020.https://support.empatica.com/hc/en-us/articles/202581999-E4-wristband-technical-specifications.
Schuurmans AA, de Looff P, Nijhof KS, Rosada C, Scholte RH, Popma A et al. Validity of the Empatica E4 wristband to measure heart rate variability (HRV) parameters: A comparison to electrocardiography (ECG). J Med Syst 2020; 44: 1–11.
Kleckner IR, Jones RM, Wilder-Smith O, Wormwood JB, Akcakaya M, Quigley KS et al. Simple, transparent, and flexible automated quality assessment procedures for ambulatory electrodermal activity data. IEEE Trans Biomed Eng 2017; 65: 1460–1467.
Li X, Kane M, Zhang Y, Sun W, Song Y, Dong S et al. Circadian Rhythm Analysis Using Wearable Device Data: Novel Penalized Machine Learning Approach. J Med Internet Res 2021; 23: e18403.
Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther 2005; 85: 257–268.
de La Torre J, Puig D, Valls A. Weighted kappa loss function for multi-class classification of ordinal data in deep learning. Pattern Recognit Lett 2018; 105: 144–154.
Duran A, Dussert G, Rouvière O, Jaouen T, Jodoin P-M, Lartizien C. ProstAttention-Net: A deep attention model for prostate cancer segmentation by aggressiveness in MRI scans. Med Image Anal 2022; 77: 102347.
Czodrowski P. Count on kappa. J Comput Aided Mol Des 2014; 28: 1049–1055.
Alavi M, Biros E, Cleary M. A primer of inter-rater reliability in clinical measurement studies: Pros and pitfalls. J Clin Nurs 2022; 31: e39–e42.
Lin T-Y, Goyal P, Girshick R, He K, Dollár P. Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision. 2017, pp 2980–2988.
Buda M, Maki A, Mazurowski MA. A systematic study of the class imbalance problem in convolutional neural networks. Neural Netw 2018; 106: 249–259.
Li L, Jamieson K, DeSalvo G, Rostamizadeh A, Talwalkar A. Hyperband: A novel bandit-based approach to hyperparameter optimization. J Mach Learn Res 2017; 18: 6765–6816.
Loshchilov I, Hutter F. Decoupled Weight Decay Regularization. In: International Conference on Learning Representations. 2019https://openreview.net/forum?id = Bkg6RiCqY7.
Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 2014; 15: 1929–1958.
Föll S, Maritsch M, Spinola F, Mishra V, Barata F, Kowatsch T et al. FLIRT: A feature generation toolkit for wearable data. Comput Methods Programs Biomed 2021; 212: 106461.
Friedman J, Hastie T, Tibshirani R. Sparse inverse covariance estimation with the graphical lasso. Biostatistics 2008; 9: 432–441.
Tibshirani R. Regression shrinkage and selection via the lasso. J R Stat Soc Ser B Methodol 1996; 58: 267–288.
Foygel R, Drton M. Extended Bayesian information criteria for Gaussian graphical models. Adv Neural Inf Process Syst 2010; 23.
Haslbeck J, Waldorp LJ. mgm: Estimating time-varying mixed graphical models in high-dimensional data. ArXiv Prepr ArXiv151006871 2015.
Haslbeck JM, Waldorp LJ. How well do network models predict observations? On the importance of predictability in network models. Behav Res Methods 2018; 50: 853–861.
McHugh ML. Interrater reliability: the kappa statistic. Biochem Medica 2012; 22: 276–282.
Kennedy SH. Core symptoms of major depressive disorder: relevance to diagnosis and treatment. Dialogues Clin Neurosci 2022.
Weiner L, Doignon-Camus N, Bertschy G, Giersch A. Thought and language disturbance in bipolar disorder quantified via process-oriented verbal fluency measures. Sci Rep 2019; 9: 1–10.
Saha S, Baumert M. Intra-and inter-subject variability in EEG-based sensorimotor brain computer interface: a review. Front Comput Neurosci 2020; 13: 87.
Sun C, Shrivastava A, Singh S, Gupta A. Revisiting unreasonable effectiveness of data in deep learning era. In: Proceedings of the IEEE international conference on computer vision. 2017, pp 843–852.
Krishnan R, Rajpurkar P, Topol EJ. Self-supervised learning in medicine and healthcare. Nat Biomed Eng 2022;: 1–7.
Joyce DW, Kormilitzin A, Smith KA, Cipriani A. Explainable artificial intelligence for mental health through transparency and interpretability for understandability. Npj Digit Med 2023; 6: 6.
Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G et al. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In: Advances in Neural Information Processing Systems 32. Curran Associates, Inc., 2019, pp 8024–8035.
Biewald L. Experiment Tracking with Weights and Biases. 2020.https://www.wandb.com/.
Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O et al. Scikit-learn: Machine Learning in Python. J Mach Learn Res 2011; 12: 2825–2830.
Epskamp S, Cramer AO, Waldorp LJ, Schmittmann VD, Borsboom D. qgraph: Network visualizations of relationships in psychometric data. J Stat Softw 2012; 48: 1–18.
Epskamp S, Borsboom D, Fried EI. Estimating psychological networks and their accuracy: A tutorial paper. Behav Res Methods 2018; 50: 195–212.

Table 1. This work is the first in personal sensing for MDs attempting to infer the full symptom profile, providing actionable clinical information beyond
a single reductionist label, and it also stands out for the relatively large
sample size (the largest among studies where MD acute phase clinical
evaluation was not retrospective). Previous studies recruiting patients with
either a DSM or an International Classification of Diseases (ICD) MD diagnosis
and using passively collected wearable data are reported. F%: Percent Females;
Mage: mean age; SDage: standard deviation age.

	Device(s)	Num. Patients	Patients Features	Task
This work	Empatica E4	75	MDD, BD; M_age=44.16 SD_age=14.42 F_%=56	HDRS and YMRS items multi-task regression
Côté-Allard et al.²¹	Empatica E4	47	BD; M_age=44 SD_age=15 F_%=67.24	Mania vs Euthymia binary classification
Ghandeharioun et al.²³	Empatica E4 and Android Phone	12	MDD; M_age=37 SD_age=17 F_%=75	HDRS total score regression
Pedrelli et al.²²	Empatica E4 and Smartphone	31	MDD; M_age=33.7 SD_age=14 F_%=74	HDRS total score regression
Jacobson et al.²⁵	Actiwatch	23	MDD; M_age=48.2 SD_age=11.0 F_%=43	Depression detection binary classification
Tazawa et al.²⁴	Silmee W20	45	MDD, BD; M_age=52.1 SD_age=13.2 F_%=46.7	Depression detection binary classification
Nguyen et al.²⁶	Actiwatch	45	MDD, SCZ; M_age=44.70 SD_age=11 F_%=73.33	Disease detection binary/multi-class classification
Lee et al.²⁷	Fitbit Charge Hr 2 or 3 and Smartphone	270	MDD, BD; M_age=23.3 SD_age=3.63 F_%=54.5	Mood episode prediction binary classification

Table 2. Clinical-demographic characteristics of the study population (N = 75).
According to the DSM-5, an MD can be categorized as either a major depressive
episode or a manic episode. As a bridge between these two, the DSM-5 admits
a mixed symptoms specifier (MX) to cases where symptoms from both polarities
are present. SD: standard deviation; IQR: inter-quartile range.

	mean (sd)		median (iqr)
age	44.66 (14.42)		45.00 (24.50)
hdrs (total)	7.27 (6.94)		4.00 (6.00)
ymrs (total)	7.21 (8.75)		3.00 (10.00)
number of subjects (%)
sex	male: 33 (44)	female: 42 (56)
mood state assessment(s)	mde-mdd: 9 (12) eu-mdd: 3 (4) 1: 75 (100)	mde-bd: 12 (16) me: 28 (37) mx: 7 (9) eu-bd: 16 (21) 2: 44 (59) 3: 22 (29) 4: 8 (11)

Table 3. Quadratic Cohen’s κ ranges from 0.775 on “somatic symptoms gastrointestinal” and to 0.492 on “anxiety psychic” (mean of 0.609). Item level

QCK across HDRS and YMRS items. See Table A.3 for marco-averaged F1 scores.

Item QCK	H1 0.642	H2 0.624	H3 0.694	H4 0.534	H5 0.595	H6 0.512	H7 0.629	H8 0.604	H9 0.508	H10 0.492
Item QCK	H11	H12	H13	H14	H15	H16	H17	Y1	Y2	Y3
	0.636	0.775	0.582	0.594	0.691	0.637	0.574	0.602	0.590	0.627
	Y4	Y5	Y6	Y7	Y8	Y9	Y10	Y11
	0.629	0.591	0.572	0.582	0.588	0.755	0.602	0.566

The authors have declared there is NO conflict of interest to disclose

SupplementaryMaterials.docx

Download PDF

Journal Publication

published 26 Mar, 2024

Read the published version in Translational Psychiatry →

Editorial decision: revise
23 Oct, 2023
Review #3 received at journal
18 Oct, 2023
Reviewer #3 agreed at journal
27 Sep, 2023
Review #1 received at journal
10 Aug, 2023
Reviewer #2 agreed at journal
03 Aug, 2023
Reviewer #1 agreed at journal
01 Aug, 2023
Reviewers invited by journal
01 Aug, 2023
Submission checks completed at journal
10 Jul, 2023
First submitted to journal
07 Jul, 2023
Editor assigned by journal
07 Jul, 2023

You are reading this latest preprint version

Automated mood disorder symptoms monitoring from multivariate time-series sensory data: Getting the full picture beyond a single number.

Status:

Journal Publication

Version 1

Abstract

Figures

Introduction

Methods

2.1 Data collection and cohort statistics

2.2 Data pre-processing

2.3 Evaluation Metrics

2.4 Model Design

2.5 Learning from imbalanced

2.6 Model training and hyperparameter tuning

2.7 Non-deep-learning baseline model

2.8 Prediction Error Examination

2.9 Channels Importance

Results

4.1 Post-hoc diagnostics

4.2 Channels contribution

Discussion

Declarations

References

Tables

Additional Declarations

Supplementary Files

Status:

Journal Publication

Version 1