Analysis 1 – UK Validation
Results from Pearson correlation analyses indicated that ATI was linked with survey anxiety, r(20)= .68, 95%CI[.34,.86], p < .001, as well as depression measures r(20)= .83, 95%CI[.60,.92], p < .001 (see figure 1). ATI was also correlated with average search frequency for mental health-related terms r(20)= .46, 95%CI[.03,.75], p = .04, although this last measure was not substantially related to either survey anxiety r(20)=.13, 95%CI[-.33,.54], p = .59 or depression measures, r(20)= .34, 95%CI[-.13,.68], p = .15.
Yet, because our data are essentially time series, we needed to rule out non-independence of observations as a potential source of bias. To do so, we regressed all our measures on time and extracted the remaining residuals of these analyses. All variables exhibited linear decay, β = -.82, 95%CI[-1.10,-.54], p < .001 for ATI, β = -.94, 95%CI[-1.11,-.78], p < .001 for anxiety and β = -.71, 95%CI[-1.06,-.36], p < .001, for depression – at the exception of average frequency of mental health searches β = .23, 95%CI[-.71,.25], p = .33. We then re-ran Pearson correlations using the residuals. Although the link between ATI and depression held, r(20)= .61, p = .006, the relationship with anxiety reversed, r(20)= -.50, p = .03. suggesting a potential artefactual origin (i.e. from data non-independence).
This first validation analysis therefore provided evidence for predictive, concurrent and construct validity of the ATI. Not only this measure was strongly linked with survey measure of depression and, but it also displayed a similar dynamic over time than what was found in the UCL COVID-19 Social Study’s results30. However, the sample size was small and we still needed to establish cross-cultural validity of the ATI before using it as a proxy for population mental health in France. We therefore performed a second validation.
Analysis 2 – France Validation
In France, the larger number of observations and the data structure allowed to proceed with finer grained analyses. The only difference was that depression and anxiety measures in that sample reflect provisional diagnoses for disorders, not scores (i.e. percentage of individual above scale cutoff scores). To account for clustering within regions (n = 12) and to directly incorporate time-related trends (survey waves, n = 16), mixed-effects models were computed for each survey measure allowing both slopes and intercepts to vary according to the following specifications:
(1) depression = 1 + ATI + wave + (1 + ATI + wave | region)
(2) anxiety = 1 + ATI + wave + (1 + ATI + wave | region)
Results replicated what we found in the UK, with 1% ATI increase associated with +.23 % depression diagnoses 95%CI[.09,.37], t(22) = 3.30, p = .003, while it was not robustly linked with anxiety β = .20, 95%CI[-.01,.38], p = .076 (full model tables are available at https://osf.io/2yh65/?view_only=e750deb7ac0a46358cab870e3fb1d260).
Interestingly, all three variables seemed to display a polynomial dynamic, with an initial decline and stabilization until week 10 (June 8th) and a progressive increase again (see figure 2). This was confirmed by further mixed-modelling according to the following specifications:
(3) depression = 1 + wave + wave2 + wave3 (1 | region)
(4) anxiety = 1 + wave + wave2 + wave3 (1 | region)
(5) ATI = 1 + wave + wave2 + wave3 (1 | region)
Although anxiety rates displayed a second-order (quadratic) trend, β = .22, 95%CI[.07,.35], p = .005 (cubic parameter p > .10, see full model on the OSF project page), both ATI and depression rates displayed similar cubic behavior with respectively β = .01, 95%CI[.006,.013], p < .001 and β = .02, 95%CI[.01,.03], p < .001. This provided further evidence for the specificity of ATI – which mimicked depression dynamics, and the polynomial trend observed was indicative of something ongoing around the same time the 2nd wave of the pandemic started in France. Having established the relevance of using ATI as a proxy for mental health (and more specifically depression) in both the UK and France, we proceeded to our main analysis of the pandemic’s effect on population mental health.
Analysis 3 – Joint Pandemic Impact Analysis
For this third analysis, we combined weekly ATI data from 2019 and 2020 (n = 104) in both France and the UK (total n = 208) As can be seen in figure 3, ATI dynamics are quite similar across countries, and most likely reflect a strong impact of the COVID-19 pandemic, at least during the first wave.
We then set out to test this hypothesis using Bayesian structural time-series models30. In short, these models allow to train Markov chain Monte Carlo algorithm to quasi-experimentally generate the estimate of an exogeneous event’s impact on a time series of interest relative to a time-series hypothesized to be unaffected by the said event. To do so, we extracted and averaged a series of 19 randomly generated words from Google Trends (using https://randomwordgenerator.com/) matched in English and French so as not to contain words with accents ( “position,” “money,” “seed”, “participate,” “node,” “asylum,” “dominate,” “size,” “passion,” “question,” “ant,” “suntan,” “mass,” “clash,” “file,” “bold,” “band,” “addition,” and “attachment”).
After specifying the pre and pandemic periods based on reports of the first cases to appear in each country, (week 56 in France and 57 in the UK), we ran the Bayesian models (syntax, full tables and data available on the project OSF page). Our analyses indicate that, in line with the visual effect on figure 3, the pandemic did increase ATI by an absolute average 3.2%, 95%CI[2.1,4.2], p = .001 in France and by an average 3.7%, 95%CI[2.9,4.4], p = .001 in the UK (see figure 4).
Overall, we had strong evidence that the ATI was impacted by the pandemic, but we wished to further investigate this phenomenon. We decided to focus a last analysis on data during the pandemic exclusively, factoring in both new weekly cases and deaths per million inhabitants from the COVID-19 Data Repository by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University (available at https://github.com/CSSEGISandData/COVID-19) in France and the UK (n = 96). A final mixed-effects model was therefore computed according to the following (full model available on the OSF project page):
(6) ATI = 1 + cases + deaths + weeks (1 | country)
Despite a general tendency to go back to baseline over the weeks, β = -.18, 95%CI[-.25,-.10], p < .001, ATI increased significantly as a function of COVID-19 deaths β = .14, 95%CI[.14,.21], p < .001 , but not cases β = -.001, 95%CI[-.001,.001], p = .40 (see figure 5)