Need: To avert the looming mental health crisis for millions of families
COVID-19 has a significant consequence on mental health across a vast population as per experts . Mental health & mental wellness concern is emerging as a significant and urgent need for a vast majority of the world population as per a recently published report by the World Health Organization (WHO) . The COVID-19 pandemic is causing monumental effects on mental wellbeing worldwide .
Experts call for an urgently deployable intervention, as per publications in JAMA  and Lancet [2 , 32]. Recent editorials in reputed Nature Medicine  and Lancet  have highlighted the need for action as there is mounting evidence of a widespread impact of the pandemic on mental health. United Nations has recommended actions to ensure people and societies are better protected from the mental health impact of COVID-19 . The Lancet position paper  calls for multi-disciplinary research priorities. Rising to this challenge will require integration across disciplines and sectors as per Lancet . Given the need for timely intervention for millions of families, a scalable solution will be necessary. The opportunity and necessity to contribute to research to find solutions to address this challenge are immense, given the pandemic’s vast impact . This paper contributes to this gap and supports the research community to avert the crisis by contributing an open-source AI.
Challenge: Timely mental health intervention for millions of families
Fear about the perceived threat of COVID-19 infection, loss of a job, physical isolation from friends due to lockdown, quarantine of family members, closures of schools, fear of infecting loved ones by nurses caring for COVID-19 patients, uncertain future, risks due to asymptomatic virus are affecting mental health [1, 2, 31, 32]. Millions of such families require early & timely health intervention .
Hence, the three dimensions of the challenge to be addressed by the proposed AI are
- Scalable: The scale to impact millions of families, and
- Timely: Early & timely intervention with 24x7 availability of help
- Privacy: Protection of information on individual’s situations, her fears etc.
In Oct 2020, the Lancet editorial  said it is unclear how the world will deal with this forthcoming mental health crisis, given the shortage in the capacity to respond. WHO found  disruption of mental health services across many countries, especially for the vulnerable. With millions of families impacted by the COVID-19 pandemic, the shortage of capacity, and the criticality of providing timely health intervention, combined with the privacy requirements, make this challenge of averting the forthcoming mental health crisis into a monumental challenge. Also, privacy needs can’t be ignored as mental health therapy often involves handling sensitive personal information.
As per WHO , 1 person dies from suicide every 40 seconds. Around 1 in 5 adolescents have a mental disorder as per WHO . As per many editorials in Nature Medicine and Lacent Global Health journals, there is an increasing alert on the fallout of COVID-19 pandemic on global mental health [1, 2, 39, 40]. The United Nations reported that the mental health and mental wellbeing of societies have been severely impacted by COVID-19 . The May 2020 editorial of Nature Medicine  called for researchers across multiple domains to address the impact of COVID-19 on the world’s mental health. The Nov 2020 editorial of Lancet Global Health  says 1 billion people suffer from a mental disorder. The Lancet editorial  also restates that sooner or later, health systems will be challenged to face a widespread demand for mental health due to COVID-19. By May 2020, United Nations published a policy brief  strongly urging the need for action on mental health. By Oct 2020, WHO calls the world to action on mental health. In short, multiple experts and leaders have forewarned about the impact of the pandemic on mental health and have called for action during the last few months.
As a response to this call for action, this paper contributes an AI to enable research communities to design a solution to avert the impending mental health crisis. Though this paper demonstrates an AI solution and develops a prototype implementation in the English language, future researchers may extend it to other languages, given the spread of COVID-19 across countries.
Existing literature and novelty in this paper
Novel application of GPT2 for CBT:
- There are not many research publications on the applications of the latest advancements in AI for mental health therapy. Specifically, the gap is in the application of state of the art NLG such as GPT2.
- While there is a lot of work in applications of state of the art NLU for mental health diagnosis , there is a gap in applying state of the art NLG for mental health therapy.
- While research papers had attempted therapy using NLG, old Deep Learning techniques were applied. However, the opportunity to apply the latest Deep Learning techniques remain untapped. The power of the latest advances such as GPT2 opens up the opportunity to create human-like English narratives. More significantly, the possibility of conditioning the language generated using transfer learning allows AI based therapy to help individuals. This breakthrough potential was untapped in the literature. This untapped opportunity offers a clear path to avert the looming mental health crisis from the COVID-19 pandemic. This paper presents this untapped opportunity and reports the conceptual advance by proposing an AI based architecture, its feasibility and a prototype with reproducible results, and an open-source contribution to help fellow researchers.
Very few research publications have explored the application of recent advances in NLG (Natural Language Generation) for mental health therapy. While the advancements in Deep Learning for NLG are staggering in recent years, the potential uses of applying these AI advancements to create solutions for mental health remains untapped. An advancement in the year 2019 in the field of Deep Learning based NLG is GPT-2 . GPT-2 (Generative PreTrained Transformer)  is a powerful language model built using Attention mechanism  based Transformers . GPT-2 is a language model that could perform natural language processing applications such as answering questions, completing text, reading comprehension, text summarization. GPT2 is capable of generating human-like language. It was demonstrated to create fake news  or to create poetry  or generate image captions . This was possible due to transfer learning capability , recently made possible in Transformers architecture  based NLG, opening up the “imagenet moment in NLG”. However, there is a lack of research papers on the application of GPT-2 for mental health therapy, especially at a time when its application can transform the way the world responds to the titanic challenge of looming mental health crisis. This paper is an attempt to bridge this gap, which can lead to a breakthrough approach to avert the forthcoming crisis.
A search on Google Scholar for the keywords combination of “Cognitive Therapy” and “GPT2” shows feeble search results. A Google scholar keyword search on “Cognitive Behaviour therapy Deep Learning GPT2” or “mental health therapy Deep Learning GPT2” or “Cognitive therapy Deep Learning GPT2” or “CBT Transformers GPT2” shows a few research publications, and hence the gap identified is presented in Table 1.
To the best of our knowledge, this is the first research paper to apply latest AI approaches such as GPT2 in Cognitive Behavior Therapy . This paper’s work on this unexplored area opened the doors to an AI architecture that is capable of averting the future mental health crisis from COVID-19 pandemic. One of the mental health therapies is CBT or “Cognitive Behaviour Therapy” . CBT is a therapy technique that can help people find new ways to behave by changing their thought patterns [7, 8, 9]. A novel NLG based Cognitive Behavior therapy model is proposed in this paper.
There is substantial research publications in NLU (Natural Language Understanding) such as BERT  for diagnosis of sentiment  or sensing of emotions . However, there are only very few publications on NLG for mental health therapy. A survey paper  took stock of a decade of studies with 139 papers shows a lot of effort has happened on diagnosis, but there is significant scope for future research on novel applications of NLG for mental health. A paper in Nature scientific reports  also shows efforts on the topic of diagnosis. It shows progress in application of Deep Learning for classification problem statements to classify emotions/mental health conditions. However, the opportunity to apply Deep Learning in the synthesis using NLG is a relatively unexplored research theme in the context of mental health, and even more specifically in the context of CBT.
There are attempts to use NLG for therapy using old Deep Learning techniques such as LSTM. However, not many papers have explored the use of latest Deep Learning techniques for mental health therapy. Transformers  based language models represent the latest advancements in Deep Learning based NLG. While there are attempts to use NLG for therapy earlier, such as in year 2017 , not many research papers explored the latest advancements in NLG. Since GPT2 emerged only in 2019 , the opportunity to apply state of the art NLG models such as GPT2 has opened up. While 2019 saw the evolution of GPT-2, 2020 saw the introduction of a more powerful language model called GPT-3. Though GPT-3  is a successor to GPT-2, it is not suited for edge AI due to its massive size. Since GPT-3 is only available as a cloud API requiring transmission of user’s thoughts via the internet, it is not suitable to meet the privacy requirements in a mental health solution. Due to the power of transfer learning [13, 27] since the ‘Imagenet moment in NLP’ arrived, it is possible to produce a language model to suit a particular requirement such as poetry  . Transformers based NLP architectures are capable of generating AI performance as close to humans. For instance, it may be hard for humans to find if an AI generated a piece of fake news . This power of AI to generate short text narratives close to human performance can be tapped into for designing novel health therapy solutions. The application of GPT2 like architectures for generating human-like narratives in mental health therapy solutions is a game-changing idea.
More scalable than existing approaches
- The need for a pandemic scale solution has been stated earlier.
- In contrast to cloud-based crawling of social media posts, the proposed on-device AI architecture opens the possibility of providing mental health solutions to a larger population.
Scalability is a significant need in the context of pandemic scale mental health crisis from COVID-19 for millions of families across the globe . This paper makes a unique contribution by proposing an approach that is much more scalable than existing approaches. In addition, there is another unique dimension in this paper that enables a significantly larger percentage of the population to be offered mental health compared to approaches in the existing literature. The architecture proposed in this paper offers a leap in the addressable proportion of the world population. To the best of our knowledge, this is the first paper to propose a novel edge AI architecture to do mental health on the user’s smartphone. This edge AI based architecture is presented in Figure 6. While most existing literature  focus on predicting mental health from publicly accessible social media posts, this limits the percentage of population to those netizens who are actively posting public tweets on personal sensitive thoughts. In contrast, this paper proposes an architecture where AI inference happens on the user’s smartphone, thus allowing the opportunity to screen a larger population. Gboard by Google  is a downloadable predictive keyboard for smartphone users, as shown in the screenshot of Figure 4. Similar to AI embedded Google predictive keyboard on Android smartphones , which auto-corrects spelling errors as users type in words, the proposed approach to embed AI on smartphone will enable significant scalability.
A unique idea of intelligent activation of therapy
- The conceptual advance in “NLU activated GPT2 based therapy” is another unique aspect of the contributed AI architecture.
Once a BERT classifier detects a sequence of depressing thoughts on user’s personal smartphone, the GPT2 predictive keyboard can be activated to provide self-help to weed off thoughts that can lead to habituation of depression. This is also first time in the literature where an AI based therapy is activated intelligently by another AI that keeps a tap on the candidate’s mental health. Once a BERT based classifier detects a sequence of depressing thoughts, this GPT2 predictive keyboard can be activated on the user’s personal smartphone to provide self-help to weed off thoughts that can lead to habituation depression. This novel AI architecture is presented in Figure 6.
More privacy safe than existing approaches
- While cloud chatbots based counseling has been attempted, the concern is in sending sensitive data on one’s personal thoughts into a cloud based server. In contrast, this paper develops a privacy-safe approach from the grounds-up from designing an appropriate architecture such that one’s sensitive data doesn’t leave the boundaries of her personal smartphone.
Privacy safety design from the ground-up is a crucial requirement for digital mental health counseling solutions. The privacy safety of the proposed approach makes this approach to AI-based therapy further unique in the literature. While current attempts to counseling employ an AI chatbot [11, 19, 24, 41], the user’s private data is sent across the public internet to the cloud based chatbot engines. In contrast, in this paper’s proposed architecture, the AI inference happens on-device in the personal smartphone. Privacy safety aspect in mental health approach is illustrated in Figure 6. As per this figure, the user’s sensitive personal information doesn’t leave her personal smartphone device’s boundaries. The proposed solution is privacy safe compared to other approaches, and this is illustrated in the video in URL.
A unique design to enable Early Intervention & Protection of sensitive data
- Early intervention in mental health for millions is critical during a pandemic.
- Handling of sensitive personal data should be done with utmost care.
- There are limitations of employing cloud-based chatbot based approach. Cloud chatbots are not conducive for early intervention.
- In contrast to the cloud chatbots based approach, on-device AI inference is proposed. Adding a mental health AI to a smartphone’s keyboard allows for widespread deployment and early intervention at scale to aver the pandemic scale crisis.
The significance of early intervention in mental health is stated in JAMA . The significance of proactive and early intervention can’t be understated . Hence this limits the adoption of chatbots. Chatbots  require initiation by user base, limiting early intervention & scalable user adoption. Given the scalability needed to activate mental health wellness for millions of families, a chatbot may not yield user adoption. In the early stages of depression, the affected individuals may not be aware that they need help. This lack of awareness of one’s own mental health status in the early days of depression  can stop an individual from asking for help from a web-based chatbot.
In contrast to chatbots, approaches such as Gboard [14, 16] can enable a systemic proactive early intervention at scale for the global population. The expert review in  calls out the cons of collecting personal mental health data. From a privacy point of view, patients’ personal data can’t be sent over the internet to cloud-based chatbots. In contrast, the proposed approach is AI that run on-device. This AI is an NLG based CBT inspired self-help therapy, packaged in the form of a predictive keyboard such as Gboard . The proposed idea is illustrated in Figure 2 and Figure 4.
Knowledge gap in the literature
Novelty / What is unique in this paper
Lots of existing literature
Mental Health Diagnosis Application of NLU (BERT based analysis of a sentence spoken/tweeted)
Gap/ Opportunity to contribute
Mental Health Therapy Application of NLG to influence the person’s thoughts
Notable contribution is the proposed GPT2 architecture for CBT (Refer Figure 6)
Table 1 : Related Literature and gaps for contribution
A call for action to avert the forthcoming mental health was issued recently by the United Nations (UN) . Towards addressing this mental health crisis fuelled by COVID-19 pandemic , contributions in this paper are as follows.
A scalable AI solution for mental health intervention: Given the pandemic’s impact, the proposed solution needs to be scalable to millions of families. It also needs be a timely health intervention for millions of people. It also should be privacy safe. This work proposes and demonstrates an AI-based strategy with a reference solution in response to this call for multi-disciplinary research on mental health . The focus of this paper is applying the latest advancements in AI for therapy, though the mental health solution covers both therapy and screening. The solution to avert the crisis is based on both therapies using NLG (Result #1) and screening using NLU (Result #2). Both therapy and screening can be seen in Figure 1. The proposed mental health solution and the accompanying AI architecture is demonstrated with a working lab prototype with reproducible results. A lab prototype is implemented and open-sourced to enable many interested research communities to avert the crisis.
Novel application of GPT2 in therapy: This paper proposes and demonstrates a novel AI based approach to provide mental health therapy. The novelty is regarding the applications of recent advances in Deep Learning based Natural Language Generation (NLG) to one of the commonly used mental health therapy called Cognitive Behaviour Therapy (CBT). As described earlier in the literature review section, this paper explored an untapped potential of applying the state of the art advances in language models to create a mental health therapy solution. The possibility of conditioning the language generated using transfer learning allows AI based therapy to help individuals develop a positive way of thinking and seeing situations. Like auto-correction of spellings in a smartphone’s predictive keyboard, this AI “auto-corrects” the narratives being typed by a mentally depressed person. Natural language generation generates text as user starts typing his thoughts, so that the user can view the situation from a “positive lens”. A NLG enabled keyboard can be downloaded on to the user’s smartphone that can offer care instantly, enabling early correction of unhealthy thoughts. The proposed AI embedded mental health predictive keyboard is illustrated in Figure 4. Since GPT2 inference computation happens in less than few seconds, this coaches the user’s to have healthy thoughts while maintaining privacy. Since few research publications explored the applications of state of the art AI for language generation in self-help based therapy techniques such as CBT, this paper contributed to this gap. It proposed an AI architecture based on Transformers based NLG, proposed novel AI approaches such as NLU activated NLG. It also demonstrated the feasibility of the proposed AI with a prototype implementation. The proposed AI opens the door to develop a AI-based solution to avert the looming pandemic scale crisis for millions of families with a timely health intervention. It is also designed to ensure privacy through an on-device AI inference, even though CBT often involves handling of very sensitive personal information. A pre-trained GPT2 model was fine tunned to generate human like English based on training on a synthetic dataset. Three different GPT2 models were compared, concluding a small-sized language model may be good choice for on-device AI inference. Novel ideas such as NLU activation of GPT2 based therapy ensures timely intervention in a wide deployment for a large population. A prototype of BERT activated GPT2 based therapy is demonstrated. The demonstration shows the GPT2 based self-help therapy is activated only when appropriate. Experts have alerted us of a crisis sooner or later . In this backdrop of the forthcoming crisis, this AI demonstrates a significant leap forward in developing an approach to prevent the impending mental health crisis. This work shows it is possible to use AI to prevent the looming mental health crisis.
Experts have called for multi-disciplinary teams  to get involved in averting the crisis. Accordingly, to encourage research communities to avert the forthcoming mental health crisis, the paper contributes an open-source AI prototype. While this paper focuses on the Deep Learning aspects of the solution, future multi-disciplinary research teams can extend this AI from the mental health perspective. Though this prototype is on English, future AI researchers can extend this AI to many world languages as many countries need this solution.
Idea summarized: Novel AI application to avert the mental health crisis
A key idea is an application of state of the art Deep Learning based Natural Language Generation (NLG) for mental health therapy. There has been amazing progress in attention technique based NLG models, such as Transformers based NLG architectures. A language model is basically a Deep Learning model that is able to look at part of a sentence and predict the next word. GPT-2 is a language model to produce human-like text. The opportunity to employ transfer learning techniques in NLG arrived very recently. The recent advances in Transformers based NLG model such as GPT2 allow for AI performance similar to human like performance. However, the potential of apply these Deep Learning advancements for improving mental health remains untapped in the literature. This paper demonstrates the potential to employ Transformers based NLG architectures to produce AI solutions in mental health therapy. Using Transfer learning, a GPT2 model was fine-tuned to generate short text sentences to help depressed individuals to look at the situation from a different point of view. A BERT based NLU detects if a person is entering depression and then activates the therapy module.
The idea of applying NLG for therapy is presented in Figure 2. In this idea, an AI resides in the smartphone of the user and helps to improve mental wellness by suggesting flexible language narratives. Similar to the auto-correction of spellings in a smartphone’s predictive keyboard, the proposed AI solution “auto-corrects” the narratives being typed on the user’s personal smartphone, as illustrated in Figure 4. This paper infuses AI into one of the popular mental health therapy approach, called Cognitive Behaviour Therapy (CBT) . CBT is a therapy technique to help people find new ways to behave by changing their thought patterns.
With the power of transfer learning and human like text generation capabilities based on GPT2, the potential to apply NLG in therapy is significant. A lab prototype demonstrates this result, and a short video recording of a demo is presented online at URL, https://sites.google.com/view/ai-in-mental-health .
Potential impact in future: Enabling research communities
GPT -2’s ability to generate conditional synthetic text samples of unprecedented quality combined with transfer learning opens up an opportunity for mental health professionals to scale their impact to support millions of people on a continuous online basis. This is the potential that can be unlocked to avert the challenge of mental health impacted by the COVID-19 pandemic. This paper presents this potential by developing and demonstrating state of the art AI.
Researchers can take advantage of this opportunity to apply state of the art AI techniques to improve mental health. Further to presenting novel ideas, the paper also contributes an entire AI prototype solution in open source. The scope of the prototype implemented is specified in Table 2. This is intended to encourage interdisciplinary research communities in their future research.
Organization of the paper
[Table 2 is in the supplementary files section.]
As presented in Table 2, Contribution #1 is around enabling a solution to avert the crisis, Contribution #2 is around novelty. The paper’s results section is organized as follows: The primary result of this paper is Result #1. While Result #1 presents the novel application of AI for mental health, Result #3 expands the implementation of result #1 from a Deep Learning viewpoint. NLG in CBT is presented as both Result#1 and Result #3, while NLU in screening is presented as Result #2. Future research by the community can expand this AI into a real-world deployment to avert the crisis.
The next section on Results presents the three results. For each of the three results, the respective methods, discussions on literature is organized in each sub-section.