Mental Health Monitoring System Using Facial Recognition, PEN Test and IQ Test

The human face is an important part of an individual’s body and plays an important role in knowing the individual’s mood. The face is where a human expresses all his basic emotions. In the existing system, they examine the mental state manually by assessing them but have many disadvantages like we cannot predict any accurate solutions based on the assessment score because we might be not sure what kind of emotions the human user would be all time. To overcome this problem, a novel system is proposed to suggest an effective solution for predicting the mental state dynamically, we propose a hybrid architecture invoking facial based emotion sequence, PEN test and IQ test. By consistent monitoring of a human’s emotion and subjecting to PEN and IQ tests, the human’s mental state is routed. Combination of the above three techniques provides promising results for mental state and self-control.


Introduction
Facial expressions can be considered not only as the most natural form of displaying human emotions but also as a key non-verbal communication technique. The processing of emotional facial expressions is modulated by personality; for example, neuroticism, a dimension of the ve-factor model of personality has been found to play a role in this regard. Neuroticism is characterized by the tendency to be anxious, nervous, and hostile. Neuroticism is also considered to be a risk predictor for depression.Dementia, attention deficit hyperactivity disorder, schizophrenia and obsessive compulsive disorder. In schizophrenia, cognitive impairments are often found in multiple areas, including visual information processing; attention; working memory; short-term memory and learning; executive functioning; speed of processing; reasoning and problem solving; context processing and social perception and cognition. The impairment of executive functions called "dysexecutive syndrome", are common in neurological patients and are related to brain dysfunction specifically in the prefrontal cortex. Individuals who have an impairment of executive functions show problems of starting and stopping activities, a difficulty in mental and behavioral shifts, an increased distractibility and difficulties in learning new tasks. The executive functions, defined as higher order cognitive functions needed for performing complex tasks, are often impaired also in patients with Obsessive Compulsive Disorder (OCD) which are characterized by the impairment of several skills such as attention, planning, problem-solving and behavioral control. No single profile of cognitive deficits has been found to characterize all patients. The majority have impaired ability in at least one area of functioning and a standardized platform for assessing neurocognitive functioning is an important aspect of comprehensive treatment and research for this and other conditions.
In this world, lot of people are facing numerous problems which result in depression and mental illness and the users are motivated to innovate a solution integrating image processing and machine learning techniques to predict the mental illness by recognizing people's emotions and by conducting IQ and PEN tests to measure the mental illness and provide relevant suggestions.The objective of the proposed system is to develop a feasible system which could provide promising results for detecting depression, mental state. In the proposed, the users use hybrid architecture of facial emotion sequence and psychometric tests to identify mental disorders. In this project, the user uses real time inputs for processing.
Mental Illness has a profound impact on people's functioning, health and quality of life. Detecting early warnings of depression or any other mental illness is challenging. The proposed system provides a hybrid architecture invoking facial based emotion sequence, PEN test, IQ test. By consistent monitoring of a human's emotion and subjecting to PEN and IQ tests, the human's mental state is routed. Combination of above three techniques provides promising results for depression, mental state and self-control. In our proposed system, the emotions are continuously monitored based on which the information for the classi cation of mental illness of the person is obtained. Further, using the information obtained, it conducts a psychology test to diagnose the severity of the mental condition. It combines these outputs with a psychometric study which consists of an IQ test and a Personality test. The output of the combination of these three parameters are classi ed to determine the probabilities of identifying the mental and provide recommendations such as " lacks concentration " "Need to be focused" "Seems depressed need some entertainment" a person can hold.

Literature Review
Jyoti Kumari [1] et al. discusses the Detection of mental disorders, and synthetic human expressions. The author mentioned that the two common methods used predominantly in the literature for FER automatic systems depend on geometry and appearance. The author provides a quick scan for facial expression recognition. A comparative study was also performed using various feature extraction techniques in the JAFFE dataset.Limitations: Analyzing facial expressions has a major drawback -humans can control the simulation to some extent, so recognition results may be falsi ed, intentionally or unintentionally. G. Kalaivani [2] et. al discusses Viola Jones techniques and image cropping techniques for extracting and delineating areas of the mouth.The proposed segmentation techniques are applied and compared to the method found which is suitable for segmentation of the oral region, and then the oral region can be extracted by means of contrast extension and image segmentation techniques. After extracting the mouth area, the facial feelings are ranked based on the white pixel values in the mouth area extracted from the face image.Limitations: Traditional image segmentation techniques are more fragmentation and have high noise sensitivity.
Khan [3] et al. discuss Emotion recognition as an important area of work for improving human-machine interaction. The complexity of the emotion makes the task of acquisition even more di cult.Deep learning technology with neural networks increased the machine's success rate with regard to emotion recognition. Modern works with deep learning technology have been implemented with different types of human behavior inputs such as auditory and visual inputs, facial expressions, body gestures EEG, and related brainwaves and signals.The authors have attempted to explore the relevant important work, its techniques, the effectiveness of the methods and the scope for improving the results.Limitations: Deep learning lacks common sense. This makes systems fragile and when errors do occur, the errors can be very large.
Ravichandra Ginne [4] et al. have surveyed various Facial expression recognition (FER) techniques which has become an active research area that nds many applications in areas such as human and computer interfaces, human emotion analysis, psychoanalysis, medical diagnostics, etc. The common techniques used for this purpose depend on geometry and appearance.Deep convolutional neural networks (CNN) have been shown to outperform traditional approaches for various visual recognition tasks including recognition of facial expressions. Despite efforts to improve the accuracy of FER systems using CNN, current methods may not be su cient for practical applications.This study includes a general review FER of systems using CNN and their strengths and limitations which helps us to further understand and improve FER systems.Limitations: Improper encoding of object's position and orientation. Lack of the ability to be spatially unchanged for the input data.
Turetsky [5] et al. have examined potential Event-Related Responses(ERP) to emotional faces in schizophrenic patients and controls to determine when the patient's abnormalities occurred in the temporal treatment course. Subjects indicated whether each face was "happy," "neutral," or "sad." The excited potential data was obtained using a 32-channel system EEG.Four ERP components were identi ed that are compatible with P100, N170, N250 and P300. Group differences were noted for the N170 component "face processing" that underlies the structural coding for facial features, but not for the next N250 component "effect on modi cation".Although P300 abnormalities were found, the variation in this component was explained by the earlier N170 response.The limitation is that the recording is of poor spatial accuracy.
Gibbons RD[6] et al. have explored recent developments in computerized adaptive diagnostic screening and computerized adaptive testing for the presence and severity of mental health disorders such as depression, anxiety, and mania.The statistical methodology is unique in that it is based on the multidimensional element response theory (severity) and random forests (diagnosis) rather than the traditional mental health measurement based on the classical test theory (simple score) or the one dimensional element response theory.The limitations of the system is that a complex model requires much larger samples of people. Shojaeilangari [7] at al. have proposed an approach called extreme scattered learning, which has the ability to co-learning a dictionary (set of rules) and a nonlinear classi cation model.The proposed approach combines the discriminative power of an extreme learning machine with the reconstructive characteristic of sparse representation to enable accurate classi cation when presented with noisy signals and incomplete data recorded in natural environments. The author has also introduced a distinctive and stable new local spatio-temporal descriptor.The proposed framework is able to achieve state-of-the-art recognition accuracy in both spontaneous and affective facial emotions databases.The limitations of the system is that a complex model requires much larger samples of people. and Dynamic FER has a higher recognition rate than static.
Singh [9] et al have examined the acknowledgment of outward appearances with a mix of neural organization for the acknowledgment of various facial feelings. People are t for delivering a large number of facial activities during correspondence that shift in intricacy, power, and importance. This paper investigations the restrictions with existing framework Emotion acknowledgment utilizing cerebrum movement. The author has utilized a current test system and accomplished 97% exact outcomes. Emotion acknowledgment utilizing cerebrum movement framework. The Proposed framework relies on the human face as we probably are aware face additionally mirrors the human cerebrum exercises or feelings. The author uses neural organization for better outcomes.The limitations is that the recognition from mind action may bring about poor spatial goal and misdiagnosed may bring about vein. Zhang [13] has done a survey on Emotion recognition system using facial expressions. The Approaches considered are -DBN, Deep Belief Network. For Feature extraction, MLDP-Modi ed and Local Directional Patterns are considered. The Classi ers used are GDA-Generalized Discriminant Analysis. The Database considered by the author is RGB images and Depth images. The Performance given by RGB is 93.33% and Depth is 96.25%.
Elzbieta kukla [14] et al has introduced a method that uses a series of neural networks to recognize facial expressions.As an input, the algorithm receives a natural image of the face and returns the emotion expressed by the face. To determine the best classi ers for recognizing speci c emotions, single-and multiple-layered networks were tested. The experiments covered different resolutions of the images displaying the faces as well as the images, including the areas of the mouths and eyes.On the basis of the results of the tests, a series of neural networks are proposed. The series introduces six basic emotions and a neutral expression.The Limitations are Black box, development period, amount of data and calculation cost.
Carlos Busso [15] et al have done an analysis of Emotion Recognition using Facial Expressions. The interaction between humans and computers would be more natural if computers were able to recognize an emotion. The author discusses the two methods decision level and feature level integration.Using a recorded database of an actress, four emotions were categorized: sadness, anger, happiness, and the neutral state.Using the markers on her face, detailed facial movements were captured with motion capture, in conjunction with simultaneous recordings of speech.The results reveal that the system based on facial expressions gave better performance than the system that relied only on the audio information of the studied emotions.The results also show the complementarity of the two methods and that when the two methods are combined, the performance and robustness of the emotion recognition system improves signi cantly. The limitations are technological complexity, Security risks and High time consumption.
Anil[16] et al. have done a survey on face and face expression recognition. Facial expression recognition (FER) for recognizing mental state, human interaction with computers, understanding human behavior, etc. are some of its applications.The author discusses specially designed facial expression recognition algorithms. The authors also provided a brief overview of the feature extraction method for facial expression recognition techniques. The author had discussed some techniques for recognizing facial expressions such as corrected geodetic texture conversion, Curvelet feature extraction, word bag method, local vector number pattern, regional scoring technique, gradient feature matching etc. which are used to recognize facial expressions. The limitations found are the estimator is not robust with respect to outliers or extreme skew and the defect has led to the development of semi-robust variants Liliana [17] extends the deep convolutional neural network (CNN) approach to the task of facial expression recognition.This task is done by detecting the occurrence of facial action units (AUs) as a subpart of the facial action coding system (FACS) that represents human emotions.In fully connected CNN layers, The author uses an organizing method called "leakage" which has been proven effective in reducing over-assignment.This research uses the Cohn Kanade Extended (CK +) dataset collected to experiment with facial expression recognition. System performance earns an average accuracy rate of 92.81%. The limitations are that FACS classi cation requires extensive training, time consuming and is subjective and thus prone to bias. This feature makes investigations into large samples di cult.

Santhosh Kumar[18] et al discussed Deep Learning Approach for Emotion Recognition from Human Body
Movements with Feedforward Deep Convolutional Neural Networks. Recognition of feelings from body movements has the advantage of recognizing a person's feelings from any view of the camera and also recognizing feelings, if the person is too far from the camera.Body movements can transmit emotional states more strongly than other studies.The authors have stated that emotional state is recognized from the movement patterns of the whole body using the neural network architecture of the deep gyrus gyrus. The limitations are the risk of over tting data and complex models.
Huiyuan Yang [19]  Pearson's correlation technique and Photosynthesis Pattern (PPG). This signal is simpler than that of other physiological signals.Interval values from Normal to Normal (NN) for HRV were used to extract the time domain features, and the PPG normal signal was used to obtain the frequency domain features.These statistical features were combined with deep learning features extracted from the convolutional neural network (CNN).The PPG signal and NN interval were used as inputs for CNN to extract the features. The limitations is that the convolutional neural network is signi cantly slower due to a process like maxpool.
AlMarri [22] et al. discusses a deep learning-based object detection algorithms which are implemented to classify the emotions expressed in a face in real time captured by a webcam.A Typical CNN will classify images without specifying regions within the image, which can be seen as a limitation towards better understanding of network performance based on different training options. Rapid region-dependent convolutional neural network, which is an object detection algorithm. The author discovers real-time facial expressions by classifying suggested areas. Fast R-CNN is trained using a high-quality video database, consisting of 24 actors, expressing faces of eight different emotions, obtained from processed images from 60 videos per actor. . GabrielleSimcock [25] et al. have done a study, which examined the associations between psychological and physical problems and facial recognition in early adolescence. The computerized emotion recognition task are completed where they identi ed images of 40 faces that show expressions of anger, fear, sadness, happiness, or a neutral expression.The results showed that increased symptoms of anxiety, depression, and somatization were signi cantly associated with fewer correct responses to angry expressions.These symptoms have also been associated with the rapid and more accurate recognition of expressions of fear. However, there was no correlation between mental health and recognition of sad pain. Finally, an increase in psychological and / or physical symptoms was also associated with better identi cation of neutral expressions. The author concluded that the young men with increased psychological and / or physical problems showed a processing bias for negative expressions of anger and fear but not sadness.
Maja Pantic[26] et al. had proposed an Expert system using human emotion recognition syllables -HERCULES. The proposed method is an automated nonverbal communication system, which is designed to analyze the displayed facial expression, based on a psychological study called FAC and interpreted in terms of six basic emotions. HERCULES now accepts measurements made manually on a full-face image and displays a description of the displayed facial expression, its quanti cation and interpretation in terms of the six basic emotions and quanti cation.The Kernel is designed to accept automated face measurements and process them in parallel. A fully automated version of the system in real time in addition to handling inaccurate, duplicate and dangerous entries is still under development. The constraints of the proposed system are angle dependency and load dependency. The public o ce RV is not completely representative and thus may increase or decrease the public o ce RV Filippo La Paglia [27] et al have done a Study in Obsessive-Compulsive Disorder (OCD) and Schizophrenic Patients using Psychometric Assessment Using Classic Neuropsychological and Virtual Reality Based Test. The author had discovered that the assessment of neurocognitive functioning could be a crucial task in clinical settings. In several disorders, psychological feature impairment precedes the onset of behavioural symptoms and psychological feature decline could be a major issue contributive to practical incapacity.The author has done this study to judge the manager functions by scrutiny the evaluations obtained employing a psychology battery with the one obtained victimization the computer game version of the Multiple Errands check (V-MET). The study population enclosed 3 groups: ten patients stricken by neurotic Compulsive Disorder (OCD); ten Schizophrenic patients; ten healthy Controls. The limitations identi ed in the results are few government issues in clinical samples.

Olga Mich[28] et al. discusses the Knowledge Base for Computer-Aided Diagnosis of Mental Disorders
Based on Psychometric Tests. The content may be a key component of the many call support systems. The authors tend to describe the content enforced within the Copernicus system, which is a tool for computer-aided diagnosing of mental disorders supported knowledge coming back from psychology tests, within the current version, coming back from the American state Multiphasic questionnaire (MMPI) test.This tool uses a spread a spread ways in which for differential inhume inhume diagnosing.The content embodied within the tool is of various character.The special attention is concentrated on new components further within the last version of the Copernicus system. Polaris Koi [29] et al critically examine the promise of improving convicts' capacity responsibility by neuro enhancements of self-control to see whether the special characteristics of the inmate population make a difference in the analyses. As improving self-control by means of neuro interventions seems plausible, the authors checked whether it is or could be a justified measure in court rulings. The authors concluded that, even if there are cases in which neurointerventions were warranted in the context of the stated goals of the criminal court. Decreasing recidivism and rehabilitating the offenders to the society, due to the range of individual variability in the constitution of self-control.The prescription of specific neurointerventions of self-control falls outside the scope of legitimate court rulings. Susmita Moitra [30] et al. had surveyed various Image Processing Facial Expression Recognition techniques.. The authors stated that Human facial expressions convey a lot of information visually rather than articulately. Facial expression recognition plays a crucial role in the area of human-machine interaction. Recognition of facial expression by computer with high recognition rate is still a challenging task.Two popular methods utilized mostly in the literature for the automatic FER systems are based on geometry and appearance. Facial Expression Recognition usually performed in four-stages consisting of pre-processing, face detection, feature extraction, and expression classi cation.The authors have applied various deep learning methods (convolutional neural networks) to identify the key seven human emotions: anger, disgust, fear, happiness, sadness, surprise and neutrality.

Existing System
Existing systems are highly complex in terms of time and storage for recognizing user behavior. Existing system focuses on only manual assessment which takes time and doesn't provide promising results as the user's emotions keep varying based on personal, situational problems. Existing system doesn't invoke biometric based emotion recognition of the users. Existing system doesn't focus on user facial expression with psychometric test based automatic individual criminal behavior prediction.
The drawback of the existing system is that the detection of mental illness through system monitoring has never taken place, the accuracy for emotion recognition is less and there is no evaluation of IQ and Personality in the existing system.

System Model Mental Health Monitoring system using Facial recognition, PEN test and IQ test
Mental Illness has a profound impact on people's functioning, health and quality of life. Detecting early warnings of depression or any other mental illness is challenging. The proposed system provides a hybrid architecture invoking facial based emotion sequence, PEN test, IQ test. By consistent monitoring of a human's emotion and subjecting to PEN and IQ tests, the human's mental state is routed. Combination of above three techniques provides promising results for depression, mental state and self-control. In our proposed system, the emotions are continuously monitored based on which the information for the classi cation of mental illness of the person is obtained. Further, using the information obtained, it conducts a psychology test to diagnose the severity of the mental condition. It combines these outputs with a psychometric study which consists of an IQ test and a Personality test. The output of the combination of these three parameters are classi ed to determine the probabilities of identifying the mental and provide recommendations such as " lacks concentration " "Need to be focused" "Seems depressed need some entertainment" a person can hold.
The advantages of the proposed system are having PEN and IQ tests with a set of evaluating questions. It uses a web-cam to capture the emotion of the current users. Based on the nal report, the suggestions are given to suppress the users mental state.

System Architecture
Facial expressions can be considered not only as the most natural form of displaying human emotions but also as a key non-verbal communication technique. If e cient methods can be brought about to automatically recognize these facial expressions, striking improvements can be achieved in the area of human computer interaction. Research in facial emotion recognition has been carried out in hope of attaining these enhancements. In fact, there exist other applications which can bene t from automatic facial emotion recognition. Arti cial Intelligence has long relied on the area of facial emotion recognition to gain intelligence on how to model human emotions convincingly in robots. Recent improvements in this area have encouraged the researchers to extend the applicability of facial emotion recognition to areas like chat room avatars and video conferencing avatars. The ability to recognize emotions can be valuable in face recognition applications as well. Suspect detection systems and intelligence improvement systems meant for children with brain development disorders are some other bene ciaries. Figure 1 shows the Architecture of Mental The input to the system is the Human Face, which can be captured using a web cam. This image undergoes image enhancement, where tone mapping is applied to images with low contrast to restore the original contrast of the image.

Binarization:
All RGB and gray scale images are converted into a binary image. Figure 2 shows the Binarization of a Human Face. This preprocessed image is fed into the face detection block.

Segmentation
A bounding box is formed over every feature of the face that contributes to an emotion. Each bounding box is derived using face coordinates. These boxes are thus segmented and studied further to derive an emotion. Figure 3 speci es the Full Face segmentation.

ii. FEATURE EXTRACTION
The facial image obtained from the face detection stage forms an input to the feature extraction stage. To obtain real time performance and to reduce time complexity, for the intent of expression recognition, only eyes and mouth are considered. The combination of two features is adequate to convey emotions accurately. Finally, a corner point detection algorithm was used to obtain the required corner points from the feature regions. Figure 4 speci es the Eye and Eyebrow segmentation of a Human face.

Eye Extraction
The eyes display strong vertical edges (horizontal transitions) due to its iris and eye white. Thus, the Sobel mask is applied to an image and the horizontal projection of vertical edges can be obtained to determine the Y coordinate of the eyes.

Eyebrow Extraction
Two rectangular regions in the edge image which lies directly above each of the eye regions are selected as the eyebrow regions. The edge images of these two areas are obtained for further re nement. Now sobel method was used in obtaining the edge image since it can detect more edges than roberts method. The term psychometrics refers to the design and interpretation of tests that measure psychological values such as aptitude, ability, personality, memory, happiness and intelligence. These tests are designed for, and given in, mental health, education and employment settings. In this project, we measure the IQ value and the personality traits of the person. Hans Eysenck's personality test is used to measure personality based on three dimensions Psychoticism, Extraversion and Neuroticism (PEN). Personality disorders like depression, anxiety, dissociation, positive and negative, Agreeableness and Openness, Neuroticism, Psychoticism etc can be detected and preventive measures can be addressed at the early stage itself.

vi. CLASSIFICATION OF MENTAL ILLNESS
By considering the output of Psychometrics and mental state as parameters, the probability and the nature of criminal behavior / mental disorder is classi ed as self-destructive and harmful to society.

Advantages of Mental Health Monitoring system
The Mental Health Monitoring system bears the following advantages that are as follows: PEN and IQ tests are considered with a set of evaluating questions for better accuracy It uses a web-cam to capture the emotion for the current users.
Based on the nal report, the suggestions are given to suppress the users mental state.

Results And Discussion
The proposed system has provided the classi cation of mental illness. In order to evaluate the results, IQ and Personality are tested using IQ and PEN Tests and then emotion recognition is measured. According to the results provided the proposed system provides relevant suggestions to the user. Integration of python packages provides promising results for our proposed methodology.      Figure 11 shows the Detection of mental illness for the given user.

Conclusion And Future Enhancement
It is essential to get accurate and reliable identi cation of stress and it requires a valid analysis and experimental methodology framework. The main contribution of the proposed system is developing an experimental model for successfully identifying stress at multiple levels. This proposed system uses python packages to give accurate results for our research on facial emotion based system along with psychometric tests such as IQ and PEN to evaluate the problem solving capability and the nature of users character in this world and if the user is facing some mental disturbance they are provided with relevant suggestions so that they can change their view in the things they face and try be happy in all the situations.
The proposed system can be enhanced to detect more disorders. Also, the system can be enhanced to help doctors, counsellors, therapists to identify and detect stress in patients using the framework.The proposed system can also be made into a complete software for the companies to identify their employees mental health on a daily basis. Figure 1 Architecture of Mental Health Monitoring system using Facial recognition, PEN test and IQ test  Eye and Eye Brow Segmentation  Emotion detection using webcam  Detection of Mental illness