A Novel Approach to the Classification of Human Emotion and Analysis of Complete Mental State Using Brain EEG Signals

doi:10.21203/rs.3.rs-1651932/v1

Download PDF

Research Article

A Novel Approach to the Classification of Human Emotion and Analysis of Complete Mental State Using Brain EEG Signals

https://doi.org/10.21203/rs.3.rs-1651932/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

In the area of brain-computer interface, Intelligent emotion detection based on Electroencephalography brain signals is of great significance. Currently, deep learning algorithms like DNN, CNN, and SVM have significantly improved detection and prediction accuracy in many fields. However, deep learning and SVM have certain limitations in perceiving global dependence. In the present scenario, most of the deep learning models rely on pre-processing, extracting features, and network topology but still are not sufficient to provide satisfactory accuracy for the small and noisy database. Overlapping in target classes and boundaries causes low performance of SVM no matter the dataset is highly dimensional with fewer samples. In this research study, the focus of the novel approach is on developing a classification strategy for working on more emotion types. A “Mean of Mean “algorithm is proposed to completely analyze mental state by considering all the features in the features set. Emotion is first classified into one of the quadrants from four quadrants of emotion by comparing with the referential mean and then depending on the intensity of arousal the emotion is further classified into 12 subtypes by using the MIN Max range. The proposed algorithm performed better compared to other algorithms and provide a wide range of emotion types. When compared to current research of multi-class emotion identification, the experimental findings demonstrated that the suggested technique is extremely competitive. The average accuracy rate was above 90%, and it provided a comprehensive assessment of the mental condition. On the emotional spectrum, a person's emotional state is assessed.

Mean of mean

referential mean

MIN MAX Range

To much extent, Emotion represents the mental state of the human being. It has real meaning for inner thoughts. If a human being could understand the emotion of another person, many complications can be sorted out. Understanding emotions can provide a prominent dimension to effective communication between people. Nowadays people are interested in understanding other emotions without really having language communication. It is not just because of interest but it is the need of current age for several reasons. In the hassle bustles of the urban lifestyle, individuals can’t connect in person frequently and express themselves. There are many reasons which are attracting the attention of researchers to intensively work on emotion detection areas. With increasing stress and work pressure, people are experiencing an unstable mental state which leads to the development of mental disorders and depression. It is the need of the present situation to perform regular mental state check-ups at the workplace. There is one more area where serious work on emotion recognition is needed which is for disabled people. These people are not able to express their emotions either through language or through facial expressions. Hence to provide a way of communication, therapies, and proper treatment to these people intense work on this topic is needed.

There are several techniques to capture physiological signals responsible for the emotional change. Emotional information captured by the brain-computer interface using brain encephalographic signals has true meaning. It's also more trustworthy and useful for persons who can't fully communicate themselves through facial expressions or other physiological cues. At present, the methods of emotion detection mainly use facial expressions, text, speech, and physiological signals. Facial expressions, texts, and voices are easily affected by people's subjective factors, while physiological signals are not easily controlled by subjective, and can reflect people's true emotional state more objectively.

Therefore, more and more scholars use physiological signals, especially EEG signals for emotion recognition. EEG is the method in which electrical activity on the scalp has been recorded. It is a non-invasive technique that uses electrodes placed along the scalp. It records voltage variations caused by ionic current within brain neurons as a result of a stimulus or action potential at the time. The voltage measured is in microvolts. In general, EEG refers to the recording of the brain's spontaneous electrical activity over some time, as recorded from multiple electrodes placed on the scalp. These electrodes are placed on the specific position of the brain and relative voltage readings are measured.

It is a multidisciplinary area where the major fields involved are medical signal processing, artificial intelligence and machine learning techniques, and brain-computer Interface. Because of its various uses in the medical profession, gaming business, marketing, E-learning, and psychological tests, emotion detection has gotten a lot of interest in the previous decade.

Earlier studies were either fixated on developing feature extraction techniques or testing different classification models for accurate emotion detection on existing databases like DEAP and SEED[1]. It is used to extract characteristics from a set of EEG channels that are the same for all subjects. DNN, CNN, and SVM models are widely used for classification and prediction. The emotional signals in these databases are recorded in a controlled setting over a set period of time. However, recording all of the signals in a predetermined time span is not possible. The functioning of the brain is complex and varies from person to person and from one emotional state to the next.The excitation time interval for particular emotion is different for each subject. Hence Real research work is expected on the emotional signals captured in the natural environment by considering major applications as a primary focus.

However, deep learning and SVM have certain limitations in perceiving universal dependence. The performance of these models depends on primary stages of emotion detection like pre-processing, extracting features, and network topology used for classification. But still, these models are not able to achieve a reasonable level of accuracy for the small and noisy database. Overlapping in target classes and boundaries causes low performance of SVM though the dataset is highly dimensional[2].

The focus of this research study is to develop a novel classification technique for any size of the database. These Emotional signals are captured by using the Epoc 14 channel EEG Headset. After applying a stimulus for invoking specific emotion, the time required for experiencing intense emotion is varied from person to person. By considering this fact that signals of variable length are recorded and features are extracted and the referential mean file is created by taking an average of each emotion type. The classification strategy depends on the minimum distance of the calculated mean of the test emotion signal with a referential mean. This can be implemented for all the features in the feature set. Manually voting is applied to classify emotion for mostly matched emotion types. In the following section methodology used in the proposed method is mentioned in detail.

Emotion recognition using brain EEG signals has gained extensive attention in the last decade. It is a multidisciplinary field that has a direct connection with psychology, brain study, and artificial intelligence. Past studies used primary stages of the recognition system and used different feature extraction and feature selection methods. Many researchers had been working on existing databases like SEED and DEAP. [3] proposes a deep generalised canonical correlation analysis with an attention mechanism (DGCCAAM). Emotion is divided into five categories using an emotion-related attention mechanism for feature fusion. The feature is extracted using a short-time Fourier transform (STFT). EEG data are analysed for differential entropy (DE) properties. In this research study, the SEED-V database is used. 82.11% accuracy is achieved with three modalities. In[4], the method is proposed to identify emotions by using six statistical parameters of EEG signals using the DEAP dataset. PCA and ReliefF algorithms are used for channel selection. 81.87%. accuracy is achieved by using SVM for classification. In[5] researcher used PCA to reduce the feature vector dimension on the SEED database. SVM is used for the classification of + ve, -ve, and neutral emotions. The accuracy was computed at 85.85%. In[6] multimodal approach for classification is used on the DEAP database. Data from 14 electrodes were used and 168 features of each subject were extracted. KNN, SVM, and decision tree classifiers are used for classification. KNN came up with the best outcome. Valence was assessed to be 77.54 percent, while dominance was 79 percent. Arousal was 78.88% using the SVM classifier for the same features. To boost accuracy, the multi-class classification approach is utilised in[7]. For feature selection, the researcher employed the particle swarm optimization (PSO) technique. The algorithm's performance is improved using the multi-stage linearly decreasing inertia weight (MLDW) approach. High arousal-high-valence, high arousal-low-valence, low arousal-high-valence, and low arousal-low-valence are the four emotional states classified by SVM. The calculated average accuracy was 76.67 percent. Sofen Gannouni[8] has presented a novel channel selection approach. It works by estimating signal epochs via zero-time windowing. The numerator group delay function is used to extract spectral information from an EEG signal with excellent temporal precision. The voting method was used to choose the channels. It locates epoch times and chooses the channel with the most votes. The DEAP database is used to test QDC and RNN. With 9 emotions, the calculated accuracy is 89 percent. The main motivation behind the mean of the mean proposed algorithm is that method should be effective for smaller databases and also it should be useful for classifying many emotions. The mean of the mean algorithm is specifically used for classification. It works effectively for one to several features. The first referential mean file is created for calculating distance and the hen MIN MAX range of main emotion classes is computed. Emotion is classified as angry, calm, happy, or sad by checking the Minimum distance with reference mean of four main emotions. It is further divided into subclasses by plotting it on the MIN Max Range of main emotion classes. This procedure was repeated for all the features in the feature set. If there are many extracted features, the voting technique is used to classify emotion finally. In the following section methodology of the proposed algorithm is explained in detail.

Primary steps in any detection and prediction system are data set acquisition, pre-processing, feature extraction, and classification. The Novel Approach is used at the classification stage. The main objective behind it is the strategy must be applied for the small size of the database as well as there should not be any confusion while selecting features for the feature set.

In the previous studies, different feature selection techniques are used to create a reduced feature set and test the accuracy of the system. It is observed that the computed accuracy varies each time for different feature combinations. Hence this novel method used the voting technique for labeling emotion. It computes emotion class for each feature and finally used the voting technique to label emotion. The emotion class which got maximum votes would be labeled by that emotion. In Table 1, the initial strategy of the proposed method is mentioned.

Table 1

INITIAL STRATEGY
Equipment used:	Emotive Epoc 14 channel headset is used.
Channels	AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, AF4
Sampling Frequency	128Hz
Dataset:	Dataset: [120[14(128recording time in seconds)] 120: Emotion signals 1 to 30 Angry, 31 to 60 Calm, 61 to 90 Happy, 91 to 120 Sad Dimension of each signal, [14] columns (channels) [128recording time in seconds] rows
The method used for emotion elicitation	Ground truth method
Stimulus used	Audio, Video, Thoughts
Pre-processing	Filtering, Independent component analysis
Feature Extraction	Statistical modeling
Features	“['Mean', 'STD', 'VAR', 'SKEW', 'Kurtosis', 'IEEG', 'MAV', 'MAV1', 'MAV2','SSI','VEEG','RMS','DASTD','AREG','HA','HM','HC','WL','POWER',' PERODOGRAM1','PERODOGRAM2','ENVOLEPE','PSD']”[2]

A. ANNOTATIONS AND TERMINOLOGY:

The key terms and annotations are introduced in this section. They are essential for comprehension. the proposed method. Figure 1 represents a mathematical model which needs to be understood first before going into detail about annotations and terminology.

Dependent Variable Set: The dependent Variable Set is the set of output variables. In this case, it is the Emotion Class.
Independent Variables Set is the set of input variables. In this case, it is the feature set.
Parameters: Referential mean file and Min-Max range.
Forcing Function: Uncontrolled Extraneous.

In Fig. 2, the terminology used in the mathematical model is mentioned. The details are mentioned below.

The Independent variable set is denoted by EQ. It is the set of Emotional States.
EQ1: Angry state, EQ2: Calm State, EQ3: Happy State, EQ4: Sad State

$$EQ=\left\{ EQ1,EQ2,EQ3,EQ4\right\}$$

$EQ1=\{AQ1, AQ2, AQ3\}$ = {Nervous, Angry, Annoying},
$EQ2=\{CQ1, CQ2,CQ3\}$ = { Relaxed, Peaceful, Calm},
$EQ3=\{HQ1, HQ2,HQ3\}$ = {Pleased, Happy, Excited}
$EQ4=\{SQ1, SQ2,SQ3\}$ = {Sleepy, Bored, Sad}}.
EQ= { {Nervous, Angry, Annoying}, { Relaxed, Peaceful, Calm}, { Pleased, Happy, Excited}, { Sleepy, Bored, Sad}}.

$$EQ=\{\left\{AQ1, AQ2, AQ3\right\}, \left\{CQ1, CQ2,CQ3\right\}, \left\{HQ1, HQ2,HQ3\right\}, \left\{SQ1, SQ2,SQ3\right\}\}$$

EL1 is the set of electrodes.
EL1 = {AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, AF4}
The dependent variable set is denoted by FS1. It is the feature set.
RF_Meanofmean is a referential mean file. It is used as a Parameter for the classification of emotion in 4 main types Angry, calm, happy, and sad.

Angry Mean = {A_{m1………………………}..A_24}

Calm Mean = {C_{m1…………………………}C₂₄}

Happy Mean = {H_{m1………………………}.H_m24}

Sad Mean = {S_{m1……………………………}S_m24}

Min-MaxRange_file is a range of Angry, Calm, Happy, and Sad emotions. It is used as a parameter for the classification of emotion in 12 subtypes.
Forcing Function:

"In this scenario, participants were experiencing a quick shift in sentiment or difficulties in categorising the emotion during the Signal Collection stage. It is very difficult to get the signal at an intense emotional arousal state. Un- Even the shape of the skull of subjects is also responsible for variations in recordings.”[2]

B. REFERENTIAL MEAN FILE:

Step1: 24 features are extracted from 120 emotional signals. A feature set file of [120*24] dimension is created.

Step2: Mean of Angry, Calm, Happy, and Sad is computed for each feature.

Angry Mean = {A_{m1………………………}..A_24}
Calm Mean = {C_{m1…………………………}C₂₄}
Happy Mean = {H_{m1………………………}.H_m24}
Sad Mean = {S_{m1……………………………}S_m24}

A mean of a mean file of [4*24] dimension is created. Where each row represents the mean of the mean of Angry, Calm, Happy, and Sad Emotions for each feature. Table 2 represents the mean of the mean file of 24 features of Angry, Calm, Happy, and Sad emotions.

Table 2

MEANOFMEANFOR24FEATURE
	Angry	Calm	Happy	Sad
Mean	-5.89E-19	-4.45E-19	5.31E-19	-4.07E-19
STDDEv	0.163943	0.166845	0.146946	0.143131
VAR	0.030517	0.030052	0.024944	0.024471
Skew	0.259426	0.702313	0.569911	1.23472
Kurtosis	24.89917	10.22524	30.1283	31.99606
IEEG	836.3248	842.7331	575.7628	687.0578
MAV	1.29E-05	1.95E-05	1.73E-05	1.23E-05
MAV1	0.058442	0.062112	0.044852	0.049922
MAV2	1638.734	1629.98	1101.094	1362.672
SSI	247.0576	212.9055	157.6071	182.2315
VEEG	0.030517	0.030052	0.024944	0.024471
RMS	0.163933	0.16683	0.146933	0.143122
DASTD	0.00109	0.001248	0.000995	0.000488
AREG_PXX	0.837401	0.631482	0.642969	0.887735
HA	0.030517	0.030052	0.024944	0.024471
HM	0.132829	0.127513	0.130701	0.129529
HC	1.281373	1.321213	1.307146	1.288851
WL	0.056774	0.061703	0.039255	0.014373
Bandpower	0.030513	0.030046	0.02494	0.024468
Peridogram1	0.001396	0.001511	0.001078	0.000898
Peridogram2	150.0333	129.9333	125.3	142.6333
Envlope	-15.2272	-12.0899	-13.7224	-17.6222
Hilbert	1.31E-07	1.17E-06	3.24E-07	9.54E-07
PSD	3.05E-05	3.15E-05	1.95E-05	1.84E-05

C. MIN MAX RANGE FILE

Figure 3 represents the emotion classification wheel. It shows emotion classification in different emotion types depending on the intensity of the emotion. High to Low – Positive to negative for angry, calm, happy, and sad emotions. This wheel is used to label emotion subtypes by using MIN MAX Range.

MIN to MAX[Angry],MIN to MAX [Calm],MIN to MAX[Happy]and MIN to MAX [SAD]

In Table 3 MIN MAX range of each emotion class is mentioned. The calculated feature value of the test signal is matched with this range and emotion is further classified into subtypes

Table 3

MIN MEAN MAX RANGE OF EMOTION
Emotion	Angry			Calm			Happy			Sad
Sub Emotion	Nervous	Angry	Annoying	Relaxed	Peaceful	Calm	Pleased	Happy	Excited	Sleepy	Bored	Sad
Range	Min	Mean	Max	Min	Mean	Max	Min	Mean	Max	Min	Mean	Max
Mean	-1.18E-17	-5.89E-19	2.99E-18	-1.47E-17	-4.45E-19	9.23E-18	-8.31E-18	5.31E-19	1.45E-17	-5.06E-18	-4.07E-19	5.00E-18
STDDEv	6.66E-02	1.64E-01	3.02E-01	1.05E-01	1.67E-01	3.01E-01	5.18E-02	1.47E-01	2.66E-01	4.56E-02	1.43E-01	3.19E-01
VAR	4.43E-03	3.05E-02	9.14E-02	1.10E-02	3.01E-02	9.03E-02	2.68E-03	2.49E-02	7.10E-02	2.08E-03	2.45E-02	1.02E-01
Skew	-5.34E + 00	2.59E-01	4.42E + 00	-3.80E-01	7.02E-01	2.70E + 00	-1.57E + 00	5.70E-01	4.43E + 00	-3.12E + 00	1.23E + 00	1.11E + 01
Kurtosis	3.16E + 00	2.49E + 01	1.07E + 02	4.00E + 00	1.02E + 01	2.81E + 01	4.01E + 00	3.01E + 01	1.70E + 02	3.74E + 00	3.20E + 01	2.18E + 02
IEEG	2.39E + 02	8.36E + 02	1.88E + 03	2.21E + 02	8.43E + 02	2.09E + 03	1.62E + 02	5.76E + 02	1.59E + 03	1.84E + 02	6.87E + 02	1.58E + 03
MAV	3.25E-06	1.29E-05	3.04E-05	4.24E-06	1.95E-05	1.14E-04	8.82E-07	1.73E-05	5.71E-05	1.93E-06	1.23E-05	5.66E-05
MAV1	3.31E-05	5.84E-02	2.69E-01	1.11E-03	6.21E-02	3.59E-01	1.19E-03	4.49E-02	1.99E-01	3.31E-05	4.99E-02	2.63E-01
MAV2	3.46E + 02	1.64E + 03	3.93E + 03	4.75E + 02	1.63E + 03	3.85E + 03	2.53E + 02	1.10E + 03	2.83E + 03	2.53E + 02	1.36E + 03	3.15E + 03
SSI	3.49E + 01	2.47E + 02	7.19E + 02	4.67E + 01	2.13E + 02	5.71E + 02	3.03E + 01	1.58E + 02	5.71E + 02	1.81E + 01	1.82E + 02	5.35E + 02
VEEG	4.43E-03	3.05E-02	9.14E-02	1.10E-02	3.01E-02	9.03E-02	2.68E-03	2.49E-02	7.10E-02	2.08E-03	2.45E-02	1.02E-01
RMS	6.66E-02	1.64E-01	3.02E-01	1.05E-01	1.67E-01	3.01E-01	5.18E-02	1.47E-01	2.66E-01	4.56E-02	1.43E-01	3.19E-01
DASTD	3.23E-04	1.09E-03	2.19E-03	5.77E-04	1.25E-03	2.35E-03	1.43E-04	9.95E-04	1.85E-03	1.84E-04	4.88E-04	9.31E-04
AREG_PXX	7.43E-02	8.37E-01	2.29E + 00	8.51E-02	6.31E-01	2.36E + 00	3.03E-02	6.43E-01	4.13E + 00	9.91E-03	8.88E-01	7.96E + 00
HA	4.43E-03	3.05E-02	9.14E-02	1.10E-02	3.01E-02	9.03E-02	2.68E-03	2.49E-02	7.10E-02	2.08E-03	2.45E-02	1.02E-01
HM	1.12E-01	1.33E-01	1.51E-01	9.50E-02	1.28E-01	1.46E-01	1.07E-01	1.31E-01	1.51E-01	1.10E-01	1.30E-01	1.54E-01
HC	1.19E + 00	1.28E + 00	1.39E + 00	1.20E + 00	1.32E + 00	1.57E + 00	1.21E + 00	1.31E + 00	1.50E + 00	1.17E + 00	1.29E + 00	1.38E + 00
WL	1.01E-02	5.68E-02	1.46E-01	1.88E-02	6.17E-02	1.23E-01	9.27E-03	3.93E-02	7.02E-02	8.63E-03	1.44E-02	1.96E-02
Bandpower	4.43E-03	3.05E-02	9.14E-02	1.10E-02	3.00E-02	9.03E-02	2.68E-03	2.49E-02	7.10E-02	2.08E-03	2.45E-02	1.02E-01
Peridogram1	1.23E-04	1.40E-03	7.61E-03	2.03E-04	1.51E-03	1.16E-02	2.36E-05	1.08E-03	3.09E-03	2.90E-05	8.98E-04	4.79E-03
Peridogram2	8.80E + 01	1.50E + 02	2.64E + 02	2.50E + 01	1.30E + 02	3.80E + 02	3.70E + 01	1.25E + 02	4.61E + 02	3.00E + 01	1.43E + 02	3.24E + 02
Envlope	-4.26E + 01	-1.52E + 01	3.04E + 00	-3.50E + 01	-1.21E + 01	9.95E-01	-5.72E + 01	-1.37E + 01	-1.93E + 00	-4.64E + 01	-1.76E + 01	-3.25E + 00
Hilbert	-1.09E-14	1.31E-07	2.47E-06	-5.37E-15	1.17E-06	2.01E-05	-3.06E-15	3.24E-07	4.41E-06	-8.73E-15	9.54E-07	9.04E-06
PSD	2.84E-07	3.05E-05	2.92E-04	8.38E-07	3.15E-05	3.94E-04	1.07E-08	1.95E-05	8.46E-05	7.01E-09	1.84E-05	2.31E-04

D. ALGORITHM

- Classification of emotion in 4 main classes [Angry, Calm, Happy, Sad]

1. Create a Referential mean file of 4*24 dimension.

2. Classification of emotion in 4 classes

a. Test signal is given as an input.

b. Calculate 24 features of the test signal

c. Match value of the first feature of the Test signal with Angry_mean, Calm_mean, Happy_mean, and

Sad_mean, Calculate the distance between Test_mean and referential mean

d. Find out the minimum distance

T_{meanofmean1 =} Min( Test_mean – Angry_mean, Test_mean – Calm_mean, Test_mean – Happy_mean, Test_{mean -}Sad_mean)

If T_meanofmean1 is near to Angry_mean, it will be denoted as Angry emotion

If T_meanofmean1 is near to Calm_mean, it will be denoted as Calm emotion

If T_meanofmean1 is near to Happy_mean, it will be denoted as Happy emotion

If T_meanofmean1 is near to Sad_mean, it will be denoted as Sad emotion

e. Label the output emotion of the test signal for the minimum distance emotion class.

f. Repeat the above step for all 24 features and calculate the mean of the mean for all the features.

[T_meanofmean1, T_{meanofSTDEv2,} T_meanofVAR3, T_meanofSKEW4, T_{meanofKurtosis5,} T_meanofIEEG6, T_meanofMAV7, T_meanofMAV18, T_meanofMAV29, T_meanofSSI10, T_{meanofVEEG11,} T_meanofRMS12, T_{meanofDASD13,} T_{meanofAREG_PXX14,} T_meanofHA15, T_meanofHM16, T_meanofHC17, T_meanofWL18, T_{meanofBANPOWER19,} T_{meanofPERODOGRAM20}, T_{meanofPERODOGRAM21}, T_{meanofENVOLEPE22,} T_{meanofHilbert23,} T_meanofPSD24]

3. Emotion type is labeled for each feature. A set of 24 emotions of 24 features is being computed as the result of the above steps.

4. Voting is applied to classify emotion for mostly matched emotion types.

Finally, one emotion is detected for the given test signal as a result of the above classification strategy.

Classification of emotion in 12 emotion classes

{{Nervous, Angry, Annoying}, {Relaxed, Peaceful, Calm}, { Pleased, Happy, Excited}, { Sleepy, Bored, Sad}}.

Find out the range of each emotion class. Plot calculated value of feature on the MIN Max Range of main emotion classes. This procedure was repeated for all the features in the feature set. In Fig. 5 MIN MAX Range of Mean Feature is shown.
Find out the complete mental state of the test emotion signal based on the votes achieved by each emotion class. In Table 4, the complete mental state analysis of the Sample input signal is shown.

In Table 5 Average accuracy for each feature using the mean of the mean algorithm is mentioned. It is calculated by dividing the total number of correct predictions by the total number of samples in each emotion class. Emotion class had been labeled for the given input signal for each feature. The emotion class which is voted most by features is decided as the final label. In the below table, the accuracy of prediction for angry, calm, happy, and sad emotion class for each feature is calculated. It is observed that

Table 5

ACCURACY OF 24 FEATURES IN PERCENTAGE
Accuracy in Percentage
Feature	Anger	Calm	Happy	Sad
Mean	43.33333	0	56.66667	10
STDDEv	10	43.33333	0	53.33333
VAR	43.33333	10	10	63.33333
Skew	56.66667	20	20	50
Kurtosis	10	90	0	20
IEEG	13.33333	46.66667	63.33333	20
MAV	13.33333	33.33333	0	63.33333
MAV1	0	40	70	3.333333
MAV2	43.33333	10	66.66667	16.66667
SSI	43.33333	10	63.33333	6.666667
VEEG	43.33333	10	10	63.33333
RMS	10	43.33333	0	53.33333
DASTD	13.33333	60	30	90
AREG_PXX	0	100	0	0
HA	43.33333	10	10	63.33333
HM	46.66667	43.33333	3.333333	6.666667
HC	43.33333	40	0	6.666667
WL	13.33333	56.66667	33.33333	100
Bandpower	43.33333	10	10	63.33333
Peridogram1	6.666667	30	3.333333	66.66667
Peridogram2	53.33333	6.666667	63.33333	6.666667
Envelope	3.333333	60	6.666667	46.66667
Hilbert	90	16.66667	3.333333	3.333333

In Table 6, prominent features for each emotion class are mentioned. The accuracy of classification is more than 50% by using these features.

Table 6

PROMINENT FEATURES OF EMOTION CLASS
Emotion	Prominent Feature
Anger	Peridogram2, Skew, Hilbert
Calm	Kurtosis, DASTD, AREG_PXX, WL, Envolope
Happy	Mean, IEEG,MAV1,MAV2,SSI, Peridogram2,
Sad	STDEv, VAR, MAV, VEEG, RMS, DASTD, HA, WL, BANDPOWER, Peridogram2,

By using the proposed approach emotion can be classified into 12 emotion classes and also present mental state of the person can be analyzed. Hence in this research study, a completely new approach is proposed to classify emotion into 12 subtle types. It also provides a complete analysis of the mental state. With this algorithm emotion detection accuracy is being calculated for each feature in the feature set. Emotion can be classified into four main classes Anger, Calm, Happy and Sad by calculating the distance of mean, and emotion is further classified into subtypes by calculating the range of Minima, and Maxima. It works incredibly well for small datasets also. The proposed model is tested with the existing models and the results are presented in Table 7.

Table 7

COMPARISON BETWEEN THE PROPOSED METHOD AND RELATED WORKS.
Author	Number of Emotion Types	Feature extraction algorithm	Number of electrodes	Classifiers	AVG Accuracy (%)
Ackermann, P., Kohlschein[9]	3 emotions (anger, surprise, other)	Minimum redundancy maximum relevance (mRMR)	32	SVM-Random Forest	60
Pane, E. S., Hendrawan[10]	4 emotions (happy, sad, angry, and relaxed)	Time and frequency domain features	5	Decision tree algorithm	81.64
Ahirwal, M. K. & Kose,[11]	4 emotions (angry, sad, happy, and relaxed	Time-domain features, frequency domain features, and entropy	32	ANN	93.75
Ren, F., Dong[12]	4 emotions	The probability distribution for wavelet packet coefficients	3	SVM	70.5
Ranganathan, ., Chakraborty[13]	9 emotions	Spectral features	32	DBN	79.2
Liu, Y. & Fu[4]	9 emotions	The fusing of 6 statistical features	32	SVM	81.87
Sofen Gannouni[8]	9 emotions	ZTWBES(zerotime windowing – based epoch estimation and relevant electrode identification	Adaptive	RNN	89.33
Proposed method	12 types of emotions	Time and frequency domain features	14	The Mean of Mean Algorithm	90%

This research study presented a new approach to emotion recognition using EEG signals. The suggested method compares the distance between the referential mean and the calculated mean using the Mean of Mean algorithm. Our findings revealed that the proposed strategy was a viable option in this sector, outperforming earlier research. During the categorization of Angry, Calm, Happy, and Sad emotions, the Mean of Mean algorithm achieved an average accuracy of 90%, 100%, 70%, and 100%, respectively. Emotion identification using EEG waves necessitates a combination of expertise from neuroscience, engineering, computer science, and psychology. However, existing emotion identification algorithms require more research in neuroscience and psychology, or a multi-modal approach that integrates EEG-based emotion recognition models with image processing-based approaches.

Those who are unable to communicate their feelings and those who operate under stress and pressure might benefit from this activity. "Getting reliable data for emotion categorization is a big difficulty in this project." Labeling emotions is an important undertaking since people's moods are so volatile. To solve this, the number of signal collecting trials for each emotion class might be increased. The size of the database should be enormous to improve the accuracy of classifiers." [2]. "By using IoT-based technologies to construct wearable devices, this research study may be further extended." The person's unstable emotional condition can be communicated to caregivers, and early intervention can be done. More work on delicate emotions such as pride, jealousy, and anger.

Funding: This research received no external funding.

Institutional Review Board Statement: This research study was conducted according to the guidelines of the Declaration of Helsinki. This research study is not at all being conducted for diagnosis or treatment of disease in any patients. This research study is purely used for research application and personal use.

Data Availability Statement: In this research study own database is used which is collected in the database collection phase.

Conflicts of Interest: The authors declare no conflicts of interest.

Kimmatkar, N., Vishnupant, & Vijaya, B. (2018). Babu. "Human emotion classification from brain EEG signal using the multimodal approach of the classifier." Proceedings of the International Conference on Intelligent Information Technology. 2018
Kimmatkar, N., Vishnupant, & Vijaya Babu, B. (2021). "Novel Approach for Emotion Detection and Stabilizing Mental State by Using Machine Learning Techniques. " Computers, 10(3), 37
Liu, W., et al. (2019). "Multimodal emotion recognition using deep canonical correlation analysis." arXiv preprint arXiv:1908.05349
Liu, Y., & Fu, G. (2021). Emotion recognition by deeply learned multi-channel textual and EEG features. Future Generation Computer Systems, 119, 1–6
Rahman, M., Asadur, et al. (2020). "Employing PCA and t-statistical approach for feature extraction and classification of emotion from multichannel EEG signal.". Egyptian Informatics Journal, 21(1), 23–35
Nawaz, R., et al. (2020). Comparison of different feature extraction methods for EEG-based emotion recognition. Biocybernetics and Biomedical Engineering, 40(3), 910–926
Li, Z., et al. (2020). Enhancing BCI-based emotion recognition using an improved particle swarm optimization for feature selection. Sensors (Basel, Switzerland), 20(11), 3028
Gannouni, S., et al. (2021). Emotion detection using electroencephalography signals and a zero-time windowing-based epoch estimation and relevant electrode identification. Scientific Reports, 11(1), 1–17
Ackermann, P., et al. (2016). "EEG-based automatic emotion recognition: Feature extraction, selection and classification methods." 2016 IEEE 18th international conference on e-health networking, applications, and services (Healthcom). IEEE,
Pane, E., Septiana, Engineering, B., et al. (2017). (ICICI-BME). IEEE,
Ahirwal, M., Kumar, and, Mangesh Ramaji Kose. "Emotion recognition system based on EEG signal: A comparative study of different features and classifiers." 2018 Second International Conference on Computing, Methodologies, & Communication (2018). (ICCMC). IEEE,
Ren, F., Dong, Y., & Wang, W. (2019). Emotion recognition based on physiological signals using brain asymmetry index and echo state network. Neural Computing and Applications, 31(9), 4491–4501
Ranganathan, H., Chakraborty, S., & Panchanathan, S. (2016). "Multimodal emotion recognition using deep learning architectures." 2016 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE,
Nisha Vishnupant Kimmatkar and Vijay Babu (2022). “The Study of Emotional Brain to Detect Emotions Using Brain EEG Signals and Improving Accuracy of Emotion Detection System Using Feature Selection Techniques."2022 5th International Conference on Machine Vision and Applications ICMVA 2022. Association for Computing Machinery, New York, NY, USA, 22–33. https://doi.org/10.1145/3523111.3523115

Table 4 is available in the Supplementary Files section.

No competing interests reported.

Table4.jpg

Download PDF

Version 1

posted

You are reading this latest preprint version

A Novel Approach to the Classification of Human Emotion and Analysis of Complete Mental State Using Brain EEG Signals

Status:

Version 1

Abstract

Figures

I. Introduction

Ii. Related Work

Iii. Methodology

Iv. Performance Analysis Of The Proposed Method

Declarations

References

Table 4

Additional Declarations

Supplementary Files

Status:

Version 1