4.1 Data collection
There are a total of 213 photos in the Japanese Female Facial Expression (JAFFE) collection, depicting 7 different emotions (6 different expressions + 1 neutral expression). The models for these photographs are 10 different women from Japan. Sixty Japanese individuals rate each picture on six different emotional scales. Three people, Michael Lyons, Miyuki Kamachi, and Jiro Gyoba, are responsible for the creation and compilation of this database. Several scientific studies have made use of JAFFE pictures. Grayscale photos at a resolution of 256x256 are provided in uncompressed.tiff format.
The main dataset contains 213 images so in this paper for increasing accuracy and preventing possible overfitting we used data augmentation techniques with some filters, histogram, and adding noise to the image to increase dataset size. Therefore, each mental state includes 300 images. Data set volume augmented to 2100 images.
4.2 Feature extraction using the fractal model
An example of a feature extraction diagram for one of the photos is shown in Figure 4. Figure 4's blue histogram represents the frequency distribution of picture pixels. Finding the optimal normal functions for modeling the histogram is the focus of the fractal approach. From this, we may estimate the image's histogram with a sum of three red functions, each of which is represented by a green Gaussian function. Coefficients of standard green functions are equivalent to those of the resulting images in terms of their attributes. In light of this, we have catalogued four characteristics for every picture. If we can use these four operations to create the primary picture, it will look like.
4.3 Implementation of classification using SVM kernels
In this subsection, facial expressions are identified using the support vector machine (SVM) technique. In this case, the JAFFE dataset was mined for information. Three hundred face photographs were assessed for each scenario, for a total of 2100 images analyzed (75% for model training and 25% for testing and verification). Each of the seven emotions (anger, disgust, fear, happiness, neutral, sadness, and surprise) were represented by one of seven possible labels (1–7), with each image having four features as independent variables.
Different kernels were also employed to training the model and the outcomes have been compared. The following are the outcomes of the classification using a polynomial kernel of degree 5. Figure 6 illustrates the confusion matrix in training data. In this picture, the diagonal diameters show the number of accurate classifications by the trained network. Table 2 provides the results of the performance assessment criteria for the suggested technique.
Table 2. Results of performance analysis
|
Polynomial
|
RBF
|
Linear
|
|
Train
|
Test
|
Train
|
Test
|
Train
|
Test
|
Accuracy
|
69.20
|
68. 63
|
89.83
|
94.29
|
56.34
|
82.34
|
Specificity
|
83.03
|
79.38
|
89.22
|
86.51
|
49.57
|
68.58
|
Sensitivity
|
13.12
|
18.54
|
67.83
|
71.93
|
23.69
|
22.44
|
F1 score
|
15.63
|
9.55
|
62.22
|
58.89
|
22.19
|
10.12
|
Gmeans
|
23.85
|
26.37
|
73.63
|
69.12
|
29.27
|
27.49
|
Now we'll evaluate the SVM method's various kernels side by side. Table 2 summarizes the outcomes of using polynomial kernels, radial basis functions (RBFs), and linear kernels for this purpose. The training data comparison table shows that the polynomial approach with order 5 meets the most requirements. The findings show that the RBF model's SVM accuracy is 93.18 percent for the training data and 89.33 percent for the testing data. The RBF kernel has stricter performance requirements than competing techniques. It has a 96% sensitivity for diagnosing a mental state, but only an 86.67% sensitivity for a Linear kernel and an 86.03% sensitivity for a polynomial kernel.
4.4 Implementation of classification using DCNN
We'll use a deep convolution neural network to analyze facial expressions and label them accordingly. The approach presented for the deep convolutional network algorithm as according Figure 8 consists of five layers, which are: Human face photos from the JAFFE collection, which comprises 2100 images in 7 Face mode (6 face modes + 1 neutral) from 10 Japanese female models. An picture of size 256x256 with 7 face modes is used to create a 4D matrix of size 2100 by 2100 in this layer.
Table 3
The presented DCNN architecture layers
Layer
|
Name
|
Description
|
1
|
Image Input
|
256x256x1 images with 'zero center' normalization
|
2
|
Convolution
|
8(3x3) convolutions with stride [1 1] and padding 'same'
|
3
|
Batch Normalization
|
Normalization
|
4
|
ReLU
|
Rectifier
|
5
|
Pooling
|
2x2 max pooling with stride [2 2] and padding [0 0 0 0]
|
6
|
Convolution
|
16(3x3) convolutions with stride [1 1] and padding 'same'
|
7
|
Batch Normalization
|
Normalization
|
8
|
ReLU
|
Rectifier
|
9
|
Pooling
|
2x2 max pooling with stride [2 2] and padding [0 0 0 0]
|
10
|
Convolution
|
32(3x3) convolutions with stride [1 1] and padding 'same'
|
11
|
Batch Normalization
|
Normalization
|
12
|
ReLU
|
Rectifier
|
13
|
Fully Connected
|
7 fully connected layer
|
14
|
SoftMax
|
SoftMax
|
15
|
Classification Output
|
Cross entropy
|
Convolution occurs in the second layer, and it employs 8(3 3) filters to iteratively process the input matrix. In this research, the convolution matrix is constructed by adding a border of zeros to the input matrices. The array is analyzed in one pass by the filter as well. Matrix transformation is used in the third layer of normalization. The ReLU function, often seen in convolution networks, is used in the activation function layer, the fourth layer of a convolution network. After each layer of convolution, this function clears the matrix of any negative values and passes on its positive values to the subsequent layer. Pooling occurs on the sixth layer. This layer is a sampling technique for a convolutional neural network that functions on the pixel space and is capable of maximizing within the filter range. These subsequent levels are organized into three distinct classes. Figure 8 and Table 3 show that a completely linked layer does not appear until the thirteenth iteration. When this layer is applied on top of a matrix, the result is a vector that may be evaluated by the output layer. If you want to be technical, it creates 7 submatrixes (number of categories). The SoftMax layer, located at position 14, is the last output layer of a multi-layer classification neural network. The SoftMax function takes in values from a number of different rankings and outputs a single value between zero and one that is the total of all the ranks. To quantify the degree of inaccuracy associated with a given value of the dependent variable, the classification layer is used as the study's output layer. This layer influences when we come closer to the most accurate result. The MSE criteria, or average squared error, is utilized here. The repeat loop runs until it obtains the desired outcome. Finally, the output is identified by 7 labels with the names of the face expressions that decide the result.
The RMSE error generated by the DCNN software is shown in Figure 9. There were two epochs, each with 150 periods, and 300 iterations in all. On the test set, which comprises 30% of the whole data set, the ultimate accuracy of this study is 0.99. In addition, the final iteration has nearly little data loss.
In this study, we evaluate the computational endpoint based on the convergence slope, which must be horizontal (zero) in Figure 9's errors and losses perspective. The training data confusion matrix is shown in Figure 10. The figure's diagonal diameters represent the percentage of instances when the trained network correctly classified the data. Example: all 300 photos were accurately labeled as depicting rage. All the photos have been properly detected and trained, i.e. with a hundred percent accuracy and sensitivity, thanks to the confusion matrix of training data. Nonetheless, the test sample confusion matrix is shown in Figure 11. Diagnostic accuracy is close to perfect, with the lowest rate being for depression. Only one out of ninety randomly chosen photographs is incorrectly categorized as cheerful. Consequently, we can say with 99.8 percent certainty that the tests have been accurate.