Different datasets like Karolinska Directed Emotional Faces (KDEF) datasets and Kaggle's Facial Expression Recognition Challenge [8][9], FER-2013[10][16][17][18], Japanese Female Facial Expressions (JAFFE)[11], Cohn Kanade[2][12] are frequently used. Some researchers have also used datasets like Caltech faces, CMU database and NIST database[13], Taiwanese Facial Expression Image Database(TFEID)[14], KinFaceW-I and II [15], Yale[2], The MUG Facial Expression Database[19], MultiPie[20], MMI[21], GEMEP FERA [22], SFEW[23], AffectNet [24] and RAFD-DB[25] in their research on facial expression recognition. Some investigators have created their datasets for the experiment.
Revina et al.[2] listed down different types of facial expression recognition techniques with major contributions and stated that the ROI method of segmentation is best suited for pre-processing and gives 99% accuracy and the SVM classifier gives the highest accurate result. Nithya Roopa [8] used an inception net to identify facial expressions and establish 35.6% of accuracy and suggested that the same method can be applicable to real-time expression recognition.
Umarani & Srilakshmi [3] give a brief introduction to different stages, techniques, and datasets used in the Facial expression recognition system. It is mentioned that expression recognition is useful in criminal investigation, teaching and learning, healthcare, and Human-robot interface. It is also found that images can be sliced into two regions namely face region and non-face region.
Kanade et al.[4] established an Automatic Face Analysis model using permanent and transient facial features. Different facial action units like the inner, outer, and middle portions of the eyebrows, and lower and upper portions of eyelids, cheeks, and lips are used in expression recognition and they achieved 96.7% of accuracy. Research suggested that a combination of the feature-based method and template-based method will give a more accurate result.
Paweł et al.[5] proposed a method to calculate features for a 3D model using a KNN classifier and MLP neural network. The proposed method gets tested in two ways namely subject-dependent and subject-independent and it is found that the subject-independent way of testing is more versatile.
Z. Zhang and M. Li [11] Propose a cross-connected AlexNet enhanced convolutional neural network model in which the researcher added convolution and pooling layer in the original AlexNet. Mehendale, N[13] developed the FER model and uses Expression Vector in the determination of facial expressions. It is also said that background removal helps in the identification of accurate expressions. An expression matrix is used in the proposed method to distinguish five types of expression. They suggested that the proposed technique can be beneficial in applications like a lie detectors, and mood detectors.
Yue et al.[16] Designed a modular multi-channel deep convolutional neural network using the global average layer. The proposed system consists of four phases namely, Image Gaining, Face Locating, Face Recognition, and Expression identification. The proposed method uses three different channels and improves the accuracy of ExpressionNet. There is a scope for improvement in network enactment and in reducing the size of the model.
Christou & Kanojiya's [17] research aims to increase the accuracy of a FER2013 dataset. They have classified input images into seven basic expressions by applying a convolutional neural network and confirmed the success of improving the accuracy of biometric applications. The overall accuracy of their proposed method is 91.12% and, 45% and 41% recognition rate for the classification of disgust and fear.
Minaee et al.[37] proposed an algorithm built on an attentional convolution network in which the author gives special attention to remarkable areas of the face like eyes, and mouth and found 70.02% accuracy for FER2013, 99.3% accuracy for FERG, 92.8% for JAFFE and 98.0% accuracy for CK+ dataset. Goren & Wilson's[38]study shows that one can identify expressions using the geometric shape of facial parts like mouth, eye, eyebrows, nose, etc. They suggested that one can improve the perception of facial expression by focusing on the collaboration between configuration and exterior appearance which helps to decide whether exterior perception is degraded due to configurable mechanism breakdown.
Samadiani et al.[39] stated that Illumination variation, subject dependence, and head pose are three main challenges in facial expression recognition (FER) and proposed a multimodal sensor framework for the automatic detection of FER. Song Z[40] proposes a novel idea of a fusion of dual-channel i.e. ROI area division and Gabor feature to recognize facial expressions based on logical thinking and machine learning.
Lekdioui et al.[41]Proposed a new facial decomposition method using regions of interest (ROI) to detect basic emotion state recognition. The second step is to resize and partition into blocks before feature extraction. Lastly, a multiclass SVM classifier is used to infer emotions.
Kulkarni & Bagal[42] Perform a comparative study of automatic Facial Expression Recognition by compensating for the influence of age on the recognition procedure. The proposed study uses the Log Gabor filter and Gabor filter for feature mining followed by the implementation of an SVM classifier for training and classification purposes. Matre & Shah [43] Gives a brief of different techniques used in facial expression detection with their advantages and disadvantages. The author mentioned that, we can do facial expression recognition of black and white images as well as RGB images and conclude that the tensor perceptual color framework has the highest recognition rate and highest performance.
Ahmed et al.[44] Propose a convolution neural network with data augmentation. Debnath et al. [45] Offer a convNet CNN classification model using feature extraction techniques like region-based oriented FAST, local binary pattern (LBP) and rotated BRIEF (BRB) techniques. Adholiya [46]Developed a deep convolution neural network model using tf.Keras. Yao and Qiu[47] Propose hybrid features that combine convolutional neural networks and dense SIFT features for low-pixel facial images. This model uses local features like eyes, eyebrows, and mouth to detect facial expressions. Zhang et al.[18] proposes a method built on a convolution neural network as an image edge identification to detect facial expressions.
From the literature, it is found that the retrieval of facial features, like geometry and texture features, is done using a variety of traditional approaches. Appearance-based feature extraction and feature-based feature extraction are the two categories of feature extraction methodologies. Local Binary Patterns (LBP)[26], Local Directional Patterns (LDA)[27], and Fisher Face Method are, consider feature-based extraction methods. Principle Component Analysis (PCA)[3], Gabor wavelet[28], Line Edge Map (LEM)[29], and Facial Action Units (FAC)[30] are vision-based feature extraction methods. On the other hand, different feature extraction methods are developed and used by researchers in a facial expression such as Angle and Distance method[31], DSAE[32], Wavelet Entropy[33], PCA + Fisher Face + HOG[34], Facial landmarks point[35], Face Alignment with Regression Tree[36] which produce different results.
According to the aforementioned literature evaluation, there is still scope for advancement in the field of facial expression recognition. As an enhancement, the use of IoT and Human-Robot Interface (HRI) is possible to detect real-time expressions. The research focus is required on the recognition of facial landmarks. Design of Automatic face alignment component in occlusion and dark light is possible. At the time of facial expression recognition, consideration of multiple forms like age, gender, and personality attributes of the subject may improve accuracy. Genetic Property Evolution Framework designing is beneficial and may help in criminal detection, confidentiality breaches, etc. Subject dependence is a complicated issue addressed by researchers that involves a considerable range of data to acquire expressions of a subject at a particular age, but the classifier has a tendency to overfit and needs more attention in the future. Large Pose Variation, by extensive research, can be managed by 3-D models, and acquiring polarization-state imagery of faces is an efficient way of harnessing more textual and geometric data. There is a need of designing a reliable standard FER system by adding a number of layers along with a different number of filters at each in the CNN model.
In the proposed research, we have created our dataset named SMM Facial expression dataset which has captured the emotions of adults of age 18-22 only using a questionnaire. Then we instigate the CNN model on the SMM Facial expression dataset and show that we can get good accuracy in facial expression detection.