Surgical Masks Impair People's Ability To Accurately Classify Emotional Expressions, Except For Anger

Recently, the need to continuously wear surgical masks in everyday life has drawn the attention of neuroscientists and psychologists to the negative effects of face covering on social processing. A very recent but not very homogeneous literature has highlighted large costs in the ability to recognize emotions. Here it was investigated how surgical masks covering impaired the recognition of facial mimicry in a large group of 220 undergraduate Italian students. Sex differences in emotion recognition were also observed in 2 subgroups of 94 age-matched participants. Subjects were presented with 112 pictures displaying the faces of 8 actors (4 women and 4 men) wearing or not wearing real facemasks, and expressing 7 emotional states (neutrality, surprise, happiness, sadness, disgust, anger and fear). The task consisted in categorizing the emotion while indicating the recognizability degree with a 3-point Likert scale. Scores underwent repeated measures ANOVAs.


Background
It is known that surgical facemasks (used intensively and worldwide in recent times, due to the COVID pandemics) might negatively affect social processing. Impairment might concern the recognition of face identity [1,2 ], emotion reading [3,4,5,6,7,8 ], trustworthiness judgement [9], face likability and closeness impression [6], as well as speech comprehension [10]. Relatedly, it was shown that face blurring impairs the understanding of emotional signals including body language [11]. Although emotions conveyed by bodily expressions are quite easily recognizable [12], face obscuration reduces pantomime comprehension in healthy subjects, as opposed to patients with bilateral amygdala damage [13]. This indicates how facial mimicry is crucial in nonverbal communication. For example, when facial expressions are incongruent with bodily expressions (of anger, for instance) response times are much slower during a matching-to-sample task in controls [14], thus suggesting that bodily expressions are better recognized when accompanied by a face that expresses the same emotion [15]. To investigate at which extent face covering impaired social communication Grundmann and coauthors [6] performed a large study on 191 individuals of different ages and sex and found that facemasks diminished people's ability to accurately categorize facial expressions and affected the perceptions of person trustworthiness, likability, and closeness.
Generally, the mouth region is thought to be most informative for happy, surprised and disgusted expressions, while the eyes area is considered more informative for fearful and angry expressions.
Instead, both the mouth and eyes areas are informative for neutral and sad expressions [16,17]. The white sclera expansion, typical of fear display, is especially at the basis of its innate recognition, in that it strongly stimulates the amygdala nuclei [e.g., 18, 19].
Noyes et al. [2] recently explored the effect of masks and sunglasses wearing on familiar and unfamiliar face matching and emotion categorization in 100 participants. They found that, while masks did not reduce the recognition of angry faces, facial expressions of disgust, happy, fear and surprise were most affected by it. A large reduction in categorization accuracy for disgust expressions was found in particular. Sadness detection was di cult both mask less and with mask covering, so that the performance was not signi cantly impaired by masking. Results are not fully consistent across studies.
In a recent study by Marini et al. [4] investigating the impact of facemasks on emotion recognition (but only with three emotions) they showed an impaired recognition of sad and fearful expressions in the masked condition, with no effect on neutral expressions. Among the three expressions, sadness was the most affected and happiness the least affected. In this study, sadness was more hardly detected with mask covering the mouth area.
One of the problems with the available studies is that many of them digitally applied a mask or a foulard on the face picture in order to create identically expressive faces, across the masked vs. non-masked category [e.g., 1,3,4,5,6]. While this procedure might assure an optimal matching between masked and unmasked expressions, however it lacks likelihood and ecological value. Indeed, digitally applied masks are not deformed by the facial expression thus reducing the verisimilitude. Furthermore, they deprive the visual image of details that are present in the real masked face, such as mask sucking or folding. In reality, surgical masks are, for example, deformed by the vertical opening of the mouth in expressions like surprise or laughing, or during verbal speech; likewise, they are stretched horizontally for smiling. Indeed the masks adapt to, and reveal, the underneath muscular movements, which can be picked up by an observer.
In order to maintain the visibility of mask bending and stretching due to underneath facial mimicry, in this study, actors wore real surgical masks during shooting. Several repetitions and much effort was devoted to the perfect matching between expressions produced with or without masks. Pictures were taken at home during COVID pandemics lock-down. Actors were required to avoid wearing earrings, glasses, make up, hairpins, pliers, any type of hair embellishments, mustaches, beard. They were also instructed to wear a black t-shirt and gather the hair behind the head. The pictures of two actors were discarded in that showing a different mimicry in the natural vs. masked condition; their pictures were therefore used only as stimuli for the training phase, to accustom the subjects to the task, without showing them the faces selected for the experimental phase.

Participants
For each of the 7 emotions, actors were instructed to imagine a vivid emotional state, while concentrating on a speci c autobiographic scenario through the Stanislavsky method, and express it spontaneously while ignoring the presence/lack of surgical masks. They trained repeatedly in order to reach the same degree of intensity across subjects and emotions (not too subtle, not too intense, see Fig. 1 for some examples). Each of the 10 actors provided written consent and lled in the privacy release form.

Procedure
After giving active informed consent, participants to the study provided basic demographic information.
Subsequently, participants completed an emotion-recognition task that was told to last 10 minutes. Participants were instructed to observe one face at a time and respond within 5 seconds. They were advised not to miss any answer and to provide only 1 answer per face. The emotion-recognition task consisted of 112 trials. Speci cally, participants were asked to ll in a questionnaire consisting of an initial part with general demographic questions (regarding age, sex, manual dexterity, educational quali cation and e-mail address). This part was followed by the emotion-recognition task, consisting in 112 experimental trials, in which participants were rst shown a portrait photograph of an adult face to be inspected for about 2 seconds and immediately below found a list of emotions (neutrality, happiness, surprise, fear, anger, sadness, disgust), from which they had to select the correct one. Next, participants judged the extent to which they deemed the expression recognizable on a 3-point Likert scale (ranging from '1 = not much recognizable not' to '3 = extremely recognizable). The emotion was scored 0 if a different incorrect expression was selected. 5 seconds were allowed for perceiving and responding to the two queries.

Data Analysis
The individual scores obtained from each individual, for each of the 7 facial expressions and condition, underwent a 3-ways repeated-measures ANOVA whose factors of variabilities were: one between-groups named "sex" (with 2 levels, female and male), and two within-groups named "condition" (with 2 levels, natural and masked) and "emotion" (with 7 levels, happiness, neutrality, surprise, anger, sadness, fear, disgust).
In order to properly assess the statistical effect of the sex of participants (who were females in majority) in a balanced population, two subgroups of participants were created: the group of males comprised all male participants recruited (N = 47) and a blind selection of females (N = 47) chose on the basis of their date of birth (by paring each of the male with a same-age female). As a result of this blind procedure, the age of the two sub-groups was identical (males: 23.04255, fameless: 23.04255). A 3-ways repeatedmeasures ANOVA was also performed on the data relative to this sample. Factors of variabilities were one between-groups named "sex" (with 2 levels, female and male), and two within-groups named "condition" (with 2 levels, natural and masked) and "emotion" (with 7 levels, happiness, neutrality, surprise, anger, sadness, fear, disgust). Multiple post-hoc mean comparisons were performed using Tukey's test. Greenhouse-Geisser correction was applied in case of epsilon < 1 and epsilon corrected p value were computed.
Post-hoc comparisons showed that overall positive emotions were recognized more easily than negative emotions (p < 0.00001), except for anger, as shown in Fig. 2  Surgical masks (covering the nose and mouth area) strongly reduced recognizability of all emotions, as shown by the statistical signi cance of condition x emotion [F (6,1272) = 160; p < 0.00001, ε = 0.911, εcorrected p value = 0.00001], except for anger. Post-hoc comparisons showed strongly signi cant differences among all categories, except that neutral and happy expressions were equally well recognizable under the mask, although worse than angry expressions. Again, negative emotions such as disgust, sadness and anger were equally less recognizable than positive emotions without mask. Figure 3 shows the mean recognizability scores and SE values for each of the facial expressions and as a function of the masking condition. Negative emotions such as sadness and disgust, more relying on the nose and mouth area expressivity, were most penalized by mask covering. As for the last interaction (see Fig. 4 and Fig. 5 for mean values and SEs), post-hoc comparisons among means (similarly to the main ANOVA) showed that while women were better at recognizing surprise (p < 0.004) and sadness (p < 0.05), males were better at recognizing fear expressions (p < 0.005). Simple effect analysis showed that this male advantage in recognizing fear was even stronger (see Fig. 5) in the masked conditions (p < 0.004).

Discussion
In the natural (mask less) condition, positive emotions (happiness, neutrality, surprise) were recognized more accurately than negative emotions such as fear, sadness or disgust. Masking heavily affected emotion comprehension with a 31% decay in recognizability scores (namely, going from 2.31 in the natural condition to 1.59 in the masked condition, on a scale where 0 indicated the "lack of recognition" and 3 stood for "extremely recognizable"). Overall, these ndings t with previous recent literature showing how facemasks reduce emotion recognition accuracy [3,4,5,6,7,8]. In our study, face masking was most detrimental for sadness and especially disgust detection, than positive emotions such as happiness. This pattern of results agrees with previous studies, for example Marini et al. [4], nding that sadness was the most affected and happiness the least affected expression by face masking.
However, we found that mask covering did not affect the recognition of angry faces, which replicates some ndings obtained with non-digital masks by Noyes and coauthors [2] (see Fig. 7 of their paper), who also found that the mask and sunglasses conditions did not signi cantly differ in the angry expressions. The primacy of anger among the biologically relevant emotions has been shown by several studies (e.g., [20]). Conversely, the emotional display whose recognition was most affected by mask covering was disgust (also in Noyes et al.'s [2] study). Indeed, disgust's more evident markers (nasiolabial lifting and grimacing and/or nose wrinkling) were hidden by surgical masks in the masking condition. At this regard it is known that successful recognition of anger versus disgust requires one to process information located in the eye/brow region (which was disclosed) as opposed to the mouth/nose region (which was covered by masks), respectively [21]. Again, in a study by Ponari et al. [22] where emotion recognition was hampered by stimuli in which an upper or lower half-face showing an emotional expression was combined with a neutral half-face it was shown that neutral lower half-face interfered with recognition of disgust, whereas the neutral upper half (i.e., the eyes area) impaired the recognition of anger. This difference may probably explain the supremacy of anger and the poor recognition of disgust in the present study.
Women better at recognizing sadness and surprise. In our study, females outperformed males in the recognition of sadness and surprise. Several evidences in the literature consistently reported a similar pattern of results for both sadness [23,24,25] and surprise [23]. In addition, according to some investigations, women seem to be more sensitive to sadness whereas men seem to be more sensitive to anger [26,25,27].
In another study by Montagne and coworkers [23] women were reported to be signi cantly more accurate than men at identifying sadness and surprise. Furthermore, Li et al. [24]'s study, performed in 1063 participants varying in sex and age, reported that women performed signi cantly better at recognizing facial expressions of sadness and disgust.
As for the speci c effect of masking, Grundmann and coauthors [6] tested 191 participants (52.9% female) and aging form 19 to 79 years and found that emotion-recognition accuracy declined for masked (vs. unmasked) faces. More interestingly, they showed lower accuracy to being male vs. female, being old (vs. young), and to seeing an old (vs. young) target face. In a study by Calbi et al. [3] involving only three affective displays (neutrality, happiness and anger) it was found that female participants gave more negative ratings than male ones when evaluating angry and neutral facial expressions, and more positive ratings when evaluating happy facial expressions. This was discussed in terms of women' stronger sensibility to face expressivity and better decoding of emotions through facial expressions [28,29,30,31]. Consistently, Hoffmann and coauthors [32] found that women were better at identifying subtle, less intense emotions (such as sadness), but equally good at identifying clearly expressed emotions (such as fear). Apart from that, it is generally believed that women are more sensitive to emotional facial cues [31].
Men better at detecting fear. In this study, males outperformed women in recognizing fearful expressions (especially masked ones). The increased male ability to recognize fear (relying mostly on the processing of the eyes area, with the typical sclera enlargement) when faces were covered by surgical masks, might depend on the fact the eyes were even more focally attended in the masked condition, being the only uncovered face area. However, Sullivan et al. [33], investigating the percentage of time young women and men spent xating the eyes and mouth areas of facial expressions (including fear), found that both sexes spent 63.6% of their time looking at the eyes (and 36.4% of the time at the mouth) with no difference across sexes.
In the literature, a male advantage in the processing of fearful expressions is not commonly found, except for an fMRI study, observing regional brain responses to face versus shape identi cation, in which men showed more signi cant modulations by both fear and anger affective traits than women [24].
On a different verge, Riva et al. [34] have instead found that the observers' ability to detect pain in a female face was lower than their ability to detect pain in male faces, i.e., that male pain faces are more easily processed at the re exive level. Relatedly, Simon et al. [35] in an fMRI study found that observing male (vs. female) individuals expressing pain activated in the observers a much greater threat-related response, including the activation of the ventromedial prefrontal cortex, posterior and anterior insula, somatosensory areas, and amygdala. In another study, where healthy subjects were provoked by money taken by an opponent and given the opportunity to retaliate, men showed a higher amygdala activation during provocation, and the amygdala activation correlated with trait anger scores in men, but not in women [36]. As well-known amygdala nuclei are the brain structures most involved in fear and threat processing [37].
Summary. Overall, while face masking reduced the comprehension of all facial expressions but anger (conveying an aggressive display), it was most detrimental for sadness and especially disgust detection (conveying a second person, more passive negative state). The larger impairment for the recognition of the above expressions might depend on their mainly relying on the expressivity of mouth (especially sadness: [16,17]) and nose areas (especially disgust: [2,21]), which were covered by masks. Instead, the angry expression was totally unaffected by face masking.
In general, women showed a better performance for positive emotions, both in masked and natural conditions, and men for fear recognition (in natural but especially masked conditions). At this regard, it might be interesting to consider that sex differences in the hemispheric activation for emotion processing have been found. Cahill et al. [38] found that enhanced memory for emotional video clips was associated with activity of the right amygdala in men, and of the left amygdala in women. In addition, an fMRI study investigating the emotional response to odors by Royet et al. [39] found a sex difference in the activation of the left orbitofrontal cortex, which was greater in women compared to men. On the other side, Bourne and Watling [40] found that for males, but not females, greater reported use of negative emotion strategies was associated with stronger right hemisphere lateralisation for processing negative emotions. In the light of the well know right/left asymmetry for negative/positive emotions [41,42] these studies might provide the neural underpinnings for the higher male accuracy in fear recognition (right amygdala), and of the higher female accuracy for detecting subtle positive emotional cues (e.g., [3]), but further investigations are certainly needed to reach a de nitive conclusion.
More in general, our study suggests the opportunity of studying the effect of face masking with really worn facemasks (instead of digitally applied ones) because there might be a difference in the way masks elastically respond to underneath facial muscles contractions, by deforming and stretching differently as a function of the facial expression. Furthermore, the typical inhalation associated, for example, to the surprised or fearful reaction (startle response), which results in mask sucking, will not be observable with digitally applied masks.

Consent for publication
Written and signed consent for release of personal information (pictures of facial expressions) was obtained from all actors.

Availability of data and material
Anonymized data and details about preprocessing/analyses are available to colleagues upon request.

Competing interests
The authors declare that the research was conducted in the absence of any commercial or nancial relationships that could be construed as a potential con ict of interest.
Funding 42. Canli T, Desmond JE, Zhao Z, Glover G, Gabrieli JDE. Hemispheric asymmetry for emotional stimuli detected with fMRI. NeuroReport. 1998; 9, 3233-3239. Figure 1 Example of stimuli (facial expressions) produced by two of the actors in the natural and masked condition. Overall, stimuli were created by taking photographs of natural expressions of 8 actors (4 male and 4 females) concentrated on their inner imaginary emotional state, through the Stanislavsky method.

Figures
Masks were worn, not digitally recreated. This revealed, for example the large mouth opening associated with disgust, or the air intake (inhalation) characteristic of surprise or fear resulting in evident mask sucking.

Figure 2
Mean scores of recognizability (along with SE values) attributed by participants (N=214) to the various facial expressions regardless of masking condition. Scale ranged from 0 (lack of recognition) to 3 (extremely well recognizable). Mean scores (along with SE values) of recognizability attributed by participants (N=214) to the various facial expressions as a function of masking condition. Scale ranged from 0 (lack of recognition) to 3 (extremely well recognizable). Mean scores (along with SE values) of recognizability attributed by female and male participants (N=94) to the various facial expressions as a function of masking condition.