Impairment of emotional expression detection after unilateral amygdala resection

Detecting facial emotional expressions is an initial and indispensable for face-to-face communication. Neuropsychological studies on the neural substrates of this process have shown that bilateral amygdala lesions impaired the detection of emotional facial expressions. However, the ndings were inconsistent, possibly due to the limited number of patients examined. Furthermore, this processing is based on emotional or visual factors of facial expressions remains unknown. To investigate this issue, we tested the group of patients (n = 23) with unilateral resection of anterior temporal lobe structures, including the amygdala, and compared their performance under resected and intact hemisphere stimulation conditions. The patients were asked to detect normal facial expressions of anger and happiness, and articially created anti-expressions, among a crowd with neutral expressions. Reaction times were shorter to detect normal versus anti-expressions when the target faces were presented to the contralateral visual eld (i.e., stimulation of the intact hemisphere) than to the ipsilateral visual eld (i.e., stimulation of the resected hemisphere). Our ndings suggest that the amygdala plays an essential role in the detection of emotional facial expressions, according to the emotional signicance of the expressions. experiments run on a computer with a monitor using Presentation 14.9 software The resolution of the monitor 1,024 × and the refresh rate was Hz, as by a high-speed camera frames/s; EXILIM A response box a 2–3-ms RT resolution used obtain A rest a distance of 0.57 m participants monitor.


Introduction
Detecting emotional facial expressions is an initial and indispensable stage in conscious face-to-face emotional communication. Appropriate detection of others' emotional expressions allows us to understand their emotional states, and thus regulates social behavior and promotes the creation and maintenance of social relationships.
Experimental psychology studies of healthy participants have demonstrated that the detection of emotional expressions is faster than that of emotionally neutral expressions using the visual search paradigm [1][2][3][4][5][6][7][8][9][10][11] . For instance, Williams et al. 3 instructed participants to search for target faces in arrays of distractor faces and tested the effects of emotional expressions on search behavior. The reaction times (RTs) for the detection of emotional (e.g., angry and happy) expressions among neutral expressions were shorter than those for the detection of neutral expressions among emotional expressions. Some studies suggested that this e cient detection of emotional expressions is due to the emotional, but not visual, factors of the expressions 5,6 . For instance, Sato and Yoshikawa 5 instructed participants to search for normal emotional (angry and happy) facial expressions and their anti-expressions among neutral expression distractors. The anti-expressions were arti cial facial expressions that contained visual changes quantitatively comparable to normal expressions and were recognized as emotionally neutral 12 .
The RTs for the detection of normal emotional expressions were shorter than those for the detection of anti-expressions. These data indicate that emotional facial expressions are e ciently detected because of their emotional signi cance.
A few previous neuropsychological studies have examined the neural substrates of this process, and found that a bilateral amygdala lesion impaired the detection of emotional facial expressions in visual search tasks 13,14 . Speci cally, Bach et al. 13 tested two patients with bilateral amygdala damage, and a group of normal controls, on a visual search task in which participants searched for an angry target among a crowd of happy expressions or a happy target among a crowd of angry expressions. Although controls detected angry expressions more rapidly than happy ones, the patients showed the opposite pattern, detecting happy expressions more rapidly. Domínguez-Borràs et al. 14 tested a patient with bilateral amygdala damage, and a control group, on a visual search task in which participants searched for an emotional (fearful or happy) facial expression or a neutral facial expression among a crowd of neutral facial expressions. Whereas controls detected facial expressions of fear and happiness more rapidly than neutral expressions, the patient did not. Although the results are not completely consistent, collectively, these studies suggest that amygdala lesions impair the detection of emotional facial expressions.
However, some issues regarding the involvement of the amygdala in the detection of emotional facial expressions remain unresolved. First, one study 15 has reported no effect of amygdala lesions on the detection of facial expressions in a visual search task. That study tested a bilateral amygdala-damaged patient and normal controls using a visual search task, in which participants detected a fear expression among a crowd with neutral expressions, or a neutral expression among a crowd with different neutral expressions. Both the patient and the controls detected fearful expressions more rapidly than neutral ones. These results suggest that amygdala lesions may not impair the detection of emotional facial expressions. One plausible explanation for the inconsistent ndings is the small samples of the studies, which tested only one or two bilateral amygdala-damaged patients. Because such small samples do not provide reliable ndings16, investigating this issue in a group of patients is warranted.
Furthermore, whether impaired detection of emotional expressions in amygdala-damaged patients is due to emotional or visual factors remains untested. Emotional and neutral facial expressions possess differences not only in emotional signi cance, but also in physical features (e.g., oblique eyebrows in angry expressions versus horizontal eyebrows in neutral expressions). Because some studies have demonstrated that several visual features, such as oblique lines and curves, were detected more e ciently than other features, such as horizontal lines 17,18 , it may be that the abnormal detection of emotional facial expressions in amygdala-damaged patients reported in previous studies re ected problems with visual processing. Regarding this issue, some functional neuroimaging studies have demonstrated that amygdala activity in response to emotional facial expressions re ected the emotional signi cance, but not the visual features, of the expressions [19][20][21] . Based on these data, we hypothesized that an amygdala lesion may impair the detection of emotional facial expressions, even after controlling for the visual elements of the expressions.
To investigate this hypothesis, we tested a group of patients with unilaterally resected temporal lobe structures, including the amygdala (Fig. 1), using a visual search paradigm. Normal facial expressions of anger and happiness, and their corresponding anti-expressions, were the target stimuli among a crowd with neutral expressions presented to the unilateral visual eld (Fig. 2). Because the anti-expressions showed neutral emotions, but had visual feature changes equivalent to those between normal emotional and neutral expressions 12 , they allowed us to compare emotional and neutral facial expressions while controlling for the effects of basic visual processing. Because visual images presented in a unilateral visual eld are primarily processed in the contralateral hemisphere 22 , we compared the RT required to detect normal and anti-expressions between normal and resected hemisphere stimulation conditions. This paradigm has been shown to effectively reveal the effects of amygdala lesions on psychological impairment 23,24 . To con rm the emotional impact of normal and anti-expressions, we also obtained subjective ratings of the stimuli from the patients, in terms of valence and arousal, and also investigated familiarity and naturalness as possible cognitive confounding factors.

Visual search RT
The RTs obtained under each condition in the visual search task are shown in Table 1, and RT differences between the normal and anti-expression conditions are shown in Fig. 3. Two-way analysis of variance (ANOVA) on the RT differences between normal and anti-expressions, with stimulated hemisphere (resected, intact) and emotion (anger, happiness) as factors, showed a signi cant main effect of stimulated hemisphere, indicating faster detection of normal expressions versus anti-expressions when the intact hemisphere was stimulated (i.e., expressions were presented to the contralateral visual eld) than when the resected hemisphere was stimulated (i.e., expressions were presented to the ipsilateral To investigate the hemispheric functional asymmetry and resection method of this phenomenon, a fourway ANOVA were conducted for RT differences with additional between-subjects factor of resected side (left, right) and resection method (selective amygdala-hippocampus resection, temporal lobectomy

Rating
The subjective stimulus ratings are presented in Table 2. To assess the subjective emotional impact of the stimuli, the valence and arousal ratings were assessed. The familiarity and naturalness of the stimuli were included as possible covariates. The ratings were analyzed using two-way ANOVA with stimulus type (normal, anti) and emotion (anger, happiness) as factors.  Follow-up analyses for the interaction showed that the simple main effects of stimulus type were signi cant in both the anger and happiness conditions, F(1,44) = 6.84 and 11.86, p = 0.012 and 0.001, respectively, i.e., the naturalness ratings were higher for anti-angry than normal angry expressions, and for normal than anti-happy expressions.

Discussion
Our results revealed that the rapid detection of normal versus anti-expressions of anger and happiness was more evident with stimulation of the intact hemisphere than of the resected hemisphere. Because we compared the RTs to normal expressions and their anti-expressions, which had featural changes quantitatively comparable with normal expressions, visual factors were controlled for. The subjective ratings of valence and arousal con rmed the difference in emotional impact between normal and antiexpressions. The ratings of familiarity and naturalness, as potential confounders, showed different patterns from the detection performance and emotional ratings, suggesting that they did not account for detection performance. The visual search task results are consistent with previous ndings that normal angry and happy expressions were more rapidly detected than their anti-expressions in healthy participants, and that the rapid detection of emotional expressions was impaired in bilateral amygdalalesioned patients compared with normal controls 13,14 . However, inconsistent ndings were reported with respect to the detection of emotional facial expressions by bilateral amygdala-lesioned patients, and no previous study has compared the effects of visual and emotional factors in this context. To the best of our knowledge, this is the rst study showing that the detection of facial expressions of anger and happiness is impaired by unilateral amygdala lesions in accordance with emotional, but not visual, aspects of the expressions.
Our ndings indicate that the amygdala plays a crucial role in the detection of emotional facial expressions. This corroborates previous electrophysiological studies showing rapid activity in the amygdala during the processing of emotional facial expressions. For example, some intracranial eld potential recordings in the amygdala have shown that it was more rapidly activated in response to emotional than neutral expressions 25,26 . Another study indicated that the amygdala modulated activity in the visual cortex during the observation of facial stimuli 27 . Extending this literature, our ndings suggest that the amygdala is indispensable for the rapid processing of emotional facial expressions, in part due to its involvement in the conscious detection of facial expressions.
A limitation of this study was that the resected region was not restricted to the amygdala; the anterior part of the hippocampus and parahippocampal gyrus were also resected. Several lesion and functional neuroimaging studies have shown that the human hippocampus, and adjacent structures, are mainly involved in spatial and episodic memory functions, suggesting a more important role of the amygdala in emotional processing 28,29 . However, animal anatomical studies revealed interconnections between the amygdala and hippocampal/parahippocampal regions 30 and human neuroimaging studies showed functional coupling between these structures 31,32 , suggesting that the amygdala and hippocampal/parahippocampal regions may act as a functional circuit. Future studies including patients with more amygdala-speci c restriction or damage are warranted to clarify the neural mechanisms underlying rapid detection of emotional facial expressions.
In conclusion, we tested a group of patients who had undergone unilateral temporal lobe resection, including the amygdala, on a visual search paradigm in which they detected normal facial expressions of anger and happiness and their anti-expressions among a crowd with neutral expressions. RTs to normal versus anti-expressions were shorter when the target face stimulated the intact hemisphere than when it stimulated the resected hemisphere. These ndings suggest that the amygdala plays an indispensable role in the detection of emotional facial expressions, in accordance with their emotional signi cance.

Participants
The study included 23 patients (8 females, 15 males; mean ± SD age = 32.7 ± 12.8 years) with temporal lobe structures that were unilaterally resected due to pharmacologically intractable seizures. Although three additional candidates were tested, their data were not analyzed because they displayed a visual de cit (n = 1; see Procedure), withdrew from the study (n = 1), or slept during the task (n = 1). We determined sample size using an a priori power analysis. We used G*Power ver. 3.1.9.2 software 33 and assumed to contast the intact versus resected hemisphere stimulation with an α level of 0.05, power of 0.80, and effect size d of 0.5 (strong). The results indicated that 21 participants would be required. All participants had undergone the surgical procedure more than 1 year before the experiment. Seizures were well controlled in most of participants (n = 17, 3, 2, and 1 for Engel Classes 34 I, II, III, and IV, respectively), and all were mentally stable during the experiments. Twelve and eleven participants had undergone resection in the left and right hemispheres, respectively. The resection method was selective amygdalohippocampectomy, which included the amygdala, anterior part of the hippocampus, and anterior parahippocampal gyrus, in 17 participants, and anterior temporal lobectomy, which included the amygdala, anterior part of the hippocampus, anterior temporal lobe neocortex (4-5 cm from the temporal pole), and anterior parahippocampal cortex, in 6 participants. Postsurgical magnetic resonance imaging con rmed resection of the target regions in all patients (Fig. 1). Handedness was assessed using the Edinburgh Handedness Inventory 35 , and most of the participants were right-handed (n = 22). All participants had normal or corrected-to-normal visual acuity, and all provided written informed consent following a full explanation of the procedure. This study was approved by the Ethics Committee of Shizuoka Institute of Epilepsy and Neurological Disorders, and was conducted according to institutional ethical provisions and the Declaration of Helsinki.

Apparatus
The experiments were run on a Windows computer (HP Z200 SFF; Hewlett-Packard Company, Tokyo, Japan) with a 19-inch CRT monitor (HM903D-A; Iiyama, Tokyo, Japan) using Presentation 14.9 software (Neurobehavioral Systems, San Francisco, CA, USA). The resolution of the monitor was 1,024 × 768 pixels, and the refresh rate was 100 Hz, as con rmed by a high-speed camera (1,000 frames/s; EXILIM FH100; Casio, Tokyo, Japan). A response box with a 2-3-ms RT resolution was used to obtain responses (RB-530; Cedrus, San Pedro, CA, USA). A chin-and-forehead rest was used to maintain a distance of 0.57 m between the participants and the monitor.

Stimuli
From a facial expression database 36 , we selected gray-scale photographs of a female (PF) and male (PE) model with angry, happy, and neutral expression, with the teeth not showing. The models were not known to any participants. Anti-expressions were created from these normal expressions using morphing software (FUTON System; ATR, Seika-cho, Japan). First, we manually identi ed the coordinates of 79 facial feature points and readjusted them based on the coordinates of the iris of each eye. Next, the distances between each feature point of the emotional (angry and happy) and neutral facial expressions were calculated. Finally, anti-expressions were created by setting their feature positions to the same distance in the opposite direction. Two types of adjustments were made to the stimuli using Photoshop 5.0 (Adobe, San Jose, CA, USA). First, the photographs were cropped using an oval shape within the contour of the face, to eliminate factors irrelevant to the expression (e.g., hairstyle). Second, signi cant differences in contrast were eliminated, thereby removing possible identifying information. In addition, some minor color adjustments were made to a few pixels. The face stimuli were all 1.58° horizontally and 1.93° vertically. Photographs of normal and anti-expressions of anger and happiness were used as target stimuli, and photographs of neutral expressions were used as distractor stimuli. Illustrations of the stimuli are shown in Fig. 2.

Procedure
Each participant was tested individually. The experiment comprised three sessions, i.e., visual eld assessment, visual search, and rating sessions. Participants were instructed to keep their gaze on the xation cross (0.86° × 0.86°) at the center of the display when the cross was presented throughout the sessions.
Visual eld assessment Participants were assessed for possible visual eld defects in four trials. In each trial, a white xation cross (0.86° × 0.86°) was rst presented in the center of the monitor for 500 ms, followed by a target stimulus (white circle subtending 1.0°), which was presented for 200 ms in the corner of the square area where the faces were presented in the visual search task. Participants were asked to look at the xation cross, and then to point to the place where the target appeared. No participant included in the analysis showed any visual eld de cit.

Visual search task
The visual search task consisted of 512 trials presented in eight blocks of 64 trials, with an equal number of target-present and target-absent trials (i.e., 256). In the target-present trials, a target face was presented among three neutral faces, while the target-absent trials showed four neutral faces. Each target condition (normal anger, normal happiness, anti-anger, and anti-happiness for resected and intact hemisphere stimulation) was represented by 32 trials. The trial order was randomized across all conditions within a block. The interstimulus interval varied from 1,300 to 1,800 ms.
In each trial, after the xation cross (0.86° × 0.86°) appeared for 500 ms in the center of the monitor, the 2 × 2 face stimulus array (4.30° × 4.30°) was presented until participants responded. All the faces were presented in the unilateral left or right visual eld. An example of the stimulus display is shown in Fig. 2. Each facial array was comprised pictures of a single model. Participants were instructed to look at a xation cross, and then to decide whether one face was different, or all four faces were the same, by pushing prede ned buttons on a response box using their left and right index ngers, as quickly and accurately as possible. The position of the response buttons was counterbalanced across participants.

Rating task
After the visual search task, the rating tasks for the target and distractor stimuli were performed. The stimuli were presented individually. Participants were asked to rate each stimulus in terms of emotional valence and arousal (i.e., the subjective ratings of the nature and intensity of the emotional experience), familiarity (i.e., the frequency with which they encountered the facial expressions depicted by the stimulus in daily life), and naturalness (i.e., the degree to which the expression depicted by the stimulus seemed natural) using a scale ranging from 1 to 9. The order of presentation of facial stimuli and rating items during the rating task was randomized.

Data analysis
All statistical tests were performed using SPSS 16.0J software (SPSS Japan, Tokyo, Japan). The α-level for all analyses was set to 0.05.
For RT analysis, the mean RTs of correct responses in target trials were calculated for each condition and participant, with values ± 3 SD from the participant's total mean excluded as artifacts. To simplify the analyses, RT difference scores were calculated for each participant by subtracting the RT for the normal expression condition from the RT for the anti-expression condition (positive values indicate faster reactions to normal expressions). The RT difference was then analyzed using two-way repeatedmeasures ANOVA with stimulated hemisphere (intact, resected) and emotion (anger, happiness) as factors. Follow-up simple effect analyses were conducted for signi cant interactions29. To investigate the effects of possible covariates, we also conducted ANOVAs of RT differences between normal and anti-expression conditions, including the between-subjects factors of resected side (left, right) and resection method (selective amygdala-hippocampus resection, temporal lobectomy). Valence and arousal ratings were also analyzed using ANOVA.
Preliminary analyses conducted for accuracy using two-way ANOVA with the same design as the above RT analysis showed no signi cant main effect or interaction (F(1, 22) < 1.02, p > 0.10, η 2 p < 0.045). Hence, we report only the RT results, as in previous studies (e.g., [5]). Figure 1 Representative anatomical magnetic resonance image of one of our temporal lobe-resected patients.  Mean (± SE) reaction time (RT) differences between the normal and anti-expression conditions. Asterisks indicate a signi cant simple main effect of stimulated hemisphere, indicating larger RT differences when the target faces were presented to the contralateral visual eld (i.e., stimulation of the intact hemisphere) compared with the ipsilateral visual eld (i.e., stimulation of the resected hemisphere). *: p < 0.05.