Music-evoked emotions affect memory for emotional pictures

Music is widely known for its ability to induce emotions. However, to investigate music-evoked emotions, most studies rely on self-report questionnaires, which are vulnerable to bias. In the present study, we explored mood-congruency effects on memory as an indirect, nonverbal method to examine the experience of musical emotions. Participants listened to 15 music excerpts chosen to induce different emotions; after each excerpt, they were required to look at four different pictures, that could be either congruent with the emotion conveyed by the preceding music excerpt, incongruent, or neutral. After the presentation of the stimuli, participants completed a recognition task, including new pictures, already seen emotionally congruent pictures, and already seen emotionally incongruent pictures. Based on previous ndings about mood-congruency effects, we hypothesized that if individuals had felt an emotion, this would facilitate memorization of emotionally congruent pictures. Results supported this prediction, as accuracy in the recognition task was higher for the emotionally congruent pictures than for the emotionally incongruent ones. This effect suggests that music-evoked emotions have an inuence on subsequent cognitive processing of emotional stimuli, a result relevant for application in different psychology elds. Moreover, mood-congruency tasks may represent a source of evidence for the presence of music-evoked emotions.


Introduction
Music-evoked emotions has grown into a key area of research in the psychology of music, and it has also gained a signi cant place in the psychology of emotions and in affective neuroscience. Its growing role across these elds has compelled researchers to critically examine measurement approaches to musicevoked emotions [1][2][3][4]. Among the most frequently used approaches are subjective verbal reports of emotions one hand (e.g., [ 5 ]) and physiological measures on the other (e.g., [ 2 ]).
Measuring music-evoked emotions by use of verbal reports has the advantage of capturing speci c, musically relevant emotional states, such as awe, tenderness, nostalgia, or sadness. The disadvantage is that the extent to which these emotions are actually felt remains often unclear. When reporting a given emotion, listeners may describe the stereotypical emotional connotation of the music that they perceived, rather than an emotion that they have truly felt [ 5,6 ]. While perceived and felt emotions are often correlated [ 6 ], they do not necessarily have to coincide: an individual who listens to sad music can feel pleasure and positive emotions even if the music has negative valence 7 .
Physiological measures, such as skin conductance or heart rate, have a greater probability of capturing emotional arousal (i.e., felt emotions), but they lack the speci city that would be necessary to identify speci c emotional states, such as awe, nostalgia, or sadness. There is evidence suggesting that neuroimaging methods may provide more speci c emotional information than do psychophysiological measures [ 8 ]. However, such methods are time-and resource-intensive, and may therefore be di cult to use on a broad scale. Clearly, a method capable of identifying speci c emotional experiences without the biases inherent in verbal reports would go an important step beyond previous work toward achieving a valid assessment of musical emotions induction. In this study, we aimed to study music-evoked emotions behaviorally, by using a paradigm that relies on mood-congruency effects on attention and memory.
The affective state of an individual can in uence the perception of external stimuli, where to direct attention, and what will be remembered, and these phenomena are commonly referred to as "mood congruency effects" [ 9,10,11 ]. Previous research has shown that information processing is facilitated when the affective tone of the information matches the perceiver's mood. For example, a study by Niedenthal and Setterlund (1997) [ 12 ] showed that the affective state of the participants facilitated the recognition of emotionally-congruent words. Furthermore, Becker and Leinenger (2011) [ 9 ] observed that subjects were more likely to detect the appearance of an unexpected face during a tracking task when the expression of the face matched the participants' mood. Other ndings showed that individuals in a positive affective state, compared to a neutral one, direct attention to peripheral positive stimuli [ 13 ], and more often towards rewarding than to aversive information [ 14 ]. Demonstrations of affective state in uences on cognitive processes also come from clinical population studies (e.g., in mood disorders). For example, dysphoric patients hold attention for longer to negative stimuli [ 15 ] and they show increased neural response to sad stimuli [ 16 ], suggesting that they attend to information that matches their negative mood.
The semantic associative network model of memory by Bower (1981)[ 18 ] provides a possible explanation for these mood-congruency effects. According to the model, memory can be seen as a network of nodes and connections. Each emotion has its speci c node. Connected to this node are other nodes containing events, verbal labels, and any kind of information that is associated with that speci c emotion. Once the emotional node is activated, the other nodes connected to it will become more easily accessible to attention and/or memory. It is thus expected that when an individual is in a certain affective state, this state will make mood-congruent categories more salient than incongruent ones, thus in uencing where to direct attention and/or what to retrieve.
There is, however, scarce information about whether these mood-congruency effects can emerge when using music to induce an emotional state. To our knowledge, only Tesoriero and Rickard (2012)[ 18 ] investigated mood-congruency effects by using music to induce an emotional state in their participants.
Speci cally, they selected music to induce four different emotions (i.e., calmness, happiness, sadness, and fear) in four different groups of participants. Later, they tested whether the induced emotions affected the recall of emotionally congruent and incongruent narrative content. The authors observed mood-congruency effects in only one condition, that is, a superior recall of positive information over negative information in the group that listened to happy music.
As aforementioned, music can convey emotions and, eventually, also change the affective state of the individual [ 5,8,19,20 ]. We can thus hypothesize that if an individual feels an emotion in response to music, this could indirectly affect his/her cognitive processes, as already observed in the study by Tesoriero and Rickard (2012)[ 18 ]. Based on this hypothesis, in the present study, participants listened to some music excerpts, and then looked at different pictures that were either emotionally congruent, incongruent, or neutral with respect to the music presented. We chose to use pictures because they can be easily selected to match the emotion evoked by the music (e.g., without needing any interpretation); in fact, they can convey an emotion immediately, other than, for example, using narrative content that would instead require a deeper processing to recognize its emotional connotation. In a second part, we administered a recognition task with 50% of the pictures previously presented and 50% of new pictures. We hypothesized that, if participants felt the emotion conveyed by the music, this would facilitate the memorization of emotional congruent pictures, in comparison to incongruent and neutral pictures.
We also expected that some individual difference variables, such as music training, gender, current affective state, and emotional intelligence could in uence the perception of musical emotions and measured these variables. Concerning music training, previous studies observed that musicians possess superior emotion recognition abilities in music than non-musicians [ 21 ], and that they experience more intense and differentiated musical emotions than non-musicians and amateur musicians [ 22 ]. Concerning the current affective state, it is important to consider the mood of participants at the beginning of the experiment, as individuals in a negative mood were found to perceive more sadness in music that did not have a clear negative valence [ 23,24 ]. Regarding gender, a meta-analysis showed that females and males differ in terms of recognizing emotions (conveyed by voice, and/or faces/postures), with females showing higher recognition abilities than males [ 25 ]. However, less is known about gender differences in recognizing emotions in music, and a few studies did not observe any group difference [ 21,26 ]. Finally, we also assessed emotional intelligence as it has shown to be connected to emotion recognition in musical stimuli 26 . Moreover, individuals with higher empathy, a concept that is related to emotional intelligence, are better at perceiving the emotional intentions of musicians, react more intensely to music in general and especially to unfamiliar sad pieces of music than less empathic ones [ 28, 29,30 ]. In summary, we expect that (a) the emotionally congruent pictures with the music will be remembered better than the incongruent ones and that (b) some individual variables (e.g., emotional intelligence, music expertise) will increase the accuracy difference between congruent and incongruent pictures.

Method
Participants Two-hundred adults (69 females, 3 non-binary) participated in this study. They were mainly university students, with a mean age of 23.86 years (SD = 4,65). One hundred and seventeen participants reported to have received some music training in their life (M =11.08 years, SD = 4.66, years range = 3 -22). All participants had a high level of education, with 112 individuals having completed high school, 61 having a bachelor's degree, 26 a master's degree and one person a higher degree.
The minimum number of participants required to reach 80% of statistical power (predicted with g-power, [ 30 ] for a repeated-measure ANOVA with within 3 within factors, was 73.

Materials
Music. The music excerpts we selected had already been used in previous works and categorized with the Geneva Emotional Music Scale (GEMS) [ 5,8,31 ]. The GEMS is a domain-speci c scale on musical emotions consisting of nine second-order categories (Wonder, Transcendence, Nostalgia, Tenderness, Peacefulness, Joyful Activation, Power, Tension and Sadness) and three rst-order emotion factors (Sublimity, Vitality, and Unease). We used the GEMS scale as it is based on categories (e.g., tension, nostalgia) rather than on dimensions (i.e., valence and arousal). This distinction seems important as the affective state of participants was found to facilitate the recognition of emotional congruent stimuli (i.e., words) when they both (i.e., the affective state and the word) belonged to the same emotional category, but not when they had the same valence [ 12 ]. We selected 15 music excerpts, equally dived into three categories: two rst order categories of the GEMS (i.e., vitality and sublimity), and one second-order category (i.e., tension). The rst-order category "unease" (that together with "tension" includes also "sadness") was represented by tension only, rather than both tension and sadness, because the intercorrelations between the two have been found to be relatively low [ 5 ]. Moreover, we decided to use the higher-order categories sublimity, vitality, and unease (represented by tension here) rather than the nine more speci c emotional categories for two reasons: rst, the narrower the GEMS emotions the more they could be speci c to music (e.g., "transcendence" could be hardly elicited by a picture). Secondly, it would have been near impossible to achieve an experimentally viable balance of items/trials, symmetry of picture presentation, and duration of the sessions using nine categories.
Loudness was equalized across all excerpts. The complete list of the excerpts used is reported in the supplemental material.
Pictures. In order for music and pictures to be matched in terms of emotion, the pictures were taken from the EmoMadrid [ 32 ] database and were pre-tested in a pilot study, in which we asked 98 subjects (divided into four groups, each to evaluate 50 pictures) to categorize each picture according to the emotions they were representing/inducing. The rating was done on the GEMS-9, with the addition of the category "neutral". We then selected 80 pictures that were equally disturbed to the four categories (i.e., tension, sublimity, vitality, and neutral).

Memory experiment
The memory experiment was designed speci cally for the present study with the program Psychopy [ 33 ].
The rst part consisted of the presentation of the music and the pictures, and had a total of 15 trials. Each trial consisted of listening to a classical music excerpt for 45s, and then looking at four pictures appearing on the screen for 2s. One picture was always congruent with the emotion expressed by the music, two pictures were emotionally incongruent with the music, and one was neutral. There were 15 trials, equally distributed into the three categories of the GEMS (i.e., tension, sublimity, vitality) that belonged to three separate blocks. The order of the blocks was randomized across subjects as well as the order of the excerpts within each block. The pictures presented as congruent and incongruent were also randomized across subjects (e.g., picture 1 could be congruent for one subject, and the incongruent for another subject). An example of a trial is depicted in Fig. 1. After the rst part nished, and before the second part began, participants had to solve 4 arithmetic equations, and answer questions about the familiarity with the music and how much they liked it. In the second part of the experiment, participants had to perform the recognition task. This task consisted of presenting 61 different pictures, one at a time, at the centre of the screen, and the subject had to decide whether each picture had already been seen in the rst part or not. Fifty percent of pictures were already presented, and 50% were new. Among those already presented, 15 were the congruent images and 16 were the incongruent images, equally distributed to the four categories (i.e., unease, sublimity, vitality, and neutral). Accuracy was computed as the percentage of pictures correctly identi ed as "already seen" (separately for the congruent and incongruent pictures) and "new". The experiment script and the stimuli used are available on the OSF platform.
Affective state. To measure the current affective state a short form of the Positive and Negative Affect Scale (i.e., I-PANAS-SF) developed by Thompson (2007) [ 34 ] was used in this study. It contains 10 items about different affective states (e.g., "Right now I feel attentive."), and the subject has to answer to each of them on a ve-point Likert scale from "not at all" to "extremely". The score of the scale is computed separately for positive and negative affectivity.
Emotional intelligence. The Emotional Intelligence Scale [ 35 ] was used to assess the subjects' emotional intelligence. The scale includes 33 items that should be rated using a ve-point Likert scale (1 = "I strongly disagree" to 5 = "I strongly agree"). An example item is as follows: "I know what other people are feeling just by looking at them."

Demographic variables
Age, gender, education, and musical background (i.e., musical status, years of music training) were assessed with a questionnaire

Music liking and familiarity
We asked participants to rate on a scale from 1 to 4 how much they liked the musical excerpts presented, as previous studies suggested that this variable can in uence emotional experience [ 36 ]. Furthermore, we assessed whether participants were familiar with the musical excerpts, as also familiarity seems to play a role in musical emotions experience [ 37 ].

Procedure
On the starting page of the Survey platform (LimeSurvey 2.64.1), participants were informed of the nature of the tasks. This was followed by an informed consent statement that participants had to agree to in order to proceed with the study. Then, demographic information was collected, followed by the emotional intelligence and the PANAS questionnaires. At the end of the questionnaires, participants were redirected to the platform Pavlovia, where the memory experiment began. The entire duration was around 25 minutes.
The current study was approved by the ethics committee of University of Innsbruck.

Results
To answer the main hypothesis, we conducted a one-way, repeated measures ANOVA with percentage of correct responses as the dependent variable, and the picture type (i.e., incongruent, congruent, new) as within-subject factor. Secondly, to understand whether the recognition of congruent pictures vs incongruent pictures could be affected by individual variables, affective state (i.e., PANAS scores), emotional intelligence, years of music training, gender, music familiarity and music liking were included as predictors in a multiple linear regression, with the difference in accuracy between congruent and incongruent pictures as dependent variable.
In the repeated-measures ANOVA, Mauchly's test indicated that the assumption of sphericity had been violated, χ2(2) = 12.06, p = .002, therefore degrees of freedom were corrected using Huynh-Feldt estimates of sphericity (ε = .95). The results showed that there was a signi cant effect of picture type, The impact of listener features as a whole was not signi cant, as shown by a non-signi cant model in the multiple linear regression, F(7,109) = 1.03, p = .415, predicting only the 0.6% of the variance. This nding indicates that the predictors did not in uence the direction of the results (i.e., higher recognition accuracy for congruent pictures than for incongruent pictures). The single predictors and their coe cients are reported in Table 1. Only the years of music training were a signi cant predictor (B= -0.85, SE = 0.38, p = .026). The more the years of music training, the smaller the difference in accuracy for congruent and incongruent pictures. Note. Dependent Variable: Accuracy difference between congruent and incongruent pictures. PANAS pos = positive affect score; PANAS neg = negative affect score; EI = Emotional Intelligence.

Discussion
The present study aimed to assess whether using a recognition memory task with congruent and incongruent emotional pictures could serve as an indirect behavioral measure of felt musical emotions. This assumption is based on the fact that attention and memorization are enhanced when the affective state of an individual matches of emotional nature of a stimulus [ 9 ]. Thus, if subjects feel emotions while listening to music, we expected them to have better recall of emotionally congruent pictures than incongruent ones in our recognition tasks. Two-hundred participants completed an online task, where they listened to 15 music excerpts, each followed by emotional congruent pictures, incongruent pictures, and neutral ones. After this part, participants performed a recognition task, with 50% of old pictures and 50% of new pictures. Results indicated that participants were more accurate in recognizing the emotionally congruent pictures than the emotionally incongruent ones. In what follows, we will rst discuss the implications of these results for the distinction between felt and perceived musical emotions, then for mood-congruency theory, and nally with respect to the individual variables assessed.
First, the present ndings suggest that using a paradigm based on mood-congruency effects could be an alternative behavioural method for assessing music-evoked emotions. In fact, we could observe moodcongruency effects with music (i.e., higher memory accuracy for emotionally congruent pictures than for emotionally incongruent ones), suggesting that our participants were able to feel the different emotions conveyed by the music (i.e., tension, sublimity, and vitality). The fact that we used a categorical approach here (i.e., by having three different emotional categories) further suggests that mood-congruency effects go beyond a general distinction between positive/negative arousal and are, to some extent, speci c to the emotion felt. This is in line with previous ndings showing that, when listeners indicated to have felt emotions belonging to four different GEMS-categories (sublimity, vitality, tension, and sadness), the feelings tended to correlate with emotion-speci c brain activation patterns [ 8 ].
Secondly, our ndings are among the rst to show that mood congruency effects occur in response to musical emotion induction as opposed to the more commonly used non-musical mood induction procedures. Previous studies often used different techniques, such as asking participants to write about a positive or negative emotional event that happened in their lives [ 9 ], presenting emotional faces [ 38 ], giving participants a bag of candies [ 13 ], or performing hypnosis [ 39 ]. Alternatively, instead of inducing an emotional state, some studies simply recorded the actual mood of the participants at the moment of the experiment [ 12 ]. The present results thus support the use of music as a mean to induce emotions in studies on mood-congruency effects. Moreover, music has the bene t of being an easy, enjoyable approach, and an objective stimulus (i.e., everybody listens to the same music).
Finally, the fact that our regression model was not signi cant indicates that current mood, gender, music training, emotional intelligence, music liking and familiarity did not in uence the superior memory for emotionally congruent pictures over incongruent ones. These results are interesting in light of previous studies on music-evoked emotions. While some ndings showed that certain personality traits, such as empathy, musical expertise or mood lead to stronger and more differentiated emotions [ 22,23,30,40 ], in some cases these differences did not emerge [ 41 ]. It is possible that, in our case, either the assessed variables or the induced emotions did not vary substantially across subjects; in fact, the music excerpts we chose were already selected because of their ability to induce certain emotions (assessed with selfreport questionnaires and/or with brain imaging, see [ 8, 31 ]). However, it is also possible that moodcongruency effects are robust and happen despite these individual differences. In support of this, Mayer and colleagues (1990) [ 42 ] found that the strength of the emotions felt after a mood induction had no in uence on mood-congruency effects (i.e., both strong and weak emotions led to emotion-congruent content being perceived and remembered more readily). Despite the regression model being nonsigni cant, the single predictors' coe cients showed that years of music training appeared to modulate slightly memory performance for congruent and incongruent pictures. However, the direction of this modulation was not the one expected. As musicians experience more intense emotions in music than non-musicians and amateur musicians [ 22 ], we expected years of music training to increase a possible mood-congruency effect. However, the years of music training were negatively related with the accuracy difference between congruent and incongruent pictures. In other words, the more the years of music training, the less the difference between accuracy for congruent and incongruent pictures. A possible explanation is that musicians seem to have better memory skills than nonmusicians [ 43 ] so this might compensate any mood-congruency effect in remembering emotional material. However, further evidence would be needed to test this, as in the present study the effect of music training was only marginal.
Limitations. The experiment was conducted online, therefore we had no control on the environment in which the participants found themselves. Nevertheless, the high rate of accuracy with new pictures suggest that participants were actually engaged in the task. However, future studies might want to replicate the current results in a more controlled environment. Secondly, in the present study we used only classical music, and results cannot be generalized to other music genres. Finally, we could not look at possible different effect strength depending on the emotion type, and future studies should provide more trials per emotion type to understand whether mood-congruency effects are stronger with speci c emotions (e.g., with positive emotions, [ 44 ]).
To conclude, using mood-congruency effects could thus be a way to assess behaviourally if music can induce speci c emotions. Moreover, researchers interested in studying mood-congruency effects could choose to use music to induce speci c emotional states. Finally, these ndings can be also relevant in the clinical eld, for instance, with mood disorders: future studies could uncover whether these results are also replicable with dysphoric and/or depressive patients. If "happy" music can induce happy feelings and in uence consequent cognitive processes, it could become a helpful tool when treating these disorders.