Experiment 1
In order to first assess the influence of contour features on valence ratings of a set of photographs, we performed a multiple linear regression analysis, predicting valence ratings of each IAPS photograph from the contour features of the traced IAPS line drawings. The line drawings were produced by artists at the Lotus Hill Research Institute (People’s Republic of China) by tracing the original IAPS colour photographs using a custom graphical user interface. The artists were instructed to trace all important lines so that if a human observer were to look at the line drawings, they would be able to recognize the image (i.e., the scene being depicted and the objects within the scene). Using these line drawings, contour properties (orientation, length, and curvature) were computed from the geometrical information of the line drawings. See Figure 1 for an example image and its feature histograms. Each histogram was computed in eight bins, each bin corresponding to a particular range of values. All feature values included in the regression were square root transformed to reduce outliers.
The multiple linear regression analysis yielded a significant regression equation (F(22,1159) = 2.103, p = 0.002), with an R2 of 0.04. The significant predictors of valence ratings were Curvature 8 (i.e., number of pixels that belong to angular contours), β = -0.08, p = 0.02, Length 7 (i.e., number of pixels that belong to long contours), β = 0.01, p = 0.007, and Orientation 5 (i.e., number of pixels that belong to vertical contours), β = -0.03, p = 0.001.
Figure 2a shows that as the number of pixels that belong to angular contours increases, valence ratings decrease. Figure 2b shows that as the number of pixels that belong to long contours increases, valence ratings increase. Finally, Figure 2c shows that as the number of pixels that belong to vertical contours increases, valence ratings decrease.
The IAPS images are a protected image set and are therefore not allowed to be published in any capacity. However, the table below (see Table 1) contains the one-word descriptions (as provided in the IAPS database) of the top and bottom three images containing the greatest and the least, respectively, of each of the significant features with their valence scores. For example, the image containing the smallest number of pixels that belong to vertical contours) depicts a mountainous landscape, while the image containing the most of that feature is a checkerboard pattern. Note that these are not necessarily the images rated the highest and lowest in valence, though the landscape image did receive a higher valence score (7.29) than the checkerboard pattern image (5.16).
Table 1: The image descriptions and valence scores of the three images with the least and three images with the greatest number of pixels corresponding to the significant features shown in Figure 2.
|
Bottom three
|
Top three
|
High angularity
|
Dog
7.01
|
Coyote
6.27
|
Veiled Woman
5.56
|
Rug
5.06
|
Mutilation
1.45
|
Chicken
6.19
|
Long lines
|
Snake
4.26
|
Snake
3.80
|
Snake
3.70
|
Violin
6.5
|
Money
7.51
|
Agate*
5.26
|
Vertical orientation
|
Mountains
7.29
|
Abstract Art
4.97
|
Puzzle
5.40
|
Jail
3.73
|
City
6.05
|
Checkerboard
5.16
|
* Agate is a type of gemstone.
|
The results support the hypothesis that at least some image features (i.e., contour features) predict valence ratings of photographs. The presence of long contours is positively related to valence ratings, while the presence of vertical contours and angular contours is negatively related to valence. A particular image which contained many high curvature (i.e., high angularity) contours depicted a disturbing scene of mutilated and burned corpses. Images containing few high curvature contours depicted more positive stimuli, such as a dog or a beautiful veiled woman. A similar finding is observed when looking at the Orientation feature. Images containing few vertical contours depicted natural landscapes or abstract art, while those high in vertical contours depicted more cluttered scenes such as a city street, or a prison cell with large vertical bars.
A striking finding when looking at individual exemplars of the Length feature (see Table 1) is the fact that the three images with the fewest long contours all depicted snakes. This is somewhat surprising when considering that snakes, in their simplest form, are described as long curvilinear lines5. We find that, in these more realistic photographs of snakes, there are actually many short triangular contours, corresponding to the scales of the snake. This suggests that researchers may have to rethink their depictions of snakes when studying the visual features that influence snake detection. By focusing on the rapid detection of curvilinearity as a proxy for rapid snake detection, researchers may be inadvertently simply showing a fluency effect (i.e., that smooth curves are processed more quickly than sharp angles)7,28,29 rather than a snake effect.
Experiment 2
The above results reveal that the relationships between particular contour features and participants’ valence judgements exist. However, the percent of explained variance is fairly low, and several confounding factors may be driving the significant results. For example, the line drawings were extracted from the photographs via artists’ tracings. It could be the case that artists imposed their own biases when tracing the photographs, highlighting certain contours and ignoring others depending on the emotional valence of the photograph. We do not think it is a problem, since any computational edge detection algorithm would also fall victim to biases depending on its parameters. More importantly, the results found in Experiment 1 may suffer another confound: they are likely biased by the semantic content available in the images, and the fact that, for the most part, there is still a central object of focus in each image. Therefore, in experiments 2a and 2b, we sought to explore the link between visual features and emotions using scene-like stimuli that were not confounded with semantic content or the biases of artists tracing photographs in a potentially biased manner. In two online experiments, participants viewed abstract, content-free line drawings of scenes artificially generated so that they adhere to particular statistics of their visual features (see Figure 3). On each trial, four such images were displayed at once and participants were asked to rate one image as the most positive and another as the most negative (Experiment 2a; affective valence judgement) or as the most threatening and safe (Experiment 2b; threat judgement).
To assess the influence of contour features on positive and negative judgements of abstract line drawings, we performed a logistic regression (using glmer from the “lme4” package30 in R31, modelling valence judgements (positive = 1, negative = 0) as a function of Length (short, long), Curvature (low, medium, high), and Orientation (horizontal, vertical, both, diagonal), as well as the interactions between these variables, and trials nested in participants as random effects. The neutral images (i.e., those not selected as positive/safe or negative/threatening) in each trial were not included in the model.
Experiment 2a
Using likelihood-ratio tests, as implemented in the afex package32 in R, we found main effects of Orientation, χ2(3) = 1242.85, p < 0.001, Curvature, χ2 (2) = 1310.83, p < 0.001, and Length, χ2 (1) = 643.38, p < 0.001. These main effects indicate the following: 1) images containing horizontal contours are more likely to be rated as positive than images containing contours of other orientations (i.e., vertical, both horizontal and vertical together, or diagonal contours), which are more likely to be rated as negative; 2) images containing contours with low curvature (i.e., smooth/flat contours) are more likely to be rated as positive than images containing highly angular contours, which are more likely to be rated as negative; and 3) images containing long contours are more likely to be rated as positive than images containing short contours, which are more likely to be rated as negative.
In addition to the main effects, we found a significant interaction between Orientation and Curvature, χ2 (6) = 254.59, p < 0.001, where images containing smooth (i.e., low curvature) contours are more likely to be rated as positive than images containing medium and high curvature contours across all orientations, except horizontal. In other words, if the high and medium curvature contours have an average horizontal orientation, those images will be rated as more positive than if the contour orientations are not horizontal. There was also an interaction between Orientation and Length, χ2 (3) = 9.76, p = 0.020, where images that contain long contours are more likely to be rated as positive than images containing short contours, but this difference is minimized for images that contain horizontally oriented contours. Finally, there was also a significant interaction between Curvature and Length, χ2 (2) = 13.96, p < 0.001, where images that contain long contours are more likely to be rated as positive than images containing short contours, and this difference is slightly larger when the contours have low curvature compared to when they have medium or high curvature.
Finally, the three-way interaction was significant, χ2 (6) = 57.89, p < 0.001, where the length effect is minimized when contours contain medium curvature, and the orientation effect is minimized when contours contain low curvature (see Figure 4). According to these findings, an image containing short, highly angular, non-horizontal contours would be most likely to be rated as negative (for example, see Figure 3 top row, third, fifth, and seventh image from the left, marked in red), while an image containing long, low angularity, horizontal contours would be most likely to be rated as positive (see Figure 3 bottom row, second from the left, marked in green).
Experiment 2b
So far, we have explored these visual features in relation to general valence judgements, rather than threat judgements specifically. Thus, Experiment 2b explicitly asked participants to judge threat and safety. The experiment was identical to Experiment 2a, except that participants were asked to indicate which of the displayed images they found to be the most threatening and which the safest. To assess the influence of contour features on safety and threat judgements of abstract line drawings, we performed another logistic regression, as in Experiment 2a. We modelled threat judgements as a function of Length, Curvature, and Orientation, as well as the interactions between these variables, and trials nested in participants as random effects.
Using likelihood-ratio tests, we found that all main effects and interactions were significant. Firstly, the main effect of Orientation, χ2 (3) = 2366.10, p < 0.001, indicates that images containing horizontal contours are more likely to be rated as safe than images containing contours of other orientations (i.e., vertical, both horizontal and vertical together, or diagonal contours), which are more likely to be rated as threatening. The main effect of Curvature, χ2 (2) = 1940.16, p < 0.001, indicates that images containing contours with low curvature (i.e., smooth/flat contours) are more likely to be rated as safe than images containing highly angular contours, which are more likely to be rated as threatening. Finally, the main effect of Length, χ2 (1) = 4410.36, p < 0.001, indicates that images containing long contours are more likely to be rated as safe than images containing short contours, which are more likely to be rated as threatening.
The significant interaction between Orientation and Curvature, χ2 (6) = 466.40, p < 0.001, shows that images containing smooth (i.e., low curvature) contours are more likely to be rated as safe than images containing medium and high curvature contours across all orientations, except horizontal. The interaction between Orientation and Length, χ2 (3) = 29.75, p < 0.001, indicates that the effect of orientation is more pronounced when images contain short contours compared to long contours. On the other hand, the interaction between Curvature and Length, χ2 (2) = 136.44, p < 0.001, shows that the effect of curvature is more pronounced when images contain long contours compared to short contours.
Finally, there was a significant three-way interaction, χ2 (6) = 110.50, p < 0.001, where the length effect is minimized when contours have medium and high curvature and certain average orientations, and the orientation effect is somewhat minimized when contours have a low curvature. Similar to the results of Experiment 2a, these findings indicate that an image containing short, highly angular, non-horizontal contours would be most likely to be rated as threatening, while an image containing long, low angularity, horizontal contours would be most likely to be rated as safe (see Figure 5).
Free Responses
Following Experiments 2a and 2b, participants had the opportunity to give free responses to the questions “Which features do you typically associate with positive/safe images?”, and “Which features do you typically associate with negative/threatening images?”. All participants’ written responses can be found on our OSF page (https://osf.io/y3rjm/)
Consistent with rating results, participants mention features such as “straight” and “smooth” when describing which features they associate with positive or safe images, while using words such as “sharp” and “jagged” when describing features they associate with negative or threatening images. Additionally, only in the positive/safe responses do participants describe that images that looked like “landscapes” were associated with their decisions. Conversely, “abstract” or “meaningless” are only mentioned in relation to the negative/threat responses. These qualitative responses suggest that the features cueing scene structure (long horizontal lines, e.g., the horizon line) are related to the same features that influence valence and threat judgements.