How Observer Characteristics Affect How They Infer Personality Impressions of Faces

Many previous studies on facial impression inference have focused on the physical features of the face. However, few have considered the effect of the observer on impression inference. Further, many studies have treated observers’ characteristics with regards to facial impression inference as errors or individual differences and thus largely excluded it from the explanatory variables. The present study, on the other hand, focused on the observers’ characteristics and examined whether the observers’ personality traits and observational behaviors influence the impression inferences of the faces. Experiment 1 examined the relationship between observer characteristics (observational behaviors and personality traits) and facial impression inferences. We found that the observers’ personality traits strongly influence observational behaviors, but neither of those influences the impression inferences. Using hierarchical Bayesian models, we found a large proportion of faces’ random effect, confirming many previous studies that facial features indeed affect how the faces are seen. The results of Experiment 1 indicate there might have been an interactive effect of observational behaviors and faces’ physical features on impression inferences. To control the potential interactions, we instructed participants to look at particular areas of faces during impression inference tasks in Experiment 2. Although participants generally looked at the areas instructed to look at, we still found a robust relationship between participants’ personality traits and observational behaviors. The results show that participants inferred different impressions, even for the same faces, when they were instructed to look at different areas of faces. We also found significant relationships between participants’ personality traits and impression inference ratings, suggesting people with different personality traits would have different impressions for the very same faces. Introduction People are sensitive to human faces as they can recognize them more quickly and accurately than those of other animals. Face recognition is very important in many aspects of our everyday lives, and thus research on the human face has been actively conducted for a long time. For example, evolutionary preferences show that those with attractive faces were more likely to be chosen as spouses . Likewise, some studies showed that highly attractive faces were shown to be recognized faster than ones with lower attractiveness, while people look at more attractive faces longer. It has shown that people often use other individuals’ facial features to predict and judge their personality traits and other characteristics. Perceptions of other individuals in social and professional activities such as deciding who to vote for ("trustworthy" face), job hunting ("capable-ofworking" face), and being able to run a business ("competent" face) are also strongly influenced by facial features. Studies that analyzed facial features using advanced techniques, such as computational models and machine learning, partly clarified relationships between faces’ physical features and impression judgments. Although face studies appear diverse, most of them have a great deal in common as they all focus on the physical features of faces. This series of research focusing on the features of facial parts and their configurations are built on the basis of an implicit assumption that faces with particular features give particular sets of impressions uniformly to observers regardless of the characteristics of the observer who sees the target face. Differences among observers are considered as errors, and these individual differences are left out from the explanation. Whether it is acceptable to exclude observer characteristics from the explanation remains. Only a small amount of research has been conducted on facial impression inference that focuses on observers’ characteristics. Their results suggest the problematic nature of the aforementioned research that assumes a particular face leads to more or less uniform impressions. For example, there is a cultural difference in how people look at others’ faces. While Asian people generally focus on the center of the face, Caucasian people tend to look at the eyes and mouth more frequently as compared to Asian . Other research shows that observational behavior varies depending on the observer’s age. Likewise, another study showed that the observer’s personality characteristics lead to different observational behaviors for different types of figures. Finally, it has also been shown that observers with different personalities look at others’ faces differently; the more extroverted, the more attention is paid to the eyes and mouth. Those results collectively suggest that different people may look at the same object differently. When an object is looked at differently (e.g., only limited areas are paid attention to), then the very same object may be perceived differently or result in different impressions. The present research examines this possibility. In particular, we examined the relationships between the observer’s personality traits, observational behaviors, and facial impression inferences. In so doing, we used hierarchical Bayesian models to quantitatively examine the mechanism of how personality traits and observed behavior interactively influence various facial impression inferences.


Introduction
People are sensitive to human faces as they can recognize them more quickly and accurately than those of other animals 1 . Face recognition is very important in many aspects of our everyday lives, and thus research on the human face has been actively conducted for a long time. For example, evolutionary preferences show that those with attractive faces were more likely to be chosen as spouses 2,3 . Likewise, some studies showed that highly attractive faces were shown to be recognized faster than ones with lower attractiveness, while people look at more attractive faces longer [4][5][6][7] . It has shown that people often use other individuals' facial features to predict and judge their personality traits and other characteristics [8][9][10][11][12][13][14][15][16][17] . Perceptions of other individuals in social and professional activities such as deciding who to vote for ("trustworthy" face), job hunting ("capable-ofworking" face), and being able to run a business ("competent" face) are also strongly influenced by facial features [18][19][20][21][22][23][24] . Studies that analyzed facial features using advanced techniques, such as computational models and machine learning, partly clarified relationships between faces' physical features and impression judgments [25][26][27][28] .
Although face studies appear diverse, most of them have a great deal in common as they all focus on the physical features of faces. This series of research focusing on the features of facial parts and their configurations are built on the basis of an implicit assumption that faces with particular features give particular sets of impressions uniformly to observers regardless of the characteristics of the observer who sees the target face. Differences among observers are considered as errors, and these individual differences are left out from the explanation. Whether it is acceptable to exclude observer characteristics from the explanation remains. Only a small amount of research has been conducted on facial impression inference that focuses on observers' characteristics. Their results suggest the problematic nature of the aforementioned research that assumes a particular face leads to more or less uniform impressions. For example, there is a cultural difference in how people look at others' faces. While Asian people generally focus on the center of the face, Caucasian people tend to look at the eyes and mouth more frequently as compared to Asian 29,30 . Other research shows that observational behavior varies depending on the observer's age 31 . Likewise, another study showed that the observer's personality characteristics lead to different observational behaviors for different types of figures 32 . Finally, it has also been shown that observers with different personalities look at others' faces differently; the more extroverted, the more attention is paid to the eyes and mouth. Those results collectively suggest that

3/16
skewed distribution. When fitting highly skewed variables, it is recommended to use beta regression, we consider this mixed distribution to follow the zero-inflated beta distribution (ZIB) 37 .  This experiment aimed to examine how participants' characteristics (personality traits and observational behavior) influence impression inferences. We developed and fitted a model described in Figure4. Our model considers that the participants' personality traits have two routes to affect facial impression inference, a direct effect and indirect effect through observational behaviors. That is, the participants' personality traits affect how they observe others' faces, which in turn affects how they infer impressions of the faces. Bayesian estimations were performed to see what sort of relationships among the variables exists and the validity of the model. Model building The zero-inflated mixed beta regression (ZIB) model assumes that the objective variable comes from two distributions, namely the Bernoulli distribution and the Beta distribution. The Bernoulli distribution is associated with whether participants look at the areas of interest at least once or not. The Beta distribution is associated with how much participants look at the area of interest.
Equations(1) through(6) iindicate our ZIB model describing the relationship between participants' personality traits and observational behaviors for a particular area (k), where i and j are indices for participants and stimuli, respectively. More specifically, Equations (1) and (2) model the observational behavior (G i jk ) which corresponds to if area k was looked at least once by observer i for face j, and if they did, how much the area was looked at. "Whether or not the area of interest was looked at" was modeled using the Bernoulli distribution (q i jk ), and implemented as a logistic regression model. "How much the area of interest was looked at" was modeled using the beta distribution (a i jk , b i jk ). The parameters for the beta distributions were further modeled as shown in Equations(4) and (5). Linear parts of our model indicate that the participants' personality traits (p il ) influence whether or not a particular area was looked at (Eq.(3)) and how much the area was looked at (Eq.(6)). We treated this sort of effect as the fixed effect (β Z for Bernuolli and β B for Beta), and the participants' individual differences (r Z ik and r B ik ) and individual differences in facial stimuli's (r Z jk and r B jk ) as random effects.
Equations(7) through (8) i indicate our ordered logistic regression model for personality impression inferences. An ordered logistic regression model is a model where the objective variable is on an ordinal scale. There were two types of predictor variables in this model. One was the observational behaviors estimated in Equation(1) which were influenced by the participants' personality traits. Note that we also included 2-way and 3-way interaction terms of observational behavior in Equation (8). The other type was participants' personality traits. Those two types of predictor variables were assumed to have fixed effects, while there were two random effects, one for participants (r R i ) and other for faces (r R j ).
The Rstan package was used for the parameter estimations [38][39][40] . All parameters appear in Equations(1) to (8) were estimated simultaneously. The prior distribution of fixed effects followed the normal distribution with mean of 0 and a variance of 100, and the prior distribution of random effects followed the gamma distribution (α = 10, β = 10). Each model was executed with the default stan hyperparameter values; the number of chains were 4; the number of thins were 1; the number of iteration steps were 2000; and the number of warm-up steps were 1000. The number of MCMC samples obtained was 4000.
In order to confirm whether the MCMC estimations had converged, we calculated Rhat (R) for each parameter, which is often used as judgment index for convergence. As in typical MCMC estimation, we consider estimations had "convergence" when the number of chains was greater than or equals to three andR is less than 1.1 for all parameters. Based on these criteria, all parameter estimation was confirmed converged.

Experiment 2
To eliminate plausible interaction between facial features and observational behaviors, we instructed participants to look at particular areas of faces during impression inference tasks in Experiment 2. In particular, we simply instructed participants to look at either eyes, nose, or mouth, depending on experimental conditions. Restricted observations would weaken the interactive effects of facial features and observational behaviors, because, within a condition, all participants would exhibit the same if not similar observational behaviors, observing the same set of facial features. In doing so, we could also examine the effects of participants' personality traits on impression inference as participants with different patterns of personality traits would observe at the same set of facial features within conditions, which cannot be attained if they could observe the faces freely. In addition, we could also examine the effects of observational behaviors on impression inference as all participants within a condition would observe the same set of facial features regardless of their personality traits.All participants provided a written, signed informed consent. This experiment is reviewed and authorized by Chiba University Research Review Institute (authorization #202012-1). All methods were performed in accordance with the relevant guidelines and regulations. Participants The participants were 103 students with normal vision (including vision correction) from Chiba University. They were randomly assigned into one of three experimental conditions, namely Eye, Nose, and Mouth conditions. Among them, 34 participants were in Eye condition (17 female and 17 males, mean age being 21.7 years with sd. = 2.9), 34 were in Nose condition participants (23 female and 11 males, mean age being 21.3 years with sd. = 2.7), and remaining 35 were Mouth condition (20 females and 15 males, mean being was 22.8 with sd. = 4.6). A 500-yen gift certificate was provided to all participants as a reward for participating in the experiment.
The stimuli, impression inference tasks, and apparatus were identical to those of Experiment 1. The experimental procedure was also the same as in Experiment 1, except instructions reminding which condition they belonged were presented at the beginning of each session. In addition, the same data preprocessing was applied to the data obtained in Experiment 2. This experiment is reviewed and authorized by Chiba University Research Review Institute: authorized 202012-1. Data analysis and Manipulation check In order to verify whether the gaze manipulation worked as we planned, we first created a heat map of observational behavior for each experimental condition. As shown in Figure5, the observational behaviors in each condition were properly concentrated on the eyes, nose, or mouth, depending on experimental conditions. In addition, we used ZIB model to quantitatively test whether the observational behavior of each condition group was properly manipulated. ZIB model used here was almost identical to that of Experiment 1, except that its predictors were experimental conditions but not personality traits. Figure6 shows 95% HDI of posterior distributions of pair-wise comparison between experimental conditions and the control, which was the data obtained in Experiment 1 where participants observed the faces without any restriction. As shown in Figure6, participants in Eye condition observed the eyes significantly more than those in the control condition (i.e., Free condition). The same was true in other conditions, suggesting our manipulation in observational behaviors was worked as we planned. Comparing impression inferences among experimental conditions We compared whether participants in different experimental conditions had different impressions of the facial stimuli. In so doing, we run hierarchical Bayesian ordered logistic models similar to that of Experiment 1. Instead of using personality traits and observational behaviors, we used experimental conditions (including Free condition which was data obtained in Experiment 1) as the predictor variables as shown in Equations(9) and (10). As shown in Equations. (9) and (10), we used four conditions (C id in Equation (10)) as the fixed effects, and participants (r sub jO i ) and facial stimuli (r picO j ) as the random effects.
Figure7 shows pair-wise comparisons between the control and three experimental conditions, suggesting participants in different experimental conditions indeed had different impressions for the identical set of facial stimuli. That is, even for the same faces, participants acquired different impressions when they were instructed to look at particular areas of the faces. The effect was strongest when inferring stimuli's degree of extraversion -the absolute difference in impression ratings between the control and experimental conditions were largest in Nose condition. Observational behaviors and impression inference within conditions Previous analysis suggested that different observational behaviors led to different impression inferences of the faces. We now analyze how the differences emerged by analyzing relationships among participants' personality traits, observational behaviors, and impression inferences within each experimental

Results of Experiment 1
Our model consists of two sub-models. One is ordinal logistic models with the impression inferences ratings as the objective variables. The other is zero-inflated beta distribution (ZIB) models with the observational behaviors as the objective variable. For the sake of simplicity, we describe their results separately.
Table1 (Bernoulli) and Table2 (Beta) show the results of ZIB models. Only the posterior distributions of parameters whose 95% highest density interval, which is generally considered to be similar to 95% confidence interval in frequentist approach, did not include 0 were shown in these tables. Let us state that an effect was "significant" whenever the HDI of the corresponding posterior distribution did not include 0.
As shown in Table1 and 2, the observational behaviors were strongly influenced by participants' personality traits. The results can be summarized as follows: When assessing agreeableness, we found that those who were high in extraversion tended to look at the eyes (Table 1). In addition, more extraversion participants tended to look at the eyes and nose or eyes and mouth within the same session (i.e., positive interaction). Those who were high in openness tended not to look at the eyes, nose, nor mouth. When assessing conscientiousness, we found that those who were high in neuroticism tended to look at the nose. On the other hand, those who were high in openness tended not to look at the eyes, nose, nor mouth. When assessing extraversion, we found that those who were high in extraversion tended to look at the eyes. In addition, they also tended to look at the eyes and mouth within the same session. We also found that those who were high in agreeableness tended to look at the nose. In addition, they also tend to look at the eyes and nose within the same session.
When assessing neuroticism, those who were high in neuroticism tended to look at the eyes and nose. They also tended to look at the eyes and nose within the same session. We found that those who were high in conscientiousness tended not to look at the mouth. Additionally, high openness participants tended not to look at the eyes, nose, nor mouth. When assessing openness, those who are high in openness tended not to look at the eyes nor mouth.
We counted the numbers of significant random effects of both participants (r Z ik , r B ik ) and faces (r Z jk , r B jk ) in the ZIB models. Table 3 and Table 4 summarize the proportions of significant random effects. Overall, about 10.1% and 4.0% of participant and face random effects, respectively, in the Bernoulli part of ZIB were significant. Those for Beta were 5.8% and 0.04%. The small overall proportion of face random effects in the ZIB models indicates that certain facial features did not necessarily attract particular observational behaviors, confirming that observational behaviors can be significantly accounted for by participants' personality traits. Table 5 shows the results of the ordered logistic models. The results show that participants in high agreeableness tended to infer the faces were more agreeable. It turns out that faces were perceived as less conscientious if participants looked at the nose. These results suggest that the participants' personality traits and observational behavior significantly influence on personality impression inferences of faces. However, their effects were somewhat small.
We then counted the numbers of significant random effects for both participants and facial stimuli in the ordered logistic models. Table 6 summarizes the proportion of significant random effects. Overall, about 3.0% and 44.0% of participants and face random effects, respectively, were significant. These effects were above and beyond participants' personality traits and how they observed at the faces. The large overall proportions of random effects of faces indicate that certain facial features indeed led to certain impression inferences, confirming previous research that focuses on the effects of physical features on impression inferences. On the other hand, the small overall proportion of participant random and fixed effects indicate that participants' characteristics may not be associated with how they infer personality impression.

Results of Experiment 2
Tables 7 through 10 show significant personality predictors for observational behaviors for Eye (Tables 7 and 8), Nose (Table 9 and 10) conditions. Our manipulation check showed that participants, in general, looked at the facial areas where they were instructed to look at. However, we found significant personality effects on observational behaviors in Eye and Nose conditions. No significant effect was found in Mouth condition.

Eye condition
When inferring faces' conscientiousness, participants with a lower degree of openness tended not to look at the eyes even if they were instructed to look at the eyes. Participants with a lower degree of agreeableness (higher degree of extraversion) tended (tended not) to look at the mouth in Eye condition. We also found that participants with a lower degree of conscientiousness tended to look at the nose.

Nose condition
When inferring agreeableness of the facial stimuli, participants with a higher degree of agreeableness tended not to look at the mouth in Nose condition. When assessing conscientiousness, we found that participants with a higher degree of openness (neuroticism) tended (tended not) to look at the eyes, while those with a higher degree of extraversion tended to look at the

11/16
nose. When assessing neuroticism, we found that participants with a higher degree of agreeableness and/or openness tended not to look at the mouth. We also found that participants with a higher degree of openness tended not to look at the mouth.

Impression inferences
Tables 11 -13 summarize the significant effects of participants' characteristics on impression inferences for each condition. Given that within each condition, participants, in general, observed the facial stimuli in a very similar manner, there was only one significant effect of observational behaviors on impression inferences, namely looking at the nose affects conscientiousness inference in Mouth condition. There were several significant effects of participants' personality traits on impression inferences within conditions. Overall, the effects of personality traits were more noticeable than those without manipulation (Exp.1).
In particular, when participants were instructed to look at the eyes, the effects on personality traits were more noticeable. For example, when participants were instructed to look at the eyes, then those with a higher degree of neuroticism tended to rate the facial stimuli lower in their agreeableness and extraversion. Those with a higher degree of agreeableness and/or conscientiousness tended to rate the faces lower in agreeableness. In Mouth condition, those with a higher degree of openness tended to rate the faces higher in openness.

Discussion
The present research examined the effects of the participants' personality traits and observational behaviors on the impression inference of human faces. We collected observational behavior data using an eye-tracker, which were processed with a Gaussian filter and then normalized for each area of interest, namely eyes, nose, and mouth. We use the Japanese version of Ten-Item Personality Inventory to measure each participants' Big Five personality traits, which were also the personality traits we asked participants to rate for the faces they saw in the experiments. The results of Experiment 1 showed that there were robust relationships between participants' personality traits and observational behaviors, suggesting that participants with particular sets of personality traits tended to look at particular areas of the faces. However, there were weak relationships between participants' personality traits and impression inferences as well as between observational behaviors and impression inferences, suggesting that participants' characteristics did not affect how the faces were seen. Detailed model analyses showed that while the effects of individual differences of participants on observation behaviors were more substantial than those of faces, the effects of individual differences of faces on impression inference were more substantial than those of participants. In other words, in inferring faces, facial features strongly affect their impression, while participants' individual differences did not. On the other hand, observational behaviors were not influenced by the facial features (i.e., particular facial features did not necessarily attract particular observational behaviors) but by participants' personality traits. We hypothesized that the weak effects of participants' characteristics on impression inferences in Experiment 1 might have caused by a potential interactive effect between facial features and participants' observational behaviors. Experiment 2 was conducted in order to control this potential interaction by manipulating participants' observational behaviors. In particular, we asked participants to look at either eyes, nose, or mouth during impression inference tasks. The results of the manipulation check confirmed participants mostly looked at the areas where they were instructed to look at. The pair-wise comparisons confirmed that looking at the different areas of faces indeed led to different impression inferences, suggesting simply looking at the different areas of faces, regardless of individual differences in facial features, leads to different impressions. These results suggest, for example, regardless of the sizes and/or shapes of the eyes, simply looking at eyes makes people feel that a face has a higher degree of openness. This interpretation seems counterintuitive, but for the following reasons, it may be sufficiently reasonable. A study that examined the relationship between self-control and mindset showed people exhibit more self-control, manifested as gaze control, during abstract than concrete thinking tasks 41 . In other words, it is harder for humans to think concretely while controlling their gaze or vise versa. In Experiment 2, we manipulated participants' observational behavior limiting their self-control, which in turn altered their thinking mindsets as compared to those in Experiment 1. In addition, it is well known different areas of the face are associated with different emotions. The upper area of the face is associated with anger, fear, surprise, and sadness, while the lower area is associated with disgust and happiness 42 . Looking at particular areas of the face as instructed reminded them of a particular emotional state, leading to different impression inferences. Similarly, looking at particular areas remind them of particular actions. For example, looking at the mouth made participants imagine "talking," causing them to infer that the face had a higher degree of openness. The other potential explanation is that it was not looking at particular areas that led to different impressions but looking at face freely cancel out the effects of facial areas leading to the "average" impression inferences.
Our detailed analyses showed significant effects of personality traits on impression inferences when observational behaviors were manipulated. It seemed that the effects of observational behavior and personality traits, which influenced each other alternately in typical circumstances, were remarkably manifested by gaze manipulation. At the same time, as compared to Experiment 1, the relationships between personality traits and observational behavior were weakened in Experiment 2. This is probably because gaze manipulation restricted observational behaviors. Nevertheless, the robust relationships between observational behavior and personality traits existed even when the gaze was manipulated.
The present research showed that the participants' personality traits had a considerable impact on how they observed facial images. Although it is not of our interest in the present study, it is perhaps very interesting to examine the relationship in the opposite direction -can participants' personality be predicted or explained by how they observe the facial images? -in future research. In addition, utilizing time-series data, which we did not conduct here, may provide richer or more robust results and is another exciting future research. Furthermore, instead of manipulating observational behaviors or using personality traits as predictors, one would like to examine whether participants' mood, state of mind, or attitude can also affect the impression inference. Given that the very same individuals exhibited different observational behaviors in different impression inference tasks (e.g., inferring openness or neuroticism), great care needs to be paid in designing stimuli, tasks, and experiments.

Data availability
Our datasets, stan model and R code generated during this study are available on OSF at https://osf.io/bn93u/