Disentangling the contribution of individual and social learning processes in human advice-taking behavior

doi:10.21203/rs.3.rs-2736136/v1

Download PDF

Article

Disentangling the contribution of individual and social learning processes in human advice-taking behavior

https://doi.org/10.21203/rs.3.rs-2736136/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 20 Jan, 2024

Read the published version in npj Science of Learning →

You are reading this latest preprint version

The study of social learning examines how individuals learn from others by means of observation, imitation, or compliance with advice. However, it still remains largely unknown whether social learning processes have a distinct contribution to choice behavior, independent from non-social trial-and-error learning that often occurs simultaneously. 153 participants completed a reinforcement learning task, where they were asked to make choices to gain rewards. Advice from an artificial teacher was presented on 60% of the trials, allowing us to compare choice behavior with and without advice. Results showed a strong and reliable tendency to follow advice (test-retest reliability ~ .73). Computational modeling suggested a unique contribution of three distinct learning strategies: (a) individual learning (i.e., learning the value of actions, independent of advice), (b) informed advice-taking (i.e., learning the value of following advice), and (c) non-informed advice-taking (i.e., a constant bias to follow advice regardless of outcome history). Computational lesion simulations analyses provided specific regression signatures to both informed and non-informed advice taking processes that were then confirmed by empirical data. We discuss the theoretical implications of integrating internal and external information during the learning process.

Biological sciences/Psychology/Human behaviour

Biological sciences/Neuroscience/Computational neuroscience/Learning algorithms

Biological sciences/Neuroscience/Cognitive neuroscience/Decision

Two forms of learning are predominant in the process of understanding our environment. The first is an unguided trial-and-error learning process, whereby individuals learn from personal experience (henceforth individual learning)¹. The other form of learning falls under the broad definition of social learning, in which we learn by observing and imitating others, or by following explicit advice or instructions ^2–4. Social learning is considered one of the primary ways by which we learn, especially in humans but also in animals. Despite the distinction between these two forms of learning, it is not simple to decipher and disentangle the unique contribution of each of these processes in explaining human choice behavior. For example, imagine that a friend suggests that you order a particular dish at a restaurant. You might order the advised dish but, from an observer’s perspective, it is impossible to know whether this behavior was due to (a) your own personal prior experience with that type of cuisine irrespective of the advice, (b) the value you assigned to your friend’s advice because her advice panned out before due to her culinary knowledge, or (c) your tendency to comply with advice received from others, regardless of the specific context. Thus, to better understand human choice behavior, it is critical that we disentangle and estimate the influence of both social and non-social learning processes.

The literature on social learning strategies ^5,6 has dealt extensively with the influence of a single piece of advice on human choice behavior. For example, Biele et al. ⁷ demonstrated that participants have a general tendency to behave in accordance with a single piece of advice provided to them just one time. In their study, participants were shown to be more likely to choose a card from an advised deck of cards relative to a card deck with a similar positive value but for which they did not receive advice. Biele et al. ^7,8 suggested that individuals have a positive bias when evaluating the outcomes of recommended options relative to options that are not specifically recommended. Several studies support this assumption, having demonstrated that, remarkably, even a single piece of advice or instruction can influence and bias human choice behavior ^7–12. However, it is important to note that most natural environments consist of repeating socially transmitted information, where advice is given more than once, and often by the same individual.

Indeed, previous studies have suggested that individuals tend to keep track of and assess the value of repeating socially transmitted information. For example, one adaptive social learning strategy involves copying the choices of successful individuals, as the knowledge they hold is assumed to be more valuable than that of others ¹³. Furthermore, Behrens et al. ¹⁴ found that the associative neural mechanisms that are responsible for processing social information are similar to those at play during individual learning and are subject to updating and learning. Taken together, these studies suggest that agents engage not only in learning the value of their own choices, but also in learning the value of social information. However, it is not clear how these two processes interact and how one might influence the other. For example, Bonawitz et al. ⁹ demonstrated that pedagogical teaching (e.g., instructing about the specific use of a toy) limits children’s own exploration when the teacher is assumed to be knowledgeable. However, the same study also showed that children still engaged in exploration when they believed there was more to learn about the toy.

In a related line of research, researchers have explored the specific computational process that underlies individuals’ responses to repeated social advice. For example, Najar et al. ⁴ demonstrated that individuals perceive the choices of a demonstrator in a similar manner to how they perceive a positive reward. Hence supposedly, the advice acts as a driving force for behavior. In addition, they showed that only demonstrators who have proven to be knowledgeable are imitated, further supporting the idea that individuals consistently engage in a process of evaluating the information presented to them. Other works were able to demonstrate that the same reward circuits that are activated during individual learning are also activated when receiving advice in a decision-making situation ¹⁵. Moreover, Diaconescu et al. ¹⁶ suggested that people deliberate between social and individual information, while also accounting for the level of certainty or uncertainty they have in the information source. Finally, Rybicki et al. ¹⁷ exhibited support for the assumption that social and individual learning processes rely on similar neurochemical mechanisms (while also indicating the importance of the temporal primacy of the social information¹⁸). However, these studies used designs that make it difficult to disentangle the influence of different facets of social learning and individual learning on decision-making.

These studies improve our understanding of the processes by which individuals utilize one-time or repeated socially transmitted information when making choices. However, to the best of our knowledge, no study to date has examined whether a bias to follow advice (such as the one that was observed in one-time advice-giving paradigms) remains present in an environment in which advice is repeating. In the current study, we assumed that in a repeating advice-giving environment, wherein individuals are learning the values of different actions (e.g., whether they lead to a positive or a negative outcome), participants will behave in accordance with the advice given for one of three reasons. The first is due to individual learning, which reflects an internal value that was learned in a non-social manner, regardless of whether advice was provided or not. A second option is informed social learning, whereby the individual assigns value to an advised option due to a previous positive experience with the advisor. The third and final option is non-informed social learning, whereby the individual assigns value to complying with advice in general. Whereas individual learning and informed advice-taking often result from trial-and-error learning, non-informed social learning may be due to social norms, such as reciprocity or respect ^{11, 19–21}, or epistemic trust in “advisors”/”teachers”, people who are assumed to have relevant knowledge on which to base their advice ²².

In the current study, we aimed to disentangle, in a single reinforcement learning task, the influence of social learning from individual value-based learning. We introduced a novel experimental paradigm in which we randomly interleaved trials with and without advice to allow us to disentangle individual learning from social learning. We estimated individuals’ bias towards following advice in a reinforcement learning task in which advice was either revealed to or concealed from participants. Specifically, participants had to choose between two cards (from a deck of four) that were offered (i.e., a multi-armed bandit task; see Fig. 1). Cards led to monetary rewards ($0 or $1) probabilistically, and participants were asked to make choices to maximize their return. On some of the trials, an artificial teacher gave advice regarding which card should be chosen. Regression analyses demonstrated an influence of the teacher’s advice on participants’ choice behavior, such that participants were more likely to choose the card offered by the teacher when the advice was presented on screen. Furthermore, the tendency to follow advice showed good test-retest reliability, suggesting robustness across two time points. Of main interest in this work, we conducted formal computational models that mimicked the hypothesized cognitive mechanism and tested its fit to the observed data. We found evidence to suggest that participants’ behavior reflect a combination of non-informed social learning, informed social learning, and non-social individual learning. We then conducted computational lesion analyses to identify unique signatures for these processes using dedicated regression analyses. We conclude that individuals have a predominant bias to follow advice, when advice from a social agent is repeated and when individual learning occurs. We discuss the theoretical implications of integrating information learned from personal experience with external advice.

We examined the contribution of informed and non-informed advice-taking to participants’ choice behavior in a sequential reinforcement learning task (Fig. 1). In this section, we start by establishing a ‘reveal effect’ describing the causal influence of revealing the teacher’s advice to the participant on choice behavior (Fig. 2). This effect uniquely disentangles the influence of advice-giving from trial-by-trial independent learning, which was explored both at the group level and at an individual level (i.e., between sessions test-retest estimates). We then continue to describe the mathematical processes underlying individuals’ choice behavior. Specifically, we used computational modeling to estimate the contribution of three internal latent processes to participants’ choice behavior: (1) individual learning (i.e., learning from experience with the cards, independent of any advice), (2) informed advice-taking (i.e., trial-by-trial learning of the value of following the teacher’s advice), and (3) non-informed advice-taking (i.e., a fixed internal bias to follow advice regardless of choice and outcome history). Finally, we will show further support for the necessity of the two advice-taking processes. We used participants’ estimated computational parameters to simulate data. We independently lesioned participants’ informed or non-informed advice-taking tendencies and demonstrated unique regression signatures for these processes. Overall, we found strong and compelling evidence suggesting a causal effect of advice on participants’ choice behavior, both via informed and non-informed advice-taking processes.

Theory-independent analysis examining the influence of receiving advice on compliance. We first assessed whether participants demonstrated a bias towards advice-taking across trials and sessions. For this aim, we first calculated a coherence rate as the dependent variable to reflect whether participants chose the same card as the teacher (coded 0/1 when teacher and student selected two different/same cards, respectively), both when the teacher’s choice was revealed and when it was concealed. The coherence rate in the concealed condition served as a baseline, as it reflects the contribution of an individual learning process to the likelihood of agreement with the teacher’s choice, i.e., the likelihood of the participant reaching the same choice as the teacher independently. This is plausible when the participant has learned to identify the more valuable offer, which was often aligned with the choice of the teacher (concealed or revealed). This approach enabled us to examine the unique contribution of the teacher’s advice by comparing coherence rates across the revealed advice and concealed advice conditions, which we termed the reveal effect.

We therefore performed a hierarchical Bayesian logistic regression analysis, predicting participants’ coherence rates as a function of advice presentation (concealed/revealed). We found that coherence rates in the revealed advice condition were higher than the baseline rates in the concealed advice condition, suggesting a causal influence of advice on compliance. Specifically, participants were more likely to choose the card advised by the teacher when the teacher’s advice was revealed (66% coherence rate) compared to when the advice was concealed (48% coherence rate; posterior median estimation for the difference in the population was 0.29, HDI_89% = 0.25 to 0.33 probability of direction (pd) ~ 100% suggesting all of the distribution was positive; Fig. 2A/B].

To further substantiate the estimation of the causal influence of advice on participants’ choice behavior, we examined the test-retest reliability of the tendency to follow advice. Participants performed the task in two separate experimental sessions across two adjacent days (visual stimuli of cards and artificial teachers were changed across blocks and sessions). We performed a hierarchical Bayesian regression analysis and found good test-retest reliability (posterior median = 0.73, HDI_89% = 0.64 to 0.81; Fig. 2C/D). Overall, these results demonstrate a strong and reliable causal influence of receiving advice on individuals’ choice behavior in a reinforcement learning task during which advice was given repeatedly. However, these analyses did not allow us to disentangle the effects of informed from non-informed advice-taking. For this purpose, we turned to computational analyses that can explicitly model the contribution of different learning and decision-making processes.

Computational Modeling. We hypothesized that participants’ choice behavior integrates three sources of information: individual learning, informed advice-taking and non-informed advice-taking. We formulated a saturated model that included all three types of information and tested it against three nested models. Specifically, in these models we predicted the agents’ choices based on reward history and experimental conditions. For each trial, we updated the subjective values of the cards (Q-values) based on the prediction error signal²³. For clarity, we will describe the models we computed. The first model is a baseline model which only had an individual learning component (Model 1). We then added non-informed advice-taking (Models 2 & 3) and informed advice-taking (Model 4) to the models, and lastly, we computed a full model with all three components (Model 5).

Model 1 (null model)

This model assumes that participants learned the cards’ values from their own experience without considering the presented advice

(1) δ_{chosen_card} = (reward – Q_{chosen_card})

(2) Q_{chosen_card} = Q_{chosen_card} + α * δ_{chosen_card}

where α is a learning-rate (free-parameter) and δ_{chosen_card} represents the prediction error for the card selected by the agent (this equation is used in all models). Thus, this model ignores any advice revealed to the participant. In order to choose between the cards, we used a softmax policy:

(3) p(choice) =$\frac{exp(\beta \cdot {Q}_{chosen\_card} )}{\varSigma exp(\beta \cdot Qi)}$

where β is an inverse noise parameter (free parameter), and Q_i denotes the Q-values of each card offered in a current trial. Thus, this model had two population-level free parameters (α_{chosen_card}, β).

Model 2 (fixed non-informed advice-taking)

This model is similar to the baseline model only with an additional fixed bias in favor of teacher advice, when it was presented. When the teacher’s advice was presented, action values were calculated according to

(4) Qnet_{advised_card} = Q_{advised_card} + φ

(5) Qnet_{unadvised_card} = Q_{unadvised_card}

where φ is a free parameter (unrestricted and could be positive or negative) describing the individual tendency to follow advice regardless of any choice-outcome history during the task.

These Qnet values were then entered into the softmax:

(6) p(choice) =$\frac{exp(\beta \cdot {Qnet}_{chosen-card} )}{\varSigma exp(\beta \cdot Qne{t}_{i})}$

Overall, this model has three population-level free parameters (α_{chosen_card}, β, φ).

Model 3 (dynamic non-informed advice-taking)

Here, we augmented Model 2 so that on trials in which the participants’ decision was more difficult (the offered cards’ internal values were similar), they could increase non-informed advice-taking tendencies to further rely on external information. Specifically, this model assumes that the bias in Eq. (4) changes as a function of the difference between the Q-values of the two cards that are currently offered to the player.

(7) φ = φ_intercept + φ_slope* |ΔQ_cards|

φ_intercept serves as a general bias factor, and φ_slope serves as an additional slope parameter that is multiplied by the absolute difference between Q_{advised_card} and Q_{unadvised_card}. This model includes four free parameters: α, β, and two bias parameters (φ_intercept and φ_slope).

Model 4 (Informed advice-taking): This model assumes that instead of having a general preference for the advised card, participants evaluated the advice during the experimental block via Q-learning. The two options, to follow or not to follow the advice, were updated via Q-learning using a prediction error and learning rate (the same free parameter α from the other models), as shown below (Eq. (8) and Eq. (9)):

(8) δ_{follow_advice} = (reward – Q_{follow_advice})

(9) Q_{follow_advice} = Q_{follow_advice} + α *δ_{follow_advice}

Next, we calculated Qnet, which incorporated the Q-values of the cards and of following (or not following) advice, and weighted them using ω:

(10) Qnet_{advised_card} = ω*Q_{advised_card} + (1-ω)*Q_{follow_advice}

We note that the Q-values of the cards were updated using Eq. (1) and Eq. (2), and that the softmax decision function in this model is the same as in Model 2 (Eq. (6)). This model involves 3 free parameters: learning rate for the cards and for following advice (α), inverse noise parameter (β), and weighing between the Q-values of the cards and of following advice (ω).

Model 5 (informed and non-informed advice-taking): This model combines Models 2 and 4 and assumes both a general preference for the advised card, and an evaluation of the teacher through the reward history (Eq. (11)).

(11) Qnet_{advised_card} = ω*Q_{advised_card} + (1-ω)*Q_{follow_advice} + φ

Model fitting. We performed a model comparison using a leave-one-block-out approach, as described in the Method section. We calculated a difference distribution for each paired model comparison and estimated the elpd difference and the standard error of the difference distribution (using ‘loo’ package; with an elpd difference of 2 times the standard-error considered substantial²⁴). We found Model 5 to be the winning model (see Table 1). Parameter recovery is depicted in SI.

Table 1

Model comparison results – winning model compared to other models.
Model	Expected log probability difference compared to the winning model (Model 5 – informed and non-informed advice-taking)
Null model	-3938 (89.4)
Model 2 (fixed bias)	-189.5 (22.9)
Model 3 (differential bias)	-213.2 (22.7)
Model 4 (teacher evaluation)	-916.7 (59.8)
Note. Elpd (expected log probability density) was calculated using a leave-one-(block)-out cross-validation approach. An elpd difference that is larger than 4 and at least twice the standard error is considered to be significant evidence for the winning model ²⁴. Elpd difference standard errors are noted in brackets.

Estimated parameters. The estimated parameters for the winning model highlight the contribution of individual learning, informed advice-taking and non-informed advice-taking (Fig. 3). The ω parameter was estimated to be ~ 0.75 (Fig. 3C), indicating a reliance on individual learning, but also on informed advice-taking, which tracked the accuracy of the teacher. This value indicates that a highly accurate teacher could influence the decisions made by participants beyond individual learning, a finding that is in line with the reveal effect discussed above. Parameter φ was estimated to be ~ 0.2 (Fig. 3D), indicating a tendency to follow advice in a manner that is unrelated to the teacher’s accuracy. This fixed contribution is the basis of non-informed compliance and contributed to the reveal effect as well. This tendency increased the likelihood of following advice even when it went against one’s individual experience and/or when the teacher’s accuracy was low.

Associations between the winning model and empirical data. To demonstrate the association between the winning model parameters and the model-agnostic results, we estimated the individual bias to follow advice for each participant and for each simulated agent. To do so, we simulated artificial data with the same number of trials as the empirical data using individuals’ parameter estimation (we used the mean posterior for each individual and parameter). We then calculated and plotted the 'reveal effect’ from empirical data and artificial data for each subject based on the model and examined the correlation between them. We found a strong positive correlation (Pearson r = 0.89, pd ~ 100%; CI_89% = .85 to .91; Fig. 3E), showing that the model successfully replicated the behavioral results.

Computational lesion analyses. Thus far, our modeling results (Table 1) show clear evidence suggesting that individuals are using both informed and non-informed advice-taking mechanisms. Specifically, expected log probability density (elpd) suggested that using both informed and non-informed processes substantially increased our ability to predict left-out blocks. To illustrate the existence of both types of advice-taking in the empirical data, we additionally performed a lesion analysis. Here, we first simulated two data sets for each individual, based on the individual empirical parameters’ estimation. In the first set (non-informed advice-taking), we lesioned the ω parameter (i.e., ω was set to 1). In the second set, (informed advice-taking) we lesioned the φ parameters (i.e., φ was set to 0). This allowed us to examine the reveal effect in datasets that were artificially constructed using only one of the advice-taking processes at a time.

(1) Signature for non-informed advice-taking: We calculated the ‘reveal effect’ using only two trials for each participant, per block – the first time a conceal trial was presented and the first time a reveal trial was presented. We reasoned that during such an early stage in the task, informed advice-taking would not be able to produce a reveal effect. Note that each block included a novel teacher, with whom the participant did not have prior experience. For the empirical data, we found a substantial reveal effect (median = 0.62, CI_89% between 0.56 to 0.71, pd ~ 100%; see Fig. 4A). We found a similar effect in the non-informed artificial dataset (median = 0.30, CI_89% between 0.22 to 0.38, pd ~ 100%; see Fig. 4C). Importantly, we found evidence in favor of no reveal effect for the informed advice-taking artificial data set (median = -0.01, CI_89% between − 0.09 to 0.07, pd = 59.19%; see Fig. 4B).

(2) Signature for informed advice-taking: To capture a unique signature for informed advice-taking, we examined the influence of outcome on choice behavior. Specifically, we calculated a coherence repeat rate dependent variable, which reflected whether participants exhibited the same behavior (following the teacher’s advice or not) across two consecutive trials, n and n + 1 (coded 0/1 for different/same behavior, respectively). We then used Bayesian logistic regression analyses to predict coherence repeat as a function of previous outcome (rewarded vs. unrewarded) and advice presentation (concealed vs. revealed on both n and n + 1 trials), as well as their paired interaction. Note that we did not include trials in which advice presentation was different across n and n + 1 trials to allow for a more precise comparison. We reasoned that the interaction of previous outcome X advice presentation would be present only for informed advice-taking. Specifically, when advice is presented, a reward on trial n should increase coherence on trial n + 1 compared to unrewarded n trials (as can be seen in the updating of the Q_{follow_advice} value in the winning Model 5). However, in concealed advice trials, the value for taking/rejecting advice should not be updated, and so the reward on trial n should not affect coherence rates. Note that we also added teacher accuracy in the n trial as a fixed effect only with no Interaction, allowing us to control for overall coherence baseline rates which might change with teacher accuracy.

For the empirical data, we found evidence in favor of a previous outcome X advice presentation paired interaction (median = 0.06, CI_89% = 0.04 to 0.07, pd ~ 100%; Fig. 4D). This suggests, as we predicted, higher coherence repeat rates after a rewarded vs. unrewarded n trial, but only when advice was revealed. When the advice was concealed, the outcome of trial n had no influence on coherence repeat rates. A similar (but smaller) positive interaction was found for the informed artificial data (in which non-informed advice-taking was lesioned; median = 0.02, CI_89% = 0.01 to 0.04, pd = 99.10%; Fig. 4E). Importantly, the non-informed artificial data set (in which informed advice-taking was lesioned) did not show evidence in favor of a previous outcome X advice presentation paired interaction (median = 0, CI_89% = -0.1 to 0.02, pd = 85.35%; Fig. 4F).

Overall, our lesion analyses were able to show two unique regression signatures: one that is predicted only by informed, but not non-informed advice-taking and one that is predicted only by non-informed but not informed advice-taking. Importantly, both effects were observed in the empirical data set, providing further evidence for our model comparison conclusion which suggests that both types of advice-taking are required to explain the data.

Previous studies have demonstrated that learners integrate social information into their value-based decision-making. However, one prominent unresolved issue with prior studies concerns the inability to separate unguided internal decisions (individual learning) from decisions based on external advice. Moreover, we argue that following advice could be further differentiated into informed advice-taking (learning the value of following advice), and non-informed advice-taking (bias to follow advice regardless of an advisor’s previous accuracy). In the current study, we therefore opted to approach these two issues using a reinforcement learning design comprising repeated advice. With this experimental design, we were able to disentangle individual from social-based decisions, while also investigating the reliability of advice-taking behavior.

The results showed that, behaviorally, participants demonstrated a robust bias towards following advice, which was a rather stable bias across separate measurements. Computational modeling enabled us to identify the underlying mechanisms and showed that the results are likely due to both non-informed advice-taking (i.e., a fixed bias towards following advice), and informed advice-taking (i.e., learning the value of following the teacher’s advice). Analysis of the empirical data showed two complementary findings for the non-informed and informed processes, respectively: (1) participants demonstrated a bias to follow advice even on their very first encounter with the teacher; and (2) reward history had a causal effect on future advice-taking behavior. Signatures from lesion analyses using simulated lesioned data showed a double dissociation supporting the necessity of considering both processes in order to explain the empirical data. Taken together, the results suggest that an intelligent process occurs in which teachers’ outcome history (i.e., accuracy) contributes to participants’ tendency to follow their advice.

The computational mechanism found in the current study resembles that of Diaconescu et al.¹⁶, which showed that people have a social bias towards following advice, accompanied by an arbitration between social and individual sources of information. Nonetheless, here, we conceptualized the bias as a parameter that more directly increases the value of the advised option, regardless of its informativeness (i.e., likelihood that the advised option leads to a reward). Najar et al.⁴ also showed that the demonstrator’s choices directly influenced those of the player, but it was shown in an imitation process such that the players mimicked the previous choices of the demonstrator. Najar et al.’s⁴ winning value-shaping model allows for long-lasting learning of social inputs, but importantly, similar to Diaconescu et al.’s¹⁶ model, the socially oriented bias towards “advised” options is bound with the specific choice made by the demonstrator. Hence, we suggest that the bias parameter in our models (i.e., non-informed advice-taking) might capture a phenomenon that relates to the normative aspect of following others’ advice since it demonstrates a general bias that is not limited to a specific choice (see Mahmoodi et al.²¹ for a study concerning the involvement of normative and informational conformity in change of mind).

One explanation for a bias towards following advice comes from the work of Biele et al.⁸, which showed that following advice was intrinsically rewarding in and of itself, thus explaining the long-lasting influence of a single piece of advice provided one time. Moreover, Doll et al.’s¹⁰ work supported a computational model by which learners favor advice over individually-learned information, even when the advice was incorrect and in Najar et al.’s⁴ model, the demonstrator’s choice itself serves as a pseudo-reward in the computational mechanism. Presumably, such a bias could rely on epistemic trust, which represents a “trust in knowledge”, or a tendency to evaluate new social information as accurate and reliable and to integrate it into one's own learning environment ^25,26. Epistemic trust has been extensively studied in young children ^27–29, and studies show that children are able to evaluate their informants during learning, and estimate whether they are knowledgeable and trustworthy. For example, Woo and Spelke³⁰ showed that even toddlers are able to both learn from the outcome of an agent’s behavior as well as to evaluate the intention of the social agent. A unified framework by Eaves and Shafto²⁵ suggests that epistemic trust complements pedagogical inference, where informants are assumed to be trustworthy. By integrating the results of these studies, we can conclude that adaptive learning results from a combination of having trust in, and evaluating, a pedagogical agent. A social learning strategy such as “copy when model successful” allows for this type of evaluation of the informant³¹. Finally, trust has been shown to influence learning from advice, but not from observation¹¹; for example, one study showed that participants with high levels of paranoia were less likely to comply with advice, but did not differ from individuals with low paranoia levels in the extent to which they copied an observed decision. It seems that both processes are mirrored in the current results, which demonstrate a general bias to trust the advisor and follow advice (non-informed) and form an evaluation of the teacher (informed advice-taking).

The current results also relate to other works concerning social learning across different areas. For example, one line of research aims to theoretically distinguish between different avenues through which one learns from others’ behavior. It was found that, in young children, learning is often the result of “blind imitation”, such that by simply copying others, one learns new behaviors³². In contrast, another form of social learning is emulation, which is considered more resource-consuming and involves the learning of the effects or goals of the actions (as opposed to the actions themselves). In the current study, participants could have followed the teacher’s selection by blind imitation or emulation – i.e., copying the teacher’s choices in order to maximize rewards. However, the interaction between a fixed bias to follow advice and an observed pattern in which participants learned from the outcome history goes against a “blind imitation” approach, suggesting that participants deliberated the value of following advice and then made a choice of whether to emulate the teacher’s behavior or not. This is supported in the lesion analysis showing the effect of the informed advice-taking process. Nonetheless, the robustness of the bias to follow advice also suggests that participants utilized both processes when making their choices, such that the bias itself could be considered blind imitation.

Thus, one question that comes to mind concerns the boundary conditions for such a strong bias to follow advice. It should be noted that the bias to follow the teacher’s advice is highly related to what is termed informational conformity - i.e., the tendency to conform with social information in uncertain/unknown environments, where we believe that others hold valuable information³³. Indeed, previous works have shown that individuals have a higher tendency to use social information when it is harder to reach a decision on their own, or when they are less confident in their own choice³⁴. In the current study, we used a probabilistic reinforcement learning task in which the uncertainty level was high due to the shifting values of the cards across trials, which also required that participants keep track of the outcome history per card. This was done in order to increase learning demands³⁵. Hence, it is possible that a different (perhaps weaker) bias towards following advice would have been found with a different design in which the learning demands were lower.

Future studies should look further into the conditions that lead to such a strong bias. As suggested, one thing that comes to mind is the influence of decision difficulty. Second, if learners tend to follow more knowledgeable and higher-status advisors, it would be intriguing to examine how the bias to follow advice would be influenced by the accuracy of the advisor, and their social rank. Previous research has already addressed these issues to some degree, however, the current experimental design suggests the benefits of examining the contribution of the value attributed to the advisor to participants’ choice behavior, as well as the influence of the advisors’ social status. Given that the current experimental paradigm involves repeated interactions with the teacher, it is especially relevant when examining how people evaluate the value of the advisor and whether and how their accuracy influences behavior.

To conclude, in this work we were able to disentangle two types of social learning: informed and non-informed advice-taking. Our computational model supports the involvement of both processes, alongside individual learning. Finally, advice-taking was shown to be reliable across measurements, thus behaving as a trait-like decision bias to follow external advice.

Participants. We recruited participants through the Prolific online platform for a two-part experiment in return for monetary compensation. One hundred and sixty-six Prolific workers completed Session 1, and 153 completed Session 2. The final sample comprised 153 participants (mean age = 26.63, range 18 to 46; 90 men, 59 women, 4 other gender identification). Participants reported normal or corrected vision and no current or past psychiatric or neurological diagnosis. The study protocol was approved by the Research Ethics Council of Tel Aviv University and all participants signed an informed consent form before participating in the study.

The student-teacher task. Participants completed two sessions of a multi-armed bandit reinforcement learning task during which they were asked to choose between cards in order to gain rewards (Fig. 1). The task included four cards, and in each trial, the computer randomly selected and offered two cards for participants to choose from. Each card led to a reward according to a reward probability that changed gradually across trials (the temporal profile of reward probabilities was generated using a noisy random walk; see Figure S1 in Supplementary Information). Participants were informed regarding the drifting-value nature of the cards (i.e., the cards’ changing value) and were instructed to try to do their best to gain as many rewards as possible, which would be converted to a monetary bonus at the end of the experiment. In addition, the task included avatars of teachers, that latently chose a card on each trial based on a noisy estimation of the cards’ true expected values (see SI). Participants were familiarized with the teacher’s avatar at the beginning of each block and were told that the teacher would direct them, from time to time, to help them choose the best card. They were further instructed that the teachers were familiar with the card game, but do not know whether a card will produce a reward, and so it is up to the participant to decide whether to follow their advice. Importantly, the teachers’ choices were generated by the computer on all trials but revealed to participants as an advice only on 60% of the trials. The rate at which the advice was revealed was chosen based on an assessment of the required number of trials per condition. Hence, the task involved two primary conditions, concealed advice and revealed advice, which alternated randomly between trials. In “concealed advice” trials, participants were only presented with the two offered cards, and did not observe the teacher’s choice. In the “revealed advice” trials, the teacher’s choice was revealed to the participants (Fig. 1). To indicate the advice, the teacher’s avatar appeared next to the advised card, and a red X appeared next to the other card for perceptual balance. Finally, to encourage participants to keep track on both the cards’ value and the teacher advice, we slowly and randomly changed the teacher ability to pick the more valuable card (using a random walk, see SI), so that on average the teacher accuracy was 63% with a range between 53–77% accuracy.

At the beginning of the session, participants were presented with task instructions, as well as a multiple-choice quiz which they had to complete with 100% accuracy in order to continue to the card game. Participants completed the student-teacher task across two sessions (one day after the other), and at the end of the experiment, were paid a fixed amount (£2.5 per session) plus a bonus [£0.75 per session, if they completed both sessions (range: £0-£1.5, mean = £0.75)]. Each session involved three blocks with 130 trials each, and a short break was provided after 65 trials. Each block introduced a new set of visual stimuli - the cards and the avatar teachers were replaced on each block in every session. Within each block, the avatar teacher was. Each trial began with a 500 ms fixation, after which the cards (with or without the teacher’s advice) appeared until a choice was made or until 6 seconds had elapsed. After participants made their choice, the chosen card appeared on the screen for 500 ms, and the reward feedback was displayed alongside the selected card for 1000 ms. Finally, a black screen appeared for 500 ms before the next trial started (see Fig. 1).

Preprocessing. Preprocessing involved locating data points during which the participant was unengaged with the task: Participants who kept pressing the same key (key repetition on more than 90% of trials) were set for exclusion, yet none reached this criterion (0 participants excluded). Trials with unreasonably fast or slow reaction times (< 200ms or > 4000ms) were omitted (resulting in 3.75% of all trials). Participants with more than 25% excluded trials due to fast/slow reaction times (13 participants) were excluded from the analyses altogether. The remaining data set included 140 participants, with an average of 749 trials per subject, resulting in a total of 104,918 observations that were later used for Bayesian parameters updating in the main analysis.

Bayesian parameters estimation. We performed Bayesian logistic regression and reinforcement learning computational modeling analyses using 'brms', ‘rstan’, and ‘loo’ packages in R ^24,36,37. Models included population-level (fixed effects) and individual-level (random effects) parameters for all estimated models and sampled with weakly informative priors. To estimate chain convergence, we visually examined trace plots, pairs plots, and R-hat estimates and found them to show good chain convergence. We report the median, 89% CI, highest density interval (HDI), and probability of direction (pd) for parameters’ posterior distributions (logistic regression estimates are on the log-odds scale; for prior robustness checks, see SI). Note that 89% CI was selected following recent recommendations for Bayesian posterior estimates ³⁸. Specifically, the 89% CI bares no special meaning, and is as arbitrary as other estimates (i.e., 95%). Instead of using the CI as a single estimate for drawing conclusions regarding the effect of interest, we made an effort to describe the full posterior (including explicit figures) and provide more information regarding a range of CIs where the conclusion of the analysis was not clear cut. All models were estimated by comparing Expected Log-Probability Density (i.e., elpd). For computational models, we performed model comparison using a leave-one-block-out approach. For each model, we left out one of the six blocks and calculated the expected log predictive density for each trial in the left-out block [i.e., we estimated the population and individual parameter posteriors using hierarchical Bayesian modeling (using ‘stan’ Markov chain Monte Carlo) and then calculated the log predictive density for each trial in the left-out block]. This procedure was repeated for all blocks resulting in pointwise predictions for each posterior sample and across all observations (that is, an estimated elpd for each observation). We then used the ‘loo_compare’ function (from the ‘loo’ package) to perform pairwise model comparisons between each model and the model with the largest elpd. An elpd difference of 2 times the standard error was considered substantial²⁴. We further conducted posterior predictive checks to establish the conclusions arising from our model comparison results (see SI).

Data availability

All the data gathered and analyzed during this study are publicly available on https://osf.io/vn84c/.

Code availability

Regression analyses, computational modeling analysis codes and Supplementary Information are publicly available on https://osf.io/vn84c/.

Autor Contributions

The authors confirm contribution to the paper as follows: study conception and design: M.P., U.H., and N.S. Data collection: M.P., and N.S. Analysis: M.P., I.BA.; and N.S. Computational modeling: N.S. and M.P. Interpretation of results: M.P., U.H., I. BA., and N.S. Manuscript preparation: M.P., and N.S. All authors reviewed the results and approved the final version of the manuscript.

Competing Interests statement

The authors declare no competing interests.

Funding. The study was funded by the Minducate Center for the Science of Learning (Sagol School of Neuroscience, Tel Aviv University), the Israel Ministry of Science and Technology, and the Israeli Science Foundation (Grant 2536/20 awarded to the last author).

Schultz, W., Dayan, P. & Montague, P. R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).
Gweon, H. Inferential social learning: cognitive foundations of human social learning and teaching. Trends Cogn. Sci. 25, 896–910 (2021).
Joiner, J., Piva, M., Turrin, C. & Chang, S. W. C. Social learning through prediction error in the brain. Npj Sci. Learn. 2, 1–9 (2017).
Najar, A., Bonnet, E., Bahrami, B. & Palminteri, S. The actions of others act as a pseudo-reward to drive imitation in the context of social reinforcement learning. PLOS Biol. 18, e3001028 (2020).
Heyes, C. Who Knows? Metacognitive Social Learning Strategies. Trends Cogn. Sci. 20, 204–213 (2016).
Kendal, R. L. et al. Social Learning Strategies: Bridge-Building between Fields. Trends Cogn. Sci. 22, 651–665 (2018).
Biele, G., Rieskamp, J. & Gonzalez, R. Computational Models for the Combination of Advice and Individual Learning. Cogn. Sci. 33, 206–242 (2009).
Biele, G., Rieskamp, J., Krugel, L. K. & Heekeren, H. r. The neural basis of following advice. PLoS Biol. 9, (2011).
Bonawitz, E. et al. The double-edged sword of pedagogy: Instruction limits spontaneous exploration and discovery. Cognition 120, 322–330 (2011).
Doll, B. B., Jacobs, W. J., Sanfey, A. G. & Frank, M. J. Instructional control of reinforcement learning: A behavioral and neurocomputational investigation. Brain Res. 1299, 74–94 (2009).
Hertz, U., Bell, V. & Raihani, N. Trusting and learning from others: immediate and long-term effects of learning from observation and advice. Proc. R. Soc. B Biol. Sci. 288, 20211414 (2021).
Staudinger, M. R. & Büchel, C. How initial confirmatory experience potentiates the detrimental influence of bad advice. NeuroImage 76, 125–133 (2013).
McElreath, R. et al. Beyond existence and aiming outside the laboratory: estimating frequency-dependent and pay-off-biased social learning strategies. Philos. Trans. R. Soc. B Biol. Sci. 363, 3515–3528 (2008).
Behrens, T. E. J., Hunt, L. T., Woolrich, M. W. & Rushworth, M. F. S. Associative learning of social value. Nature 456, 245–249 (2008).
Campbell-Meiklejohn, D. K., Bach, D. R., Roepstorff, A., Dolan, R. J. & Frith, C. D. How the Opinion of Others Affects Our Valuation of Objects. Curr. Biol. 20, 1165–1170 (2010).
Diaconescu, A. O. et al. Neural arbitration between social and individual learning systems. eLife 9, e54051 (2020).
Rybicki, A. J., Sowden, S. L., Schuster, B. & Cook, J. L. Dopaminergic challenge dissociates learning from primary versus secondary sources of information. eLife 11, e74893 (2022).
Nafcha, O. & Hertz, U. The devil is in the task structure. eLife 11, e78930 (2022).
Dunning, D., Anderson, J. E., Schlösser, T., Ehlebracht, D. & Fetchenhauer, D. Trust at zero acquaintance: More a matter of respect than expectation of reward. J. Pers. Soc. Psychol. 107, 122–141 (2014).
Mahmoodi, A., Bahrami, B. & Mehring, C. Reciprocity of social influence. Nat. Commun. 9, 2474 (2018).
Mahmoodi, A., Nili, H., Bang, D., Mehring, C. & Bahrami, B. Distinct neurocomputational mechanisms support informational and socially normative conformity. PLOS Biol. 20, e3001565 (2022).
Pesch, A. & Koenig, M. A. Trust Matters: Measuring and Identifying a Role for Epistemic and Interpersonal Trust in Preschoolers’ Learning from Teachers. Early Educ. Dev. 0, 1–26 (2021).
Sutton, R. S. & Barto, A. G. Reinforcement learning: An introduction. (MIT Press, 1998).
Vehtari, A., Gelman, A. & Gabry, J. Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Stat. Comput. 27, 1413–1432 (2017).
Eaves, B. S. & Shafto, P. Chapter Eleven - Unifying Pedagogical Reasoning and Epistemic Trust. in Advances in Child Development and Behavior (eds. Xu, F. & Kushnir, T.) vol. 43 295–319 (JAI, 2012).
Fonagy, P., Luyten, P., Allison, E. & Campbell, C. Mentalizing, Epistemic Trust and the Phenomenology of Psychotherapy. Psychopathology 52, 94–103 (2019).
Harris, P. L. Trusting What You’re Told: How Children Learn from Others. (Harvard University Press, 2012).
Koenig, M. A. & Harris, P. L. The Basis of Epistemic Trust: Reliable Testimony or Reliable Sources? Episteme 4, 264–284 (2007).
Sobel, D. M. & Kushnir, T. Knowledge matters: How children evaluate the reliability of testimony as a process of rational inference. Psychol. Rev. 120, 779–797 (2013).
Woo, B. M. & Spelke, E. S. Toddlers’ social evaluations of agents who act on false beliefs. Dev. Sci. e13314 (2022) doi:10.1111/desc.13314.
Heyes, C. & Pearce, J. M. Not-so-social learning strategies. Proc. R. Soc. B Biol. Sci. 282, 20141709 (2015).
Want, S. C. & Harris, P. L. How do children ape? Applying concepts from the study of non-human primates to the developmental study of ‘imitation’ in children. Dev. Sci. 5, 1–14 (2002).
Deutsch, M. & Gerard, H. B. A study of normative and informational social influences upon individual judgment. J. Abnorm. Soc. Psychol. 51, 629–636 (1955).
Morgan, T. J. H., Rendell, L. E., Ehn, M., Hoppitt, W. & Laland, K. N. The evolutionary basis of human social learning. Philos. Trans. R. Soc. B Biol. Sci. 279, 653–662 (2012).
Daw, N. D., Niv, Y. & Dayan, P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8, 1704–1711 (2005).
Bürkner, P.-C. Advanced Bayesian Multilevel Modeling with the R Package brms. R J. 10, 395–411 (2018).
Carpenter, B. et al. Stan: A Probabilistic Programming Language. J. Stat. Softw. 76, 1–32 (2017).
McElreath, R. Statistical Rethinking: A Bayesian Course with Examples in R and STAN. (Chapman and Hall/CRC, 2020).

(Not answered)

SupplementaryInformationOSF.docx
Disentangling the contribution of individual and social learning processes in human advice-taking behavior - Supplementary Information

Download PDF

Journal Publication

published 20 Jan, 2024

Read the published version in npj Science of Learning →

Editorial decision: revise
11 Jul, 2023
Review #2 received at journal
12 Jun, 2023
Reviewer #3 agreed at journal
30 May, 2023
Reviewer #2 agreed at journal
21 May, 2023
Review #1 received at journal
29 Apr, 2023
Reviewer #1 agreed at journal
22 Apr, 2023
Reviewers invited by journal
21 Apr, 2023
Submission checks completed at journal
28 Mar, 2023
First submitted to journal
27 Mar, 2023
Unknown event
27 Mar, 2023
Editor assigned by journal
25 Mar, 2023

You are reading this latest preprint version

Disentangling the contribution of individual and social learning processes in human advice-taking behavior

Status:

Journal Publication

Version 1

Abstract

Figures

Introduction

Results

Discussion

Materials And Methods

Declarations

References

Additional Declarations

Supplementary Files

Status:

Journal Publication

Version 1