We examined the effect of distance and AAT score on accuracy. As expected, accuracy was higher for participants with higher AAT scores, B = 0.793, SE = 0.07, z = 108.49, p < .0001. Critically, accuracy was higher with proximal than with distal questions, B = -0.098, SE = 0.01, z = -6.80, p < .0001, (M-proximal = 0.71, SE-proximal = 0.01, M-distal = 0.68, SE-distal = 0.01). An interaction between AAT and distance, B = 0.20, SE = 0.26, z = 7.64, p < .0001, indicated that the advantage of proximal questions over distal questions was more pronounced for low-achieving examinees (Figure 2).
Study 2
In study 1 we compared to each other questions that could have had different logical structure. Study 2 aimed to examine the effects of distance on verbal reasoning while keeping the logical structure of the question constant. We used materials from a standardized AAT, and adjusted the content of the questions to refer to distant or proximal objects and situations(between participants), and to include only relevant or also irrelevant details (within participants). We also explored the possible moderating effects of AAT and of working-memory capacity on the effects of distance, relevance, and their interactions. We hypothesized that (a) AAT scores would positively correlate with performance, (b) working-memory capacity would positively correlate with performance, (c) adding irrelevant details would impair performance (compared to not adding them), and (d) replicating Study 1, proximity would enhance performance, especially for participants with low AAT scores.
Method
Participants. One hundred twenty-eight undergraduate students from a large Israeli university (91 women, Mage = 24.20, SD = 2.94) took part in the study and were paid 30 NIS (around US$8) for participation. All participants were native Hebrew speakers. Six participants did not complete the working memory task, and 10 participants did not report their AAT scores. Overall, 112 participants provided both measures. We did not have an estimate of the effect, but planned to be able to detect an effect (main effects of distance and relevance and their interaction) of medium size with a probability of .80 at a 5% level of significance. An a-priori power analysis using the G*Power calculator (Faul et al., 2009) indicated that a sample size of at least 132 was required. Aiming to meet this goal, data collection continued until the end of the semester. All participants provided informed consent.
Materials
Verbal reasoning. Questions were selected from an online pool of previous academic aptitude tests in Hebrew. We created a proximal and a distal version for each question and added irrelevant information to each version (see Figure 1 for illustration, or SOM for sample questions). The irrelevant information in the distal and proximal versions was matched in length and linguistic complexity, but questions that included irrelevant information were naturally longer than those that did not. We pretested the questions to make sure that they included proximal versus distal content (see SOM for details of the pretest).
Working memory capacity. Visual working memory capacity was measured via the standard change detection task (Luck & Vogel, 1997; Luria & Vogel, 2011a). In this task, participants were presented with an array of either four or eight colored squares, that appeared for 150 ms. After a 900 millisecond-long retention interval, one square appeared at one of the previous locations. Participants indicated whether the color of the square is the same as or different from the square presented in the same location in the original array. The task consisted of 20 practice trials, and 120 critical trials. For the detailed description of the task see (Hadar et al., 2020). Visual-working-memory capacity estimate, Kmax, was computed by separately averaging accuracy for each array size (four and eight items). These two values were then averaged to form a single parameter with a standard formula (Cowan, 2001), Kmax = S(H-F), where S is the size of the array, H is the observed hit rate (i.e., the proportion of correct answers in trials that presented a change), and F is the observed false alarm rate(i.e., the proportion of errors in trials that did not present a change). Higher Kmax scoresindicate higher capacity (i.e., more items aresimultaneously held in memory).
Procedure. Participants were randomly assigned to either a proximal or a distal condition. Participants were seated individually in a quiet room and were provided with a digital stopwatch and an answer booklet, conditions that resembled a standard AAT. Participants had 12 minutes to complete the verbal reasoning section, and were instructed to answer as accurately as they can as many questions as they can. They were also instructed to guess in case time was over. Each participant was presented with only one version of each question. Relevance was manipulated within participants, such that each participant had six questions with only relevant details, and six questions that included also irrelevant details, presented pseudo-randomly. Participants were informed that the three best performers in the test would receive a 100 NIS cash bonus (around $27). At the end of the experiment, participants provided demographic information and reported their highest attained AAT score[1]. One hundred twenty-three participants (95% of the sample) also completed the change-detection task for another study that took place on the same day, prior to the current study.
Results and discussion
As both AAT scores and working-memory capacity reflect cognitive ability, they tend to be positively associated (e.g., Alloway & Alloway, 2010). In the current sample, however, only a modest, insignificant correlation emerged, r(110) = .16, p = .087, possibly due to the restricted range of AAT scores which could have reflected the high admission threshold of students in the psychology program of the relevant university (in our sample: M = 652, SD = 62; the grand mean in the relevant population is M = 579, SD = 107). We therefore included both AAT score and Kmax in the same model.
We assessed the effect of distance, relevance, AAT score, working-memory capacity (Kmax)and their interactions on accuracy. As expected, accuracy was higher for participants with higher AAT scores, B = 0.349, SE = 0.135, z = 2.59, p = .009, as well as for participants with higher working-memory capacity, Kmax, B = 0.297, SE = 0.147, z = 2.01, p =.044. An interaction between these scores, B = 0.315, SE = 0.150, z = 2.103, p = .035, indicated that their effects intensified each other, namely, with higher AAT scores, working-memory capacity had a larger effect on accuracy, and that with higher Kmax, AAT scores had a larger effect on accuracy (Figure 3).
Distance did not significantly affect accuracy, B = -0.289, SE = 0.199, z = -1.45, p =.147, (M-proximal = 0.57, SD-proximal = 0.19, M-distal = 0.49, SD-distal = 0.21), nor did relevance, B = -0.113, SE = 0.168, z =-0.67, p =.499, (M-relevant = 0.55, SD-relevant = 0.24, M-irrelevant = 0.51, SD-irrelevant = 0.24). The only effect that involved relevance was a marginally significant three-way interaction between relevance, distance and Kmax,B = 0.482, SE = 0.250, z = 1.93, p =.054. Inspection of the interaction revealed that adding irrelevant information impaired the performance only among participants with low Kmax in the distal condition, B = 0.284, SE = 0.135, z = 2.10, p =.035.
Importantly, replicating the findings of Study 1, an interaction between distance and AAT score, B = 0.536, SE = 0.235, z = 2.28, p =.022, indicated an advantage of proximal questions over distal questions among low-achieving participants but not among high-achieving participants. The effect of distance for participants with below-average AAT score was quite sizeable: accuracy with proximal questions (M= .50) was 28% higher than with distal questions (M= .39) that had the same logical structure (Figure 4).
The interaction of AAT with relevance (p = .89), and the three-way interaction between AAT,
Study 3
Study 3 aimed to replicate Study 2 in an ecological settings, with a larger sample, and a broader range of AAT scores. We collaborated with an organization that administers nation-wide academic aptitude tests, and incorporated the experimental questions into the exam administered as an entrance requirement for higher-education institutions. As in Study 2, we manipulated distance (proximal vs. distal) between participants and relevance (relevant-only vs. irrelevant-added) within participants. Based on the results of Studies 1-2, we predicted that proximity would enhance performance, especially for participants with low AAT scores.
Method
Participants. Participants were 1,744 examinees which were randomly sampled from all examinees who took the AAT in two waves[1]. In the first wave, which took place in the summer of 2018, 870 examinees (469 women; Mage = 22.30, SD = 2.95) were sampled. In the second wave, which took place in the summer of 2019, 873 examinees (470 women; Mage = 21.76 years, SD = 2.95) were sampled. While we aimed to reach as many examinees, the organization administering the exam allocated the experimental version on the test according to their operational constraints.
Verbal reasoning. The test was comprised of several chapters that are used for calculating the AAT score, and several pilot chapters that do not count toward the final test score but are rather used for operational purposes. Examinees do not know which are the pilot chapters. The experimental questions were incorporated into one of these pilot chapters.We used the same method as in Study 2 with several changes. First, the order of questions in each verbal reasoning section was fixed. In Wave 1 we were able to administer nine new questions, and in Wave 2 we administered 10 of the questions from Study 2. Questions were different in the two waves because the rules of the administering agency do not allow repeating questions across tests.
Procedure. Participants were randomly assigned to either a proximal or a distal condition. Each examinee received a test form, comprised of several chapters. Each chapter contained 20-23 multiple choice questions with 20 minutes allotted to complete the chapter. The experimental questions were incorporated into the pilot chapter which did not count toward the examinees’ final AAT score.
Results and discussion
We examined the effect of distance, relevance, AAT score, and their interactions on accuracy. The analysis revealed that as expected, accuracy was higher for participants with higher AAT scores, B = 0.823, SE = 0.024, z = 33.72, p < .001. Distance did not affect accuracy, B = -0.037, SE = 0.023, z = -1.601, p =.109 (M-proximal = 0.66, SD-proximal = 0.47, M-distal = 0.64, SD-distal = 0.48). Adding irrelevant information had a marginal negative effect on accuracy, B = 0.076, SE = 0.039, z = -1.92, p =.055, (M-relevant = 0.67, SD-relevant = 0.47, M-irrelevant = 0.64, SD-irrelevant = 0.48).
Although the interaction between distance and AAT score was not significant,
B = 0.005,
SE = 0.023
z = .232,
p =.817, a three-way interaction between distance, AAT, and relevance emerged,
B = 0.047,
SE = 0.019,
z =
2.
43,
p =.015, (Figure 5). A follow-up analysis revealed that whereas the performance of high-achieving examinees was not affected by either relevance (
p = .503), distance (
p = .890), or their interaction (
p = .880), the performance of low-achieving examinees was impaired by adding irrelevant information
B = -0.146,
SE = 0.050,
z = -2.91,
p =.003. Also in this group, although distance did not affect performance as a main effect, (
p = .390), it interacted with relevance,
B = -0.087,
SE = 0.040,
z = -
2.16,
p =.031, such that accuracy was lower in distal-irrelevant questions,
B = -0.137,
SE = 0.063,
z = -
2.17,
p =.030. For participants with below-average AAT score, performance on proximal questions (
M= .52) was higher compared to distal questions (
M = .48) by 7%, but only when the questions were made more difficult by including irrelevant details.
General Discussion
Across three studies - in acorpus of large-scale real-life data attained from an academic aptitude test (AAT; Study 1), in the lab (Study 2) and in the field (Study 3), we examined how performance on verbal reasoning questions is affected by the examinee’s psychological distance from the content of those questions. In all studies, individuals completed verbal reasoning questions from standardized AATs. The questions were about objects that were either psychologically distal or psychologically proximal to the examinees in time, space, and social distance. In Study 1, questions included only information necessary for solving the problem, whereas in Studies 2 and 3 some of the questions also included additional, irrelevant details.
Importantly, while most previous research has been conducted in laboratory settings, here we extend these findings to real life settings in which participants took a consequential high-stakes exam that would determine their admission chances to higher-education institutions, and possibly affect their future career paths.
In Study 1 we found that people scored higher on questions that presented proximal content compared to questions that presented distal content. Notably, the facilitative effect of proximity was more pronounced among low-achieving examinees than among high-achieving examinees. In Study 2, we found the same pattern: proximity (compared to distance) enhanced performance for participants who attained low scores on the AAT. In Study 3, we predicted that proximity would improve the performance of low-achieving examinees (as in Studies 1-2), but we found this to be true only in questions that were made more difficult by including irrelevant details. Our results therefore suggest that proximity is especially advantageous (or, conversely, distance is most detrimental) for low achieving individuals and when difficulty is high.
These findings are in line with previous research which documented that proximity enhanced cognitive performance (for a review, see Brockmole et al., 2013), and extend this research in several ways. In particular, the present research focuses on conceptual distance, which is ubiquitous in any object or event as they would be inevitably described by the examiner (or imagined by the examinee) to be at some distance – to occur in a certain proximal or distal point in time and space, to be familiar to the examinee or their social group, or, conversely to denote a socially distal group.
Our results suggest that the extent to which an examinee feels proximal to the content of a question can affect performance. For example, the extent to which an examinee feels close to the British philosopher David Hume can determine how well they do on a question that invokes this historical intellectual figure. Our findings thus identify a new source of potential cultural bias in AAT questions. The finding that low-scoring examinees are more affected by distance is also informative in this regard, as it suggests that this cultural bias would impact this segment in particular.
Why does proximity improve reasoning? Some of the features that characterize proximal stimuli have been found to facilitate cognitive performance. For example, people are usually more familiar with proximal objects than with distal objects, and studies have found that familiar objects are remembered better than unfamiliar ones (Jackson & Raymond, 2008; Xie & Zhang, 2017), possibly because familiar objects rely on long-term memory processes that shield representations from interference (Brady et al., 2016). Moreover, people tend to physically hold, manipulate, and interact with proximal objects more often than with distal objects and thus would tend to have a more embodied representation of them (Lakoff, 2012; Shapiro, 2019; Wilson, 2002). According to embodied cognition theory, cognition has evolved to facilitate action, and as such, cognitive processes that support action are more accurate and efficient. It is possible, then, that reasoning is better with proximal objects because they are more easily embodied. Also, a meta-analysis of 19 neuroimaging studies (Wang et al., 2010) showed that the verbal system is involved to a greater extent with processing of abstract rather than concrete concepts, while the perceptual system is involved to a greater extent with processing of concrete rather than abstract concepts. To the extent that mental imagery or any other type of perceptual simulation are helpful for solving reasoning problem, these findings might suggest that reasoning about concrete (rather than abstract) content might be easier.
Social psychology suggests another potential mediators for the effect of psychological distance, namely, the possibility that distal content might communicate to some examinees that they do not belong to the relevant social environment. For example, it is possible that questions about David Hume, about distal planets and even about a brioche might suggest that “anybody who is not familiar with these things does not belong here”. Studies on stereotype threat have found that performance is undermined when (and to the extent that) difficulty suggest to students that they do not belong to the current social environment (Walton & Cohen, 2007).
From a practical perspective, our findings can inform users and developers of scholastic aptitude tests. Tasks akin to the one used in our studies are integral to many cognitive assessment tests, such as intelligence tests (e.g., Wechsler intelligence scales, Wechsler, 2008), aptitude tests (e.g., scholastic assessment tests; SAT), cognitive functioning tests (e.g., MoCA; Nasreddine et al., 2005), and even skills assessment tests in job interviews. Accordingly, it is imperative to understand how performance is affected by seemingly neutral aspects of test questions, such as whether they pertain to distal or proximal objects.