Integrated perceptual decisions rely on parallel evidence accumulation

Summary The ability to make accurate and timely decisions, such as judging when it is safe to cross the road, is the foundation of adaptive behaviour. While the computational and neural processes supporting simple decisions on isolated stimuli have been well characterised, in the real-world decision-making often requires integration of discrete sensory events over time and space. When crossing the road, for example, the locations and speeds of several cars must be considered. It remains unclear how such integrative perceptual decisions are regulated computationally. Here we used psychophysics, electroencephalography and computational modelling to understand how the human brain combines visual motion signals across space. We directly tested competing predictions arising from influential serial and parallel accounts of visual processing. Using a biologically plausible model of motion filtering, we find evidence in favour of parallel integration as the fundamental computational mechanism regulating integrated perceptual decisions.


Introduction
All voluntary behaviours arise from a decision-making process of one kind or another.
Decision-making has been widely studied in the laboratory by psychologists (Ratcliff et al., 2016;Summerfield & Tsetsos, 2015;Zhou et al., 2021), economists (de Hollander et al., 2020;Heng et al., 2020;Khaw et al., 2021;Prat-Carrabin & Woodford, 2022;Woodford, 2020) and neuroscientists (Donner et al., 2009;Kristan, 2008;Lafuente et al., 2015;O'Connell et al., 2018;Shadlen & Kiani, 2013) over several decades.Significant progress has been made in understanding the computational and neural processes involved, particularly in the realm of simple perceptual decisions on elementary sensory stimuli (Forstmann et al., 2016;Hanks & Summerfield, 2017;O'Connell et al., 2018;Ratcliff et al., 2016).A characteristic shared by almost all variants of these decision-making paradigms is that just a single stimulus -a patch of moving dots (Loughnane et al., 2016), a static grating (McIntyre et al., 2022) or a face (Philiastides et al., 2011) -is task-relevant.In the real world, however, perceptual decisions typically require integration of discrete sensory events over time and space.When deciding to cross the road, for example, the directions and speeds of several cars must be considered.Much less is known about these more complex, 'integrative' perceptual decisions (Cheadle et al., 2014;Gardelle & Summerfield, 2011;Rangelov et al., 2021;Rangelov & Mattingley, 2020;Wyart et al., 2012Wyart et al., , 2015)).Here we used psychophysics combined with brain imaging and computational modelling to understand how human observers combine visual motion signals in different spatial locations in the service of integrated perceptual decisions.
Sequential-sampling evidence-accumulation models of perceptual decision making (Heathcote et al., 2019;Ratcliff et al., 2016) posit that sensory evidence is repeatedly sampled over time into an abstract decision variable.A decision is made when the accumulated evidence in favour of a specific choice -such as the direction of coherent motion in a random-dot kinematogram (RDK) -reaches a threshold.Electrophysiological recordings in animal models (Licata et al., 2017;Shadlen & Kiani, 2013;Waskom et al., 2018) and brain imaging studies in humans (Heekeren et al., 2008;O'Connell et al., 2012;Twomey et al., 2016) have uncovered the neural signatures of evidence accumulation in the responses of individual neurons and wider neural populations, lending neurobiological support to these models.
Recently, multivariate computational modelling of whole-scalp electroencephalography (EEG) data has permitted researchers to characterise neural responses in a feature-specific manner, indexing how well the brain represents sensory inputs in near real-time, as decisions unfold (Garcia et al., 2013;Smout et al., 2019;Tang et al., 2018).Closely mimicking evidence accumulation, these feature-specific responses have a gradual onset and increase monotonically toward an asymptotic threshold (McIntyre et al., 2022;Rangelov et al., 2021;Rangelov & Mattingley, 2020).Unlike other neural correlates of evidence accumulation, feature-tuning analyses can isolate neural responses to several concurrently presented stimuli.For instance, in recent work we found significant motiontuning to a task-relevant RDK, but not to a spatially overlapping, task-irrelevant RDK in a different colour which was of equal salience (Rangelov et al., 2021;Rangelov & Mattingley, 2020).The fact that such feature-specific analyses capture neural responses to decisionrelevant inputs, rather than passive responses to all inputs regardless of task demands, suggests that feature-tuning analyses can reveal the temporal dynamics of evidence accumulation during decision-making.Here we used feature-tuning analyses of human EEG data to characterise evidence accumulation in a novel, integrative decision-making task.
As a model of integrative decisions, we developed a variant of the dot-motion discrimination paradigm which required integration of two spatially discrete sources of evidence in support of a single behavioural choice.Healthy adult human observers (N = 33) were shown pairs of RDKs, one in each of the two visual fields (left, right; Figure 1A).On each trial, a brief, suprathreshold coherent-motion event was shown simultaneously in each RDK.The directions of the two motion signals were systematically varied within and between trials (Figure 1B), and observers were required to judge the average direction of motion of the two RDKs by adjusting a response dial.For example, if the motion direction on the left was toward 11 o'clock and the direction on the right was toward 1 o'clock, the average direction would be 12 o'clock (Figure 1B).While relatively simple, the averaging task captures the salient attributes of many integrative perceptual decisions, namely, the need to process two or more discrete sources of evidence in order to make a single behavioural choice.Relative to more complex integrative decisions, the advantage of the averaging task is that sources of evidence to be integrated vary along the same physical attribute (here, motion direction) rendering the integration process experimentally and computationally tractable.
Hypothetically, sensory evidence from different locations in the visual field could be sampled and accumulated into a decision variable either serially, one at a time, or in parallel, all at once.The distinction between serial and parallel processing has attracted interest across a range of domains, from early vision (Treisman, 1999) to response selection (Pashler, 2000) and, more recently, in the context of perceptual decision-making (Kang et al., 2021).These studies have suggested that early visual processes such as scene segmentation operate in parallel, whereas later processes such as response selection operate serially.It remains unclear, however, whether integrative evidence accumulation is inherently a serial or parallel process.To adjudicate between these possibilities, we simulated the accumulation of inputs by motion-tuned sensory neurons using a biologically plausible model of motion filtering (Adelson & Bergen, 1985;Waskom et al., 2018).In a separate analysis, we assumed that evidence accumulation was itself either serial or parallel, and the simulated data were compared with participants' observed brain activity.

Results
Behavioural analyses.Participants' responses were quantified in terms of error magnitude (i.e., the reproduced average motion direction, relative to the true average of the two motion signals for that trial).The distributions of error magnitudes were unimodal (Figure 1C), centred on zero and fairly narrow, indicating that observers were able to make decisions on the average direction of motion, despite the spatial separation between the left-and right-sided RDKs.To independently characterise random guessing and noisy target responses, observers' error magnitudes were fitted using mixture distribution modelling (Bays & Husain, 2008;Zhang & Luck, 2008).We modelled the noisy target responses as a combination of veridical and reversed motion percepts (e.g., left veridical/right reversed, etc.).Participants guessed in 8% of trials (SEM = 2%), and experienced motion reversals in 9% of trials (SEM = 3%).The estimated precision of target responses was fairly high (KM/SEM = 9.89/1.26),equivalent to 55° full-width at half-maximum on average.For reference, the average tuning width of direction-selective neurons in macaque area MT is 60° (Treue et al., 2000), suggesting that participants performed the task well.By experimental design, the angular distance between the two component motion signals varied randomly between trials.We asked whether participants' judgements of the average motion direction on each trial might be more accurate for cases in which the two component motion signals were more similar in their directions, compared with trials in which the difference between component-motion directions was larger.To characterise the effect of angular distance, trials were binned using a median split, and mixture distribution models were re-fit separately for small angular distances (67° on average) and large angular distances (113° on average).The effect of angular distance was negligible for all fitted parameters as revealed by one-sample two-tailed t-tests (guesses: 13.5% and 8.9% for small and large distance, respectively; reversals: 7.2% and 9.8%; precision of target responses: 10.84 and 10.59, all p > .10).These results suggest that the angular distance between component motion signals have a minimal effect on the accuracy of participants' integrated decisions.
Mass-univariate EEG analyses.Continuous EEG data were pre-processed offline using the FASTER pipeline (Nolan et al., 2010) and segmented into 1 s epochs, time-locked to the onset of coherent motion.In a first step, to determine whether individual electrodes carried any feature-selective information about the motion signals, we regressed time-resolved voltage at individual electrodes on the left and right component motions and the average motion direction using the same methods as in our previously published work (Smout et al., 2019).With the exception of a few brief, phasic responses following signal onset, neural activity recorded from individual electrodes was not selective for either of the component motion signals (left, right) or their computed average.Importantly, there was no evidence for selective responses over frontal electrodes, which would have been apparent had participants made eye movements to track the motion signals.
Population-tuning EEG analyses.We chose to measure neural responses using EEG because this method affords millisecond-level temporal resolution, and because we and others have shown that time-resolved direction-selective responses to brief dot-motion stimuli can be reliably decoded from EEG recordings using multivariate decoding to characterise patterns of brain activity across all electrodes (Brouwer & Heeger, 2009;Garcia et al., 2013;McIntyre et al., 2022;Rangelov et al., 2021;Rangelov & Mattingley, 2020)  For each trial (i.e., test), the EEG epoch was compared ("-" symbol) with the average (i.e., the ERP) of other, training trials using multivariate Mahalanobis distance (Wolff et al., 2017).Training trials were binned relative to the motion direction in the test trial, and the comparisons were derived per bin resulting ("=" symbol) in a time-resolved measure of similarity between the test trial and bins of training trials.
Ideally, the similarity matrix should yield a bell-shaped profile across bins with a peak at the tested angle.
Population-level motion tuning is estimated as the width of the similarity profile across bins.A tuning strength of 0 indicates no tuning (i.e., a uniform distribution).(B) Estimated similarity matrices for left-side and rightside component motion signals and their average.White contours delineate areas within which the similarity estimate was significantly different from 0 at pFDR-corrected < .05.Bell-shaped curves denote weight profiles used to compute an aggregated measure of motion-specific brain activity (i.e., tuning strength, see Methods).
All three analysed motion directions (the left and the right component motions and their average) were concurrently and robustly represented in patterns of brain activity (Figure 2B).To characterise the overall time-course of direction-specific neural activity, an aggregate index of tuning strength was computed as a weighted sum of similarities across bins (see Figure 2B and Methods).Direction-specific responses started at about .2s after motion onset and remained strong for the duration of the epoch (Figure 3A).The onset time is comparably late (Di Russo et al., 2002) for the direction-specific activity to reflect early, sensory responses (e.g., N1/P1), while it closely matches other well-documented correlates of decision-making such as the central-parietal positivity, i.e., the CPP (McIntyre et al., 2022;O'Connell et al., 2012;Rangelov et al., 2021;Rangelov & Mattingley, 2020;Twomey et al., 2016).Interestingly, tuning to the derived, average motion direction -which was never actually presented in the displays -was stronger than tuning to either of the two component motion stimuli displayed in the left and right hemifields.show fitted tuning using three parameters: t0 -the tuning onset; v -the rate of increase in tuning; and Amaxthe maximum tuning.Shaded areas show ±1 within-participants SEM (Morey, 2008).Lower panel shows the FDR-corrected p-values (from a one-sample t-test against zero) of the tuning strength averaged across all three motions.(B) Comparison of tuning to different motion directions across the three fitted parameters.The fitting was performed using jack-knifed data (Miller et al., 1998) 3A).To quantify the separation between neural tuning profiles, we subtracted bin similarities across different bin distances and different tuning profiles (see Methods for details).For the left motion direction, for example, the bin similarity in the tuning profile for the left motion at a bin distance of zero (upright teal triangle in Figure 3C) was subtracted from the bin similarity in the tuning profile for the average direction at the bin equal to the angular distance between the left and the average motion direction (inverted violet triangle in Figure 3C) on that trial.This quantity reflects the trial-specific separation between tuning profiles for the left motion direction and the average motion direction, which we call a "decision function", or DfL.To facilitate comparisons between participants, participant-specific DfL scores were z-scored across trials.
If the brain represents only the left motion direction, the DfL distribution should be centred on zero.If, by contrast, the brain represents the left motion direction separately from the average direction, the DfL distribution should be shifted away from zero.As shown in the upper panel of Figure 3D, the grand-average histogram showing DfL scores (teal bars) exhibit a clear negative shift.An analogous decision function was computed for the average motion direction (DfA).This involved subtracting the bin similarity in the tuning profile for the left component at the bin equal to the angular distance between the left motion direction and the average (teal square in Figure 3C) from the bin similarity in the tuning profile for the average motion direction at zero distance (violet circle in Figure 3C).The participant-specific DfA scores were again z-scored across trials.If the brain represents only the average motion direction, then the DfA distribution should be centred on zero.If, by contrast, the brain represents the average direction separately from the left motion direction, the DfA distribution should be shifted away from zero.The upper panel of Figure 3D clearly shows that the estimated DfA distribution was shifted positively.We used the same approach to compare tuning profiles for the right component motion direction and the average (Figure 3D, lower panel), and obtained a very similar pattern of negative and positive shifts in the decision functions.
Taken together, these results support the notion of distinct neural representations for all three motion directions (Figure 3D).To quantify the degree of separation between distributions, we derived receiver-operating-characteristic (McNicol, 2005) curves per participant.Figure 3E shows the grand-average ROC curves for discriminating between the average and the left component motion direction (teal curve in Figure 3E), and between the average of the right component motion (read curve in Figure 3E).Both ROC curves clearly deviate from chance (dashed diagonal line), indicating a reliable separation between neural representations of each component motion signal and the average.From the ROC curves, we estimated of the area-under-the-curve scores, which were significantly above chance (AUCM/SEM = .69/.06 and .69/.07 for the left and right component motion signals, relative to .50 for random guessing, both tone-sample ≥ 2.93, both ptwo-tailed < .01).
Simulations of neural population tuning.To investigate whether the neural representation of the derived average direction relied upon parallel or serial spatial integration of the component motion signals, a biologically plausible model of motion filtering (Adelson & Bergen, 1985;Kiani et al., 2008;Waskom et al., 2018) was used to simulate responses of motion-tuned integrator neurons to different types of sensory input (Figure 4A).The model assumes that motion-tuned neurons operate as spatiotemporal filters such that they preferentially respond to a specific motion direction (i.e., the preferred direction) within a specific area of visual field.Similar to the multivariate decoding analyses described above, characterising response profiles of a bank of neurons tuned to different motion directions in the same area can reveal population tuning strength to different motion directions.
Using the code that generated the visual displays shown to participants, different motion stimuli were created to mimic either serial or parallel integration of component motion signals (Figure 4A,left).The left and right RDKs were algorithmically shifted to the centre of the display so that they overlapped.For serial sampling, only the left or the right RDK was shown on any frame (see Methods).The sampling was simulated at 10 Hz and 5 Hz, which broadly overlap with the frequencies at which perceptual and attentional systems are thought to sample sensory input (VanRullen, 2016).For parallel sampling, the two RDKs were shown together on every frame.Simulated visual displays served as an input to a bank of neurons that differed in their preferred motion direction.To simulate accumulation of sensory evidence, the estimated neural activity was summed along the time axis, which yielded a robust monotonic increase in activity for neurons tuned to the analysed motion direction and a flat profile for neurons tuned to the opposite direction (Figure 4A, right).Simulated motion energy (Figure 4B, see Methods) was estimated separately for the three possible sampling mechanisms (parallel, 5 Hz serial, 10 Hz serial) and for the three different motion directions (left, right, average).In all scenarios, the time-course of accumulated motion energy closely mimicked the expected temporal dynamics of evidence accumulation, with a gradual onset and rise toward an asymptotic threshold.Further, the rate of accumulation was higher for the average motion direction than that of the component motion signals, independently of the integration mechanism.Finally, the simulated motion energy for the average motion direction reached a higher asymptote than either of the two component motion directions.Broadly speaking, simulating accumulation of neural responses to different motion directions revealed that all three hypothetical integration mechanisms were consistent with the EEG results.To test these observations, the jack-knifed time-traces were modelled using three parameters: the onset, the rate, and the maximum amplitude (see Methods).Repeatedmeasures ANOVAs for the onset parameter, conducted separately for each integration type, revealed mixed results.For the 10 Hz serial model, there was a statistically significant main effect of the motion direction (Fc = 3.44, p = .034),with the slowest onset for the right component motion (82 ms) relative to the left (40 ms) and average motion (60 ms) signals, which were not different from each other (p > .10).For the 5 Hz serial model, by contrast, the effect of motion direction on the onset parameter was not statistically significant (p = .347).Finally, for the parallel model, the onset was slower for the two component motion signals (63 ms on average) than for the average motion (60 ms, Fc = 9.22, p < .001).
Repeated-measures ANOVAs for the rate parameter, by contrast, were consistent across different scenarios: there was a statistically significant main effect of motion direction (all Fc > 7.68, all p > .001 ) reflecting a higher accumulation rate for the average motion direction (2.31 a.u./s) relative to the component motions (1.98 a.u./s on average), which were not statistically different from each other (all p > .244).Similar to the rate parameter, repeatedmeasures ANOVAs for the maximum parameter revealed a statistically significant effect of motion direction (all Fc > 38.13, all p > .001)reflecting a higher maximum for the average motion direction (1.86 a.u. on average) relative to component motion signals (1.58 a.u. on average), which were not different from each other (all p > .10).Taken together, the analyses of the fitted parameters revealed that the simulated evidence accumulation for the average motion direction was superior to that of the component motions.Critically, the three simulated models were comparable, showing that both serial and parallel integration models could account for the observed EEG data.
Analyses of noise correlations.To determine which mechanism provides a better fit to the obtained data, in a final set of analyses we focused on noise correlations between tuning strengths for different motion directions.Under serial sampling, whenever one motion is sampled (e.g., left), the other motion signal (e.g., right), by definition, is ignored.
Strong tuning to the sampled motion should therefore coincide with absent or weak tuning for the ignored motion stimulus, thus predicting a negative correlation between the two.

Discussion
We characterised the neural mechanisms that support integration of spatially discrete motion signals in the service of a single, integrated decision in human observers.In contrast to typical perceptual decision-making tasks11/7/2023 9:26:00 am which use a limited stimulus set and require coarse motion discrimination (e.g., left vs right), our task used a large range of motion directions and required a precise estimate of the average of two discrete coherent motion signals shown in the left and right visual fields.To avoid floor effects for the averaging task, we used high motion coherence stimuli in the peripheral RDKs.The behavioural data showed that observers performed the task well.
Previous psychophysical and brain imaging studies in humans (Anderson & Burr, 1987, 1989;Born & Bradley, 2005;Huk et al., 2002) have found that the extent of the visual field over which motion signals are integrated in the sensory visual cortex is relatively small, subtending 8-10 dva.As our peripheral stimuli were positioned 16 dva apart, it is unlikely that observers' behavioural choices relied on a single population of sensory neurons that aggregated input from both patches.Rather, integration of peripheral signals most likely relied on a network of neurons, tuned to relevant regions within the left and right visual fields and, potentially, a pool of integrator neurons.suggesting that the population tuning strength reflects the dynamics of evidence accumulation.This onset was sufficiently early (~200 ms) to eliminate the possibility that motion tuning reflected response preparation rather than evidence accumulation.
Consistent with the parallel model, population tuning analyses revealed that component motion signals were processed at a similar rate and that tuning to the average motion direction unfolded concurrently with tuning to the component signals.In fact, tuning to the average motion direction was stronger than tuning to either of the component motion signals.While possible, it is unlikely that stronger tuning to the average motion direction reflected motor response preparation.Preparing a response should, if anything, result in a monotonic increase in tuning with a peak immediately prior to response onset, as we have observed in previous work (McIntyre et al., 2022).By contrast, in the present study we found that motion tuning decreased somewhat immediately prior to participants' responses, suggesting that stronger tuning to the average motion direction relative to its components was not simply a consequence of motor preparation.Hypothetically, stronger to the average motion direction than to its components might result from a passive pooling of motion signals across the whole visual field.This seems unlikely, however, because we have previously shown (Rangelov et al., 2021;Rangelov & Mattingley, 2020) that when two overlapping RDKs are presented at fixation and only one is task-relevant, the brain represents only the task-relevant motion stimulus, as if the task-irrelevant stimulus was not present in the display.These results strongly suggest that population tuning analyses primarily track task-relevant rather than passive pooling of sensory input across the visual field.It is possible that sensory brain regions encode only unfiltered sensory inputs, predicting that sensory areas should exhibit faster and stronger feature-specific responses to component motion signals relative to the average motion direction.While the limited spatial resolution of EEG is not optimal for spatially restricted tuning analyses, its superior temporal resolution was essential for characterising the time-course of integrated decisionmaking, which was the focus of the present study.Future studies using functional magnetic resonance imaging or magnetoencephalography, which have a superior spatial resolution, could focus instead on revealing which specific neural areas are involved in representing raw, unfiltered sensory evidence and which areas represent integrated perceptual decisions.Hypothetically, integration could occur in motion-tuned sensory cortex, e.g., V5/hMT+ (Newsome et al., 1989), in associative brain areas, e.g., parietal cortex (Shadlen & Kiani, 2013), or even in motor areas, e.g., primary and supplementary motor cortex, (Purcell et al., 2010).Given the characteristics of our displays, and considering the typical receptive field size of motion-sensitive visual neurons in the human brain (Anderson & Burr, 1987), it is unlikely that integration occurred in sensory areas, at least not during the initial feedforward sweep of visual information.Integration in lateralized sensory cortices would predict a lag between representations of the component signals (left, right) and the integrated representation (average), reflecting the lag involved in transmitting signals between the cerebral hemispheres (Saron & Davidson, 1989).We found no such lag in our EEG recordings.Alternatively, representations of the component motion signals might be integrated at the level of associative and/or motor brain regions.
A defining feature of our task is that participants had to sample and integrate evidence from two discrete locations in space.The absence of negative correlations for neural processing of different component motion events suggests that participants sampled sensory input in parallel between visual hemifields.Existing research suggests that parallel sampling across hemifields is possible.For example, monitoring discrete signals is easier between visual hemifields than within hemifields (Bland et al., 2020;Strong & Alvarez, 2018).The integration of discrete signals that are sampled in parallel could take place in at least two different ways.In one, the sampled evidence could be accumulated into two different decision variables, one for each RDK, until a decision about the direction of each component motion is reached.On this account, the average motion direction would be computed only after both component motion signals have been processed, predicting a delay for the average relative to the component representations.Our findings are inconsistent with this prediction.Alternatively, the sampled evidence could be accumulated in parallel into a single decision variable as if the two RDKs were virtually shifted so that they overlapped in space.This account predicts that representations of the two component motion signals and their average should overlap in time, as we observed (see Figure 3A).Overall, our findings with the parallel integration model.
A recent study addressed a related but distinct aspect of perceptual decision-making.Kang et al. (2021) had their participants judge different attributes of a single RDK stimulus, such as its colour or motion direction.In contrast to the results presented here, they found that evidence accumulation for these different attributes takes place serially.Crucially, however, in their task the same stimulus feature (e.g., blue) was mapped to different behavioural responses, raising the possibility of a response selection bottleneck (Pashler, 2000), rather than true serial evidence accumulation.By contrast, here we held stimulus-response mapping constant throughout and allowed observers ample time to rotate the response dial to its desired position.More broadly, it is possible that the computational mechanisms that regulate integration of sensory evidence within attributes (e.g., motion signals) are different from those that control integration between attributes (e.g., across motion and colour).This is likely to be an important distinction for future studies.
Analyses of both behavioural and neural data revealed a negligible impact of angular distance between component motion signals on participants' representation of the average motion direction.It is likely that the high level of motion coherence used in the present study (80%) rendered the task relatively easy even at large angular distances.
Hypothetically, the preferred integration mechanism might change as signal strength decreases.To alleviate performance costs, for example, participants might switch to serial integration instead of using parallel processing.While future work might focus on characterising integration mechanisms at lower signal strengths, the current study demonstrates that parallel evidence accumulation is possible and indeed preferred, provided strength is sufficiently high.
Taken together, our results suggest that sensory inputs from the two hemifields are sampled in parallel and accumulated into a single decision variable.This is a striking feature of decision-making mechanisms because previous studies have shown that, when integration is not necessary (Rangelov et al., 2021;Rangelov & Mattingley, 2020;Wyart et al., 2015), evidence accumulation can successfully disregard concurrent but task-irrelevant signals.The neural mechanisms that dynamic adjustments in contributions of discrete inputs to evidence accumulation probably rely on dynamic reconfigurations of synaptic weights (Duncan, 2001;Duncan & Miller, 2002) connecting different brain regions.
Future studies should focus on characterising the role of distinct functional neural pathways in evidence accumulation processes.

Methods
Participants.42 healthy, human adults (22 female, Mage/SDage = 23/5 years) took part in the study.They were compensated for their participation at a rate of $20 per hour.Participants had corrected-to-normal or normal vision, and all were right-handed as assessed by the Edinburgh Handedness Inventory (Oldfield, 1971).Sample size was determined in advance in accordance with our recent studies (McIntyre et al., 2022;Rangelov et al., 2021;Rangelov & Mattingley, 2020) which used a similar data-analytic approach while accounting for a projected participant attrition rate of 10-15%.No a priori exclusion criteria were applied.
Six participants were excluded due to data corruption and a further three participants were excluded as outliers on the basis of brain imaging data (see below), leaving 33 complete data sets (15 females, Mage/SDage = 24/5 years).The study was approved by the Human Research Ethics Committee of The University of Queensland (Approval 2016001247) and was conducted in accordance with the Human Subjects Guidelines of the Declaration of Helsinki.Written informed consent was obtained from all participants prior to experimental testing.
Each patch comprised 150 dots (diameter = .16dva, lifetime = 50 ms) moving at 8 dva/s.The dots moved randomly for .5-1s, after which 80% of dots in each patch started moving coherently for .8s, followed by a blank screen for .2s.Coherent motion events were displayed well above the normal perceptual threshold for motion at fixation (Scase et al., 1996) because the RDKs presented in the periphery and the task required averaging of the two signals rather than discrimination of either component-motion alone.The motion direction of the left patch was randomly and uniformly sampled on every trial from directions spanning the full circle (0-2p radians).The motion direction of the right patch was randomly and uniformly sampled from directions shifted ±(.25p-.75p)radians relative to the motion of the left patch.Participants were instructed to monitor the motion directions of both patches while maintaining fixation.
A response display appeared at the end of each trial, and comprised a central, grey circle contour (diameter = 8 dva, RGB = [125,125,125]) together with a radial blue line (length = 4 dva, RGB = [0, 0, 255]).Participants were instructed to reproduce the average motion direction of the left and the right coherent motion events by adjusting the orientation of the blue line using a computer mouse.They had 3 s to respond, after which a new trial started.
Prior to the main testing session, participants completed a training session comprising one block of 85 trials.The trial structure during training was identical to that of the main session, with the exception that a feedback display was added at the end of every trial for 1 s, showing the correct answer.In the main testing session, participants completed three blocks of 96 trials (i.e., 288 trials) separated by short rest breaks.The whole testing session took approximately 45 min to complete.As participants performed the task, their brain activity was recorded using electroencephalography (EEG).Participants also had inactive electrodes attached to their scalp for a transcranial electrical stimulation (TES) protocol, the results of which are not reported here.The active TES and sham sessions were completed on different days and the TES to session assignment was randomized across participants using Latin square.Experimenters were aware of the TES (active vs sham) per session, while participants were not informed about the nature of the TES.
Apparatus.The stimuli were presented on a 24-inch LED monitor (screen resolution 1920x1080 pixels, refresh rate 60 Hz).Participants were seated approximately 57 cm in front of the monitor.The stimuli were generated using Matlab with PsychToolbox on a Dell Precision T1700 computer with Quadro K4000 graphics card, running Microsoft Windows 7 Enterprise, 64 bit.Brain activity was recorded using a 24-bit, BioSemi ActiveTwo EEG system comprising 128 Ag/AgCl scalp electrodes positioned in the 10-20 layout with a resolution of 31.25 nV (±262 mV recording range) at a sampling rate of 1024 Hz.
Behavioural analyses.For every trial, error magnitudes were computed as the angular difference between the reproduced and the average motion direction.To characterise observers' ability to perform the task and their guess rates independently, distributions of the observed error magnitudes per participant were analysed using mixture distribution modelling (Bays & Husain, 2008;Zhang & Luck, 2008) using maximum likelihood optimisation.The distribution of error magnitudes was modelled (Eq. 1) as a combination of random guessing, characterised as a uniform distribution ( ! "# ), and noisy target responses ( $,& ).The mixture of the two components was modelled using a guessing coefficient ().
For several participants, the distribution of error magnitudes was multimodal, with prominent peaks at 0, ±.5p, and p, suggesting that these participants frequently perceived one or both presented motion events in the opposite direction, i.e., they experienced motion reversals.To characterise these multimodal distributions, noisy target responses were modelled (Eq.2) as a combination of four possible types of reversals (e.g., left motion reversed/right motion veridical, etc.).mixture of the four reversal types was modelled using a reversal coefficient (), reflecting the probability of a motion reversal of either the left or the right motion direction.Each reversal type was modelled as a von Mises distribution defined by location (µ ∈ {, .5,0, −.5 }) and precision () parameters.A higher  indicates a better ability to integrate sensory input across hemifields independently of guessing or motion reversals.
(1) 3 5 EEG analyses.The MNE-Python (Gramfort et al., 2013) library was used for EEG data analyses.Continuous recordings were pre-processed offline using the automated FASTER pipeline (Nolan et al., 2010).Briefly, the EEG channels were re-referenced using the average of all electrodes.Using a 4th order IIR Butterworth filter, the data were bandpass-filtered in the range Hz and notch-filtered at 50 Hz to remove electrical line noise.Bad channels were identified and interpolated using adjacent electrodes (9 out of 128 channels on average across participants , SD = 2).All channels were again re-referenced to the average electrode.Independent component analysis (ICA) was used to identify and remove eye- ) was computed between the test trial and the binned training trials using the regularised covariance (Ledoit & Wolf, 2004) between electrodes estimated from the binned training trials.The resulting distance was transformed into an index of similarity (ρ -,. ) by removing the mean across bins and reversing the sign 3).The multivariate analyses were used to characterise motionspecific responses to the left motion stimulus, the right motion stimulus, and the average of the two.

)
The multivariate analyses resulted in a matrix of similarities across all bins and time samples per trial (Ρ !/ -234 × 6 4789:;4 ).The similarity matrices were temporally smoothed using a Gaussian window (SD = 16 ms).A uniform profile of similarities across bins per time sample would indicate no motion-specific pattern of responses at that time.By contrast, a bellshaped profile with low values for the extreme bins (i.e., ±) and high values for the central bin (i.e., 0) would indicate a strong motion-specific response.To characterise the similarity profiles, an aggregate measure of motion-tuning strength () was derived as a dot product of the similarity matrix and the cosine-transform of the bin centres (cos( !×!/ )) (Eq.4).A tuning strength of zero would indicate a uniform profile; its absolute value increases monotonically as the profile becomes more bell-shaped.To compare tuning between different motion directions, the time-series were averaged across trials per participant and fitted to an exponential function (Eq.5) defined by three parameters:  , , denoting the time when the tuning starts to increase; , denoting the rate at which the tunning increases; and A 87< , denoting the maximum tuning, i.e., the asymptote.The three parameters were estimated using the jack-knifed data (Miller et al., 1998).
(4)  6 4789:;4 = Ρ !/×6 4789:;4 ⋅ cos( !×!/ ) (5)  .∈{!,…,6) = A 87< −  +A(.! +) To quantify the degree of overlap between neural tuning profiles for the component motion signals and their average, we first averaged (Eq.6) the bin similarity estimates (ρ -,., see Eq. 3) across all trials per participant within a time window of 0.  Simulation analyses.To investigate neural mechanisms of sensory integration, responses of hypothetical motion-tuned neurons were simulated using a biologically-plausible model of motion filtering (Waskom et al., 2018).Briefly, this model simulates responses of motiontuned neurons which respond preferentially to a specific motion direction (e.g., 12 o'clock) within a specific region of visual field (e.g., around fixation).Visual displays comprising dot motion stimuli were generated using the same code as used in the experiment proper.Only the .8s periods of the left-and right-sided coherent motion events were generated, together with .2s of blank display at the end.In addition, the left and right patches were algorithmically shifted to the centre of the display, mimicking pooling of sensory input across space by a single, integrative neuron.Since the stimuli were presented around fixation, all simulated neurons were preferentially tuned to the central area of the visual field.The sensory input was simulated in two different ways (Figure 4A).Parallel sampling involved presenting the left and the right patches together (i.e., overlapping) on every frame.Serial sampling, by contrast, involved presenting either the left patch alone or the right patch alone on every frame.Two sampling frequencies were used: 10 Hz sampling involved presenting, e.g., the left patch for 100 ms, then randomly choosing which patch to present for the next 100 ms, and so on.For 5 Hz sampling, the same procedure was used, the only difference being that a patch was presented for 200 ms. 100 trials were simulated, and each trial had three different sampling versions (parallel sampling, 5 Hz serial and 10 Hz serial).For every trial, 16 neurons were simulated, differing in their preferred motion directions.The preferred motion directions spanned a full circle, ranging from completely opposite to the motion signal shown in the display (e.g., − preferred motion for 0 signal) to similar to the motion signal (e.g., −.2 preferred motion for 0 signal).To emulate evidence accumulation, cumulative sums of the simulated neuronal activity were computed along the time axis.The accumulated signals were analysed separately for the left motion patch, the right motion patch, and the average of the two motion signals.
Similar to the multivariate feature-specific analyses, the simulation analyses yielded a matrix of simulated accumulated responses Α !/ 3;LGM34 × 6 4789:;4 .The activation matrix was temporally smoothed using a Gaussian window (SD = 16 ms) and normalized by dividing the raw activity with the grand average across bins and time samples.The neural population tuning was characterised using Eq. 5, yielding an estimate of simulated tuning strengths ( c ).To compare tuning between the left motion patch, the right motion patch, and the average of the two, the tuning time-traces were fitted to a cumulative uniform distribution (Eq.9) defined by three parameters: the onset ( , ), the offset ( 87< ) and the height ( 87< ).
The slope () of the fitted function was estimated using Eq.The fitting was conducted using the jack-knifed data (Miller et al., 1998).
(9)  c .∈{!,…,6) = g 0,  <  ,  87< i  −  ,  87< −  , j ,  ∈ [ , ,  87< ]  87< ,  >  87< (10)  =  87<  87< −  , Noise correlations.To further investigate whether signals in the left and right patches were integrated in a parallel or serial manner, the correlations between the instantaneous tuning noise for the left and the right motion patches were analysed.If, for example, only the left motion patch is sampled at any given moment, then tuning to the left motion events should be robust and substantial.Since in this example the right motion stimulus is not sampled at all, tuning to the right motion event should only reflect noise in the sensory input, predicting a negative correlation between the two.By contrast, the parallel model predicts no correlation as the tuning noise is independent across different motion signals.To compute the noise correlations, the tuning time-traces across trials were de-meaned relative to the average of all trials.Next, the correlation coefficient was computed (Spearman's ) between the residual tuning to the left and the right motion events across trials and per time sample.
Finally, the median correlation coefficient across time samples was computed as the overall estimate of noise correlations.To estimate distributions of the noise correlations, the correlations for 100 simulated trials were bootstrapped by random and uniform sampling with replacement 10,000 times.In addition to estimating noise correlations between tuning to the left and the right motion signals, noise correlations for the average motion representations were also examined.These analyses were performed separately for three different sampling algorithms (5 Hz serial, 10 Hz serial and parallel sampling).
While the analyses of noise correlations in the simulated neural responses would demonstrate that different sampling algorithms predict different patterns of correlations, it is unclear which sampling algorithm is implemented by the brain.The final analysis characterised the noise correlations between tuning strengths estimated using the recorded EEG activity.As with the analyses described above, the tuning time-traces were first demeaned.Next, the de-meaned time traces across all trials per participant were used to compute correlation coefficients between tuning to different analysed motion directions, separately per time sample.Finally, the median correlation across time samples was computed as the estimate of the overall correlation coefficient.The distributions of correlation coefficients across participants were qualitatively compared with the predictions of the serial and parallel models generated on the basis of the simulated neural responses.
Data and code availability.The collected data together with the code used for analyses and reporting are available at the eSpace repository of the University of Queensland under a "reuse with acknowledgment" licence (https://doi.org/10.48610/5485667).

Figure 1 .
Figure 1.Illustration of the experimental paradigm, the motion sampling algorithm and the main behavioural results.(A) Example displays from a typical trial.Dots are enlarged for clarity.Arrowheads indicating the direction of coherent motion events were not present in the actual displays.Participants judged the average direction of the two component motion signals in the left and right visual fields.Feedback on accuracy was provided during the training session only.(B) The coherent motion signal on the left was sampled randomly from the full circle (0-2p radians).The motion signal on the right was sampled from directions shifted by ±(.25p-.75p)radians relative to the motion on the left (white quadrants).The average motion panel on the right shows the correct, expected response for the two component motion signals depicted.See Methods and text for details.(C) Distribution of error magnitudes averaged across all participants (black bars) together with the estimated true target responses and guesses using mixture distribution modelling (pink lines).Note: dva = degrees of visual angle.
. The decoded signals (Figure 2A, Methods) correspond to the aggregated activity of hypothetical neural populations responding preferentially to different motion directions.The profile of the decoded signals resembles a bell-shaped curve defined by the match between the preferred and analysed motion directions.We used this curve to quantify population-level motion tuning to the left and right component motion signals, as well as to the derived (but never shown), average motion direction.Comparing the tuning time-courses across different motion directions allowed us to compare neural representations of the different component motion signals and their average, and thus to characterise the neural mechanisms underlying integrated decision-making.

Figure 2 .
Figure 2. Motion-tuning analyses of the EEG data during 1 s epochs of the motion-integration task, time-locked to the onset of coherent motion signals in the two visual fields.(A) Illustration of the multivariate motiondecoding EEG analyses.For each trial (i.e., test), the EEG epoch was compared ("-" symbol) with the average

Figure 3 .
Figure 3. Time-resolved motion-tuning strengths and tuning profiles to different motion directions.(A) Estimated tuning strengths for the component motion signals and their average.Upper panel shows motiontuning strengths, with 0 on the y-axis indicating no tuning.Solid lines denote estimated tuning, dashed lines . Note: "ns" not significant, * p < .05,** p < .01. (C) Grand-average bin similarity profiles for component motions and their average.Profiles are averaged over a time window of 0.2 -1.0 s, during which tuning strength was statistically significant.The profiles of the component motion signals (teal and red) are shifted by ±.25p relative to the profile of the average direction, which was the mean angular distance between component motions and their average.Symbols denote bin similarities used for estimating the decision function for the left component motion and the average direction (DfL and DfA, see text and Methods for more details).(D) Grand-average distributions of z-scored decision functions for component motion signals and their average.Upper panel shows contrast between the left component and the average direction.Lower panel shows the contrast between the right component and the average.(E) Grand-average receiver-operating-characteristic (ROC) curve across participants for discriminating between the average and component motion signals using the decision functions shown in panel D. Error bars denote ±1 SEM.The dashed line denotes the expected ROC curve if the decision functions overlapped, i.e., if discriminating between either component motion signal and the average were at chance.We considered two ways in which the brain could process component motion signals and their average.On the one hand, it might represent the component motion signals independently of the average motion direction.On the other, it might represent the average motion direction only, with tuning to the component motion directions merely reflecting their correlations with the average motion direction.To address this issue, we quantified the mean profiles of bin similarities for the two component motion signals (Figure 3C, teal and red lines) and their average (violet) per participant by averaging bin similarities (Figure 2B) across all trials and all time samples within a time window of 0.2 -1.0 s after the onset of coherent motion signals (during which motion-tuning was statistically significant, as shown in Figure

Figure 4 .
Figure 4. Simulation of neuronal population tuning to the left-and right-hemifield component motion signals and their average.(A) Left: Illustration of the display sequence under serial and parallel assumptions.Right: Idealised time-resolved response profile of a bank of motion-tuned neurons.(B) Estimated accumulated sensory evidence (i.e., normalized motion energy, see Methods) as a function of the accumulation algorithm (separate panels) and analysed motion directions (different lines).Shaded areas denote the bootstrapped 95% confidence interval (CI).Full lines denote estimated motion energy, the dashed lines show fitted motion energy using three parameters: t0 -the onset; v -the rate of increase; and Amax -the maximum.(C) Pairwise noise correlations (Spearman's rho) between tuning to the left and right component motion stimuli and the average motion direction.The first three panels from left to right show the correlations between simulated motion energy (panel B) under different accumulation mechanisms for 100 simulated trials.The violin plots show bootstrapped variability from resampling 1000 times with replacement.The rightmost panel shows the correlations between multivariate tuning strengths (panel B) for the observed EEG data.Violin plots show the Under parallel sampling, by contrast, the two component motion events are sampled concurrently.Their respective tuning strengths should be independent, thus predicting no correlation.The parallel model is also consistent with positive correlations between component motion signals, as random fluctuation in attentional states would affect processing of both.Figure4Cshows pairwise correlations between the simulated accumulated tuning for all three motions across the three accumulation types.As hypothesised, there was a strong negative correlation the tuning noise for the left and the right component motion signals under the serial assumption, and no correlation between component motion signals under the parallel assumption.In addition, the serial model predicts only a weak positive correlation between tuning to the average motion and tuning to either the left or the right component motion signal, whereas the parallel model predicts strong positive correlations.To test these predictions, pairwise tuning-noise correlations were computed for the observed EEG data (Figure 4C, rightmost panel).Mimicking the simulated parallel model, these analyses revealed no correlations between tuning to the left and right component motion signals, and strong positive correlations between tuning to the average and the two component motion signals.By contrast, the noise correlations in patterns of brain activity were inconsistent with the predictions of the two serial models (5 and 10 Hz).

Functional
brain activity during integrative decision-making was characterised by robust feature-specific responses to both component motion signals.The onset of the featurespecific responses was comparable to other neural correlates of decision-making, such as the CPP (O'Connell et al., 2012; Rangelov & Mattingley, 2020; Twomey et al., 2016), Due to the relatively high level of motion coherence we employed, evidence accumulation for component motion signals might have unfolded serially and finished within the first 200 ms of signal onset.If this were the case, the feature-specific neural responses we describe would not reflect the temporal dynamics of evidence accumulation.Such a rapid and serial accumulation process seems unlikely for two reasons.First, while most previous studies of visual motion discrimination used a single RDK presented at fixation, we presented two concurrent patches in the periphery (16 degrees of visual angle apart), thus rendering evidence accumulation more difficult for participants.Second, previous research measuring speeded responses required only categorical decisions on motion direction (e.g., left versus right), whereas our participants had to reproduce the average motion direction of the two component motion signals along a continuous scale, a much more demanding task.Moreover, if computation of the average motion direction was completed within the first 200 ms, it is unclear why the brain should maintain separate representations of the component motion signals and their average throughout the course of the trial, as our results show.Finally, assuming a rapid serial accumulation would still predict negative correlations between decoding of component motion signals, which is inconsistent with the results of our analyses.In the context of the present study and the data-analytical framework we used, our findings render a fast, serial evidence accumulation process unlikely.
Τ (,& =  "  #,( + [(1 − )] .*#,(+ [(1 − )] +.*#,( + (1 − ) "  ,,( movement and other artefacts.The ICA yielded 70 components on average across participants (SD = 12), of which 9 bad components (SD = 2) were identified and removed on average across participants.The data were segmented into 1 s epochs time-locked to the onset of coherent motion, down-sampled to 256 Hz, baseline-corrected using the average amplitude in the -.05 to .05 s time window, and linearly detrended.Since the coherent motion signals were preceded by a variable buffer period (.5-1 s) containing random motion, the time window for baseline correction was selected to minimize the influence of neural responses to the random motion signals.There was no reason to believe that including 50 ms post-stimulus would interfere with the ability to decode motion signals since we have shown in our previously published work using a similar paradigm that significant motion decoding does not arise until around 200 ms after the onset of coherent motion signals.The segmented data were spatially filtered using a surface Laplacian and temporally smoothed using a Gaussian window (SD = 16 ms).Bad epochs were identified and excluded from further analyses (14 epochs per participant on average, SD = 6), leaving 274 epochs on average across participants for further analyses.The final step of the FASTER processing pipeline involved z-scoring of the ERPs over all conditions per participant relative to other participants.Participants with an absolute z-score higher than 3 (i.e., > 3 standard deviations from the sample mean) were considered as outliers and removed from further analyses.Three participants were identified as outliers on the basis of the EEG data and excluded from further analyses.tuning analyses.Analyses of the segmented EEG data focused on characterising patterns(Wolff et al., 2017) of responses across all electrodes.When analysing responses to the left motion direction, for example, for every trial  (i.e., test trial), the remaining trials (i.e., training trials) were sorted into 16 equidistant bins spanning the full circle (i.e., bin centres Î {-p, …, -.6p, …, -.2p, …, .2p,…, .6p,…, p}), relative to the angular difference between the left motion presented in the test and the training trials.For each bin  and time sample , a multivariate Mahalanobis distance (δ -,. 2 -1.0 s.This yielded a mean tuning profile ( C,- SSSSS) across bins () per analysed motion direction ().Next, to quantify the overlap between neural representations of different motion directions, we computed a difference (i.e., a decision function, or ) between mean bin similarities across different bin distances and tuning profiles.When estimating the overlap between profiles for the left component motion and the average motion direction (i.e.,  D , Eq. 7), the bin similarity ( :;E.,,SSSSSSSS) of the tuning profile for the left component motion for  = 0 was subtracted from the bin similarity ( 7F;G7H;,I SSSSSSSSSSSS) of the tuning profile for the average direction for  = , or the angular distance between the component motion direction and the average motion direction ( :;E.and  7F;G7H; Linear interpolation was used for values of  which were not elements of the set of bin centres  !×!/ (see Eq. 4).The decision function for the average motion direction (i.e.,  ' , Eq. 8) was estimated by subtracting the bin similarity ( :;E.,ISSSSSSSS) for the tuning profile for the left component motion for  =  from the bin similarity ( 7F;G7H;,, SSSSSSSSSSSS) for the tuning profile for the average direction for  = 0. Analogous analyses were conducted to characterise the overlap between tuning profiles to the right component motion and the average direction.
D =  7F;G7H;,I SSSSSSSSSSSS −  :;E.,,SSSSSSSS,  =  :;E.−  7F;G7H; (8)  ' =  7F;G7H;,, SSSSSSSSSSSS −  :;E.,ISSSSSSSS The vectors of function estimates (i.e.,  D , etc.) were scaled across trials per participant using z-scoring.If the brain represents either the component motion signals alone, or their average alone, the of decision function estimates should be centred on zero and overlap across component motions and their average.To characterise the degree of overlap between  D and  ' distributions, and between  K and  ' distributions, we estimated receiver-operating-characteristic (McNicol, 2005) curves per participant, and computed area-under-the-curve as an index of the overlap.Values around .5 indicate close overlap between neural representations of component motions and their average, whereas values >.5 indicate distinct and separable representations.