Behavior-related visual activations in the auditory cortex of nonhuman primates

While it is well established that sensory cortical regions traditionally thought to be unimodal can be activated by stimuli from modalities other than the dominant one, functions of such foreign-modal activations are still not clear. Here we show that visual activations in early auditory cortex can be related to whether or not the monkeys engaged in audio-visual tasks, to the time when the monkeys reacted to the visual component of such tasks, and to the correctness of the monkeys’ response to the auditory component of such tasks. These relationships between visual activations and behavior suggest that auditory cortex can be recruited for visually-guided behavior and that visual activations can prime auditory cortex such that it is prepared for processing future sounds. Our study thus provides evidence that foreign-modal activations in sensory cortex can contribute to a subject’s ability to perform tasks on stimuli from foreign and dominant modalities.


Introduction
There is accumulating evidence that activity in sensory cortical regions traditionally thought to be unimodal, i.e., early auditory cortex (AC), visual cortex, somatosensory cortex, and olfactory cortex, can be modulated or even evoked by stimuli from modalities other than their dominant one (Atilgan et al., 2018;Atteveldt et al., 2014;Bauer et al., 2020;Bizley et al., 2007;Chandrasekaran et al., 2013;Delano et al., 2010;Fishman and Michael, 1973;Fu et al., 2003;Ghazanfar et al., 2005;Gnaedinger et al., 2019;Hennesy et al., 2022;Kayser et al., 2008Kayser et al., , 2010;;Kobayasi et al., 2013;Lakatos et al., 2007;Lehmann et al., 2006;Lurilli et al., 2012;Morrell, 1972;Varga and Wesson, 2013;Wesson and Wilson, 2010;Wu et al., 2023;Zhou and Fuster, 2000).These foreign-modal effects on sensory cortex can depend on features of the foreign-modal stimuli and have been proposed to have several functions: they may (1) support the perception of stimuli from another sensory modality, e.g., the perception of sound sources in an auditory scene (Atilgan et al., 2018;Schroeder et al., 2001;Selezneva et al., 2018); (2) facilitate the integration of stimuli from different sensory modalities and in this way contribute to the perception of multisensory objects (Ghazanfar et al., 2005); (3) contribute to a subject's ability to perform tasks on stimuli from the dominant modality (Brosch et al., 2005) or from a foreign modality (Morrell, 1972;Vasconcelos et al., 2012).However, there exists a paucity of neurobiological evidence linked to behavior to support the proposed functions.The best evidence exists for foreign-modal activations in deaf cats, in which they contributed to the superior visual abilities of these animals because these abilities were lost when the AC was deactivated (Lomber et al., 2010).By contrast, studies addressing functional roles of foreign-modal activations in normal subjects are not conclusive.Foreign-modal activations were found to depend on whether subjects engaged in a task, on the subjects' ability to discriminate foreign-modal stimuli, and on the type of task (Brosch et al., 2005;Vasconcelos et al., 2012).However, both studies did not control for potential confounds, such as arousal level, gaze or movements.Therefore, it is possible that foreign-modal activations have no functional roles for performing tasks, as suggested by Lemus et al. (2010) who found no relationship between foreign-modal activations and the ability of subjects to discriminate foreign-modal stimuli.
The current study addresses the still open question whether foreign-modal activations have functional roles when normal subjects perform tasks.To mitigate potential confounds we utilized an J o u r n a l P r e -p r o o f experimental design that included measurements of eye position and local field potentials (LFPs) as well as a rigorous data analysis.Because the occurrence of foreign-modal activations may depend on the type of task and on the stimuli, we used several tasks and stimuli.We required monkeys to perform different audio-visual (AV) tasks and recorded spiking activity and LFPs from their early AC, i.e., a region in which visual activations have been extensively studied.The visual stimuli were important for performing the tasks because they indicated to the monkeys when a trial started, that they could trigger sounds, and how these sounds needed to be processed and mapped onto motor responses.To shed more light on potential functional roles of visual activations in AC, we compared AC with the ventrolateral prefrontal cortex (vlPFC), a region that responds both to visual and auditory stimuli (Huang and Brosch, 2016) and is causally involved when subjects perform AV memory tasks (Plakke et al., 2015).

Experimental overview
Using a Go-NoGo paradigm with positive reinforcement, we trained two monkeys on four AV tasks which had a common trial structure (Figure 1A).Each trial started with the illumination of LED lights.
For the trial to continue, the monkeys had to report the presence of this visual stimulus by grasping a touch bar within 4 s.Holding the grasp for ~1.5 s triggered sounds which the monkeys had to discriminate by releasing the bar at the appropriate time to earn a water reward.
To identify potential functional roles of visual activations in AC, 1) we tested whether these activations carried information about the monkeys' performance on the visual component of the AV tasks.To this end, we compared activations in trials in which the monkeys either reacted (engaged trials) or did not react to the light (non-engaged trials) by grasping the bar.We also compared activations in trials with short and long reaction time (RT) and calculated the trial-bytrial correlation between RT and the activation.
2) We tested whether visual activations carried information about the monkeys' performance on the auditory component of the AV tasks by comparing activations in trials in which the monkeys reacted either correctly or incorrectly to the sounds.
J o u r n a l P r e -p r o o f 3) We tested whether visual activations occurred in different tasks and for different visual stimuli.
We used AV tasks which differed in the cognitive demands that were required to process sounds and in which the light stimuli provided information about these demands.For the design of the tasks we were also guided by our intention to address additional questions in our experiments, i.e., the neuronal bases of auditory working memory and of audio-motor associations (Huang et al., 2016(Huang et al., , 2019)).
Owing to the difficulty of training monkeys to perform multiple tasks in the same behavioral session, we split our study into two parts, in each of which the monkeys performed two AV tasks (Figure 1A).In Part I, the monkeys performed two tasks requiring auditory working memory (Task 1 and Task 2).In Part II, which was conducted after the completion of all recordings of Part I, the monkeys performed two other tasks: one also requiring auditory working memory (Task 3) and the other requiring auditory reference memory (Task 4).In each part, we used different light stimuli in the two tasks.To ease their discriminability, the stimuli differed in various physical features.In Task 1 and Task 3, we used two red LEDs, located at the monkeys' left side.In Task 2 and Task 4, we used two green LEDs, located at the monkeys' right side.

Experimental model and subject details
Two monkeys (Monkey C and Monkey L; Macaca fascicularis) from a group-housed colony participated in this study.Both monkeys were implanted with a head holder to allow atraumatic head fixation and with two recording chambers, one over the right AC and the other over the left PFC, to allow access to the corresponding cortical region with microelectrodes (for details of the implantation, see Brosch and Scheich, 2008).This study was approved by the authority for animal care and ethics of the federal state of Sachsen-Anhalt and conformed to the rules for animal experimentation of the European Communities Council Directive (86/609/EEC).
J o u r n a l P r e -p r o o f 2.3.Methods details

Behavioral paradigms and stimuli
The two monkeys were each trained to perform four audio-visual (AV) tasks (Task 1, 2, 3, and 4; Figure 1A).In all tasks, a trial started with the illumination of two LED lights.Within the following 4 s, the monkeys had to grasp a touch bar and to hold onto it.After ~1.5 s, a sequence of two 200-ms sounds, S1 and S2, separated by a silent delay of 800 ms, was presented.Depending on the task and the sound sequence, the monkeys were required to either release the bar within a specific time window after S2 (Go response) or to continue holding the bar (NoGo response) to earn a water reward.In non-engaged trials in which the monkeys did not grasp the touch bar within 4 s after light onset, the LED lights were turned off.In correct trials, the LED lights were turned off when the reward was delivered.In incorrect trials, the LED lights were turned off when the monkeys released the bar.In all conditions, the LED light off period lasted 5 s.Both monkeys performed the tasks with their left hand.
We split our study into two parts, which were conducted in different behavioral sessions.In Part I, we used Task 1 and Task 2. In Part II, Task 3 and Task 4 were used.In each session, the two tasks were distinguished by the light stimuli and performed in separate, alternating blocks of ~140 trials.
In Task 1 and Task 3, two red LEDs, each with an angular size of 4°, were used.The LEDs were located below and ~25 cm away from the monkeys' forehead, one at 20° and the other at 50° to the left of the monkeys.In Task 2 and Task 4, two green LEDs of the same size were used, located to the right of the monkeys at positions mirror-imaged to those of the red LEDs.The switching the LEDs did not generate sounds, which was verified by measurements with a free-field ½ inch microphone (40AC, G.R.A.S., Vedbaek, Denmark) located close to the center of the monkeys' head.
The sounds were presented bilaterally through free-field loudspeakers (Karat 720, Canton, Weilrod, Germany) at a sound pressure level of 67 dB SPL.In Part I of the study (Figure 1A), 3-kHz and 1-kHz tones were used as S1 and S2, resulting in four possible sound sequences.The sound sequence presented in a trial was randomly chosen from the four sequences.In Task 1, a Go response was required if both S1 and S2 were 1 kHz and a NoGo response otherwise.In Task 2, a Go response was required if both S1 and S2 were 3 kHz and a NoGo response otherwise.In Part II of the J o u r n a l P r e -p r o o f study, the sound sequence presented in a trial was randomly chosen from a set of nine sequences.
Task 3 required auditory working memory during the delay between S1 and S2.In this delayed-matchto-sample task, 0.4-, 1.2-, or 3.6-kHz tones were used as S1 and S2.A Go response was required if the frequencies of S1 and S2 were identical and a NoGo response otherwise.Task 4 required auditory reference memory.For S1, the same three tones as in Task 3 were used.For S2, either a 20-Hz click train, an 80-Hz click train, or a broadband noise burst was used.A Go response was required if S2 was the noise burst and a NoGo response was required if S2 was a click train, irrespective of the identity of S1.
Before or after the monkeys performed the tasks, we measured the best frequency and the frequency response area of each unit by presenting 400 pure tones with 40 different frequencies, typically covering a range of eight octaves in equal logarithmic steps (Brosch and Scheich, 2008).We also assessed the sound selectivity of the units by presenting a variety of complex sounds such as noise bursts, dog barking, monkey coos and human vocalizations (Huang and Brosch, 2016).

Electrophysiological recordings
We used two multi-channel microelectrode manipulators (Thomas Recording, Giessen, Germany) to simultaneously record action potentials (0.5 -5 kHz) and local field potentials (LFPs; 0.1 -140 Hz) from AC and PFC.Each manipulator was equipped with up to seven microelectrodes (80-µm diameter).By using the built-in spike detection tools of the data acquisition systems (BrainWave, DataWave Technologies, Minneapolis, USA; or Alpha-Map, Alpha Omega, Nof HaGalil, Israel), the action potentials of a few neurons (multiunit) recorded from a given microelectrode were isolated and their time stamps and waveforms were stored.Single-unit activity was extracted off-line by performing a principal component analysis on the waveforms of the action potentials.
The AC recordings were conducted in the core fields, mainly in field A1.The position of A1 was determined by the spatial distribution of the best frequencies across the recording sites and the information about the electrode tracks through the brain, e.g., whether or not the electrodes passed through parietal cortex into AC (Brosch et al., 2005;Kaas and Hackett, 2000).For PFC, the recordings were conducted in the ventrolateral region (vlPFC).The vlPFC was identified (1) by its anatomical J o u r n a l P r e -p r o o f location ventral to the principal sulcus and anterior to the arcuate sulcus (Szabo and Cowan, 1984;Romanski and Goldman-Rakic, 2002), and (2) by the characteristics of its responses to the pure tones and complex sounds presented in the passive condition (for details, see Huang and Brosch, 2016).

Pupil position recordings
We recorded the pupil positions of the monkeys, with a rate of 25 frames/s, by means of a video camera (CCD CG-811P; Fujitsu, Tokyo, Japan) and Pinnacle Studio 10 (Pinnacle Systems, Mountain View, CA, USA).The video recordings were performed simultaneously with the electrophysiological recordings.

Data analysis
Data analyses were performed off-line using MATLAB (MathWorks, Natick, MA, USA).Our analyses included only units, or sites, that responded to the 40 pure tones or to the complex sounds when the monkeys were passively exposed to these auditory stimuli.The exact number of units, sites and trials used in each analysis varied because their selection depended on the specific research question.At least four trials were used for each condition.

Spike ratios
For each unit, we first computed a post-stimulus time histogram (PSTH) with a bin size of 20 ms and triggered to the onset of light stimuli by averaging the spike rates across the trials of a condition (e.g., Figure 1B).The light spike ratio was then defined as the mean spike rate across all bins during the 400-ms window right after light onset divided by the mean spike rate during the 400-ms window right before light onset.In addition, we defined the baseline spike ratio as the mean spike rate during the 400-ms window right before light onset divided by the mean spike rate during the window from 800 to 400 ms before light onset.A permutation test with an alpha value of 0.05 was used to test whether a spike ratio was statistically different from 1.
J o u r n a l P r e -p r o o f 2.4.2.Population spike rates For a given condition, we computed the time course of the population spike rate of the units that were selected for a specific research question.For each unit, we normalized the PSTH of a condition to the mean spike rate during a reference window.We then averaged the normalized spike rates across the selected units.The reference window was the 400-ms window right before light onset for the PSTH relative to light onset.It was from 400 to 800 ms after bar touch for the PSTH relative to bar touches in engaged trials.It was from 1340 to 940 ms before bar touch for the PSTH relative to spontaneous bar touches, which the monkeys occasionally made after a trial.We chose the latter window in order to match the separation between the reference window and bar touch in engaged trials, i.e., the median RT of 938 ms.

Population correlation coefficients between spike rate and reaction time
We computed population correlation coefficients between spike rate and RT using the following steps.
For each unit and for each 20-ms bin relative to light onset, we computed the Spearman correlation coefficient between the spike rate and the RT across trials.For this computation, only engaged trials with RT >920 ms were used.Their number ranged from 22 to 719 for different units, with a median of 329 trials.The correlation coefficients of individual units were then averaged across units after subtracting the mean correlation during the 400-ms window right before light onset for each unit.

Analysis of neuronal responses to sounds in the audio-visual tasks
For each unit, we computed PSTHs by averaging the spike rates across trials relative to the onset of different sounds that were presented at S1 or S2 during task performance.A unit was considered to be responsive to a given sound if, in at least one of the PSTHs, the spike rates in a minimum of two consecutive bins from 0 to 300 ms after sound onset deviated more than three standard deviations from the mean spike rate during the 100-ms period directly before sound onset.
J o u r n a l P r e -p r o o f

Bandpass filtering of local field potentials
The LFP recorded from a given site was bandpass filtered into six separate frequency bands by using fourth-order Butterworth filters with a bandwidth of 0.8 octaves at the center frequencies of 2.5 ('delta'), 5 ('theta'), 10 ('alpha'), 20 ('beta'), 40 ('low-gamma'), and 80 Hz ('gamma').The filtering was performed on the continuous LFP recorded during a behavioral session before it was cut into individual trials.Because this study focused on the amplitude of the LFP, and not on its phase, we calculated the root mean square (RMS) value of each data sample for each LFP frequency band.We used the RMS value rather than the power to ease the comparison between the LFPs and the spiking activity.

LFP ratios
Similar to the spiking activity, for a given LFP band at a given site, we first averaged the RMS values across the trials of a condition to obtain how the evoked LFP amplitude evolved over time relative to light onset (e.g., Figure 1D).The light LFP ratio was then defined as the mean evoked LFP amplitude across all data samples during the 400-ms window right after light onset divided by the mean during the 400-ms window right before light onset.The baseline LFP ratio was the mean amplitude during the 400-ms window right before light onset divided by the mean amplitude during the window from 800 to 400 ms before light onset.

Population local field potential amplitudes
For each site, we first normalized the evoked LFP amplitude in a condition to the mean amplitude of all data samples during a reference window.We then down-sampled the normalized evoked LFP amplitude to 50 samples/s (to have the same temporal resolution as for the spiking activity) and averaged the normalized evoked LFP amplitude across the selected sites to obtain the population LFP amplitude.The reference window was the 400-ms window right before light onset for the light evoked amplitude.It was from 400 to 800 ms after bar touch for the LFP amplitude relative to bar touches in L, respectively), using the same methods as in Huang and Brosch (2024).The number of eye positions obtained here was smaller than the number used in Huang and Brosch (2024), because the current study included only sessions in which the monkeys performed both Task 1 and Task 2. Eye positions ranged from 1 to 20 pixels in the horizontal plane.We estimated that one pixel value is equal to 6.6 visual degrees.

Statistical tests
Depending on research question, we conducted different statistical tests to compare conditions, using an alpha value of 0.05.For multiple comparisons, the alpha value was corrected using Bonferroni Correction.

J o u r n a l P r e -p r o o f
We performed chi-square tests to find out whether the percentages of units or sites whose activity varied with eye position were significantly higher than 5%, the value expected by chance.The tests were performed for each of the 81 time bins from 1000 ms before to 2240 ms after light onset.
When we compared the neuronal population activity between two conditions, we conducted permutation tests on each of the 20-ms bins during a given time window.The time window was from 0 to 400 ms after light onset for the comparison of the light evoked activity.It was from 940 ms before to 0 ms after bar touch for the comparison of the activity relative to bar touches in engaged trials.This time window was used because it was equivalent to the median RT of 938 ms between light onset and bar touch.The same window was used when we tested whether the neuronal activity relative to spontaneous bar touches was significantly different from 1.
The significance of the correlation coefficients between the neuronal activity and the RT was determined by using permutation tests, which were conducted on each of the 20 bins during the 400ms time window after light onset.

Light stimuli can evoked spiking activity in auditory cortex
The current study shows that light stimuli can evoke spiking activity in the AC of monkeys, which confirms and extends earlier observations (Brosch et al., 2005(Brosch et al., , 2015)).Figure 1B and 1C show how the spiking activity of an example single unit (SU) and an example multiunit (MU) changed after light onset when the monkeys performed the AV tasks.To quantify these changes, we computed the light spike ratio by dividing the mean spike rate of a unit during the 400-ms window after light onset by that before light onset.The light spike ratios of the two units were 1.4 and 1.1, reflecting increases in spike rate after light onset.The two ratios differed significantly from 1 (p<0.05,permutation tests).
Light spike ratios significantly different from 1 were found in 322 of a total of 763 SUs (42.2%, Figure S1Ai) and in 778 of a total of 1244 MUs (62.5%, Figure 2A) that were recorded from AC during 399 sessions in which the two monkeys performed AV tasks.The majority of the significant light spike ratios was >1 and the minority was <1 (207 vs. 115 SUs; 620 vs. 158 MUs).In the following we show J o u r n a l P r e -p r o o f that only significant light spike ratios >1 were due to light stimulation because they cannot be explained by confounds or artefacts.
We found that the significant light spike ratios >1 were not only due to random fluctuations or to an ongoing increase in the spike rate that had started already before light onset.We estimated the incidence of such baseline changes by calculating the ratio between the mean spike rates during the two consecutive 400-ms windows directly before light onset.This revealed that 15.6% of the SUs and 24.8% of the MUs had significant baseline spike ratios >1 (Figure S1Ai and 2A).These percentages were significantly smaller (p<0.05,chi-square tests) than respective percentages of significant light spike ratios >1 found for SUs (27.1%) and MUs (49.8%).Thus, the spiking activity of many neurons in AC significantly increased after light onset.In contrast, the spiking activity in AC did not significantly decrease after light onset because light spike ratios <1 could be explained by random fluctuations or by spike rate decreases starting already before light onset.For SUs, the percentage of significant light or baseline spike ratios <1 did not differ (15.1% vs. 11.7%;p>0.05).For MUs, significant light spike ratios <1 did not occur more often, but rather occurred less often, than significant baseline spike ratios <1 (12.7% vs. 24.6%;p<0.05).Consequently, light spike ratios <1 will not be interpreted further.
The significant light spike ratios >1 were also not due to artefacts that were potentially produced by turning on the light, including sound artefacts.This was revealed when we calculated the percentages of units with significant light spike ratios >1 for non-engaged trials in which the monkeys did not react to the light.These percentages were small (3.5% of 544 SUs and 9.6% of 790 MUs, Figure S1Aii and 2B) and much smaller than the percentages for the spike ratios computed from all trials in the behavioral session.Moreover, the percentages observed in non-engaged trials were not distinguishable from the percentages of units with significant spike ratios >1 obtained in a condition in which no artefacts synchronized with the light could be present, i.e., in the 800-ms window before light onset (4.0% for SUs and 9.4% for MUs; p>0.05, chi-square tests; Figure S1Aii and 2B).This indicates that, also in engaged trials, the significant light spike ratios >1 were not due to artefacts.Some studies have reported that eye position can affect both ongoing and sound-evoked spiking activity in AC (Fu et al., 2004;Werner-Reiss et al., 2003; but see Huang and Brosch, 2024).We therefore tested whether the significant light spike ratios >1 can be accounted for by the eye positions J o u r n a l P r e -p r o o f of the monkeys around light onset.Eye position differed, on average, 6° in the horizontal plane and 4° in the vertical plane between the two 400-ms windows used to calculate the light spike ratio.These differences are in a range similar to the one for which eye position effects have been reported.We therefore examined, by using Kruskal-Wallis tests (α=0.05),whether the spike rate of a unit varied with eye position.These tests were performed for each of the 81 time bins from 1000 ms before to 2240 ms after light onset and for each of the 302 SUs and the 449 MUs for which such data were available.
For all time bins, the percentage of units whose spike rate varied significantly with eye position was close to 5%, the value expected by chance if the spike rate is independent of eye position (Figure S1Aiii and iv for SUs; Figure 2C and D for MUs; chi-square tests, α=0.05/81).This indicates that the differences in eye position before and after light onset cannot explain the significant light spike ratios.This lack of eye position effects also implies that in experiments like ours, with head and light positions fixed, the spike rate in AC is largely invariant to which parts of the retina were illuminated by the light stimulation.
Spiking activity in AC can be related to and precede body movements, including grasping (Brosch et al., 2005).Because the monkeys touched the bar with RTs between 400 ms and 4000 ms after light onset (median RT of 938 ms), it is possible that movement-related activity was included in the 400-ms window after light onset and accounted for the significant light spike ratios >1.To clarify this issue, we determined the time range within which spiking activity in AC was exclusively related to bar touch.For each unit, we sorted all trials into trials with RTs shorter or longer than the median RT in the behavioral session and computed the time course of the spike rates separately for the two types of trials.After averaging the spike rates across the 763 SUs (Figure S1Av) and the 1244 MUs (Figure 2E), we found that the population spike rates were increased for several hundred milliseconds before bar touch and that they did not differ between trials with short and long RTs from 520 ms and 480 ms before bar touch for SUs and MUs, respectively (α=0.05/47, permutation test).This means that starting from about 500 ms preceding bar touch, the spike rates were independent of when the light was illuminated and thus were exclusively related to bar touch.
An alternative way to estimate the time range during which the spiking activity was exclusively related to bar touch was to analyze spontaneous bar touches, which the monkeys occasionally made after a trial.The population spike rates relative to such bar touches were significantly >1 at the earliest These two analyses indicate that spiking activity in AC can be related to bar touch as early as 520 ms before bar touch.Therefore the 400-ms window after light onset may contain such activity in trials with RTs <920 ms.Because these trials constituted almost half of all trials and to exclude the possibility of contamination by bar touch related activity, we recalculated the light spike ratios for all 763 SUs and 1244 MUs by including only trials with RTs >920 ms.For this recalculation, we also considered that spike rates did not increase in non-engaged trials and thus excluded these trials.We found significant light spike ratios >1 in 155 SUs (20.3%, ranging from 1.04 to 1.98), and in 506 MUs (40.7%, ranging from 1.03 to 1.87).The median light spike ratios across these units were significantly greater than the median baseline ratios (p<0.05,permutation test; 1.17 vs. 1.00 for SUs; 1.13 vs. 0.99 for MUs).In addition, the vast majority of these units had larger light spike ratios than baseline spike ratios (95.5% of the SUs and 92.1% of the MUs; blue dots in Figure S2A).We therefore conclude that light stimulation caused an increase in the spiking activity of these units and henceforth called them visual units.
We note that all visual units identified here are bisensory because we considered only units that were recorded from sites in AC that responded to the passively presented pure tones or complex sounds.Although all recorded units were auditory, only about 70 % of them responded also to the sounds in the AV tasks while the remaining (206 SUs and 369 MUs) did not do so, which raises the question whether these two groups of units differed in their sensitivity to the light stimuli in the tasks.
Significant light spike ratios >1 were found in both groups of units, more frequently in the MUs responding to the task sounds than in the MUs not responding to the task sounds (43.5% vs. 33.9%;p<0.05, chi-square test) while similarly often in the two groups of SUs (19.8% vs. 21.8%;p>0.05).This indicates that, in AC, there exist neurons that respond to both the auditory and the visual components of AV tasks and neurons that respond only to the auditory or only to the visual component of the tasks.
J o u r n a l P r e -p r o o f

Visual activations in auditory cortex occur in different tasks and for different visual stimuli
The results presented thus far were obtained by pooling the trials in different AV tasks and with different light stimuli.To answer the question whether visual activations in AC occur in different tasks and with different visual stimuli, we computed light spike ratios of the 155 visual SUs and the 506 visual MUs for different conditions.We found significant light spike ratios >1 in all four tasks (colored bars in Figure 3; see Figure S3 for individual monkeys).About half of these visual units had significant light spike ratios in only one of the two tasks the monkeys performed in the same behavioral session (red and green bars).Because we used a red light on the monkeys' left in one task and a green light on the monkeys' right in the other task, our findings suggest that these visual units in AC are selective either for the physical features of the visual stimulus or for its behavioral meaning, i.e., how to process the upcoming sounds in the task.Many other visual units had significant light spike ratios in both tasks the monkeys performed in the same session (yellow bars).These units may represent the common behavioral meaning of the visual stimuli in the tasks, e.g., that the monkeys had to touch the bar to continue the trial.

Visual activations in auditory cortex are related to the monkeys' behavior in audio-visual tasks
Our finding that more units had significant light spike ratios >1 in engaged than in non-engaged trials (20.3% vs. 3.5% for SUs; 40.7% vs. 9.6% for MUs) indicates that visual activations in AC were related to whether or not the monkeys reacted to the light by touching the bar.The existence of this relationship is further supported when we compared the light spike ratios in these two types of trials for the units that had significant light spike ratios >1 either in engaged or in non-engaged trials.We found that the vast majority of the units had larger light spike ratios in engaged trials (93 of 116 SUs; 239 of 320 MUs; Figure S2B), which led to a significantly larger median light spike ratio across all the units in engaged than in non-engaged trials (1.16 vs. 1.02 for SUs; 1.11 vs. 1.01 for MUs; p<0.05, permutation tests; Figure 4A and B; see Figure S4A and B for individual monkeys).
To examine whether there are other relationships between visual activations in AC and the monkeys' behavior in the tasks, we investigated whether the visual activations were related to the time when the monkeys reacted to the light.For each of the 155 visual SUs and 506 MUs, we computed J o u r n a l P r e -p r o o f the light spike ratios separately for trials with RTs shorter or longer than the median RT, considering only trials with RT >920 ms.The median light spike ratio across all visual units was greater in trials with short RTs than that in trials with long RTs (1.19 vs. 1.17 for SUs, Figure 5A; 1.13 vs. 1.10 for MUs, Figure 5D; see Figure S5A and B for individual monkeys), which was significant for MUs (p<0.05,permutation test).We confirmed both findings when we compared the light spike ratios of individual units between the two types of trials (Figure S2C).Because light spike ratios were computed by dividing the spike rate before and after light onset, the differences in ratios either reflected that the spike rates after light onset were higher in trials with short RTs, or that baseline spike rates were lower in these trials.We could exclude the latter possibility because the baseline spike rates were not lower, but higher, in trials with short RTs.
Because the above results suggest that visual activations in AC were related to the monkeys' RT to the light, we compared the time course of population spike rates of the visual MUs in trials with short and long RTs.Consistent with the results from the light spike ratios, population spike rates during the 400-ms window after light onset were generally higher in trials with short RTs than in trials with long RTs (Figure 5E).Spike rate differences were significant (p<0.05/20,permutation tests) at the earliest 180 ms after light onset.This indicates that mainly the late part of the visual activations was related to the monkeys' reaction to the light.This bias towards the late activation could be the reason why the median light spike ratio of the SUs did not significantly differ between the two types of trials when the entire 400-ms time window was analyzed.We therefore also examined the activations of the 155 visual SUs with a finer temporal resolution by computing population spike rates.The population spike rates during the 400-ms window after light onset differed significantly (p<0.05/20) between trials with short and long RTs and the significant difference was found 320 ms after light onset (Figure 5B).Therefore, also mainly the late part of the visual SU activations was related to the monkeys' reaction to the light.
Our results suggest that visual activations in AC could contribute to the monkeys' reaction to the light in the AV tasks.If so, there should be trial-by-trial relationships between the spiking activity and the RT in individual units.To test this possibility, we computed for each visual unit the correlation coefficient between the spike rate and the RT, separately for each 20-ms bin from 400 ms before to 800 ms after light onset, following the approach used by Chandrasekaran et al. (2013).Figure 5C and J o u r n a l P r e -p r o o f F show the correlation coefficients averaged across the 155 visual SUs and the 506 visual MUs, respectively, after subtracting the mean correlation coefficient during baseline for each unit.The correlations were more negative during the 400-ms window after light onset than during baseline, which was significant as early as 320 ms after light onset in SUs and 80 ms in MUs (p<0.05/20,permutation tests).This suggests that there were trial-by-trial relationships between the spiking activity and the monkeys' reaction to the light: the stronger the spiking activity, the faster the reaction.The most negative correlation was about -0.03 for SUs and -0.04 for MUs, similar to the values previously observed in AC (Chandrasekaran et al., 2013) and PFC (Wimmer et al., 2014).This was observed about 400 ms after light onset, suggesting that the spike rates predicted best the monkeys' reaction to the light at this time.Based on these results, we consider that the visual spiking activity in AC could contribute to the acceleration of the monkeys' reaction to the light.We note that because trials with RTs <920 ms were excluded from our analyses, the visual spiking activity does not include activity that is time-locked to the monkeys' motor response.The visual spiking activity is rather related to the visual stimuli themselves or to the process of mapping visual stimuli onto motor responses.
In addition to carrying information about the monkeys' behavior to a foreign-modal stimulus, visual activations in AC also carried information about the monkeys' behavior to the dominant-modal stimuli, i.e., the upcoming sounds.This was revealed when we compared the light spike ratios in trials with correct behavioral responses to sounds with the spike ratios in incorrect trials.The comparison was limited to the 146 visual SUs and 467 visual MUs with sufficient data for both types of trials.The median light spike ratio across all visual units was greater in correct trials than in incorrect trials (1.18 vs. 1.13 for SUs; 1.12 vs. 1.11 for MUs; Figure 6A and C; see Figure S6A and B for individual monkeys), which was significant for the visual SUs (p<0.05,permutation test).These differences between correct and incorrect trials were further supported when we compared the light spike ratios of individual units (Figure S2D).They were also reflected in the population spike rates which were significantly higher in correct trials (Figure 6B and D;p<0.05/20,permutation tests).Such a significant difference was found in one early and two late time bins during the 400-ms window for the SUs and in three late time bins for the MUs.We therefore consider that mainly the late part of visual activations in AC contribute to the mapping of sounds onto appropriate motor responses.
J o u r n a l P r e -p r o o f

Effects of visual stimuli on local field potentials in auditory cortex
To find out whether visual activations were also present in neuronal signals other than action potentials and whether such activations were also related to the monkeys' behavior we analyzed LFPs.Because foreign-modal stimuli can affect different frequency bands of the LFPs in early sensory cortex (Kayser et al., 2008;Lakatos et al., 2007Lakatos et al., , 2009)), we performed this analysis on the amplitudes of the six LFP frequency bands, centered on 2. 5, 5, 10, 20, 40, or 80 Hz.For each band and for each of the 1130 recording sites in AC, we computed the light LFP ratio as the ratio of the mean RMS values of the LFPs during the 400-ms windows right before and after light onset, disregarding phase information.In all bands, we found light LFP ratios that were significantly different from 1.However, only for the 80-Hz ('gamma') and the 10-Hz ('alpha') band, we found that the significant light LFP ratios were due to light stimulation, rather than to other factors.We therefore report only the results obtained from these two bands.

Gamma activity
The results obtained from the gamma band mirrored the results obtained from the spiking activity, in particular those from the MUs.Only significant light gamma ratios >1, and thus only increases in gamma activity, were related to light stimulation because they could not be solely explained by ongoing changes in the activity (Figure S1B, i), artefacts (ii), eye position (iii, iv) or grasping (v, vi).Such visual gamma activity was found at 520 sites (46.0% of all 1130 sites; blue dots in Figure S2A).It was related to whether or not the monkeys reacted to the light (Figure 4C, S2B and S4C), was significantly greater in trials with short RTs than in trials with long RTs (Figure 5G-I, S2C and S5C) and was significantly greater in trials in which the monkeys reacted to sound correctly than in incorrect trials (Figure 6E, 6F, S2D and S6C).It was found in each of the four tasks and for the two visual stimuli (Figure 3 and S3).Owing to these similarities, we consider that visual gamma and spiking activity in AC have similar functional roles, consistent with the generally tight correlation between the two signals (Ray and Maunsell, 2010;Ray et al., 2008).
J o u r n a l P r e -p r o o f

Alpha activity
The results obtained from the alpha band mirrored partially the results obtained from the spiking activity.Only increases in alpha activity after light onset could not be solely accounted for by confounds or artefacts (Figure S1C) and were thus related to light stimulation.Visual alpha activity was found at 737 sites (65.2% of 1130 sites; blue dots in Figure S2A) and in all four tasks and for the two visual stimuli (Figure 3 and S3).It was also related to the monkeys' behavior to both the light (Figure 5J-L, S2B, S2C and S5D) and the sound stimulation (Figure 6G, 6H, S2D and S6D).These similarities suggest that visual alpha activity has functional roles overlapping with those proposed for the spiking activity.
The visual alpha activity, however, also differed from the spiking activity and thus may have other functional roles.First, visual alpha activity was significantly lower in trials with correct behavioral responses to sounds than in trials with incorrect responses (Figure 6G, 6H, S2D and S6D), suggesting that increased visual alpha activity was associated with degraded mapping of sounds onto appropriate motor responses in the AV tasks.
Second, visual alpha activity was also present in non-engaged trials in which the monkeys did not touch the bar after light onset (Figure S1Cii).More sites had significant light alpha ratios >1 in these trials than in a condition without light stimulation (Figure S1Cii; p<0.05, chi-square test).This finding indicates that light stimuli reached the monkeys' eyes in some of the non-engaged trials and that the visual stimulus itself was sufficient to drive alpha activity in AC.In addition, there was no clear difference in the median light alpha ratio between the non-engaged and the engaged trials (p>0.05,permutation test; Figure 4D; see Figure S4D for individual monkeys).These observations indicate that the absence of visual spiking activity in non-engaged trials (Figure 2B and S1Aii) cannot be explained by the possibility that the light stimuli did not impinge on the retina in these trials.
Third, there might be an early time period during which the population alpha activity was slightly lower, but not higher, in trials with short than with long RTs (p>0.05/20,permutation tests; Figure 5K).This relationship was also reflected in a brief period with a significant positive trial-by-trial correlation between the alpha activity and the RT (p<0.05/20; Figure 5L).Thus the early visual alpha activity might be associated with a retardation of the monkeys' reaction to the light.This is in contrast to the J o u r n a l P r e -p r o o f late visual alpha activity (starting 200 ms after light onset) which, like the spiking activity, was associated with an acceleration of the monkeys' reaction to the light.
In contrast to spiking activity, alpha activity was not related to bar touch.In engaged trials, the population alpha activity was not exclusively related to bar touch even around the time the monkeys touched the bar (Figure S1Cv).Moreover, the population alpha activity only weakly varied relative to spontaneous bar touches (Figure S1Cvi).

Visual activations in prefrontal cortex
To understand functional roles of visual activations in AC further, we compared AC with vlPFC.We recorded neuronal activity from this region during 280 behavioral sessions in parallel with the activity from AC.Using the same analyses as for the AC data, we found significant light ratios in both the spiking activity and the LFPs in vlPFC.Only for the spiking activity, but for none of the six LFP bands, we found that the significant light ratios were related to light stimulation.
The results obtained from the spiking activity in vlPFC (Figure 7 and S7) were largely similar to those obtained from AC.Only the increases in the activity were related to light stimulation because they could not be solely explained by ongoing changes of the activity (Figure S7A and S7B, i), artefacts (ii), eye position (iii, iv) or grasping (v, vi).This visual activity was related to the monkeys' reaction to the light (Figure 7D-G and S7Ci-iv) and was found in all four tasks and for the two visual stimuli (Figure 7C).Visual spiking activity was also present in non-engaged trials (Figure S7Bii), which is similar to the alpha activity in AC.
Different from AC, visual spiking activity in vlPFC was not related to the monkeys' behavioral response to sounds (Figure 7H and I; S7Cv and vi).The light spike ratios did not differ significantly between trials in which the monkeys responded correct or incorrectly to sounds (p>0.05,permutation tests; Figure 7H and S7Cv).Also, the population spiking activity of the visual units during the 400-ms window after light onset did not differ significantly between correct and incorrect trials (p>0.05/20,permutation tests; Figure 7I and S7Cvi).Finally, different from the spiking activity but like the alpha J o u r n a l P r e -p r o o f activity in AC, the spiking activity in vlPFC was only weakly related to bar touch (panels v and vi in Figure S7A and B).

Discussion
The current study extends the knowledge of neuronal activity in early sensory cortex that is evoked by stimuli from a foreign modality.We find such activations in the spiking activity and in the LFPs in AC in different AV tasks and for different visual stimuli.The visual activations are related to the monkeys' behavioral responses to the light and the sound stimulation.This suggests that foreign-modal activations in early sensory cortex have functional roles for performing tasks also in normal animals and not only in sensory deprived animals (Lomber et al., 2010).

Visual activations in auditory cortex are related to the mental state of animals
The current study confirms and extends our earlier observations of visual spiking activity in AC (Brosch et al., 2005(Brosch et al., , 2015)).We show that more spikes, particularly during the late part of the 400-ms time window after light onset, can be considered to be evoked by light stimuli and that there are more visual units than previously identified in AC (Brosch et al., 2005).We also show visual activations in SUs and in LFPs.The latter observation indicates that visual stimuli also encoded in neuronal signals other than action potentials in AC.Most importantly, we find that the occurrence of visual activity in AC is related to whether the monkeys engage in AV tasks.
The absence of visual spiking activity in non-engaged trials in our study appears to be in conflict with the presence of visual responses in the AC of passively stimulated awake or anesthetized animals (Bigelow et al., 2022;Bizley et al., 2007, Kobayasi et al., 2013, Morrill and Hasenstaub, 2018).
However, there may be no conflict because it can be questioned whether the non-engaged trials constituted a passive condition.The monkeys used in our study were highly trained on the AV tasks and the light stimuli likely had a behavioral meaning for them both in engaged and non-engaged trials, which were interleaved during a behavioral session.If this conjecture is true one wonders why visual spiking activity was absent in non-engaged trials and not observed more frequently than reported in J o u r n a l P r e -p r o o f passive conditions.The conflict may resolve if one assumes that visual stimuli drive AC in passive conditions and that in active conditions this drive is modulated by the animals' mental state such that visual responses are amplified in engaged trials and attenuated in non-engaged trials.In some conditions, e.g., when animals perform a purely visual task (Selezneva et al., 2018), the attenuation may be so strong that no visual responses are observed in AC.It is unlikely that the modulations take place outside AC and affect the visual input to AC from the ascending auditory pathway (Itaya and van Hoesen, 1982) or from visual and multisensory cortices (Bizley et al., 2007;Budinger et al., 2006;Henschke et al., 2015).This is supported by finding that visual alpha activity was present both in nonengaged and engaged trials (Figure 4D) and by the fact that the alpha band of LFPs largely reflects synaptic inputs (Buzsáki et al. 2012).Therefore, it is likely that the modulations of visual activations take place inside AC, possibly, by a gating mechanism which results in a strong relationship between the spiking output and the subject's behavior (Figure 4A and B).

Putative functional roles of visual activations in auditory cortex
We consider different functional roles for the visual activations in AC.Their link to whether and when the monkeys reacted to the light stimulation suggests that visual activations in AC carried information about the animals' performance on the visual component of the AV task, meaning that AC temporarily becomes part of the neuronal network that is involved in visually guided behavior.This cross-modal recruitment would complement findings that the AC is recruited for visual functions in deaf animals (Lomber et al., 2010).A similar functional role of foreign-modal activations has also been proposed for visual cortex in purely tactile tasks (Vasconcelos et al., 2012).
The link of the visual activations in AC to the monkeys' response to sounds suggests that visual activations carried information related to the monkeys' performance on the auditory component of the AV task, meaning that they inform AC about future sounds and the rules according to which these sounds should be processed and mapped onto motor responses.Such cross-modal priming may facilitate future auditory processing in AC over a time scale of seconds.It differs from cross-modal modulation of auditory processing in AC by concurrent visual stimuli, which acts on a subsecond time J o u r n a l P r e -p r o o f scale and has been proposed to contribute to the perception of multisensory objects (Atilgan et al., 2018;Schroeder et al. 2001).
The functions of visual activations in AC proposed here are derived from the findings obtained when monkeys performed AV tasks in which light stimuli signaled the beginning of a trial, the time window within which a bar touch was required to trigger sounds, as well as how to process future sounds and map them onto motor responses.It is Iikely that cross-modal recruitment also occurs in other AV tasks with different visual stimuli and with different visually guided behavior (Brosch et al., 2015).It is also Iikely that cross-modal priming occurs not only in our AV tasks in which visual stimuli provide information about how future sounds need to be memorized or discriminated, but also in AV tasks in which sounds need to be processed in other ways, e.g., need to be categorized or discriminated from visual stimuli (Brosch et al., 2005(Brosch et al., , 2015)).It is unclear whether visual activations have similar functional roles in AV tasks in which visual stimuli do not provide information relevant for auditory processing, in tasks without auditory components, and in passive conditions.
The current study suggests that AC is part of a neuronal network that operates not only on the auditory but also on the visual component of AV tasks.To shed more light on the potential contribution of the visual activations to the performance of AV tasks, we compared AC with a different part of the neural network and selected the vlPFC because it is causally involved in the performance of AV tasks (Plakke et al., 2015).Because we found that visual activations in vlPFC were related to similar aspects of AV tasks as those in AC, we consider that vlPFC has functions overlapping those of AC.Thus, vlPFC may also be recruited for visually guided behavior and may also carry information about the rules according to which future sounds should be processed and mapped onto motor responses.
However, compared to AC, visual activations in vlPFC are only weakly linked to the monkeys' task engagement and not linked to the correctness of the monkeys' responses to sounds.These observations suggest that AC and vlPFC are on a similar functional level in the network dealing with the rules by which sounds should be processed and are on different levels in the network controlling visually and auditory guided behavior.
J o u r n a l P r e -p r o o f 4.3.How foreign are visual stimuli to auditory cortex?
The current study suggests that, in addition to carrying information about low-level aspects of visual stimuli, such as their position or brightness (Bizley et al., 2007;Kobayasi et al., 2013, Morrill andHasenstaub, 2018;Wallace et al. 2004), AC activations carry information about high-level aspects of visual stimuli, such as the motor response prompted by them and the rules by which future sounds should be processed.Because AC also carries information about both low-level and high-level aspects of auditory stimuli (Scheich et al., 2011;Weinberger 2011), one could argue that there is no principal difference regarding the information about visual and auditory stimuli present in AC and that visual stimuli are not foreign to AC.
However, there are also arguments that visual and auditory activations in AC differently contribute to perception and task performance.(1) Many more neurons in AC respond to auditory than to visual stimuli and most neurons that respond to visual stimuli also respond to auditory stimuli (Bigelow et al., 2022;Bizley et al., 2007;Brosch et al., 2005;Morrill and Hasenstaub, 2018).The dominance of auditory processing is further highlighted by the finding that visual activations in the AC of mice are limited to infragranular layers (Bigelow et al., 2022) and that manipulations of AC activity change exclusively auditory perception (Bhaya-Grossman and Chang, 2021;Slonina et al., 2022;Tsunada et al., 2016).(2) Auditory activations are less strongly related to the state of animals than visual activations.The former vary only moderately with mental state (Knyazeva et al., 2020) and do not disappear even under deep anesthesia.In contrast, visual activations can even be absent in nonengaged conditions (e.g., Figure 2B).Thus auditory processing is the basic and immutable mode of AC while visual processing is a volatile mode to which AC switches upon demand.(3) Visual and auditory activations vary differently with the mental state of animals.During task engagement visual activations are amplified (Figure 4A and B), whereas auditory activations are suppressed (Knyazeva et al., 2020;Otazu et al., 2009).

Foreign-modal activations in early sensory cortex
Foreign-modal activations are found in AC also for tactile stimulation (Fu et al., 2003;Lemus et al., 2010;Schroeder et al., 2001) and are also found in visual, somatosensory, and olfactory cortices J o u r n a l P r e -p r o o f (Fishman and Michael, 1973;Gnaedinger et al., 2019;Headley and Weinberger, 2015;Lemus et al., 2010;Morrell, 1972;Varga and Wesson, 2013;Wesson and Wilson, 2010;Wu et al., 2023).They can also be related to low-level aspects of foreign-modal stimuli, such as their spatial position on the skin and the frequency of sounds (Fishman and Michael, 1973;Fu et al., 2003).However, whether these foreign-modal activations are also related to high-level aspects of foreign-modal stimuli and have functional roles in performing multisensory tasks, as proposed here for visual activations in AC, is unclear.Because tactile activations in AC and auditory activations in somatosensory cortex have been reported to be unrelated to a subject's perceptual judgement (Lemus et al., 2010), it could be that foreign-modal activations only in certain sensory cortices are related to high-level aspects of foreignmodal stimuli in specific tasks.
J o u r n a l P r e -p r o o f engaged trials.It was from 1340 to 940 ms before bar touch for the LFP amplitude relative to spontaneous bar touches.2.4.8.Population correlation coefficients between the local field potential and the reaction time Similar to the spiking activity, we computed the Spearman correlation coefficient between the downsampled evoked LFP amplitude of a site and the RT across trials.The correlation coefficients of individual sites were then averaged across sites after subtracting the mean correlation during the 400ms window right before light onset for each site.2.4.9.Analysis of eye position For a given behavioral session, we analyzed the video file that was recorded to obtain the position of the monkeys' right eye at different times from 1000 ms before to 2240 ms after light onset.Eye position was obtained from 2,672,126 video frames recorded during 56,290 trials of 107 behavioral sessions from the two monkeys in Part I of the study (90 and 17 sessions from Monkey C and Monkey J o u r n a l P r e -p r o o f 220 ms before bar touch in the 314 SUs and at the earliest 380 ms before bar touch in the 731 MUs (Figure S1Avi and 2F; α=0.05/47, permutation test).

Figure 1 .
Figure 1.Audio-visual tasks and examples of visual activations in auditory cortex.(A) Scheme of the AV tasks.After the illumination of LED lights, the monkeys had to grasp and hold a bar to trigger the two sounds S1 and S2, which they had to discriminate by releasing the bar at the appropriate time in order to obtain reward.The color and spatial position of the light, the sounds used for S1 and S2,

Figure 2 .
Figure 2. Multiunit activity in auditory cortex is evoked by light stimulation.(A) Cumulativeprobability distributions of light (black) and baseline (purple) spike ratios of 1244 multiunits.Dots mark ratios significantly different from 1.More multiunits had significant light ratios >1 than baseline ratios >1, which was not the case for significant ratios <1.Spike ratios were computed by using all trials in a session, irrespective of whether the monkeys reacted to the light (engaged trials) or not (non-engaged trials).(B) Cumulative probability distributions of light spike ratios and baseline spike ratios are similar in non-engaged trials.(C, D) Percentages of multiunits whose spiking activity varied significantly with horizontal or vertical eye position at different time points.At all times, these percentages were close to the chance level of finding such multiunits (dashed line).(E) Normalized

Figure 3 .
Figure 3. Visual activity in auditory cortex occurs in four audio-visual tasks and for two visual stimuli.Histograms showing the percentage of visual (closed bars) and non-visual (open bars) units, or sites, in the two parts of the study.Red bars denote visual units or sites with significant light ratios >1 only for the red light in Task 1 or Task 3. Green bars denote cases with significant light ratios >1 only for the green light in Task 2 or Task 4. Yellow bars denote cases with significant light ratios >1 for both the red and the green light in the two tasks tested in the same behavioral session.Gray bars denote cases with significant light ratios >1 only when trials with the red light and trials with the green light in the same session were pooled.Task 1 and Task 2 were performed in Part I of the study.Task 3 and Task 4 were performed in Part II of the study.Trials with RT >920 ms were used for the analyses.

Figure 4 .
Figure 4. Visual activity in auditory cortex is related to whether or not the monkeys engage in a task.Light ratios were larger in engaged (black) than in non-engaged trials (gray) for single units (A), multiunits (B), and gamma activity (C), but not for alpha activity (D).J o u r n a l P r e -p r o o f

Figure 5 .
Figure 5. Visual activity in auditory cortex is related to the monkeys' reaction to the light.Left column: Cumulative probability distributions of light ratios in trials with short (orange) and long RTs (blue) for visual single units (A), multiunits (D), gamma activity (G) and alpha activity (J).Light ratios were larger in trials with short RTs for all neuronal signals.Middle column: Normalized neuronal population activities during the 400-ms window after light onset were greater in trials with short RTs than in trials with long RTs (stars mark significant differences).Right column: Population trial-by-trial correlation coefficient computed between the neuronal activity and RT at different times relative to light onset (stars mark correlation coefficients significantly different from 0).

Figure 6 .
Figure 6.Visual activity in auditory cortex is related to the monkeys' reaction to sounds.Left column: Cumulative probability distributions of light ratios in trials in which the monkeys reacted to sounds correctly (black) or incorrectly (red).In correct trials, light ratios were larger for visual single units (A), multiunits (C), and gamma activity (E), but smaller for alpha activity (G).Right column:Normalized neuronal population activity during the 400-ms time window after light onset was greater in correct than in incorrect trials for visual singe units (B), multiunits (D), and gamma activity (F).The relationship was opposite for alpha activity (H).
J o u r n a l P r e -p r o o f

Figure 7 .
Figure 7. Visual spiking activity in ventrolateral prefrontal cortex.(A, B) Visual spiking activity of an example single unit and an example multiunit.(C) Visual single unit and multiunit activity occurred in four AV tasks and for two visual stimuli.(D) Light spike ratios were larger in trials with short RTs to the light (orange) than in trials with long RTs (blue).(E) Normalized population spike rates during the 400-ms time window after light onset were greater in trials with short RTs than in trials with long RTs.(F) Population trial-by-trial correlation coefficients computed between spike rate and RT.(G) Light spike ratios were similar in engaged (black) and non-engaged trials (gray).(H) Light spike ratios were similar in trials with correct (black) and incorrect (red) responses to sounds.(I) Normalized population spike rates during the 400-ms time window after light onset did not differ in trials with correct and incorrect responses to sounds.