The major brain networks of human visual consciousness

Understanding consciousness is one of the most important and challenging questions in modern 2 science. Existing theories have pursued single unifying mechanisms but do not succeed in 3 explaining consciousness. Importantly, the neural circuits that distinguish messages that arrive 4 from the outside world and attain consciousness have remained unknown. Here we identify 5 signals throughout the entire brain at high spatiotemporal resolution specifically related to 6 consciousness. To accomplish this, we combined a large sample size of electrical and neuroimaging data with a novel experimental approach to remove confounding signal unrelated 8 to consciousness 1-3 . We discovered three major brain networks driving conscious visual 9 perception. First, we found increases in signal detection regions in visual, fusiform cortex, and 10 frontal eye fields; and in arousal/salience networks involving midbrain, thalamus, nucleus 11 accumbens, anterior cingulate, and anterior insula. Second, we found increases in frontoparietal 12 attention and executive control networks and in the cerebellum. Finally, we found decreases in 13 the default mode network. Our results identify subcortical and cortical networks designed for 14 signal detection, attentional amplification, and perceptual processing that together can explain 15 visual consciousness. These findings provide evidence that understanding consciousness can be 16 reframed as requiring multiple overlapping brain networks to produce consciousness of visual 17 events 4 .


1
Understanding consciousness is one of the most important and challenging questions in modern 2 science. Existing theories have pursued single unifying mechanisms but do not succeed in 3 explaining consciousness. Importantly, the neural circuits that distinguish messages that arrive 4 from the outside world and attain consciousness have remained unknown. Here we identify 5 signals throughout the entire brain at high spatiotemporal resolution specifically related to 6 consciousness. To accomplish this, we combined a large sample size of electrical and 7 neuroimaging data with a novel experimental approach to remove confounding signal unrelated 8 to consciousness 1-3 . We discovered three major brain networks driving conscious visual 9 perception. First, we found increases in signal detection regions in visual, fusiform cortex, and 10 frontal eye fields; and in arousal/salience networks involving midbrain, thalamus, nucleus 11 accumbens, anterior cingulate, and anterior insula. Second, we found increases in frontoparietal 12 attention and executive control networks and in the cerebellum. Finally, we found decreases in 13 the default mode network. Our results identify subcortical and cortical networks designed for 14 signal detection, attentional amplification, and perceptual processing that together can explain 15 visual consciousness. These findings provide evidence that understanding consciousness can be 16 reframed as requiring multiple overlapping brain networks to produce consciousness of visual 17 events 4 . Consciousness is central to human experience yet is not easily explained. Theories of 2 consciousness typically emphasize a single mechanism hoping to solve the mystery linking brain 3 activity to conscious experience. However, single-mechanism theories or models have so far not 4 achieved convincing success. Instead, we posit that consciousness is best understood through a 5 synergistic combination of multiple mechanisms overlapping in space and time. Just as several 6 processes in biology together distinguish living from non-living things, multiple mechanisms in 7 neuroscience combine to separate conscious from non-conscious neural activity. Specifically, 8 conscious awareness of discrete events lies at the temporal nexus of attention and memory, both 9 major fields in modern neuroscience that contribute to understanding consciousness. We 10 hypothesize that systems crucial for consciousness include: (1) attention mechanism mediating 11 signal detection, dynamic modulation of arousal, and bottom-up plus top-down attentional 12 control, overlapping in space and time with; (2) systems that limit competing signals (e.g., 13 through reduced default mode network activity); and, finally, (3) hierarchically organized 14 systems that fully process signals for memory encoding and subsequent report 4 .

16
To investigate these multiple systems, a comprehensive approach is needed to identify 17 activity throughout the brain at high spatial and temporal resolution specifically related to 18 consciousness. We aimed to overcome several limitations of prior studies. For example, 19 subcortical regions are poorly understood in human conscious perception and are often relegated 20 to preconscious state-based precursors of consciousness [5][6][7] . We therefore investigated dynamic 21 changes in both subcortical and cortical systems using techniques with complementary strengths. 22 These included scalp electroencephalography (EEG) with large sample size and depth recordings 23 from the human thalamus providing direct measurements of neural activity at high time 24 resolution, and functional magnetic resonance imaging (fMRI) with large sample size analyzed 25 with data-driven approaches providing comprehensive mapping of the whole brain. Most 26 importantly for linking these neural measures with consciousness, we used a threshold visual 27 perception task to measure brain signals produced by physically identical stimuli that are either 28 perceived versus not perceived, coupled with a unique innovation to remove the confound of 29 overt report. When participants are asked to overtly report whether they have perceived a 30 stimulus this introduces post-perceptual processes (e.g., decision making and motor planning) 31 that can confound signals linked to consciousness, even when the report is delayed by several 32 seconds after the stimulus 1-3 .

34
Identifying visual conscious perception without report 35 To address the confound of post-perceptual processing we developed a novel no-report paradigm 36 using transient changes in pupillometry and eye tracking to classify stimuli as perceived or not 37 perceived without overt report 8 (fig. S1-9; Tables S4-6). In a previously established Report 38 Paradigm 9 , participants were repeatedly shown identical faces at 50% perceptual threshold (Fig. 39 1A). This resulted in approximately equal numbers of perceived and not perceived stimuli based 40 on overt report of stimulus presence and location ( fig. S6, 7). We formed a novel combined 41 Report + No-Report Paradigm by maintaining the report, task-relevant stimuli from the Report perception of no-report stimuli was determined by classification of pupillometry and eye tracking during the task. Specifically, we found pupil dilation, blink rate increases, and microsaccade rate 1 decreases for consciously perceived visual stimuli, present irrespective of task relevance ( Fig.   2 1C, D). Pupillometry and eye tracking have been previously implemented as covert measures of 3 consciousness 10,11 . However, the dynamics reported here are unique because they do not rely on 4 a task sequence (e.g., no-report stimuli always first) 12,13 , changes in the stimulus (our stimuli are 5 all identical), perceptual switching (e.g., binocular rivalry) 10 , nor stimulus type (e.g., stimulus 6 modality), as we have found similar pupil, blink, and microsaccade responses for perceived 7 auditory and tactile stimuli 8,14,15 . 8 9 Report-independent event-related potentials in conscious visual perception 10 Our first goal was to investigate report-dependent and report-independent brain signals at high 11 time resolution. We recorded the following well established event-related potentials (ERPs) for 12 perceived stimuli in the report data, in temporal sequence: (1) N100, (2) VAN (visual awareness 20 S10E). Our novel paradigm thus strengthens previous findings that early ERPs, particularly the 21 VAN, are seen even under no-report or task-irrelevant conditions. Therefore, these early signals 22 reflect the scalp neurophysiological signatures for report-independent conscious perception 20-22 . 23 Furthermore, report and task-relevance introduce report-dependent changes dominated by later Thalamic awareness potential 28 Having established early report-independent signals of conscious perception, and later report-29 dependent signals likely related to post-perceptual processing, we next sought to investigate 30 subcortical signals selective for conscious perception and to determine their timing. A key 31 subcortical brain structure for arousal and consciousness is the intralaminar thalamus 24 . Patient 32 participants with chronically implanted deep brain recording and stimulation devices (RNS® 33 System, NeuroPace, Inc.; Natus NeuroWorks, Inc.) for the treatment of epilepsy provided unique 34 access to this region 8 . We simultaneously recorded cortical electrophysiology from scalp EEG 35 and subcortical signals from thalamic depth contacts ( Fig. 2D; fig. S12, 13; Tables S2, 3) while 36 participants completed the Report Paradigm. We found a novel, biphasic thalamic awareness 37 potential (TAP) highly selective for perceived stimuli with an onset at ~250ms and initial peak at 38 ~430ms post-stimulus presentation (Fig. 2E). TAP was localized to channels within or along the 39 lateral border of the intralaminar thalamus ( Fig. S2D; fig. S12). TAP was also selectively present 40 for perceived auditory stimuli in one participant (participant 1 in Table S2) who completed an 41 analogous perceptual threshold auditory task in a separate study from our group (fig. S14) 14 .

43
We next investigated the timing of TAP relative to scalp ERPs. First, we noted that the 44 scalp ERPs recorded in the patient participants were similar to those of the healthy participants, 45 despite different recording systems and sample sizes (fig. S13). We found that TAP preceded P3, 46 but followed the N100, VAN, and N2 (Fig. 2F). Therefore, the timing of TAP was later than ERPs found in the prior experiment to be report-independent (VAN), and fell within or earlier than ERPs that were report-dependent (N2, P3). We did not directly test whether TAP was 1 report-dependent because of limited recording time with the patient participants. However, 2 because we hypothesized that TAP is one node in a broad subcortical arousal and salience 3 network participating in attention state dynamics and consciousness 25,26 , we next used fMRI in a 4 large cohort of healthy participants to investigate cortical and subcortical conscious perception-5 linked dynamics, with and without overt report. 6 7 Report-independent fMRI changes in conscious visual perception 8 We found a broad network of subcortical and cortical regions showing report-independent fMRI 9 changes especially at earlier times after stimulus presentation. Perceived versus not perceived To further investigate the main large-scale networks involved in conscious perception with and 3 without report and the temporal profile of these networks, we used temporal correlation-based k-4 means clustering across the entire brain 8 . Data-driven clustering of statistically significant voxels 5 for report perceived versus not perceived fMRI signals revealed three anatomically and 6 functionally distinct networks: (1) early positive, (2) late positive, and (3) late negative (Fig. 4A-7 F; fig. S17D, E, F). The early positive network has a peak at 3-4 seconds after stimulus onset and 8 includes subcortical and cortical detection, arousal, and salience networks (DAS). They include 9 FG, PMFG, MT, Th, NA, AC/SMA, AI, cerebellar vermis, and subregions of the Str, AIPL, SPL 10 and MP (Fig. 4A). The late positive network peaks ~6 seconds after stimulus onset and includes 11 task-positive networks (TPN) such as AMFG, OFC, FP, cerebellum Crus I and II, and subregions 12 of the Str, AIPL, SPL, and MP (Fig. 4B). The late negative network has a trough at 6-8 seconds 13 after stimulus onset and occupies DMN regions that are exclusively cortical, including the 14 VMFC, PC, PIPL, and ALT (Fig. 4C). Thus, three major and distinct brain networks for 15 conscious perception emerge from the fMRI data entirely based on BOLD timecourse dynamics.

17
How does overt report change the signal timecourses in these networks? Analysis of 18 mean timecourses for all voxels within each network revealed that only TPN was on the whole 19 different between report and no-report data at later times ( Fig. 4D-F). However, subregion 20 analyses showed that all three networks contained both regions that agreed and regions that 21 differed between report and no-report data (Fig. 4G-J; fig. S18-20). Importantly, the fMRI 22 timecourse for Th (>94% DAS voxels) did not differ between report and no-report conditions 23 (Fig. 4G). In contrast, the PMFG/FEF (>87% DAS voxels), DIPL (>82% TPN voxels), and 24 VMFC (>79% DMN voxels) had greater signal amplitude and duration for report versus no-25 report conditions (Fig. 4H-J). Similar to the conjunction and difference analyses already 26 discussed (Fig. 3E, F; fig. S16), additional timecourse analyses confirmed that some brain areas 27 (e.g., left motor cortex; fig. S19) are only significantly involved in report data, whereas many 28 more regions are shared between report and no-report data at early times, but show persistent or 29 larger signals in report data at later times (e.g., DIPL, SPL, MP; fig. S18-20).

32
We found that multiple mechanisms at the nexus of attention, signal processing and memory 33 formation contribute to conscious visual perception. Report-independent signals are early and 34 transient, and on EEG include the N100 and VAN. Report-independent signals from fMRI with 35 the identical behavioral paradigm and many of the same participants as EEG, showed early and 36 transient, but widespread involvement of three major subcortical and cortical brain networks. 37 The importance of subcortical networks in conscious perception was further supported by our 38 identification of the TAP through direct recordings from the human thalamus. Meanwhile, 39 report-dependent signals were late and persistent, and on EEG include the P2/N2 and P3/LP; and These findings support an approach to studying consciousness based on investigation of 46 multiple overlapping systems in neuroscience, rather than looking for a single theoretical model. 47 This approach has succeeded in the study of other important and complex biological functions (e.g., reproduction and digestion) where no single model is proposed, and instead multiple 1 contributing processes are found that provide key features. Our identification of specific report-2 independent subcortical and cortical systems overlapping in space and time is an important step 3 forward towards understanding the neural mechanisms of consciousness. These systems can be 4 placed along a proposed timeline consistent with the three major networks found in our data 4 . 5 First, (1) Detection/arousal/salience (DAS): when a visual stimulus appears, activation of V1 6 interacts with FG, FEF, and other regions for signal detection [27][28][29][30] . A dynamic transient pulse in 7 subcortical arousal (e.g., MT and Th) and emotional/motivational systems (e.g., NA) amplifies 8 and facilitates bottom-up attentional salience and top-down attentional control 25,31,32 . Next, (2) 9 Default-mode network (DMN): switching off of the DMN and related circuits can reduce 10 competing signals to prevent interference with conscious perception 9,[33][34][35] 1961-1972, doi:10.1111/j.1460-9568.2011.07696.x (2011 (Table S2)