Weaving a story: narrative formation over prolonged time scales relies on cortical cohesiveness of distributed brain networks

doi:10.21203/rs.3.rs-155155/v1

PDF

Research Article

Weaving a story: narrative formation over prolonged time scales relies on cortical cohesiveness of distributed brain networks

https://doi.org/10.21203/rs.3.rs-155155/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted 10 Feb, 2021

You are reading this latest preprint version

Forming narratives are of key importance to human experience, as it allows one to render large portions of information into relatively compacted stories for future retrieval, and give meaning to otherwise fragmented occurrences. The neural mechanisms that underlie coherent narrative construction of causally-connected information over prolonged temporal periods are yet unclear. Participants in this fMRI study observed consecutive scenes from a full-length movie either in their original order, enabling causal inferences over time, or in reversed order, impeding the formation of coherent narratives. In between scenes, we presented short periods of blank screens for analysis of post-encoding processing. Using multivariate pattern analysis (MVPA) and inter-subject correlations, we aimed at exploring functional connections among brain networks that take part in enabling coherent narrative construction. We detected two separate networks that discerned between coherent and incoherent narrative conditions: a network associated with information processing, consisting of primary and high-order association cortices that showed increased functional connectivity among themselves, as well as a distributed fronto-parietal network. In addition, hippocampal activation was found to correlate with portions of the visual processing hierarchy in a time dependent manner, parallel to the unfolding of the narrative. We suggest that with the unfolding over time of causal relationships among presented scenes, top-down modulation of posterior association cortices and frontoparietal networks enables the integration and binding of incoming and previously acquired information to form coherent and concise narratives.

Cognitive Neuroscience

Psychology

narrative

functional MRI

causal inference

functional connectivity

By forming narratives, humans are able to bind large amounts of discrete events into meaningful representations. The ability to produce narratives is pivotal to one’s sense of personal and cultural identity, as well as to the creation and construction of memories ^1,2. A key defining feature of narrative is causation ³. In "Poetics", Aristotle emphasized the importance of coherence in narrative, which results from logical, causal connections between occurrences ⁴. Accordingly, literary accounts define narrative as a series of actions and events that unfold over time according to causal principles ⁵. For a coherent narrative to form, events must occur in a logical order, wherein actions and episodes that lead to other events must take temporal precedence given the conflation of logical (if x then y), causal (because x then y), and temporal priority (first x then y) ^5,6.

Neural correlates of causal inferences have typically been studied in short time scales. For example, generating inferences during the reading of short texts correlated with activation in superior temporal gyrus ⁷, and regions in prefrontal cortex ascribed to retrieval and working memory were found to mediate processes essential to causal inferences during narrative comprehension ^8,9. Narrative construction of naturalistic events often necessitates, however, causal inferences in the range of several minutes and beyond. Here we manipulated causality on the time scale of several minutes, aiming to delineate neural networks that support coherent narrative construction with causal connections over prolonged time scales.

Narrative formation is essential to the representation of past events, a notion dating back to Pierre Janet (1859–1947), who posited two parallel processes that co-occur during narrative construction. Specifically, ongoing events are perceived and encoded online, and in parallel, the meaning of what had happened - the narrative - is generated ¹⁰. Accordingly, we expect that narrative formation would instigate (1) an online process of perceiving the continuous stream of information, and (2) a parallel process of integrating incoming information with recently accumulated narrative information according to causal inferences.

In recent years, narratives were used as a tool to study various cognitive processes involved in story comprehension, such as event segmentation ^11–15 and narrative shifts ¹⁶. Studies focusing on the understanding of narratives differentiated between brain networks processing short segments of information versus full narratives ^17,18. Particular emphasis was devoted to the default mode network (DMN), which appears to display unique activation patterns, depending on the high-level meaning of conveyed scenes, rather than on their physical attributes ^18–20. Taken together, these studies identified brain regions that correlate with the outcome of narrative formation. However, the process of narrative formation, i.e., how discrete units of information, processed by distributed brain regions, are bound together through causal relationships across prolonged time scales to form a cohesive meaning is yet unclear.

To explore behavioral and neural correlates of narrative formation, we presented participants with 22 scenes from the classic film ‘Bicycle Thieves’ (Vittorio de Sica, 1948) that was edited in a way that preserved the central plot. The scenes were presented either in their correct order, meeting the requirements of the definition of narrative, or in reverse order, breaking temporal causation between events and impeding the formation of a coherent narrative ⁶. As events that unfold over time and require binding of information are hypothesized to be processed not only during the event, but also upon event offset ²¹, we introduced short blank epochs between successive movie scenes, aiming to shed light on processes of narrative formation that may occur during those periods.

Since one of the defining features of an unfolding narrative is the temporal evolution of its plot, we applied an analysis strategy that considers dynamic time-courses (functional connectivity), rather than univariate approaches of average signal strength, which are blind to temporal changes. We applied a multi-voxel pattern analysis (MVPA) approach, under the assumption that patterns of inter-brain connectivity measures, rather than average activation strength, would discern mental states associates with coherent vs. incoherent narrative conditions. We used a whole-brain multi-variate pattern analysis of functional connectivity measurements followed by inter-subject correlations (ISC) and seed-based connectivity analysis (SAC) based on the ROIs that were delineated by the MVPA, as well as on predefined anatomical delineation of the hippocampus, during scene presentation and inter-scene ('blank') periods and across time. The analysis yielded a set of regions that differed between the groups albeit their exposure to similar audio-visual input. We demonstrate that the presentation of scenes in coherent vs. incoherent order engages two key networks; one that processes and interprets the continuous stream of events, and a second network that integrates complex relations and contextual information to represent the global narrative across prolonged time scales.

Reversing scene order impeded the formation of a coherent narrative

Participants were scanned in an fMRI environment while viewing movie scenes that were presented either in their chronological order (coherent group) or in reversed order (incoherent group), thus comprising or omitting a fundamental feature of narrative coherence, i.e., the causal connection between events (Fig. 1A). As indicated by their self-reports, the bulk of participants in the incoherent narrative group indeed noticed that the scenes were out of order. Nevertheless, their narrative understanding, as derived from their written description of the story, did not meet the predefined criteria for plot comprehension. Thus, only 33 percent of the participants in the incoherent group correctly described the fundamental components of the plot, which included the main characters, the main protagonist’s goal, and the main dramatic development in terms of causality, compared to 83 percent of the coherent narrative group (Fig. 1B). The lack of coherent understanding was also supported by the participants’ responses to a question about the causal connection between two dramatically loaded successive scenes. These two scenes were presented in the middle of the movie, perceived in both groups at a similar time interval after the experiment’s onset. Seventy-eight percent of the coherent narrative group correctly understood the causal connection between the restaurant scene (where the father treats the kid to a fancy meal) and an aversive event portrayed in the previous scene (the father slaps the kid in anger), compared to only 5.5 percent of the incoherent narrative group (see Fig. 1B). Nonetheless, although the incoherent group did not –according to their subjective reports –succeed to form a correct and coherent narrative, the vast majority of them (82 percent) sensed that they understood the story, that is, they a subjective feeling of following or constructing a story. Notably, neither the affect that was evoked during the movie (as rated in the post-movie questionnaire) nor the willingness to participate in future follow-up research (high willingness) differed between the groups. Moreover, pupil dilation in both groups was shown to synchronize with scene onsets and offsets, indicating engagement of both groups throughout the experiment (Figure S1).

Differential functional connectivity patterns between coherent and incoherent groups

We performed multivariate pattern analysis (MVPA) to illuminate brain areas with differentiable functional connectivity measures during coherent vs. incoherent conditions. The analysis was performed separately for the 'movie' and inter-movie’ (blank) periods. During the movie periods, the between-group MVPA comparison yielded differences in the supra-calcarine (4–86 10) and cerebellum crus 2 (-32 -74 -38) (p < 0.005, cluster-size p-FDR corrected-p < 0.05). The same analysis for the blank periods yielded differences in the supra-calcarine (MNI x,y,z peak coordinates 4–88 12), cerebellum crus-2 (38 22 − 2), supra-marginal gyrus (56 − 42 26), left putamen (-28 -10 8), cingulate gyrus (-10 -4 38), lingual gyrus (-30 -54 20), fusiform (28–74 -8), Mid FG (-30 0 62) (see Fig. 2).

Table 1

**Regions identified by the multi-variate pattern-analysis (MVPA) to differ between coherent and incoherent narrative groups.** MNI coordinates indicate voxels of peak statistical differences. BA – Brodmann area, PFC – prefrontal cortex
Coherent > Incoherent during ‘movie’ periods	MNI
1. Right calcarine sulcus (BA 17)	4–86 10
2. Cerebellum crus II	-32 -74 -38
Coherent > Incoherent during ‘blank’ periods	MNI
1. Right calcarine sulcus (BA 18)	4–88 12
2. Right anterior insula (BA 13) / ventrolateral PFC (BA 47)	38 22 − 2
3. Right supra-marginal gyrus (BA 40)	56 − 42 26
4. Left putamen	-28 -10 8
5. Middle cingulate gyrus (BA-24)	-10 -4 38
6. Left Lingual gyrus (BA 19)	-30 -54 -4
7. Right Fusiform gyrus (BA 19)	28–74 -8
8. Left middle frontal gyrus (BA6)	-30 0 62

Inter–subject correlations

Since MVPA is an omnibus test, it can provide information regarding regions that differ in between-group functional connectivity. Further analysis is required in order to explore the nature and direction of these differences. We therefore explored the regions-of-interest (ROIs) outlined by the MVPA analysis using inter-subject correlation (ISC) analysis, to test for differences in group cohesion during coherent vs. incoherent narrative conditions (Hasson ref). Specifically, we performed an ISC analysis within each of the above-mentioned regions for each group separately. Mean ISC values significantly differed between the groups in several of the ROIs outlined by the MVPA (see Fig. 3), depicting significantly higher within-group time-series’ correlations in the coherent-narrative group compared to the incoherent-narrative group. Intriguingly, this effect was found in regions involved in both sensory processing and perception as well as in high cognitive functions. As such, activation in sensory and perception centers in occipital and occipito-temporal cortex was highly correlated within subjects in the coherent narrative group and less so in the incoherent narrative group. The supramarginal gyrus, middle-frontal gyrus and cingulate gyrus, involved in higher cognitive functions, also depicted higher inter-subject correlations in the coherent narrative group (Fig. 3B).

Seed-to-voxel analysis of MVPA regions

To further explore neural networks that differentiated between coherent and incoherent narrative processing, we used the MVPA between-group regions as seeds in a seed-based correlation analysis (SCA). For each region, we computed its functional connectivity with the rest of the brain for each subject and compared the results between the groups, creating a map of regions that differentiate between coherent and incoherent narrative groups during the scenes (movie) and during inter-scene periods (), using a threshold of p < 0.005, cluster level: p < 0.05, FDR corrected.

The calcarine sulcus, which appeared to differ between the groups during both movie and blank periods, was highly correlated with the precuneus, cuneus, and right inferior frontal gyrus (see Fig. 4A & Table 2). The cerebellum crus II, which appeared in the MVPA analysis only during the movie periods, but not during the blank periods, was found to be highly correlated with a distributed array of cerebral regions that largely overlap the frontoparietal network (Fig. 4B & Table 2).

Table 2

Regions delineated in the seed-based correlation analysis with the calcarine and cerebellum crus II as seed regions.
Seed-based correlation analysis (SCA) – Calcarine seed	MNI
Anterior cuneus (BA 18) and posterior precuneus (BA 7)	-4 -74 24
Right frontal operculum (BA 44)	38 14 6
Seed-based correlation analysis (SCA) – Cerebellum crus II seed	MNI
Left superior parietal lobule (BA 40)	-42 -44 40
Right superior parietal lobule (BA 7)	32–42 40
Right frontal pole (BA 10)	46 48 0
Left fusiform gyrus (BA 37)	-52 -62 -14
Left middle frontal gyrus (BA 46)	-50 30 20
Left Inferior frontal gyrus (BA 44)	-48 8 22
Left inferior temporal gyrus (BA 37)	62 − 48 -26

Heightened functional connectivity between the hippocampus and visual processing regions during coherent narrative formation

We examined functional connectivity between the left and right hippocampi and the rest of the brain during the movie and blank periods and between the groups. During the inter-scene periods, the right hippocampus was more correlated with the cuneus and lingual gyrus in the coherent compared to incoherent narrative condition. During scene presentations, group differences were found in the lateral occipital cortex, the supra-marginal gyrus, and the angular gyrus. The left hippocampus showed no difference in functional connectivity between the groups during the scene periods, but was significantly higher correlated with the precuneus in the inter-scene periods in the coherent narrative condition (see Fig. 5 and Table 3).

Table 3

Functional connectivity of left and right hippocampi with the rest of the brain during scene and inter-scene periods.
Right hippocampus seed - Movie	MNI
Supra-marginal gyrus (BA 40), angular gyrus (BA 39)	44–44 48
Right hippocampus seed - Blank	MNI
Cuneus (BA 17), lingual gyrus (BA 19)	14–72 16
Left hippocampus seed – Blank	MNI
Bilateral Cuneus (BA 17), Posterior precuneus (BA 7), Lingual gyrus (BA 19)	-16 -88 42
Right orbitofrontal cortex (BA 11)	14 28–32

We characterized the functional connections of the left hippocampus with four regions along the visual processing hierarchy by computing correlations in 15 bins of 50 TRs each (100-sec) of the experimental timeline. As depicted in Fig. 6, hippocampal functional connections were significantly stronger in the coherent vs. incoherent narrative condition with the posterior precuneus (F_1,36=7.59, P < 0.01) and anterior cuneus (F_1,36=6.75, P < 0.05). In contrast, the posterior cuneus and fusiform gyrus showed the opposite pattern, wherein functional connectivity with the hippocampus was higher in the incoherent compared to the coherent narrative condition (F_1,36 = 4.2, P < 0.05, and F_1,36 = 6.71, P < 0.05, respectively). The anterior cuneus and fusiform gyrus also showed main effects for time bins across groups (F_14,504 = 2.52, P < 0.005, F_14,504 = 2.29, P < 0.005, respectively). All the four regions showed an interaction effect of time by condition, such that the functional connectivity between each region and the hippocampus changed over time as a function of condition (posterior precuneus – F_14,504 = 2.55, P < 0.001; anterior cuneus - F_14,504 = 2.75, P < 0.001; posterior cuneus - F_14,504 = 2.05, P < 0.05, and fusiform gyrus - F_14,504 = 2.82, P < 0.001). These findings show that hippocampal connections with dorsal cuneus and precuneus areas are stronger during coherent conditions, whereas posterior and ventral visual processing areas gradually weaken their functional connections with the hippocampus in the coherent condition and strengthen them in the incoherent condition (Fig. 6B-D).

Narratives are central to human experience, enabling the representation and communication of complex constructs, ranging from personal memories to cultural identities ^22,23. Nevertheless, the neural mechanisms that underlie the formation of narratives are still largely unknown, particularly as they pertain to the binding of events across prolonged time scales to form a coherent story. In the current study, we tackled this issue by using an edited version of a full-length movie, which encapsulates core features of narrative characteristics in a cinematic medium ^5,24. The difference between the conditions was designed to control for the core components of narrative formation, as reversing scene order should distort their causal connection between the scenes, thus impairing narrative formation. We found that inferring causal connections between scenes that unfold over prolonged timescales is expressed by increased functional connectivity in fronto-parietal regions and across the visual processing hierarchy, which co-activate with the hippocampus during coherent narrative formation, particularly during post-scene blank periods.

As stated at the outset, a primary feature of coherent narrative formation relies on causal inferences among successive units of information ²⁵. Accordingly, interfering with causality by breaking temporal order on a large temporal scale impeded the capacity to construct a coherent narrative. The failure of participants in the incoherent condition to form a coherent narrative despite their effort to bind otherwise fragmented events, stresses the substantial role of temporal features during the actual (rather than reconstructed) experience in causality inference and coherent narrative formation ^26,27. The fact that participants in the incoherent condition experienced the scenes devoid of clear causal connections does not necessarily imply that they did not produce a narrative. As previously shown in the comprehension of texts, changing the temporal order of cause and effect does not necessarily prevent their retroactive understanding, yet does affect processing time ²⁶ and narrative comprehension ²⁸. The written descriptions from both groups contained information from the entire scene timeline, and pupil dilation was synchronized with scene onsets throughout the whole experiment, indicating that both groups remained engaged in the experiment throughout, yet only the coherent group succeeded in forming a coherent narrative.

Our data analysis was guided by two main strategies: An explorative approach that focused on delineating functional connectivity profiles within and between groups and ROIs, and a theory-driven approach, focusing on functional connectivity with the hippocampus, due to its pivotal role in memory construction and temporal coding. For the explorative approach, we applied inter-subject correlations (ISC) and seed-to-voxel functional connectivity, based on the regions yielded by the MVPA analysis. MVPA is an omnibus analysis that does not supply information regarding the directionality and inter-connections among the regions it points to, requiring complementary investigation ²⁹. We therefore carried out an ISC analysis, which demonstrated a higher within-group similarity in regions related to sensory processing and perception, as well as in high order cortical areas in the middle frontal gyrus, supramarginal and cingulate cortex, involved among other things in logical inference and theory of mind.

Previous studies have revealed that high-order cortical areas tend to be more correlated among participants who share mutual understanding of narrative content ^18,30,31. The current findings suggest that not only high-order cortical regions are sensitive to narrative structure, but also brain areas that are engaged in sensory processing, implying continuous top-down regulation. Primary and secondary sensory regions were previously shown to activate in synchronization among participants watching the same natural stimuli (Hasson et al., 2004), yet here we show preferable synchrony related to coherent narrative formation.

The supra-calcarine cortex, which was delineated by the MVPA test, was found to increase its functional connectivity in the coherent vs. incoherent condition during both movie and blank periods, showed heightened correlations with the cuneus, precuneus, and right IFG (Brodmann area 44) in the coherent condition. The latter area, which is considered homologous to Broca’s area in the left hemisphere, has been implicated in the mirror neuron system ³², and particularly in processing the goals and intentions of others according to their biological motion ³³. In his Poetics, Aristotle stated that narrative is the mimicry of action (mimesis), which is the ability to simulate someone else’s behavior, and to identify and empathize with them. This early notion regarding the relationship between mimicry and narrative organization resonates with our findings that regions of the mirror system are involved in understanding the goals of others by interpreting their actions ³⁴. This feature may be particularly critical in understanding prolonged sequences of social events, which their understanding relies heavily on tracking the protagonists’ behaviors, as in the current experiment.

The precuneus is functionally and structurally connected with visual processing regions, including the cuneus ³⁵, with which we found it to be preferably correlated in the coherent narrative condition. The precuneus plays a pivotal role in high cognitive functions ³⁶, including retrieval of episodic memories, simulating future scenarios ^37,38, and imagery processes that support the understanding of others’ intentions ³⁹. Its activation during the processing of a continuous story has previously been attributed to its capacity to process relatively long and meaningful chunks of information ⁴⁰, as well as in segmentation and integration of present occurrences with the recent past ^13,41. In the current study, the visual system serves as the medium through which the narrative is perceived. Considering the role of the precuneus in the cognitive functions mentioned above, its higher correlation in the coherent narrative group with the primary visual cortex may imply reciprocal relations between these regions, suggesting both bottom-up processing and top-down regulation. Specifically, sensory information acquired via bottom-up processing systems may be met with top-down input regarding previous information and expectations acquired throughout the event.

Comparing coherent vs. incoherent narrative conditions yielded the detection of a distributed network that was functionally correlated with the cerebellum’s crus II area. This network largely overlaps with the frontoparietal network, which has previously been shown to co-activate with the cerebellum’s Crus II ^42,43, an area considered as a supramodal zone of the cerebellum ⁴⁴. Serving as a key control system, the frontoparietal network supports information integration in diverse cognitive contexts ⁴⁵, and was shown to play a role in context-dependent narrative comprehension ⁴⁶. Corroborating our findings, frontoparietal regions were found to correlate with chronological vs. inconsistent information in a narrative comprehension task ⁴⁷, an effect attributed to working memory demands. Notably, the network’s functional cohesiveness was more prominent during movie compared to blank segments, implying its involvement in online processing of incoming information. We suggest that in the context of narrative construction, the frontoparietal network is engaged in online processing of incoming streams of information of current occurrences.

Hippocampal functional connectivity during narrative formation

The essential role of the hippocampus in long-term memory formation is well-documented. Of particular relevance here is the role that the hippocampus and surrounding cortex play in binding sensory and temporal information to enable the formation of episodic memories, characterized by multi-modal sensory information that propagates over time ⁴⁸. Considering its crucial role in binding sensory and temporal information into meaningful units, the hippocampus contributes to the formation of event representation ^49–51. This unique feature of the hippocampus can be attributed to its role in mapping spatial layouts according to environmental cues ⁵². A recent framework proposes that the hippocampus binds both item and context information that constitute a given event ⁵³. The binding of item and context information may occur both during and after an event has terminated ²¹. In the current study, right hippocampal activation was preferentially correlated with lateral parietal regions (supramarginal and angular gyri) in the coherent narrative condition during scene presentation. During inter-scene epochs, both hippocampi showed heightened functional connectivity with occipital visual processing regions, including the cuneus and lingual gyrus during the coherent vs. incoherent condition. Taken together, these functional connections may support the binding of information processed by temporo-parietal regions that support the understanding of biological motion and interpretation of others’ goals ³³, as well as for integrating multimodal sensory information into meaningful concepts in the angular gyrus ^54,55.

The fact that the hippocampus was more correlated with visual areas in inter-scene periods, in the absence of sensory input, may point to the mechanism that supports continuous processing and memory formation of ongoing occurrences ⁵⁶. Such processes seem to be impaired in extreme conditions that may lead to dissociative states, e.g., post-traumatic stress disorder (PTSD). In such cases, the sense of continuity is lacking, manifested by fragmented episodes that fail to converge to a coherent narrative outline ^57,58.

In the current study, the scenes, and therefore the narrative that they portray, unfold over a prolonged time period. The narrative in turn does not consist merely of a situation, but rather of an extended story that encompasses short episodes embedded in a broader contextual storyline. This resonates with a suggested framework regarding two sensory-cognitive processing time-scales that enable humans to process both incoming streams of information during event encoding, as well as a ‘slow process’, which unfolds continuously in the background and binds complex information at much longer timescales ^27,59. This framework is supported by studies that revealed a connection between post-stimulus activation and subsequent memory of short audiovisual scenes ²¹, suggesting an information binding mechanism that extends beyond online sensory processing.

As the default mode network, and particularly its central hub in the precuneus, are imperative in binding and integrating incoming information with semantic knowledge and episodic memories ^60–63, the precuneus may play a prominent role in plot weaving. Indeed, previous studies have demonstrated the involvement of the precuneus, especially when narrative unfolds across prolonged timescales ²⁷. Thus, the enhanced hippocampal - precuneus functional connections we found in the coherent narrative condition, particularly during post-scene periods, may reflect information processing that contributes to the abovementioned ‘slow process’, enabling narrative formation across prolonged timescales. Thus, information presented in a coherent manner may be bound in multiple levels throughout the unfolding of a prolonged event, and fed into the hippocampus after initial processing has already occurred in association cortex. In contrast, processing of incoherent information that lacks clear causal relations may lack top-down integration and lead to termination of processing already at early stages of sensory processing. We suggest that the process of narrative formation may serve to organize otherwise context-less information into large chunks of meaningful information, a feature that is mediated by functional connections between high-order association cortex and the hippocampus. Traumatic memories, which are manifested by fragmented flashbacks of raw representations of past occurrences, are an example of sensory information that is detached from a coherent story and continuous time-line.

In conclusion, narratives are of great importance not only to normal human functioning but also to mental health. A better understanding of the neural mechanisms that support narrative construction holds the potential for detection and interventions for those suffering from severe mental illnesses. The current study provides new insights into the processing of information that unfolds over long durations of time, unearthing wide-range cerebral co-activity that subserves narrative construction.

Participants

Thirty-eight healthy individuals (mean age 27 ± 3.7 y) participated in the study. Two separate groups of 19 subjects were assigned to either the Coherent-Narrative or to the Incoherent-Narrative group (8 and 9 females respectively in the coherent and incoherent narrative groups). All participants had normal visual acuity (without glasses or corrective lenses, enabling eye tracking). None of the subjects were familiar with the movie presented during the scan. The experimental procedures were approved by a Institutional review board of Tel-Aviv University and the Ethics committee of the Chaim Sheba Medical Center, Tel-Hashmoer, Israel, as required by Israeli law. All subjects provided written informed consent prior to the experiment, and all methods were performed in accordance with the relevant guidelines and regulations.

Stimuli and experimental design

In the MRI scanner, participants watched either the coherent or incoherent narrative (experimental conditions), which consisted of 22 scenes (mean scene length: 55.15 ± 23 sec) edited from the movie "Bicycle Thieves" (Italy, 1948), interleaved by 10 sec blank screens. Briefly, the movie portrays a young father, who manages to obtain a pair of bicycles that are required for his employment. On his first day at work, his bicycles are stolen (the plot's turning point), and from that moment on the main protagonist tries to retrieve them with his son. The unfortunate events that follow feed his desperation, which culminate in a failed attempt to steal someone else's bicycles, just to be caught by the police and shamed in front of his son. The experiment lasted 25 minutes, wherein the scenes were identical across groups and differed only in their order of presentation, such that in the coherent-narrative group, scenes were presented in their correct order - from first to last, and in the incoherent narrative the same scenes were presented in reversed order, from last scene to first. The movie was edited using "Movie-Maker" software (Microsoft ©), and were presented using Presentation Version 20.1 software (www.neurobs.com).

Behavioral assessment

Immediately following scanning, comprehension of the story was assessed using a questionnaire targeting narrative comprehension and personal ratings regarding the movie. Participants were asked whether or not they felt they understood the movie’s plot and how confident they were about their answer. They were also asked to briefly describe the main characters and the general story line. Their accounts were subsequently analyzed according to their mentioning of key plot features; namely, the main characters, the main protagonist’s goal, and the main dramatic development in terms of causality. Given that the plot includes a protagonist that aims at achieving a specific goal, we examined the participants' understanding regarding this process. In addition, they were asked more specifically about the casual connection between two specific successive scenes that occurred in the middle of the movie; the first depicted the main protagonist slapping his child, and the second showed a compensation scene wherein the father took the kid to a fancy restaurant. This assessment was designed to probe the participants’ understanding of the event’s causation - a central component in the definition of coherent narrative.

MRI data acquisition and preprocessing

Whole-brain imaging was performed in a 3T Siemens Magnetom MRI system (Siemens Medical Systems, Erlangen, Germany) using a 16-channel head coil. Blood-oxygenated-level-dependent (BOLD)-sensitive T2*-weighted functional images were acquired using a single shot gradient-echo EPI pulse sequence (TR = 2000 ms, TE = 30 ms, flip angle = 82°, 64 axial slices, 2mm³, FoV = 192 × 192 mm, interleaved slice ordering) and corrected online for head motion. Participants completed the task in two fMRI runs, each lasting ~ 12.5 minutes. The first two volumes were discarded to allow for equilibration effects. Visual stimuli were presented on a screen behind the scanner using Presentation software (www.neurobs.com), and were viewed through a mirror attached to the head coil. Following functional imaging, a high-resolution T1 scan was acquired for anatomic normalization.

Imaging data were slice-time corrected and realigned using the SPM12 package (Wellcome Institute of Cognitive Neurology, London). Functional volumes were co-registered, normalized to the Montreal Neurological Institute (MNI) template brain, and smoothed with an 8 mm³ isotropic Gaussian kernel. We assessed task-related functional connectivity using the CONN toolbox (v17) ⁶⁴. The implemented CompCor routine was carried out for each participant, aimed at identifying principal components associated with white matter (WM) and cerebrospinal fluid (CSF), which were segmented. These components, as well as realignment correction information were entered as covariates in the first-level model.

Functional connectivity analysis

In order to detect brain regions that differed in their functional connectivity between the two conditions (i.e., the two groups), we carried out the following analysis steps: 1. A first-level multivariate pattern analysis (MVPA) of pairwise connections between all measured voxels separately for film and blank segments (voxel-to-voxel analysis); 2. Second-level between-group analyses that tested for differences in connectivity between the groups using F-tests; and 3. A seed-based correlation analysis (SCA), using the regions that the MVPA analysis yielded as seeds. Correlations were computed between these regions-of-interest (ROIs) and the rest of the brain (seed-to-voxel analysis). These correlations served to form statistical maps that revealed differences between the coherent and incoherent narrative groups, and served for characterizing the temporal dynamics of narrative related processes between the groups.

Multivariate pattern analysis

Multivariate pattern analysis (MVPA) enables to differentiate among cognitive states based on activation / co-activation patterns across voxels rather than within specific voxels, hence avoiding the exclusion of sub-threshold voxels that may still contain meaningful information as part of a wider pattern ⁶⁵. Pertinent to functional connectivity, MVPA allows for detection of group differences without asserting presumptions concerning a-priori seed regions. Using the MVPA routine, voxel-to-voxel functional correlation matrices were computed across the entire dataset of each subject. Subsequently, the connectivity matrices were reduced into three principle components, which explained the most variance of the matrix ^64,66. The measures of this analysis were then entered into a second-level general linear model (GLM) to assess group differences between the functional connectivity components. This analysis was separately performed for film and blank epochs.

Inter-subject correlation analysis

The continuous nature of movie-sequences does not lend itself to the application of standard fMRI analyses that require multiple repetitions of discrete event types. For exploring the neural dynamics of coherent-narrative processing, we therefore assumed a model-free inter-subject correlation (ISC) approach, which has previously been utilized to examine neural processes associated with naturalistic experimental designs ^67,68. The ISC analysis was performed on the ROIs delineated by the MVPA results (Table 1), such that for each ROI, we calculated the correlations between the mean time-series of all subject-pairs within each group, yielding a correlation matrix for each ROI for each group. We subsequently compared the mean correlation values for each ROI between the two conditions.

Seed-based correlation analysis

Since MVPA is an omnibus test that informs on differences between groups but not on the source of the differences ⁶⁹, we performed a complementary post-hoc exploration to characterize the MVPA effects and delineate brain networks that support coherent narrative formation. To characterize the differences between the coherent and incoherent conditions, we performed seed correlation analyses (SCA), using the resulting MVPA regions as seeds ^70,71. The SCA was carried out for each subject by computing Fisher-transformed correlation coefficients between the mean seed time courses and all other voxels. Following this, a second-level analysis was performed, where between-group t-tests were computed on the resulting correlation values (using a height threshold of P < 0.005 and cluster level of P < 0.05, FDR corrected).

Theory-driven analysis

In addition to the explorative MVPA approach detailed above, we performed a seed correlation analysis using the left and right hippocampi as anatomically-delineated seed regions. For each participant, we calculated correlations between the mean time-series of each hippocampus with the rest of the brain, based on CONN's integrated atlas. Following this, we compared the correlations between the groups and sought for significant differences during 'movie' and 'blank' parts separately. To explore the relationship between hippocampus and key regions that showed differential functional connectivity between the conditions, and considering that the unfolding over time is inherent to the concept of narrative, we tested hippocampal correlations with the posterior precuneus, anterior and posterior cuneus, and fusiform gyrus across bins of 100 sec (15 segments in total) for each subject and for each group. We then performed a mixed-design analysis of variance for each condition separately using ROI and time as repeated-measures factors and condition as a between-group factor.

Eye-tracking

Eye-tracking was performed during scanning using an EyeLink 1000 system (SR Research, Canada) installed at the rear end of the scanner. Gaze positions and pupil diameter were sampled with a frequency of 500 Hz. Calibration was performed prior to each scan. Due to technical issues of data quality, eye tracking was successful in 26 participants (13 from each group). For the purpose of this study, pupil diameter (PD) was analyzed throughout the experiment by calculating for each participant the percent PD change from baseline, which was defined as the average PD of the 1.5 minutes prior to the onset of the first scene of each run. Percent PD changes were subsequently averaged within each condition, and plotted to assess physiological responses during the unfolding of the experiment in both groups.

Author Contribution Statement

A.A. constructed the experiment, carried out the fMRI experiments, and analyzed the data. A.M. oversaw the project and wrote the manuscript with A.A. Both authors reviewed the manuscript.

Competing interests statement

The authors declare no competing interests.

Sarbin, T. R. The narrative as root metaphor for psychology. in Narrative psychology: The storied nature of human conduct 2–21(Praeger, 1986).
Hoerl, C. Episodic memory, autobiographical memory, narrative: On three key notions in current approaches to memory development. Philos. Psychol. 20, 621–640 (2007).
Hayles, N. K. & Richardson, B. Unlikely Stories: Causality and the Nature of Modern Narrative. South Atl. Rev. 64, 140 (1999).
Barthes, R.. Introduction to the structural analysis of narratives. in A barthes reader (ed. Sontag, S.)251–295(Hill and Wang., 1982).
Mar, R. A. The neuropsychology of narrative: Story comprehension, story production and their interrelation. Neuropsychologia. 42, 1414–1434 (2004).
Graesser, A. C. et al. Advanced Outlines, Familiarity, and Text Genre on Retention of Prose. J. Exp. Educ. 48, 281–290 (1980).
Virtue, S., Haberman, J., Clancy, Z., Parrish, T. & Beeman, M. J. Neural activity of inferences during story comprehension. (2006). doi:10.1016/j.brainres.2006.02.053
Mason, R. A. & Just, M. A. How the Brain Processes Causal Inferences in Text A Theoretical Account of Generation and Integration Component Processes Utilizing Both Cerebral Hemispheres. 15, (2004).
Kuperberg, G. R., Lakshmanan, B. M., Caplan, D. N. & Holcomb, P. J. Making sense of discourse: An fMRI study of causal inferencing across sentences. (2006). doi:10.1016/j.neuroimage.2006.06.001
Onno van der hart, Nijenhuis, E. & steele, K. The Haunted self: Structural dissociation and the treatment of chronic traumatization. (W.W. Norton&Company 2006).
Richmond, L. L. & Zacks, J. M. Constructing Experience: Event Models from Perception to Action.Trends in Cognitive Sciences21, (2017).
Kurby, C. A. & Zacks, J. M. Segmentation in the perception and memory of events. Trends Cogn. Sci. 12, 72–79 (2008).
Zacks, J. M. The brain’s cutting-room floor: segmentation of narrative cinema. Front. Hum. Neurosci. 4, 1–15 (2010).
Baldassano, C. et al. Discovering Event Structure in Continuous Narrative Perception and Memory. Neuron. 95, 709–7215 (2017).
Zacks, J. M. et al. Human brain activity time-locked to perceptual event boundaries. Nat. Neurosci. 4, 651–655 (2001).
Whitney, C., Krach, SörenKlann, J., Weis, S., Huber, W. & Kircher, T. Neural correlates of narrative shifts during auditory story comprehension. Neuroimage. 47, 360–366 (2009).
Yarkoni, T., Speer, N. K. & Zacks, J. M. Neural substrates of narrative comprehension and memory. Neuroimage. 41, 1408–1425 (2008).
Yeshurun, Y. et al. Same Story, Different Story: The Neural Representation of Interpretive Frameworks. Psychol. Sci. 28, 307–319 (2017).
Chen, J. et al. Shared memories reveal shared structure in neural activity across individuals. Nat. Neurosci. 20, 115–125 (2017).
Simony, E. et al. Dynamic reconfiguration of the default mode network during narrative comprehension. Nat. Commun. 7, 1–13 (2016).
Ben-Yakov, A., Eshel, N. & Dudai, Y. Hippocampal immediate poststimulus activity in the encoding of consecutive naturalistic episodes. J. Exp. Psychol. Gen. 142, 1255–1263 (2013).
Brockmeier, J. Remembering and forgetting: Narrative as cultural memory. Cult. Psychol. 8, 15–43 (2002).
Hirst, W. & Manier, D. Towards a psychology of collective memory. Memory. 16, 183–200 (2008).
Tomasulu, F. P. " Bicycle Thieves ": A Re-Reading. Cine. J. 21, 2–13 (2007).
Graesser, A. C., Singer, M. & Trabasso, T. Constructing inferences during narrative text comprehension. Psychol. Rev. 101, 371–395 (1994).
Briner, S. W., Virtue, S. & Kurby, C. A. Processing Causality in Narrative Events: Temporal Order Matters. Discourse Process. 49, 61–77 (2011).
Tylén, K. et al. Brains striving for coherence: Long-term cumulative plot formation in the default mode network. Neuroimage. 121, 106–114 (2015).
Ohtsuka, K. & Brewer, W. F. Discourse Organization in the Comprehension of Temporal Order in Narrative Texts. Discourse Process. 15, 317–336 (1992).
Williams, L. J. & Abdi, H. Tukey ’s Honestly Signiflcant Difference test (HSD).Encycl. Res. Des.2–7(2010).
Nguyen, M., Vanderwal, T. & Hasson, U. Shared understanding of narratives is correlated with shared neural responses. Neuroimage. 184, 161–170 (2019).
Baldassano, C., Hasson, U. & Norman, K. A. Representation of real-world event schemas during narrative perception. J. Neurosci. 38, 9689–9699 (2018).
Kilner, J. M., Neal, A., Weiskopf, N., Friston, K. J. & Frith, C. D. Evidence of mirror neurons in human inferior frontal gyrus. J. Neurosci. 29, 10153–10159 (2009).
Van Overwalle, F. & Baetens, K. Understanding others’ actions and goals by mirror and mentalizing systems: A meta-analysis. Neuroimage. 48, 564–584 (2009).
Buccino, G. et al. The neural basis for understanding non-intended actions. Neuroimage. 36, T119–T127 (2007).
Cavanna, A. E. & Trimble, M. R. The precuneus: A review of its functional anatomy and behavioural correlates. Brain. 129, 564–583 (2006).
Margulies, D. S. et al. Precuneus shares intrinsic functional architecture in humans and monkeys. Proc. Natl. Acad. Sci. 106, 20069–20074(2009).
Buckner, R. L., Andrews-Hanna, J. R. & Schacter, D. L. The brain’s default network: Anatomy, function, and relevance to disease. Ann. N. Y. Acad. Sci. 1124, 1–38 (2008).
Stillman, P. E. et al. Neurological evidence for the role of construal level in future-directed thought. Soc. Cogn. Affect. Neurosci. 12, 937–947 (2017).
Mar, R. A. The neural bases of social cognition and story comprehension. Annu. Rev. Psychol. 62, 103–134 (2011).
Lerner, Y., Honey, C. J., Silbert, L. J. & Hasson, U. Topographic Mapping of a Hierarchy of Temporal Receptive Windows Using a Narrated Story. J. Neurosci. 31, 2906–2915 (2011).
Whitlam, J. et al. Continuous theta burst stimulation of angular gyrus reduces subjective recollection. Neuroimage. 38, 1–15 (2019).
Vincent, J. L., Kahn, I., Snyder, A. Z., Raichle, M. E. & Buckner, R. L. Evidence for a Frontoparietal Control System Revealed by Intrinsic Functional Connectivity. J. Neurophysiol. 100, 3328–3342 (2008).
Caulfield, M. D., Zhu, D. C., McAuley, J. D. & Servatius, R. J. Individual differences in resting-state functional connectivity with the executive network: support for a cerebellar role in anxiety vulnerability. Brain Struct. Funct. 221, 3081–3093 (2016).
O’Reilly, J. X., Beckmann, C. F., Tomassini, V., Ramnani, N. & Johansen-Berg, H. Distinct and overlapping functional zones in the cerebellum defined by resting state functional connectivity. Cereb. Cortex. 20, 953–965 (2010).
Naghavi, H. R. & Nyberg, L. Common fronto-parietal activity in attention, memory, and consciousness: Shared demands on integration? Conscious. Cogn. 14, 390–425 (2005).
Smirnov, D. et al. Fronto-parietal network supports context-dependent speech comprehension. Neuropsychologia. 63, 293–303 (2014).
Ferstl, E. C., Rinck, M. & Cramon, D. Y. Von. Emotional and Temporal Aspects of Situation Model Processing during Text Comprehension: An Event-Related fMRI Study. J. Cogn. Neurosci. 17, 724–739 (2005).
Tulving, E. Episodic memory: from mind to brain. Annu. Rev. Psychol. 53, 1–25 (2002).
Furman, O., Dorfman, N., Hasson, U., Davachi, L. & Dudai, Y. They saw a movie: Long-term memory for an extended audiovisual narrative. Learn. Mem. 14, 457–467 (2007).
Davachi, L. & DuBrow, S. How the hippocampus preserves order: The role of prediction and context. Trends Cogn. Sci. 19, 92–99 (2015).
Staresina, B. P., Gray, J. C. & Davachi, L. Event congruency enhances episodic memory encoding through semantic elaboration and relational binding. Cereb. Cortex. 19, 1198–1207 (2009).
Epstein, R. & Kanwisher, N. A cortical representation of the local visual environment. Nature. 392, 6–9 (1998).
Yonelinas, A. P., Ranganath, C., Ekstrom, A. D. & Wiltgen, B. J. A contextual binding theory of episodic memory: systems consolidation reconsidered. Nat. Rev. Neurosci. 20, 364–375 (2019).
van der Linden, M., Berkers, R. M. W. J., Morris, R. G. M. & Fernández, G. Angular Gyrus Involvement at Encoding and Retrieval Is Associated with Durable But Less Specific Memories. J. Neurosci. 37, 9474–9485 (2017).
Seghier, M. L. The angular gyrus: Multiple functions and multiple subdivisions. Neuroscientist. 19, 43–61 (2013).
Cohen, N. et al. Peri-encoding predictors of memory encoding and consolidation. Neurosci. Biobehav. Rev. 50, 128–142 (2015).
Crespo, M. & Fernández-lansac, V. Supplemental Material for Memory and Narrative of Traumatic Events: A Literature Review. Psychol. Trauma Theory, Res. Pract. Policy. 8, 149–156 (2016).
O’Kearney, R. & Parrot, K. Trauma Narratives in Posttraumatic Stress Disorder: A Review. J. Trauma. Stress. 19, 81–93 (2006).
Donald, M. The slow process: A hypothetical cognitive adaptation for distributed cognitive networks. J. Physiol. Paris. 101, 214–222 (2007).
Utevsky, A. V., Smith, D. V. & Huettel, S. A. Precuneus Is a Functional Core of the Default-Mode Network. J. Neurosci. 34, 932–940 (2014).
Uddin, L. Q., Kelly, A. M. C., Biswal, B. B., Castellanos, F. X. & Milham, M. P. Functional Connectivity of Default Mode Network Components: Correlation, Anticorrelation, and Causality. Hum. Brain Mapp. 30, 625–637 (2009).
Lahnakoski, J. M., Jääskeläinen, I. P., Sams, M. & Nummenmaa, L. Neural mechanisms for integrating consecutive and interleaved natural events. Hum. Brain Mapp. 38, 3360–3376 (2017).
Cooper, R. A. & Ritchey, M. Cortico hippocampal network connections support the multidimensional quality of episodic memory.1–35(2019).
Whitfield-Gabrieli, S., Nieto-Castanon, A. & Conn A Functional Connectivity Toolbox for Correlated and Anticorrelated Brain Networks. Brain Connect. 2, 125–141 (2012).
Norman, K. A., Polyn, S. M., Detre, G. J. & Haxby, J. V. Beyond mind-reading: multi-voxel pattern analysis of fMRI data. Trends Cogn. Sci. 10, 424–430 (2006).
Flodin, P. et al. Intrinsic brain connectivity in chronic pain: A resting-state fMRI study in patients with rheumatoid arthritis.Front. Hum. Neurosci.10, (2016).
Hasson, U., Nir, Y., Levy, I., Fuhrmann, G. & Malach, R. Natural Vision. 303, 1634–1640 (2004).
Nummenmaa, L., Lahnakoski, J. M. & Glerean, E. Sharing the social world via intersubject neural synchronisation. Curr. Opin. Psychol. 24, 7–14 (2018).
Williams, L. J. Tukey ’ s Honestly Signiflcant Difierence (HSD) Test 1 Overview.Encyclopedia of Research Design1–5 (2010).
Beaty, R. E., Benedek, M., Barry Kaufman, S. & Silvia, P. J. Default and Executive Network Coupling Supports Creative Idea Production. Sci. Rep. 5, 1–14 (2015).
Tortora, D. et al. Early Pain Exposure Influences Functional Brain Connectivity in Very Preterm Neonates. Front. Neurosci. 13, 1–11 (2019).

No competing interests reported.

AssoulineSupplementarymaterial.docx

PDF

Version 1

posted 10 Feb, 2021

You are reading this latest preprint version

Weaving a story: narrative formation over prolonged time scales relies on cortical cohesiveness of distributed brain networks

Status:

Version 1

Abstract

Figures

Introduction

Results

Reversing scene order impeded the formation of a coherent narrative

Differential functional connectivity patterns between coherent and incoherent groups

Inter–subject correlations

Seed-to-voxel analysis of MVPA regions

Heightened functional connectivity between the hippocampus and visual processing regions during coherent narrative formation

Discussion

Hippocampal functional connectivity during narrative formation

Methods

Declarations

References

Competing Interests

Supplementary Files

Status:

Version 1