When people surf online, they share audio-visual materials and may have similar emotional reactions. We carried out this study to explore the neural mechanism of emotional inter-brain synchronization (IBS). Multilateral interaction is more practical in IBS research than the interaction between two people. We propose a novel inspection method for emotional IBS based on brain connectivity analysis and manifold measurement. The method is expected to deal with multilateral interaction IBS and the problem of heterogeneous data brought by the data collection paradigm. Specifically, the algorithm was constructed based on causal brain connectivity analysis and Riemannian metrics (SPD). Our results suggest that participants have an emotional IBS effect in the context of audio-visual material sharing. The proposed IBS method is more adept at dealing with multiple interaction IBS and effectively reduces algorithm complexity.
Article
Preliminary Study of Emotional Inter-Brain Synchronization
https://doi.org/10.21203/rs.3.rs-3947328/v1
This work is licensed under a CC BY 4.0 License
Version 1
posted
You are reading this latest preprint version
Emotional Inter-brain Synchronization (IBS)
Manifold Metric
Multilateral Interaction
Emotional contagion refers to people's emotions spreading to others around them. Emotional contagion is the psychological process of group emotions involving a series of processes, including emotional awareness, unconscious imitation, physiological feedback, and emotional experience. Emotion consists of many components, including external expressions, such as facial expressions, voice, and posture, and neurophysiological and autonomic nervous system activities. Millions of people use social media daily as the main channel for information exchange . The content that people produce in daily social media and the emotions expressed therein may impact the emotional states of others. An increasing amount of research is focused on group emotions, meaning that researchers are concerned about how the emotions of others influence an individual.
Social networks are invisible tools that effectively promote organizational management1,2. It has been proven that emotions can be passed on via online interactions, even without verbal cues, just as the spread of information, special attitudes, and many other behaviors in social networks3. In addition, the Internet is a typical audio-visual information-sharing and non-face-to-face interaction environment. In the internet , people receive information asynchronously or synchronously and generate emotional responses. People's expressions of these feelings promote further dissemination of information on the Internet. In other words, without direct contact, people can still share their emotions online. The Internet plays the role of a medium.
Hyperscanning is currently the most efficient way to explore the neural mechanisms of social interactions by analyzing inter-brain interactions. EEG-based hyperscanning is an innovative tool for exploring neurally coupled responses in highly ecological paradigms. To date, inter-brain ( dyad mode) interactions have been studied using various experimental paradims4,5. However, social activities in daily life include binary interactions, many-to-many interactions, and one-to-many interactions. Only a few hyperscanning investigations of about three or more people’s interactions have been performed, especially in the online environment. In the case of audio-visual stimulus sharing, emotional responses can stimulate interbrain synchronization (IBS) between people. Effective hyperscanning must cope with multilateral IBS, particularly in an online environment. This is important for integrating hyperscanning with physiological and behavioral big data technologies. Multilateral interactions represent a promising direction for the study of hyperscanning.
However, multilateral interactions pose new challenges in hyperscanning research. For example, to mimic daily life interactions in hyperscanning experiments, participants should be active freely and easily, making the construction of a well-controlled experimental paradigm challenging. Additionally, more EEG acquisition devices (three or more) are required for multilateral hyperscanning studies. As the number of participants and devices increases, the total experimental cost increases significantly. Furthermore, in terms of the sampling rate and electrode distribution, more temporal and spatial heterogeneous EEG data are produced. Multiple participants can use different EEG acquisition devices online or in non-face-to-face interactions. This situation inevitably leads to data differences in temporal or spatial properties, including differences in the electrode number, electrode map, and sampling rate. Multilateral hyperscanning experiments would lack practical significance if we ignored these data differences. Most importantly, developing effective algorithms for multilateral hyperscanning is difficult.
Connectivity analysis for inter-brain synchronization based on various ideas is used in hyperscanning studies, such as phase-locking value (PLV)4,6, Partial Directed Coherence (PDC), and power spectral density (PSD)7. The existing hyperscanning is mainly based on the calculation of the signal time-frequency correlation and is more suitable for a single channel and frequency. When facing the problems of IBS in the context of multilateral participants or heterogeneous data, researchers have determined that the computational complexities of these methods increase rapidly with an increase in the number of participants and electrodes. Therefore, it is necessary to develop a new hyperscanning method to address the problems of multilateral participants or heterogeneous data.
Daily actions are commonly executed during social interactions and are determined by different mental states that express different behavioral intentions. The brains in interactive situations are coupled with the IBS. Researchers have proposed classification algorithms based on Riemannian geometry for picking up distinctive information from EEG signals produced during IBS7. Researchers have focused on the overall pattern of inter-brain synchrony (IBS) in the time dimension and conducted studies on dynamic IBS during naturalistic cooperation, indicating that creative design tasks involve more complex and active IBS between multiple brain regions in specific dynamic mental states8. Researchers measured frontotemporal activation (fNIRS-based hyperscanning) in nine participants during drumming tasks and verified neural synchronization and attention/information-sharing patterns9.
Social interactive learning refers to obtaining new information from similar populations. This was a prerequisite for cultural evolution and survival. Inspired by neurophysiological research, it has been proven that social interactive learning can be enhanced through external synchronous oscillations of brain activity. The relevant results prove that brain synchronization is a sufficient condition for improving real-time information transmission between individual pairs10. The communication between two brains during social interaction is highly dynamic and complex. Does IBS causally facilitate social interactions? However, this question cannot be answered simply using simultaneous recordings. It aims to identify the causal relationship of brain dynamics among multiple brains11. Novembre and Iannetti12 argued that studies using hyperscanning to understand social behaviors are crucial but limited to correlational analysis. They claimed that the causal role of IBS can only be understood through multibrain stimulation (MBS). However, Moreau et al. proposed that it is possible to understand the causal role of IBS with empirical approaches ranging from multi-brain to single-brain recordings, and even no-brain recordings, through adequate experimental designs and computational tools13. The hypothesis regarding the causal relationship between IBS and social behavior was verified. There are claims that IBS could be interpreted either as a mechanism that causally facilitates social interaction or as a consequence of two brains processing similar information during social interaction14.
Focusing on online emotional contagion, we studied the effects of emotional IBS using hyperscanning. Based on the above algorithm research on hyperscanning, we propose a solution for multilateral hyperscanning that uses brain features at a higher level of abstraction to replace EEG signal sequences and combines the similarity measurement method in non-Euclidean space. The purpose is to expand the data-space dimension of hyperscanning, which is currently used for IBS detection. The algorithm was based on causal brain connectivity analysis and manifold measurement and explored an effective way to upgrade the current dyad hyperscanning to the multilateral paradigm. The problems identified in this study are as follows:
1. In audio-visual material sharing, do participants with similar emotional reactions have IBS effects?
2. Which parameter (Valence or Arousal) is the dominant factor in emotional IBS?
3. Is the hyperscanning algorithm based on brain connectivity analysis and manifold distance more suitable for multilateral interaction?
4. With the gradual familiarity with the content of audio-visual materials, will the participants' emotional IBS weaken over time?
2.1 Datasets
This study included two public datasets: DEAP (Koelstra S. et al.,2012) and SEED (Ruo-Nan Duan et al.15). DEAP is a publicly available dataset used for emotion analysis. It contains EEG and peripheral physiological signals of 32 participants. During data collection，32 active AgCl electrodes were used for data collection. The EEG channel consists of Fp1, AF3, F3, F7, FC5, FC1, C3, T7, CP5, CP1, P3, P7, PO3, O1, Oz, Pz, Fp2, AF4, Fz, F4, F8, FC6, FC2, Cz, C4, T8, CP6, CP2, P4, P8, PO4, and O2. All the data were downsampled at 128 Hz. The Valence-Arousal(V-A) model was used to calibrate the emotional state parameters. Each participant watched 40 videos and contributed 40 EEG data segments, each containing one V-A tag. This study mapped scales (1-9) into three levels of Valence and Arousal states. The Valence scale of 1-3 was mapped to ‘negative,’ 4-6 to ‘neutral,’ and 7-9 to ‘positive.’ The Arousal scale of 1-3 was mapped to ‘passive,’ 4-6 to ‘neutral,’ and 7-9 to ‘active.’ We restructured the original DEAP dataset of 32 participants using video ID. Each video group (VG) contained the EEG data collected from 32 participants.
SEED is a collection of EEG datasets provided by the BCMI Laboratory. Fifteen Chinese participants participated in this study. Fifteen Chinese film clips were selected for this study. For each film clip, the label of the corresponding emotional state is ‘-1’ for negative, ‘0’ for neutral, and ‘+1’ for positive. Participants were asked to report their emotional reactions to each film clip for feedback. The data were then downsampled to 200 Hz. Fifteen trials were conducted for each experiment. Each participant watched 15 video clips during the trial. This trial was conducted three times on three different days. Each data file contains 16 arrays. Fifteen arrays contained the segmented preprocessed EEG data from 15 trials in one experiment. The 16th array contains labels corresponding to the emotional states of the 15 arrays.
2.2 Calculations
2.2.1 EEG components relevant to emotional IBS
Emotional contagion in the network is realized through users' network behaviors, such as forwarding and comments. Every occurrence of network behaviors may be the driving force of emotional contagion. Compared to face-to-face interactions, online emotional contagion is more similar to emotional IBS when audiovisual stimuli are shared. This study, estimated IBS using the phase locking value (PLV)6. Five frequency bands were considered: delta (0.5-3 Hz), theta (4-7 Hz), alpha (8–13 Hz), beta (13-30 Hz), and gamma (31-50 Hz). PLV measures the long-term synchronization hidden in EEG signals from each electrode pair by investigating task-induced changes in neural activity. Therefore, we observed transient changes in each electrode pair through the values in a PLV sequence without predefining a time window. The PLV values ranged from 0 to 1, where 0 indicated that the two signals were totally unsynchronized, and 1 indicated perfectly phase-locked oscillations between the two signals across trials. It is defined by:
where N represented the number of trials, φ was the phase and | | represented the complex modulus. i and k indicate the electrodes of participants #1 and #2 in the dyad. In this study, inter-brain PLVs were computed by examining two electrodes from two participants. The phases were extracted using Morlet wavelet transform on the signals.
2.2.2 Multilateral hyperscanning
Existing hyperscanning methods are based on correlation calculations and are more suitable for dyad mode. We proposed a hyperscanning method for multilateral interactions based on causal brain connectivity analysis and manifold distance. The similarity of brain connectivity matrices with heterogeneous dimensions cannot be directly obtained because they are located in different vector spaces. The individual brain connectivity matrix was considered a point in the manifold space using manifold space dimension reduction. The AIRM Metric was used to represent similarity between individuals. The proposed method replaces the paired multidimensional array correlation calculation with a similarity evaluation of brain connectivity matrices based on the Affine Invariant Riemannian (AIRM) metric on the symmetric positive definite (SPD) manifold. For individual brain connectivity analysis, the theoretical basis of bi-LSTM-GC was Granger Causality estimation using bidirectional LSTM recurrent neural networks (RNNs) to solve nonlinear parameters16.
This principle is expressed as follows. The symmetric positive definite(SPD) matrix sym+ D was expressed as Riemannian manifold M. Given two points of M and M 'on a Riemannian manifold, the distance D (M, M') is determined by the geodesic length between the two points. The affine-invariant Riemannian metric (AIRM), which is a Riemannian measure reflecting the true geodesic distance between two points on a Riemannian manifold, was used to represent the similarity of the brain connectivity matrix.
Where ‖·‖F represented the F norm of the matrix, and M and M’ were the two points on the Riemannian manifold. For the AIRM, converting a non-positive-defined symmetric matrix (bi-LSTM-GC matrix) into a positive-defined symmetric matrix is necessary.
3.1 Verification of emotional IBS
In this section, we verified whether emotional IBS exists in audio-visual stimulus-sharing scenes and tested the regulatory role of emotion parameters (Valence and Arousal) in emotional IBS.
3.1.1 Emotional IBS in the dyad
We regrouped the DEAP into 40 videos according to the ID labels of the videos. Each packet contained EEG data collected from 32 participants while watching the same video and the corresponding emotion parameters. This packet is called a video group (VG).
First, we calculated the overall PLV for each VG. After selecting every possible binary combination in the 32 segments of EEG data,496 pairs of combinations were formed in each VG. For each combination, we calculated the PLV values between the two EEG sequences collected from the electrodes at the corresponding positions of the different participants.
We then averaged all PLV values over the combinations and called this PLV value the total average PLV used as the baseline for emotional synchronization comparison. The total average PLVs for all VGs were obtained separately at five EEG characteristic frequencies (Delta:0.5-3 Hz, Theta:4-7 Hz, Alpha:8-13 Hz, Beta:13-30 Hz, and Gamma:31-50 Hz). Second, according to the ‘Valence’ (one of the emotion parameters), 32 segments of EEG data in a VG were divided into three parts (negative, neutral, and positive). The average PLV values of the three groups were calculated. This calculation method is consistent with that for the total average PLV calculation method, except for the data grouping operation. Third, according to ‘Arousal’ (another emotional parameter), the 32 segments of EEG data in a VG were divided into three groups (passive, neutral, and active). The average PLV values of the three groups were obtained similarly to the total average PLV.
The calculation results are listed in Table 1. Under the ‘Valence’ condition, the PLV values of ‘Neutral’ were greater than the total average PLV in the Delta, Theta, and Alpha frequency bands. The rest included a ‘Negative’ emotion in Alpha and a ‘Positive’ emotion in Gama. Participants having a ‘Positive’ emotion were prone to emotional IBS only in the Gama frequency band. Under the ‘Arousal’ condition, the PLV values under the ‘Passive’ condition were greater than the total average PLV in nearly all characteristic frequencies except Beta. Participants in the ‘Neutral’ and ‘Active’ emotion categories did not produce emotional IBS easily, except in the Beta and Gama frequency bands.
We performed a one-sample t-test to identify the frequency of significant Emotional IBS (p < 0.05). Under the ‘Valence’ and ‘Arousal’ conditions, variance analysis was applied to PLVs of 40 VGs (40×6×5) and Total Average PLV. As shown in Table 2, under the ‘Valence’ condition, there was a significant difference between Total Average PLV and ‘Neutral’ emotion PLVs in the alpha frequency band. In the beta frequency band, there was a significant difference between ‘Negative’ emotion PLVs and total average PLV. Under the ‘Arousal’ condition, the PLVs in ‘Passive’ emotion were significantly different in the alpha frequency band. In the theta frequency band, PLVs of ‘Active’ emotion were significantly different.
We synthesized the results of PLV baseline comparison and analysis of variance and extracted the frequencies adhering to the requirements of PLV greater than the Total Average PLV and significant difference (p < 0.05). Participants with ‘Excited’ and ‘Calm’ emotions were most likely to produce emotional IBS (Red Squares in Figure 1). The corresponding characteristic frequencies were the theta and alpha values. Second, there was an obvious emotional IBS (Erythema Squares in Figure. 1) in the 'Depressed,' ‘Relaxed,' and 'Neutral' emotions; that is, the PLV in the 'Valence' and 'Arousal' dimensions were greater than total average PLV at the same time, but it did not show a common significant difference. It was concluded that emotional IBS most likely occurs in the alpha band.
3.1.2 Relationship between valence/arousal differencesand PLVs
For the 40 VGs, each group consisted of 496 participant dyads. Differences in the Valence’ and ‘Arousal of each dyad were extracted. For each VG in the five frequency bands, the PLV values (averaged over all electrode-dyads) were reordered in ascending order according to the differences of ‘Valence’ and ‘Arousal.’ We averaged the sorted PLV sequence and its corresponding ‘Valence’/‘Arousal’ differences over 40 VGs. Thus far, we have obtained a more macroscopic distribution relationship between the PLVs and differences in emotional parameters. No clear linear relationship was observed between them (Figure 2).
We fused all the data trials in the DEAP. The labels of the video and participants were removed, and only the EEG data and emotional parameters were preserved. The total data included 1280 segments of EEG data. We constructed 816003 data dyads. Differences in the Valence’ and ‘Arousal of each dyad were extracted. Thus, a one-to-one relationship between PLVs and differences in emotional parameters at the five frequencies was established. Statistical analyses were applied to emotional IBS. Two-way analyses of variance (ANOVAs) of the frequency bands (delta, theta, alpha, beta, and gamma) and emotional parameters (Valence and Arousal) were conducted.
As shown in Table 3, the differences in emotional parameters had no significant effect on the distribution of PLV values (p>0.05). The PLVs could not indicate how the emotional IBS was influenced by the ‘Valence’/‘Arousal’ differences.
3.2 Multilateral hyperscanning
We propose a hyperscanning method based on brain connectivity analysis and an Affine Invariant Riemannian Metric (AIRM) on a Symmetric Positive Definite (SPD) manifold. In this section, we verified whether a hyperscanning algorithm based on this concept is suitable for multilateral interactions. Notably, PCA was performed to reduce the dimensions of the brain connectivity matrix. Based on this, we obtained the principal component characteristics in the same dimensions extracted from the brain connectivity matrices. Then, AIRM Metric was used to represent the similarity between the individuals’ emotional states.
3.2.1 Emotional IBS in the group
One VG contained 32 segments of EEG data. After obtaining the PCA features of the brain network matrices, 496 pairs of PCA feature dyads with the same dimensions were formed in each VG. We calculated the SPD values in five frequency bands within each dyad and averaged them over the 496 PCA feature dyads. This mean PLV value, called the total average SPD, was used as the baseline for emotional synchronization comparison. According to the ‘Valence,’ 32 segments of EEG data in a VG were divided into three groups (negative, neutral, and positive). The average SPD values of the three groups were calculated. The calculation method for each group was consistent with that for the Total Average SPD. Then, according to the ‘Arousal,’ 32 EEG data segments in a VG were divided into three groups (passive, neutral, and active). The average SPD values of the three groups were calculated.
As shown in Table 4, under the ‘Valence’ condition, the SPD values of ‘Neutral’ were smaller than the total average SPD in the delta, theta, and alpha frequency bands. The rest included a ‘Negative’ emotion in Delta and a ‘Positive’ emotion in Gama. Under the ‘Arousal’ condition, the SPD values of ‘Passive’ were smaller than the total average PLV in all characteristic frequencies. The rest included a ‘Negative’ emotion in beta. The results of this part were basically consistent with section 3.1.1.
3.2.2 Relationship between valence/arousal differences and SPD metrics
As described in Section 3.1.2, there were 496 participant dyads in each VG. Differences in the Valence’ and ‘Arousal of each dyad were extracted. In each VG, in the five frequency bands, the SPD values were ordered in ascending order according to the differences in ‘Valence’ and ‘Arousal’. The sorted SPD sequence and its corresponding ‘Valence’/‘Arousal’ differences were averaged over 40 VGs. Therefore, the distribution relationships between SPD values and emotional parameter differences were obtained, as shown in Figure 3.
As shown in Figure 3 (a), in the range of small differences (<1), the SPD has an inverse linear relationship with the differences in valence. In the large difference range (3-8), SPD had a positive linear relationship with the ‘Valence’ differences. In the intermediate difference range (1-3), there was no linear relationship between the SPD and ‘Valence’ differences. As shown in Figure 2 (b), in the small difference range (<1), SPD had an inverse linear relationship with ‘Arousal’ differences. In the large difference range (5-8), SPD had a stronger inverse linear relationship with ‘Arousal’ differences. In the intermediate difference area (1-3), there was no linear relationship between SPD and ‘Arousal differences.
Statistical analyses were performed to verify the significance of emotional IBS. Two-way analyses of variance (ANOVAs) of the frequency bands (delta, theta, alpha, beta, and gamma) and emotional parameters (Valence and Arousal) were conducted. As shown in Table 5, the differences in ‘Valence’ significantly affected the distribution of SPD values in frequency theta, alpha, and beta (P<0.05). The differences in ‘Arousal’ significantly affected the distribution of SPD values only in the theta frequency (P<0.05). This result proved that the proposed hyperscanning method better indicated how the emotional IBS was influenced by the ‘Valence’/‘Arousal’ differences. And the emotion parameter ‘Valence’ was the more dominant factor in emotional IBS.
3.3 Variation characteristics of emotional IBS in time dimension
In this section, we verified whether emotional IBS weakens over time with gradual familiarity with the content of audio-visual materials. The structure of the SEED dataset was relatively simple. Each participant watched 15 videos with different emotional contents in a single trial. The experiment was repeated three times on three different days. Following the introduction of the SEED, each participant performed the experiment once with an interval of approximately one week. Each trial had a total of 45 files with the extension: mats (MATLAB files).
The EEG data from all trials in SEED shared the same label sequence. The EEG data labels were unified for each trial. This raised a question. The emotional responses produced by different participants watching the same films were consistent. The emotional states produced by participants watching the same group of videos three times were the same. Owing to this series of questions, this study did not analyze from the perspective of the emotional labels of the datasets. By ignoring the labels of the participants and films, we studied the variation characteristics of emotional IBS over time.
In each trial, the EEG data collected from 62 electrodes showed artifacts in varying degrees. Therefore, the data must be preprocessed before format deformation. The preprocessing included low-SNR electrode removal and artifact data segment removal. The processed data contained different numbers of electrodes and sampling points. Therefore, heterogeneous EEG data were obtained. For each participant, the experiment was repeated thrice on three different days. The EEG data contributed by the participants on the first, second, and third days were divided into three data groups called the date group (DG), which included 225 segments of EEG data. For one segment of EEG data, the brain connectivity matrices at five characteristic frequencies were first calculated. The dimensions of the brain connectivity matrix depended on the dimensions of the processed data (electrode × sample point). Therefore, there may be dimensional differences in the data contributed by different participants, even for the same participant in different trials. This further led to dimensional differences in brain connectivity matrices. After the principal components were extracted, all brain connectivity matrices were transformed into unified dimensions, the 225 PCA features were organized into dyad combinations, and SPD measures were obtained for each dyad. We analyzed the overall SPD value distribution characteristics over three days.
In Figure 4, the rows represent the five characteristic frequencies. Columns represent time dimensions. The x- and y-axes of each subgraph represent the data dimension because the processed data were heterogeneous, and the data dimension deviated. The z-axis of the subgraph marks the SPD between the brain-connected matrices. The figure clearly shows that the emotional differences (SPD values) between participants decreased with time at all five frequencies, and the emotional responses of the participant groups to the same video stimulus materials gradually became similar. This pattern was slightly weak in the Gama frequency band but decreased daily, as shown in Table 6.
4.1 Existence verification for emotional IBS
The progress in hyperscanning in social neuroscience has become evident in recent years. Research paradigms for hyperscanning include six categories. The imitation task aimed to evaluate the participants’ behavior when they tried to imitate a given action or task. The conclusion showed that there is obvious brain synchronization between the person performing the task and the person imitating the task, especially in the case of mirror imitation17. The coordination tasks required participants to complete the tasks in a synchronized manner. Experiments have proven that imitation tasks promote and induce synchronization between the two brains18. Studies on eye contact/gaze experiments require participants to gaze at each other. People exchange information through eye contact, which provides an ideal basis for studying the neural mechanisms underlying this behavior through hyperscanning. Researchers have used hyperscanning and economic exchange tasks to study social interactions. Cooperation and competition are inseparable parts of human life; when participants cooperate rather than compete, they are more likely to induce brain synchronization. Interestingly, cooperation in a virtual environment can lead to greater IBS compared with the world. EEG is the most commonly used neuroimaging technique for hyperscanning19. This is due to the advantages of EEG in terms of portability and cost. Cost reduction enables researchers to conduct relatively large-scale studies and simultaneously obtain high-quality data.
The study of emotional brain synchronization has not yet become a primary research direction in social neuroscience. Several relevant studies have been conducted on this topic. Behavioral decision theory argues that unfair experiences influence unfair third-party reactions. Behavioral decision-making theory emphasizes the importance of individuals’ emotional experiences in subsequent decisions20. It has been verified that fathers and their children synchronize their brain activity during naturalistic interaction21. During conversations, people often adjust their emotions at any given time. Smirnov D.22 reveals emotional consistency between the speaker and audience during a conversation. Research in organizational management has shown that experiencing positive emotions together can help promote social interactions between individuals.23.
Traditional hyperscanning focuses on the distribution of brain synchronous components in the frequency and brain regions and relevant research to conclude. In this study, to realize the brain synchronization calculation of a large number of subjects, PLV was extended to the whole brain synchronization measurement. This approach loses a certain degree of spatial brain synchronization. However, it retained the overall numerical characteristics, indicating that emotional IBS was more stable.
4.2 A new hyperscanning method for multilateral interaction
The time dimension of an algorithm refers to the time required to execute the current algorithm, which is referred to as time complexity. The time required by the algorithm was directly proportional to the number of statements executed. Basic statements executed by the algorithm should be screened out to solve the time complexity problem. The most frequently executed statements are basic, usually the innermost loops. Then, we calculated the order of magnitude of the number of executions of these basic statements. According to the multiplication rule, the overall time complexity of the algorithm is obtained by multiplying the time complexities of several parts of the algorithm.
In the present study, two methods were used to analyze emotional brain synchronization. Traditional PLV method. Emotional IBS was estimated using the phase-locking value (PLV). PLV can be used to investigate task-induced long-term synchronization in neural activity changes to measure the degree of synchronization in every electrode pair. Therefore, in this algorithm, the basic statement used to measure the time complexity of the algorithm is the PLV calculation between the electrodes; that is, the order of magnitude of the innermost loop depends on the number of electrodes. The order of magnitude of the second-layer loop depends on the number of test pairs, whereas that of the third-layer loop depends on the number of frequency bands. These three-time consumption parts follow the multiplication principle, forming the overall time complexity.
We proposed a hyperscanning method based on brain connectivity analysis and manifold metrics. The SPD matrix is expressed as Riemannian manifold M. Given two points of M and M 'on a Riemannian manifold, the distance D is determined by the geodesic length between the two points. The AIRM was used to represent the similarity of the brain connectivity matrices. In this algorithm, the brain connectivity matrix can be regarded as a feature extraction step, not a step of brain synchronization calculation. The brain connectivity matrix itself can also be used for other calculation purposes and can appear in the data system as a data reserve. The basic sentence used to measure the time complexity is the manifold metric between brain connectivity matrices; that is, the order of magnitude of the innermost loop depends on the number of brain connectivity matrix pairs (the number of test pairs). The order of magnitude of the second-layer cycle depends on the number of frequency bands. Therefore, the multilateral hyperscanning method proposed in this study has only a two-layer loop in the macro view. The time complexity of the algorithm is reduced compared to that of the three-layer loop of the traditional hyperscanning method. Simultaneously, we transformed the object of brain synchronous calculation from the electrode pair to the principal component feature of the brain connectivity matrix, which can effectively avoid the heterogeneous problem of EEG data caused by differences in the position and number of EEG electrodes.
4.3 Emotional IBS over time
This study focuses on the emotional contagion from the contact between people in physical space and network media. Network behavior then promotes the spread and change of emotional information, further leading to the fermentation of group-network emotional behavior. Online emotional contagion is more likely to produce a post-truth phenomenon than the physical space. The main reason for this includes the anonymity of cyberspace, which makes it easier to avoid responsibility by spreading emotional information online. The excessive use of information filtering and recommendation algorithms causes people to receive increasingly homogeneous information. When an increasing number of people are infected with the same emotion, they can form network groups quickly and produce group polarization. However, whether it is behavior, emotion, or information transmission, time cannot be ignored. However, few studies have examined the relationship between emotional contagion and time. The literature (time-emotion) found that, under the condition of happiness, the difference in the reaction time of subjects gradually narrowed with the progress of the trials. These findings suggest that happiness may gradually increase interpersonal coordination over time to promote better cooperative success. This study also focused on the relationship between emotional contagion and time. With the help of the special properties of the SEED, this study verified the changes in emotional IBS over time. The EEG data was collected on three different days but in the same audio-visual materials environment. The results show that the emotional differences (SPD values) between participants decreased with time in all five frequencies. The emotional responses of the participant groups to the same video stimulus materials gradually tended to be similar.
- Ferrara, E. & Yang, Z. Measuring emotional contagion in social media. PLOS ONE 10, e0142390. doi:10.1371/journal.pone.0142390 (2015).
- Wang, Q. et al. The influence of social networks on team performance: moderating effect of emotional contagion. Open J Soc Sci 08, 553-565 (2020).
- Lu, K. et al. Cooperation makes a group be more creative. Cereb Cortex 29, 3457-3470. doi:10.1093/cercor/bhy215 (2019).
- Hu, Y. et al. Inter-brain synchrony and cooperation context in interactive decision making. Biol Psychol 133, 54-62 (2018).
- Kawasaki, M. et al. Sensory-motor synchronization in the brain corresponds to behavioral synchronization between individuals. Neuropsychologia 119, 59-67 (2018).
- Lachaux, J. P. et al. Measuring phase synchrony in brain signals. Hum Brain Mapp 8, 194-208 (1999).
- Simar, C. et al. Hyperscanning EEG and classification based on Riemannian geometry for festive and violent mental state discrimination. Front Neurosci 14, 588357. doi:10.3389/fnins.2020.588357 (2020).
- Li, R. et al. Dynamic inter-brain synchrony in real-life inter-personal cooperation: A functional near-infrared spectroscopy hyperscanning study. NeuroImage 238, 118263 (2021).
- Tao, L. A. et al. Team-work, Team-brain: Exploring synchrony and team interdependence in a nine-person drumming task via multiparticipant hyperscanning and inter-brain network topology with fNIRS. NeuroImage 237 (2021).
- Pan, Y. et al. Dual brain stimulation enhances interpersonal learning through spontaneous movement synchrony. Soc Cogn Affect Neurosci. 16, 210–221 (2021). doi:10.1093/scan/nsaa080 (2020).
- Roads, B. D. & Love, B. C. Learning as the unsupervised alignment of conceptual systems. Nat Mach Intell 2, 76–82. doi:10.1038/s42256-019-0132-2 (2020).
- Novembre, G.& Iannetti, G. D. Hyperscanning alone cannot prove causality. Multibrain stimulation can. Trends Cogn Sci 25, 96–99. doi:10.1016/j.tics.2020.11.003 (2021).
- Moreau, Q.& Dumas, G. Beyond Correlation versus Causation: multi-brain Neuroscience Needs Explanation. Trends Cogn Sci 25, 542–543. doi:10.1016/j.tics.2021.02.011 (2021).
- Novembre, G. & Iannetti, G. D. Proving causality in hyperscanning: multibrain stimulation and other approaches: response to Moreau and Dumas. Trends Cogn Sci 25, 544-545 (2021).
- Duan, R.-N. et al. Differential entropy feature for EEG-based emotion classification Proceedings of the 6th International IEEE EMBS Conference on Neural Engineering (NER) 2013, 81-84 (2013).
- Li, T. et al. Analyzing brain connectivity in the mutual regulation of emotion–movement using bidirectional Granger causality. Front Neurosci 14, 369 (2020).
- Astolfi, L. et al. Raising the bar: can dual scanning improve our understanding of joint action? NeuroImage 216, 116813 (2020).
- Chen, D. et al. Gamma-band neural coupling during conceptual alignment. Hum Brain Mapp 43, 2992–3006. doi:10.1002/hbm.25831 (2022).
- Chen, Y. & Farivar, R. Natural scene representations in the gamma band are prototypical across subjects. NeuroImage 221, 117010 (2020).
- Xie, E. et al. Neural mechanisms of the mood effects on third-party responses to injustice after unfair experiences. Hum Brain Mapp 43, 3646–3661. doi:10.1002/hbm.25874 (2022).
- Nguyen, T. et al. Interpersonal neural synchrony during father-child problem solving: an fNIRS hyperscanning study. Child Dev 92, e565–e580. doi:10.1111/cdev.13510 (2021).
- Smirnov, D. et al. Emotions amplify speaker–listener neural alignment. Hum Brain Mapp 40, 4777-4788. doi:10.1002/hbm.24736 (2019).
- Lu, K. et al. The hyper-brain neural couplings distinguishing high-creative group dynamics: an fNIRS hyperscanning study. Cereb Cortex 33, 1630–1642. doi:10.1093/cercor/bhac161 (2023).
Koelstra, S. et al. DEAP: A database for emotion analysis using physiological signals. IEEE Trans Affective Comput 3, 18-31 (2021).
Santamaria, L. et al. Emotional valence modulates the topology of the parent-infant inter-brain network. NeuroImage 207, 116341. doi:10.1016/j.neuroimage.2019.116341 (2020).
Zheng, W.-L. & Lu, B.-L. Transactions on autonomous mental development (IEEE TAMD) 7, 162-175 (2015) Investigating Critical Frequency Bands and Channels for EEG-based Emotion Recognition with Deep Neural Networks, accepted by IEEE.
Table 1. Average PLVs over Video Groups
|
Condition |
Delta |
Theta |
Alpha |
Beta |
Gama |
|
|
Valence |
Negative |
1.499 |
1.566 |
1.887 |
1.634 |
1.553 |
|
Neutral_V |
1.734 |
1.595 |
1.909 |
1.740 |
1.585 |
|
|
Positive |
1.499 |
1.568 |
1.833 |
1.732 |
1.601 |
|
|
Arousal |
Passive |
1.580 |
1.600 |
1.954 |
1.736 |
1.590 |
|
Neutral_A |
1.505 |
1.564 |
1.804 |
1.774 |
1.545 |
|
|
Active |
1.496 |
1.482 |
1.734 |
1.706 |
1.595 |
|
|
Total Average PLV |
1.513 |
1.572 |
1.848 |
1.744 |
1.570 |
|
Table 2. Analysis of Difference Significance between PLVs of 40 VGs and Total Average PLV
|
Condition |
P Value |
Delta |
Theta |
Alpha |
Beta |
Gama |
|
Valence |
Negative |
0.793 |
0.923 |
0.374 |
0.048 |
0.717 |
|
Neutral_V |
0.204 |
0.219 |
0.044 |
0.847 |
0.478 |
|
|
Positive |
0.503 |
0.843 |
0.642 |
0.642 |
0.130 |
|
|
Arousal |
Calm |
0.379 |
0.124 |
0.006 |
0.710 |
0.379 |
|
Neutral_A |
0.652 |
0.720 |
0.157 |
0.190 |
0.211 |
|
|
Excited |
0.656 |
0.035 |
0.204 |
0.490 |
0.657 |
Table 3. ANOVAs of PLVs
|
P value |
Delta |
Theta |
Alpha |
Beta |
Gama |
|
Valence |
0.401 |
0.368 |
0.239 |
0.319 |
0.365 |
|
Arousal |
0.398 |
0.474 |
0.258 |
0.383 |
0.535 |
Table 4. Average SPD Values over Video Groups
|
Condition |
Delta |
Theta |
Alpha |
Beta |
Gama |
|
|
Valence |
Negative |
32.96 |
34.99 |
37.68 |
38.28 |
39.53 |
|
Neutral_V |
30.12 |
31.98 |
34.41 |
35.83 |
41.22 |
|
|
Positive |
34.96 |
36.84 |
39.21 |
39.85 |
36.22 |
|
|
Arousal |
Passive |
29.47 |
31.88 |
34.31 |
34.62 |
35.92 |
|
Neutral_A |
32.47 |
34.21 |
36.72 |
33.39 |
38.81 |
|
|
Active |
34.33 |
35.68 |
38.23 |
38.86 |
40.66 |
|
|
Total Average SPD |
32.77 |
33.51 |
35.51 |
34.71 |
36.48 |
|
Table 5. ANOVAs of SPD Values
|
P value |
Delta |
Theta |
Alpha |
Beta |
Gama |
|
Valence |
0.074 |
0.039 |
0.044 |
0.046 |
0.073 |
|
Arousal |
0.129 |
0.034 |
0.062 |
0.295 |
0.317 |
Table 6. Numerical Characterization of SPD Values in Three Days
|
Dis_Day_Freq |
Dim. |
Mean |
Max |
Std. |
Dis_Day_Freq |
Dim. |
Mean |
Max |
Std. |
|
dis_F_Delta |
84*84 |
0.7512 |
7.9520 |
1.0845 |
dis_F_Beta |
84*84 |
0.7294 |
4.6469 |
0.9585 |
|
dis_S_Delta |
64*64 |
0.6474 |
5.7449 |
0.9072 |
dis_S_Beta |
63*63 |
0.6392 |
3.6327 |
0.8407 |
|
dis_T_Delta |
53*53 |
0.5401 |
3.3065 |
0.6890 |
dis_T_Beta |
53*53 |
0.5627 |
2.9348 |
0.7154 |
|
dis_F_Theta |
84*84 |
0.7296 |
5.0098 |
0.9708 |
dis_F_Gamma |
63*63 |
0.8890 |
13.1677 |
1.3672 |
|
dis_S_Theta |
64*64 |
0.6415 |
3.5897 |
0.8436 |
dis_S_Gamma |
53*53 |
0.7041 |
6.9368 |
1.0121 |
|
dis_T_Theta |
53*53 |
0.5782 |
3.2800 |
0.7453 |
dis_T_Gamma |
52*52 |
0.5759 |
3.6124 |
0.7272 |
|
dis_F_Alpha |
85*85 |
1.1646 |
19.6959 |
1.8986 |
|
|
|
|
|
|
dis_S_Alpha |
64*64 |
0.8750 |
9.4995 |
1.2501 |
|
|
|
|
|
|
dis_T_Alpha |
53*53 |
0.8130 |
5.2643 |
1.0599 |
|
|
|
|
|
No competing interests reported.