The general experimental setup comprises the following steps: the setup of the VR headset, the development of the VR video, the acquisition of EEG signals along video visualization, and data analysis. The main steps involved in data analysis are presented in Fig. 1 and detailed in the next sections.
Visual Stimulus
The VR video was created recurring to the cross-platform game engine Unity (Unity Software Inc., San Francisco) and consists of a calm scenario of a forest, at dusk light, containing typical elements such as trees, flowers, and grass (Fig. 2.a), light breeze movements, and soft background forest sounds. The goal was to create a calm scenario with few distracting elements to reduce strong modifications in brain activity, besides the ones related to the threat stimulus. The scene was presented as if the observer were walking straight ahead along the forest path. The visual stimulus was projected using a VR headset (Reverb G2, HP, Palo Alto, California) comprising lenses and headphones.
The video lasts 5 m 35 s, and at 2 m 3 s the threatening stimulus is presented (Figs. 2.b and 2.c). It consists of an unexpected rock coming towards the viewer that ends up obscuring the scene and then disappears. The rock is in the scene for about 1 s and is also accompanied by a crashing sound. To identify whether the participant's reaction to the threat stimulus and the associated brain activity relate to defensive mechanisms or just to the sensory inputs, two additional stimuli were introduced. To segregate the effect of the crashing sound accompanying the threat stimulus, at 3 m 14 s, the same sound was reproduced but without the visual stimulus (without the rock). Also, at 4 m 45 s, a small bird crosses the scene (Fig. 2.d), representing a neutral visual input.
EEG Data Acquisition
To register the participant's brain activity, an electrode cap (Electro-Cap International, Eaton, Ohio, United States) following the international 10–20 electrode placement system was used. Activity was registered from 19 channels including: Fp1, Fp2, F7, F3, Fz, F4, F8, T3, C3, Cz, C4, T4, T5, P3, Pz, P4, T6, O1, and O2. Signals were acquired at a sampling frequency of 512 Hz and stored on a computer for posterior analysis. Both, VR video and EEG records were synchronized to identify the stimuli’ times.
Participants and Protocol
The study was approved by the Ethics Committee of the Faculty of Medicine of the University of Coimbra under the protocol CE-017/2023, and all procedures were performed in accordance with the relevant guidelines and regulations. Participants were informed about the study protocols, except for the content of the video. Each participant who agreed to enroll signed an informed consent. The inclusion criteria consisted of young adults between 18 and 40 years old. The exclusion criteria were the presence of visual deficits, neuropathological conditions, cardiac pathologies, and the use of neuropharmaceuticals. No further gender, social, or educational conditions were considered. The study included 26 eligible volunteers, and data was anonymized with a numeric code to preserve participants' identity.
The involvement of each participant contemplated only one session, where the participant was connected to the EEG system to record brain activity while simultaneously viewing the VR video through the VR headset (Fig. 3). The whole procedure (including questionnaires, participant preparation, and video projection) took about one hour, and no posterior follow-up was required. During the procedure, participants were comfortably seated in a chair and were instructed to stay as still as possible, not talk, and stay calm during the video visualization. At the end, they were asked about their personal experiences while watching the video, specifically, if the threat stimulus was observed and if they felt threatened by it.
Signal Processing and Feature Extraction
All processing and subsequent analysis of the acquired EEG data was implemented using MATLAB programming language.
The EEG frequency range of interest was considered between 0.5 Hz and 80 Hz. An infinite impulse response (IIR) notch digital filter of second order was applied to exclude the frequency component of 50 Hz related to the power-line interference. Additionally, a second IIR notch filter of second order was applied at 37 Hz since this frequency component, related to external sources, was also evidenced in the signal spectrum. To eliminate the lower (< 0.5 Hz) and higher (> 80 Hz) frequency components, the data was bandpass filtered, applying a finite impulse response (FIR) filter of minimum-order (i.e., the order is automatically computed to be the minimum the FIR filter must have to meet the specifications).
To remove the remaining artifacts (including eye blinking, eye movements, muscular activity, and cardiac artifacts) it was used independent component analysis (ICA). The number of computed independent components was the same as the number of EEG channels (i.e., 19). The identified components related to artifacts were eliminated, and the EEG was again reconstructed.
The following step consisted of extracting features from the filtered data. The relative power spectral density (rPSD) values were obtained for each EEG channel and for each frequency band: delta (0.5 to 4 Hz), theta (4 to 7 Hz), alpha (8 to 12 Hz), beta (13 to 30 Hz) and gamma (30 to 80 Hz). To compute the PSD values, the signals were first segmented in 1 s windows with an overlap of 50%, using a Hamming window. Then, for each segment, the PSD value of each frequency band was computed by adding all the PSD values within each frequency band. Finally, the rPSD value for each band resulted from dividing the corresponding PSD values by the sum of all the PDS values. Therefore, 95 features were obtained, corresponding to each frequency band (5 bands) and each EEG channel (19 channels).
Identification of Relevant Brain Areas
The next step concerns the study of the activated brain areas during a threatening situation. To discard activations associated exclusively with sensorial perception, the activated brain areas for the two control stimuli (crashing sound and bird stimuli) described in the Visual Stimulus section were also considered. Being so, the neutral state (pre-stimulus) and each of the stimulus states (post-stimulus) were analyzed. Related to the time window for the neutral state, it was used the average of the rPSD values between 20 s and 100 s (before the appearance of the first stimulus), aiming to reduce variability in the neutral condition. Regarding the time window for each stimulus, several consecutive post-stimulus windows were considered to account for the total duration of the stimuli, as well as for possible delays in reaching the different brain areas. For the threat stimulus, the time windows considered were 122.5 s to 123.5 s, 123 s to 124 s, and 123.5 s to 124.5 s. Meanwhile, for the crashing sound stimulus, the intervals were 193.5 s to 194.5 s, 194 s to 195 s, and 194.5 s to 195.5 s. Lastly, in the case of the bird stimulus, the time windows included were 285.5 s to 286.5 s, 286 s to 287 s, 286.5 s to 287.5 s, and 287 s to 288 s (the additional window accounting for the longer duration of this stimulus).
Firstly, the One-sample Kolmogorov-Smirnov test (with a significance level of 0.05) was applied to the rPSD values of each frequency band and EEG channel of all the participants, to depict if the data followed a normal distribution.
The subsequent step consisted of identifying the relevant EEG channels and frequency bands. A channel is considered relevant if the associated rPSD values exhibit significant differences between the neutral state (time interval without stimuli) and the stimuli states (time intervals starting at the beginning of each applied stimulus). For this, the Kruskal-Wallis nonparametric multiple comparison test was applied, with a significance level of 0.05. For each of the three stimuli separately (threat, crashing sound, and bird), the Kruskal-Wallis test was conducted among the rPSD data concerning the neutral condition and each of the consecutive post-stimulus time windows, as shown in Fig. 4.
After the multiple comparison tests, a multiple comparison correction using the Bonferroni method was applied, with α set to 0.05.
Of the 26 participants, 6 did not react or feel threatened by the stimulus and were excluded from the analyses. However, the data from these 6 participants were posteriorly used for comparison, following the same statistical analysis first performed for the 20 participants who felt threatened.
Functional Connectivity Between Relevant Brain Areas
The next step was to find the existence and strength of connections between the activated brain areas (i.e., the areas identified in the previous step as being involved in the defensive reaction). In this sense, five methods of functional connectivity were implemented: coherence (COH)14,15, imaginary part of coherence (iCOH)14,16, weighted phase lag index (wPLI)17,18, mean phase coherence (MPC)19, and direct transfer function (DTF)20. The connectivity methods iCOH and wPLIi overcome the problems of volume conduction and signal-to-noise ratio (SNR). The volume conduction problem refers to the electrical activity of a source being present in more than one EEG channel due to bone and other scalp structures conduction. The SNR quantifies the relative noise content in the signal14,16. The COH, iCOH, wPLI, and MPC methods were used to find the relevant connections while the DTF method was used to find the direction of these connections.
For each connectivity method, the connectivity values for each channel pair were evaluated under two conditions: neutral state (pre-stimulus) and threat stimulus state (post-stimulus). Once again, to reduce variance within the neutral state, the connectivity was calculated for the time series data corresponding to each second of the interval comprehended between 20 s to 100 s (corresponding to a neutral part of the video), and then the resultant connectivity values were averaged. Regarding the threat stimulus state, the associated connectivity values were obtained for the time series data corresponding to the instant of the threat appearance on the video, i.e. the 122.8 s − 123.8 s time window. It is to be noted that this connectivity is independent of frequency bands, encompassing the entire spectrum of the signals (0.5 Hz to 80 Hz).
For each connectivity method, the connectivity values of each channel pair were stored in three matrices containing the average of these values across participants: one for the neutral state (pre-stimulus), one for the threat stimulus state (post-stimulus), and one for the difference between the connectivity values of the stimulus and the neutral states (post-pre). Using the post-pre matrices, for each connectivity method, the pairs of channels whose post-pre values were greater than a threshold corresponding to the 0.65-quantile (that is, the value above which 35% of the post-pre values are) were obtained. These correspond to the pairs of channels with a relevant connection. To obtain more reliable results, only the relevant channel connections that were present in at least three of the four connectivity methods were considered relevant.
The next step consisted of depicting the direction of the identified connections, that is, the direction of the information flow. In this sense, the DTF method was applied to the post-stimulus data to discover the direction of each of the previously found significant connections. The post-stimulus data was the same used for the other connectivity methods (corresponding to the 122.8 s − 123.8 s time window), and a DTF matrix containing the DTF values of each channel pair averaged over all the participants was obtained.