In the present study, we investigated whether the synchronized presentation of sound and motion reduces MS while experience a simulated bicycle ride in a VR environment, using EEG and questionnaire analyses. As a result, it showed that SSQ and FMS scores significantly decreased during the VRMS task that provided subjects with both types of additional sensory stimulation, compared to the VR task that did not provide subjects with either sound or motion stimulation. In addition, the parietal and occipital lobes exhibited significant EEG power changes in response to vestibular and visual stimuli. The ERSP of the spectral changes revealed differences in the subject's level of MS during each task. During the VR task, the dB power in the alpha, beta, gamma, and theta band increased in the parietal and occipital lobe relative to the dB power recorded during the VRMS task.
MS felt by an individual in a VR environment could be measured using a questionnaire. The results of this study found that SSQ total scores and FMS scores were higher in the VR task and lower in the VRMS task23–25. Many studies have suggested that higher SSQ total and FMS scores indicate more severe MS symptoms. In 2020, Sawada et al found that when a synchronized stimulus of engine sound and vibration was applied while subjects experienced a simulated motorcycle ride in a VR environment, SSQ and FMS scores significantly decreased, and MS was reduced11. A study that compared direct participation in a VR game to only watching showed that when subjects participated in the game, MS levels were reduced. However, there was no reduction in MS with sound stimulation only40. In 2001, Nakadai et al reported that wind sounds and proprioceptive sensory stimuli, which reflect the speed of movement, are helpful in predicting incoming visual feedback, which is increasingly likely to coincide with stored sensory integration41. In contrast, adding sound or movement to a visual has not been found to be enough to reduce MS. It has been reported that adding only one sensory stimulus to a visual VR environment has only a limited effect in reducing the discrepancy between perception and movement, which is consistent with the results of the current study11,25,41.
Several studies have shown that when MS is induced, the alpha and theta power bands near the parietal lobe show the most observable variations on EEG. These results have been related to the location of the parietal lobe, a transition region between the somatosensory and motor cortex, which is involved in the integration of spatial information, including somatosensory information from vestibular sensory system input14,23. These studies concurrence with our results. In 2019, Li et al compared EEG changes in HMD-based VR roaming scenes with various road conditions and found that the alpha and theta power in the parietal lobe increased as MS levels increased12. Previous theta wave studies have concluded that the increase in parietal theta power with MS is related to increased sensory input and motor planning integration42. It has also been suggested that theta oscillations play a role in coordinating the activity of various brain regions to update the motor plan in response to somatosensory input43. Therefore, increased alpha and theta power in the parietal lobe is thought to be associated with MS.
In 2015, Naqvi et al reported a significant increase in occipital lobe alpha power when the SSQ score increased. It has also been reported that increased alpha power in the parietal and occipital regions is likely to indicate the presence of MS23. Studies on the correlation between VR symptoms and EEG recordings showed that nausea and theta power in the occipital lobe were positively correlated 26. It has been suggested that visual information conflicts have the greatest influence and, by reducing the role of the visual domain, result in an increase in theta power. Consequently, this change in occipital lobe power could be an indirect result of multi-sensory system conflict, increasing brain load compared to the steady-state, because the multi-sensory system conflict continues to look at the virtual environment despite MS44,45.
However, Motion-sickness-induced EEG power changes are not consistent among all of the cited studies. One reason could be the different paradigms used to induce motion sickness 28,30,44. In this study, we used a combination of visual and vestibular inputs. This could be a way to increase the realism with respect to changes in EEG power for sensory impingement in a VR environment rather than a single modality scheme. Nevertheless, this study has several limitations. First, it is difficult to generalize the results of this study, because the age range of the recruited subjects (20s) is rather limited, and the sample size is small. Second, since only a limited selection of sensory stimuli types was used in this study, we suggest conducting future studies that include multi-sensory feedback, such as tactile and temperature factors. Third, the presence or absence of VR experiences, and gender differences were not taken into consideration in this study.
The present study was conducted to investigate activity changes in cerebral cortex regions and questionnaires that might be related to reductions in MS-induced in a VR environment. The results obtained indicate that discrepancies between visual perception and proprioception are associated with increases in MS. Thus, provide insights that could be used in the development of VR applications that reduce MS. Consequently, they suggest that the relationship between visual perception and proprioception is important for the user to adjust to the VR environment and that synchronization of sensory stimulation is necessary.
Table 1
General characteristics of the subjects
Number of subjects
|
Sex
(male/female)
|
Age(year)
|
Height(cm)
|
Weight(kg)
|
25
|
17/8
|
24.12(2.80)
|
172.12(8.04)
|
74.08(13.38)
|
Values represent the mean (± standard deviation) |
Table 2
Comparison of the results of SSQ total score and FMS score between the tasks
|
Condition
|
Mean
|
F
|
p
|
SSQ
|
VR
|
39.49
(14.89)
|
4.852
|
0.004*
|
VRS
|
35.31
(21.51)
|
VRM
|
34.56
(16.77)
|
VRMS
|
27.68
(14.44)
|
FMS
|
VR
|
3.51
(2.68)
|
3.042
|
0.034*
|
VRS
|
3.25
(3.22)
|
VRM
|
3.21
(3.10)
|
VRMS
|
2.16
(1.98)
|
Values represent the mean (± standard deviation)
SSQ: simulator sickness questionnaire, FMS: fast motion sickness scale, VR: virtual reality, VRS: virtual reality sound, VRM: virtual reality motion, VRMS: virtual reality motion sound; *p < 0.05.
Table 3
Comparison of the results of relative power in the parietal and occipital area between the tasks
Band
|
Condition
|
parietal
|
occipital
|
Relative power
|
F
|
p
|
Relative power
|
F
|
p
|
Alpha
|
VR
|
12.74
(5.22)
|
5.136
|
0.003*
|
10.85
(6.53)
|
4.526
|
0.006*
|
VRS
|
10.74
(5.08)
|
9.78
(5.55)
|
VRM
|
11.53
(4.06)
|
9.04
(6.06)
|
VRMS
|
9.57
(7.09)
|
8.53
(6.70)
|
Beta
|
VR
|
17.37
(4.20)
|
70.596
|
< 0.001*
|
17.25
(3.65)
|
1.482
|
0.227
|
VRS
|
16.79
(2.46)
|
16.77
(3.88)
|
VRM
|
16.84
(2.90)
|
16.55
(3.63)
|
VRMS
|
16.42
(2.83)
|
16.29
(3.70)
|
Gamma
|
VR
|
25.34
(7.72)
|
1.826
|
0.150
|
26.41
(11.46)
|
0.839
|
0.443
|
VRS
|
24.24
(7.94)
|
25.34
(7.72)
|
VRM
|
23.95
(7.94)
|
26.36
(9.79)
|
VRMS
|
21.81
(6.20)
|
23.61
(9.30)
|
Theta
|
VR
|
33.12
(6.83)
|
2.961
|
0.038*
|
34.41
(11.29)
|
3.522
|
0.019*
|
VRS
|
29.75
(11.02)
|
31.35
(11.85)
|
VRM
|
30.93
(7.72)
|
30.37
(9.48)
|
VRMS
|
28.53
(6.30)
|
29.28
(10.29)
|
Values represent the mean (± standard deviation)
VR: virtual reality, VRS: virtual reality sound, VRM: virtual reality motion, VRMS: virtual reality motion sound; *p < 0.05.