Embodiment comfort levels during Motor Imagery training combined with immersive Virtual Reality in spinal cord injury: a case report

doi:10.21203/rs.3.rs-1411361/v1

PDF

Case Report

Embodiment comfort levels during Motor Imagery training combined with immersive Virtual Reality in spinal cord injury: a case report

https://doi.org/10.21203/rs.3.rs-1411361/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted 02 Mar, 2022

You are reading this latest preprint version

Brain-machine interfaces combining visual, auditory, and tactile feedback have been previously used to generate embodiment experiences during spinal cord injury (SCI) rehabilitation. It is not known if adding temperature to these modalities can result in discomfort with embodiment experiences. In this case study, the levels of comfort with the embodiment experiences were investigated in an intervention that required a chronic pain SCI patient to generate lower limb motor imagery commands in an immersive environment combining visual (virtual reality -VR), auditory, tactile, and thermal feedback. Assessments were made pre/post and throughout the intervention (Weeks 0–5), and at seven weeks follow up. Overall, high levels of embodiment in the adapted three-domain scale of embodiment were found throughout the sessions. No significant adverse effects of VR were reported. Sessions induced only a modest reduction in the levels of pain. Overall reductions occurred in all pain scales (Faces, Intensity, Verbal) at follow up. A high degree of comfort in the comfort scale for the thermal-tactile sleeve, in both the thermal and tactile feedback components of the sleeve was reported. This study supports the feasibility of combining multimodal stimulation involving visual (VR), auditory, tactile, and thermal feedback to generate embodiment experiences in neurorehabilitation programs.

Neurology

Cognitive Neuroscience

spinal cord injury

comfort

thermal-tactile stimulation

virtual reality

brain-machine interface

chronic pain

Spinal cord injury (SCI) is often associated with impairment in multiple functions in motor, sensorial, and other body systems [1,2,3] as well as a reduction in quality of life [4]. Recent studies have demonstrated that rehabilitation programs involving brain-machine interfaces that use real-time decoding of neural activity, virtual reality (VR), exoskeletons, electrical stimulation, and tactile feedback; are useful for SCI rehabilitation not only because they allow replacing functions (e.g., walking), but also due to their ability to generate beneficial neuroplastic effects [5,6,7,8,9,10]. Namely, improvements in sensorial, motor, sexual, digestive, and urinary systems have been reported in SCI patients undergoing rehabilitation programs combining BMI training with VR, assisted gait training, and BMI controlled exoskeletons [5,8].

The precise mechanism underlying these neuroplastic effects occurring in SCI neurorehabilitation programs involving embodiment and BMI controlled avatars/exoskeletons is still under investigation [5,6,7,11,12]. It has been proposed that it may be associated with the sense of embodiment (i.e., experiencing the avatar’s as one’s own body) [13,14]. Embodiment is characterized by the senses of ownership, agency, and self-location [15]. The neural basis of embodiment involves multiple areas associated with processing of the human body or its parts, self-processing, as well as multisensory integration, namely the extra-striate area and the temporoparietal junction [16], ventral premotor cortex, medial superior temporal area, and the Rolandic operculum [17,18]. Therefore, embodiment seems to be dependent on the number and type of feedback received by the subject [19].

Following in the footsteps of these previous rehabilitation programs and expanding the number of feedback modalities, we have developed a neurorehabilitation BMI setup including tactile, thermal, visual (VR), and auditory feedback (Fig. 1). The general underlying hypothesis is that a more immersive (i.e., “realistic”) rehabilitation environment will lead to better performances in motor imagery tasks [20] and consequently to increased or earlier neuroplastic effects [5,21,22] in SCI patients. It is not known, however, if increasing the number of feedback modalities will improve the overall effects of the rehabilitation program, or otherwise may become redundant or detrimental for BMI performance. For example, increasing the number of feedback modalities could lead to information overload [23] resulting in mental fatigue in the motor imagery task [24] or work as a distractor [25], due to the excessive likeness of the avatar (i.e., uncanny valley effect) [26], or due to the additional paraphernalia worn by the patient. Also, the multimodal feedback could potentiate the effects of VR on the vestibular system, or the illusory lower limbs experience may lead to some unexpected form of discomfort [27]. In fact, while a significant effort has been made to guarantee the safety of the equipment used in these treatments (see [10] for a review), little attention has been given to the levels of comfort associated with the embodiment experience. A description of the comfort levels experienced during embodiment in neurorehabilitation is therefore relevant since it may significantly condition the development and application of future protocols.

Here, we report the levels of comfort with embodiment of lower limbs experienced by a SCI patient tested in a multimodal feedback setup. The participant was an ASIA [28] complete T4 SCI stabilized patient engaged in a rehabilitation program (Fig.2) involving the recording and analysis of neural activity, combined with the use of multimodal visual (VR), auditory, tactile, and thermal stimuli to create immersive scenarios (Fig.3 a-d). The intervention used to assess comfort is the first part of a multiphase neurorehabilitation program described elsewhere [5]. In the current phase, motor imagery content was focused on generating “Walk” versus “Stop” commands upon a visual cue appearing in the VR scenario. We expected that the embodiment experience of lower limbs, associated with the multimodal immersive environment, would not lead to significant or prolonged discomfort (with comfort considered as a score of five points or above in a seven point scale in all three domains of the adapted version of the avatar embodiment questionnaire: a) perception of body qualities, b) volitional control of movements, and c) tactile perceptions) [29], nor significant VR side effects (considered as a score above 2.0 in the 4-point Simulator Sickness Questionnaire [30]).

In addition, we also expected that the potential discomfort generated by the additional paraphernalia required to generate the thermal-tactile feedback, would be largely compensated by the quality of the immersive experiences allowed by the fully working setup, and therefore would not be described by the patient as a source of discomfort.

A total of 400 trials (200 “Walk” and 200 “Stop”) in 10 sessions, all of them using the VR setup, were acquired. Of these 10 sessions, 9 were performed with real-time decoding of neural activity. A total of 6 sessions were performed with the thermal-tactile sleeves and 4 sessions solely with the VR googles and headphones. The raw values for the embodiment questionnaire, self-reported levels of pain, comfort with thermal-tactile sleeve, and performance in each session are presented in the Supplementary Information S1.

Analysis of the 7-point avatar embodiment comfort questionnaire revealed high levels of embodiment experiences in all three domains, namely in body properties (6.167±1.621 a.u.), volitional control (6.333±1.493 a.u.), and tactile sensations (6.567±0.971 a.u.). This adapted questionnaire also revealed that the participant was comfortable with the embodiment experiences in all three domains, namely: comfortable with body properties (6.9±0.305 a.u.), comfortable with volitional control (6.7±0.651 a.u.), and comfortable with tactile sensations (6.7±0.466 a.u.). In Fig. 4 (a-i), the details of the comfort with avatar embodiment questionnaire are presented. Each panel depicts, for all sessions, the results for each item related to the embodiment experience (empty circles) and the level of comfort of the participant with that experience (filled circles). Embodiment experiences were typically smaller in the first week (i.e., the first two sessions), and then increased throughout the remaining of the intervention. Meanwhile, the levels of comfort remained high throughout most of the sessions even when the embodiment experiences values were low, or the pain values were high (also see self-reported pain below). For example, in Fig. 4 panels a, c, e, f, and i; which are associated with all three domains (each domain corresponds to one row comprising three panels), present large differences between the reported embodiment experiences and the overall level of comfort for those experiences, especially in the first sessions. In other words, the absence of embodiment experiences (which occurred only in the initial sessions), and the presence of embodiment experiences, were not associated with high levels of self-reported discomfort.

Analysis of VR side effects indicated few issues throughout the sessions (Table 1), with an overall score of 0.08125±0.2741 a.u. in the 3-point Simulator Sickness Questionnaire. The presence of side-effects was more pronounced in the first session with 46.15% (6/13 side effects reported). The most frequent side effects were general difficulty focusing (3/13=23.08% of the effects reported), general discomfort and vertigo (both with the same frequency; 2/13=15.38% of the effects reported). Each one occurring in two different sessions. Lastly, during the interviews, the participant indicated that the side effects were temporary and restricted to the session period.

The participant typically chose complex scenarios involving a mixture of grass, rock, sand beach and water (such as in Fig. 3 a-c, where different parts of the same scenario are presented). When asked, the participant indicated that he had no preference for urban or natural scenarios. The participant’s favorite scenario included the avatar’s legs virtual reflection on the side of the walking path, allowing the patient to observe the gait motion while still observing the scenario through the first-person perspective.

Neural decoding performance (Fig. 5 a), collected from a total of 9 sessions, indicated that the participant could generate the neural commands associated with “Walk” and “Stop” with an overall performance of 75±23% throughout the sessions. The performance in sessions with the use of the thermal-tactile sleeve was not above those without the sleeve (Sleeve: 82.50%±6.847; No Sleeve: 73.50%±8.407; Mann-Whitney U=5.0, P=0.2857, n.s.).

The results of the three different pain questionnaires applied during pre-intervention, the intervention sessions, the post-intervention evaluation and at seven weeks follow up (Fig. 5 b,c) indicated that the intervention was associated with a reduction in the self-reported pain level of the Faces pain scale and a temporary reduction in the remaining scales. In session 6, (a session where the thermal-tactile sleeve was used), a sudden increase in all three pain ratings was present. This session coincided with the first day of work after several weeks of sick leave for the participant and, in addition, with a particularly hot day (36ºC).

Self-reports for the Faces pain scale (Fig. 5 b, blue circles) dropped from 10 points in the baseline evaluation, to an average of 5.8±0.146 points throughout the intervention sessions, and then increasing to a final 8 points in the post-intervention evaluation. No differences were found between sessions with (filled circles) and without (empty circles) the thermal-tactile sleeve (Mann-Whitney U=6.0, P=0.3379, n.s.).

The VAS intensity of pain scale (Fig. 5 b, black circles) was 8 (in a 10-point scale) in the baseline evaluation, then dropped to an average of 7.1±0.876 throughout the training sessions. However, during the post-intervention evaluation, self-reported intensity of pain again increased to 8 points. No differences were found between sessions with (filled circles) and without (empty circles) the thermal-tactile sleeve (Mann-Whitney U=4.0, P=0.1632, n.s.).

In the Verbal pain scale (Fig. 5 b, red circles), the participant described “severe pain” during the baseline, and in 7/10=70% of the remaining sessions indicated “moderate pain”, and in one session “mild pain” (1/10=10%). Lastly, the participant reported that, since he had started the intervention, he no longer felt like “cutting his trunk with a saw to end the pain”. In four sessions, where the thermal-tactile sleeve was used, pain ratings were described as moderate, and in two sessions as severe. Meanwhile, in three sessions without the thermal-tactile sleeve, pain was described as moderate and in one case as mild.

Comparison of self-reported pain at follow up, with the pre- and post-intervention scores, revealed an overall decrease in all three scales (Fig.5 d). A two-point reduction was observed between pre-/post and follow up for the VAS intensity of pain scale (starting at 8, remaining in 8, and dropping to 6 a.u.). A four-point reduction was observed for the Faces pain scale (starting in 10, reducing to 8, and dropping to 6 a.u.); and a one level reduction in the Verbal pain scale (starting severe, remaining severe, and reducing to moderate).

Evaluation of the six sessions where the thermal-tactile sleeve was used, revealed an overall high level of satisfaction 6.75±0.4423 a.u. (7-point scale) in the four-item scale. Namely, items related to sleeve placement and sleeve usability were evaluated as 6.83±0.4082 a.u. and 7.0±0.00 a.u., respectively. Meanwhile items related to thermal comfort and tactile comfort were evaluated as 6.33±0.5163 a.u. and 6.83±0.4082 a.u., respectively. In addition, the participant indicated preferring sessions with the thermal-tactile sleeve because he would get a more precise notion of the length of the avatar’s steps due to the tactile feedback.

Analysis of the patient’s additional comments throughout the sessions suggested that the embodiment experiences had effects during the period spent in the laboratory as well as at home. Namely, the patient repeatedly described his experience of interacting with the setup as being positive, specifically through comments such as “there are not enough stars in the universe to rate how much I am enjoying this experience”, “this is the first time that I feel like I am standing up and walking in thirty years”. In addition, in one session, the patient reported having vivid dreams of performing sessions where he was walking with the avatar in an unspecified scenario.

In three sessions with scenarios involving water, the participant reported feeling his legs cold as soon as he entered the VR scenario. This was not exclusive of sessions with the thermal-tactile sleeve. This was considered as a pleasant experience by the participant. He reported being surprised but not uncomfortable, since he had never experienced cold feet or legs in the previous 30 years.

The present case study describes high comfort levels with embodiment experiences in a SCI participant during a motor imagery task combining visual, auditory, tactile, and thermal feedback. These levels of comfort were high in all three domains of the embodiment experience (i.e., body properties, volitional control, and tactile experiences) with or without the use of the thermal-tactile sleeves. Even when the motor imagery scores were low, as occurred in the first sessions, comfort levels with the embodiment experiences remained high. Also, the physical apparatus itself, (consisting of two tethered thermal-tactile sleeves, one VR headset with headphones, and an EEG cap covered by an additional disposable plastic protection) did not induce significant discomfort during embodiment experiences, nor did it increase the VR side effects. Self-reported pain levels presented a modest reduction throughout the sessions, with a relevant decrease appearing at seven weeks follow up. These findings, while limited in their scope and generalization, support the feasibility of testing visuo-auditory and thermal-tactile feedback in a SCI neurorehabilitation program.

Embodiment experiences are complex phenomena that involve multiple neural networks [13, 15,16] and which may result in discomfort related to pain or to VR side effects [31,32,33,34]. In the present study, the participant reported high levels of comfort while interacting with the avatar and simultaneously receiving four different types of feedback. The results from the questionnaires, the observations made in the sessions, and the interviews performed; all indicate that the embodiment experiences started as soon as the participant interacted with the VR environment. These experiences were then intensified by the movements of the avatar (which resulted in multiple types of feedback indicating that the legs and the feet were moving properly and touching the ground as expected in normal gait) and by the transitions occurring in the scenario (e.g., moving from a part of the scenario associated with grass to another one with sand). Despite the participant’s reports that sessions with the full setup were more engaging than sessions without the thermal-tactile sleeve, no differences in accuracy of neural decoding, embodiment comfort levels, or self-reported pain ratings were found. On one hand, these results indicate that no detrimental effects in comfort or motor imagery performance were generated by the introduction of the thermal-tactile sleeve. On the other, as no clear improvement in performance, or permanent reduction in pain was observed when the thermal-tactile feedback was used; further studies with increased samples and an increased number of sessions, will be required to evaluate the true neurorehabilitation potential of the multimodal feedback setup used here [23,5,6,24,25]. Meanwhile, the overall reduction in the levels of pain, especially at follow up, and the patient’s reports of increased engagement in sessions where the thermal-tactile sleeves were used, support the use of this multimodal approach. Lastly, the comfort questionnaire used here consisted of an adaptation of a previous embodiment questionnaire [29]. Further uses of this questionnaire, and the inclusion of the comfort evaluation, will require a proper translation and cross-cultural validation to ensure that the original characteristics of the questionnaire are maintained.

We have opted to develop highly complex and realistic scenarios with the goal of increasing immersiveness, interest, and engagement to maximize comfort and improve performance in the task [20,36]. The combination of these scenarios and the multimodal feedback resulted in very few VR side effects [30,37]. We speculate that the cause for this is because we have limited the avatar movement to a single step in each trial. We have opted for this based on preliminary results obtained in a healthy subject that had previously reported severe and persistent nausea (>24 hours) when the avatar moved several steps. In this healthy subject, reducing the number of steps revealed sufficient to significantly reduce VR side effects. It should be noted however, that using a single step approach in the avatar movement may not be as rewarding as having the avatar taking multiple steps in each trial (as indicated by preliminary data from two healthy subjects who reported such an effect when testing the setup). These results suggest that future studies in SCI patients should consider a balance between one or multiple steps to maintain high levels of motivation [38], while preventing VR side effects.

Another relevant result from the present scenarios is that the participant’s choices revealed an overall preference for scenarios with multiple types of feedback, namely for mixed scenarios with a sequence of grass-stone-sand-water. In future studies it will be important to combine psychological evaluation [39] and EEG activity to identify which type of scenarios are better suited for each participant [40,41].

At the present time it is not known if the different types of feedback used here can lead to earlier or larger neuroplastic effects in SCI neurorehabilitation, when compared to the original neurorehabilitation protocol on which the present study was based [5]. The limited set of data presented here, suggests that the ability to choose between different VR scenarios, combined with the multimodal feedback, have contributed to the participant's overall motivation, possibly due to a combination of increased engagement and immersiveness [35,42,43,44,45,46], although the participant’s psychological profile may have also played a significant role [39,47].

Self-reported pain levels varied throughout the motor imagery sessions and were reduced in all three scales at follow up. These results suggest that the embodiment experiences generated during the task have the potential to induce a temporary reduction in self-reported pain scales and, in addition, support the notion that the use of the different combinations of scenarios or types of thermal-tactile stimulation did not increase baseline pain levels. Previous studies have reported temporary reductions in pain during training with VR scenarios in SCI [48,40,50], as well as in other clinical problems [31,51] (however also consider [27] for an increase in pain). Two recent reviews of studies using VR to treat neuropathic pain in SCI [49,50] suggest that the use of VR can have immediate analgesic effects, but its long-term benefits still need to be investigated. Our patient specifically reported that the use of the set up per se did not seem to change his ongoing pain, but instead that he could separate the ongoing pain from the actual embodiment experience, and the motor imagery task. Meanwhile, pain assessment in pre-/post-intervention and at seven weeks follow up, revealed and overall reduction in all three scales. Future studies with an increased number of subjects will be critical to determine if self-reported pain reductions consistently occur, and whether patients are aware of them or otherwise, if some other variable may explain our present observations. Also, the present results indicate that it may be important to ask participants to keep a record of the frequency and intensity of pain episodes occurring at home. In addition to the values obtained in the self-reported pain scales, the participant also reported that overall pain levels, even prior to the intervention, were often correlated to the number and intensity of spasmodic lower limb activity at home [52]. Although we observed spasmodic activity in two different sessions, we could not identify any specific event that triggered such activity. Future studies with electrophysiological recordings (e.g., surface electromyography) may help determine if the spasmodic activity was related to the intervention (e.g., generating “Walk command”).

Performances in the motor imagery task, indicated that the participant could generate different neural activity patterns for the algorithm to decode “Walk” and “Stop” commands [5,8,53]. These results indicate that delaying the performance feedback towards the end of the session did not preclude the participant from performing the task.

A small number of caveats should be considered in the present study. First, we have adapted a previous questionnaire by adding the question of “How comfortable were you with that experience?” to each question of the existing questionnaire. As this adaptation has not yet been properly validated and we have only analyzed one participant, the present results cannot be generalized. Second, we have not tested the full brain-machine interface setup here. Although neural activity was decoded in real-time, the movement of the avatar was not controlled by the participant’s activity. We have opted to perform the present study without the actual control of the avatar to ensure that the participant would not loose motivation due to the potential of low performances occurring, since this could bias responses related to comfort. We have meanwhile tested this patient in the setup with real-time control of the avatar and motor imagery performances were above chance.

In conclusion, high levels of comfort with embodiment experiences were present when a SCI participant was tested for five weeks in a motor imagery task combining multimodal visual, auditory, tactile, and thermal feedback. This intervention resulted in a reduction of self-reported levels of pain at seven weeks follow-up. The setup used did not induce significant discomfort nor generated significant VR side effects. These results support the feasibility of testing the effectiveness of neurorehabilitation programs based on embodiment through the application of a motor imagery task combining multimodal visual, auditory, tactile, and thermal feedback.

The study was approved by the CES- Hospital Senhora da Oliveira Ethics committee (nº15/2020). All experiments were performed in accordance with the Code of Ethics of the World Medical Association (Declaration of Helsinki) for experiments involving humans. The participating subject voluntarily filled an informed consent. The data presented here was collected between June and September 2021 at the Hospital Senhora da Oliveira, Portugal.

The participant was a 52-year-old male with an ASIA complete T4 SCI stabilized (32 years) and a history of chronic low back pain post-lesion due to surgery (5 years). The individual results for each session are presented in Supplementary Information S1.

Timeline of experiment

As presented in Fig. 2, an initial baseline evaluation was performed followed by the intervention which lasted for a period of five weeks with two sessions per week (a total of 10 training sessions), followed by a final evaluation, and a follow up at 12 weeks.

Overall apparatus

The system used here (Fig. 1) consisted of a VR system composed of a headset with headphones (for visual and auditory stimuli) and two hand controllers (used during the habituation phase), (HTC Vive Pro Eye, New Taipei City, Taiwan), a pair of custom developed thermal-tactile sleeves capable of delivering different types of tactile and thermal stimulation (i.e., the feedback of the sole of the avatar’s feet was delivered to the forearms of the user) coupled to an external control system, allowing thermal stimulation in the temperature range of 18°C to 35°C and complex tactile stimulation patterns through 6 independent vibration actuators in each arm, and EEG set up (16 channel, V-Amp, actiCAP; Brain Products GmbH, Gilching, Germany) with real-time recording and BMI decoding developed in OpenVibe. The software synchronizing and commanding the full apparatus was developed in Max (Cycling '74, San Francisco, USA), and the VR scenarios were developed in Unity (Unity Technologies, San Francisco, USA). The details of the VR scenarios, as well as the controlling software and hardware used in this case study will be described in detail in the VR environment section.

Sessions

The intervention consisted of 10 sessions run twice a week with each session lasting between 70-90 minutes. This total time was divided in: i) a period of 10 minutes for overall evaluation of comfort with the equipment as well as to gather information from the patient regarding the previous session, overall levels of pain and stress experienced at home or work, ii) a period of 20-25 minutes where the subject interacted with the VR environment (this period was composed by three sequential phases: a) habituation, b) EEG baseline and neural data acquisition; and c) real-time decoding; that are described in detail below), iii) questionnaire application, and iv) the remaining time corresponding to participant positioning, placing and calibrating the VR headset, confirming appropriate feedback from each different component of the multimodal feedback, checking EEG signal and impedance, as well as to remove the gear and cleaning the EEG gel. Sessions were run by 2-4 researchers to ensure that the full setup was working and to reduce the amount of time necessary to set the experiment. All comments related to the comfort of the embodiment experience, the apparatus, or the experiments organization; made by the participant were annotated and analyzed a posteriori.

Comfort evaluation

Comfort evaluation was performed at the end of each session, applying the adapted questionnaire [29], the Faces pain scale [54], the Verbal pain intensity scale [55], the VAS Pain Scale [56]; and the Simulator Sickness Questionnaire [30]. The overall goal was to evaluate comfort with the embodiment experiences of lower limb movement, while guaranteeing that no increases in pain would be present, or otherwise, VR side effects would occur. In addition, because we have developed a sleeve capable of delivering thermal and tactile feedback to the user, the participant was also asked about ease of placement, ease of use, thermal comfort, and tactile comfort of the sleeve; as well as overall perceived effects in the task (e.g., if the sleeves contributed to the immersiveness of the experience).

The avatar embodiment comfort questionnaire is an adaptation of the avatar embodiment questionnaire [29] designed to evaluate the full range of experiences provided by our setup. The original questionnaire is composed by nine items set to evaluate three theoretical domains: perception of body qualities (three items), volitional control of movements (three items), and tactile perceptions (three items). In the present adaptation, we added to each original item, an additional question to evaluate the comfort of the experience. Therefore, in this adaptation each domain had three items related to the embodiment experiences (e.g., “I have felt the avatar legs as if they were my own legs”), which were now followed by three additional items evaluating the comfort of that experience (e.g., “How comfortable were you?” [with the experience of feeling the avatar legs as if they were your own legs]). The adapted questionnaire is presented in Supplementary Information S1. For analysis of the questionnaire data, the values from questions that were posed in the negative form were reversed. As an example, the question “The legs felt like they belonged to someone else”, which would have a value of 1 if the embodiment experience was maximum, was reversed such that this value now corresponded to 7 points. These specific questions are indicated with (R) in Fig. 4, panels c, f, and i.

The participant was also allowed time (5-10 minutes per session) to freely describe any additional experiences that were considered as relevant (e.g., general concerns related to the general organization of the study, changes in pain at home, etc.).

Lastly, potentially relevant variables, such as extremes in external environment temperature or significant life-events (e.g., first day of work after sick-leave) were also recorded.

VR environment

The use of VR was achieved through a setup that included googles, headphones, and two hand controllers. Altogether, a total of 16 different scenarios where available (i.e., including the initial habituation and training scenarios). These scenarios involved different types of landscapes, such that the participant could walk in one or more types of ground (grass, sand, stone, water) in the same scenario. An example of the complexity of the VR scenarios used is presented in Fig. 3. The participant was allowed to choose the scenario in each session, as well as the overall presentation of the avatar. This included choosing skin tone, clothes, shoes, etc., to increase immersiveness as well as the sense of control over the experimental environment for the participant. Even though the avatar was always in the first-person perspective, the movement of the hands (which were holding the VR hand controllers), or changes in head position, would allow the participant to observe parts of the avatar’s body during movement, more specifically, the arms and hands, lower part of torso, legs, and feet. In addition, one of the VR scenarios included a virtual mirror on the side of the track, such that the participant could see a virtual reflection of the movement of the avatar legs.

The movement animation of the avatar hands and body were developed using two distinct strategies: the hands, head and upper body movement was calculated in real-time using inverse kinematics [57] processed from the VR headset and joysticks position tracking and accelerometer sensors; the walking animation was developed as a pre-processed animation obtained from the recorded and motion capture analysis of a person using the ExoAtlet exoskeleton (walking, sit-to-stand, stand-to-sit) [58]. Therefore, the avatar walking motion simulates the motion of a user in an exoskeleton. This choice was made to prepare the patient for the more advanced phases of the training protocol [5] where an exoskeleton will be used.

Interaction with the VR environment

After having all the setup calibrated, the actual sessions started. The interaction with the VR consisted of three different phases: a) habituation, b) EEG baseline and neural data acquisition for neural network training; and c) testing real-time decoding of neural activity without control of avatar (Fig. 1 b). During the habituation period, the participant was asked to choose a scenario and to interact with it using the hand controllers that triggered the avatar steps. This period allowed identification of eventual adjustments that needed to be made to the sleeves, VR environment, or wheelchair position. At this point, the participant was also asked about potential side effects due to the interaction with the VR set up [30,37]. During the initial baseline period, 20 seconds of neural activity were recorded while the participant remained with the eyes open in the scenario to be used. This period established a clear separation between the initial period of preparation (i.e., checking multiple parameters, calibration, etc.) with the period of actual testing where it was necessary for the patient to concentrate and remain calm.

During the neural data acquisition phase, colored visual cues appeared, indicating “A trial is about to start” (grey cue; not shown in Fig. 1), “Walk” (green cue), or think about “Stop” (red cue) (Fig. 1). When a green cue appeared, the avatar performed a forward gait motion consisting of one step forward with a leg (alternating between left and right legs). From a total number of 40 trials in each session, 20 trials were associated with the cue to “Walk” and 20 trials with the cue “Stop”. At the beginning of this phase, the participant was instructed that, upon a green cue, he should try to imagine one of his legs rising and then stepping on the ground and in the following green trial he should imagine the same action for the other leg. Meanwhile, he was instructed that, when the visual cue was red, he should think about not moving and just stand in the same place (e.g., enjoying the scenario). After neural data acquisition, the neural network was trained, and the confusion matrix analyzed. If true positive values were above 70% for each category, training proceeded to the following phase. Otherwise, the neural data acquisition phase was repeated, and training proceeded.

In the last phase, neural decoding was performed in real-time using an adaptation of the Open Vibe motor imagery protocol [59] communicating with the remaining setup. Here, motor imagery was used only to train the participant. He did not have control over the avatar, and he did not have immediate feedback on his performance. This means that, independently of the subject’s neural activity, the avatar would move when a green visual cue (Walk) appeared, and the avatar would remain still when the red (Stop) visual cue appeared. Although the next step of the rehabilitation protocol involves the actual control of the avatar using neural activity, in the present case study we have specifically opted to run this intermediate phase without the brain control option. This was to allow evaluation of comfort levels during the experience of illusory lower limbs without the additional stress that could be generated by low performance during the real-time neural decoding feedback (i.e., the avatar not moving in a green cue trial). At the end of the session the subject was informed of the overall performance resulting from real-time neural decoding to allow feedback on the strategies that had been used in that session. Neural activity was decoded in real time, but no further offline analysis was performed.

Statistical analysis

Results are presented as mean±standard deviation. Arbitrary units (a.u.) were used as units for the different questionnaires. An alpha value of 5% was considered significant for hypothesis testing. Comparison of variables with and without the use of the thermal-tactile sleeve was made with Mann-Whitney U tests. Data related to pain reports is presented linearly and in the same panel, although the different pain scales are evaluating different aspects and do not necessarily represent a linear scale [60], pain data relative to the three different scales is presented in the same graph solely for the purpose of presenting simultaneously the three measures for each session.

Acknowledgments: BIAL Foundation 95/2016 (MPV, CPV, MA). This work was financially supported by National Funds through FCT – Fundação para a Ciência e a Tecnologia, I.P., under the projects UIDP/04501/2020 (MPV), UIDB/04279/2020 (CPV), FCT/IF/00098/2015 (MPV), project CISUC - UID/CEC/00326/2020 (AP), by the European Social Fund, through the Regional Operational Program Centro 2020 (AP), and the European Union’s Horizon 2020 research and innovation programme, via an Open Call issued and executed under Project EUROBENCH (grant agreement No 779963) Thertact-Fb (MPV, CPV, AV) and Thertact- NEXT(MPV). This work was supported by an Honorable mention in the 2nd Edition of the Maria Lutegarda Award in rehabilitation and Scientific Research 2021/22 by the AFID Foundation.

Author contributions: AP developed the overall setup; AP and PG developed the virtual reality setup; AP, DM and LPA developed the thermal-tactile sleeves; CPV, DM, TP, AVP, MJA, MG and MPV collected the data; CPV and MPV analyzed the data; all authors participated in the manuscript writing and agreed to the final version of the manuscript.

Data availability: The datasets generated and/or analysed during the current study are presented in Supplementary Information.

Competing interests statement: The authors declare no competing financial and non-financial interests.

1 - de Bruin, E.D., Frey-Rindova, P., Herzog, R.E., Dietz, V., Dambacher, M.A. and Stüssi, E.. Changes of tibia bone properties after spinal cord injury: effects of early intervention. Archives of physical medicine and rehabilitation, 80(2), pp.214-220 (1999).

2 - Ahuja, C.S., Wilson, J.R., Nori, S., Kotter, M.R., Druschel, C., Curt, A. and Fehlings, M.G. Traumatic spinal cord injury. Nature reviews Disease primers, 3(1), pp.1-21 (2017).

3 - Alizadeh, A., Dyck, S.M. and Karimi-Abdolrezaee, S. Traumatic spinal cord injury: an overview of pathophysiology, models and acute injury mechanisms. Frontiers in neurology, 10, p.282 (2019).

4 - Dijkers, M. Quality of life after spinal cord injury: a meta analysis of the effects of disablement components. Spinal cord, 35(12), pp.829-840 (1997).

5 - Donati, A.R., Shokur, S., Morya, E., Campos, D.S., Moioli, R.C., Gitti, C.M., Augusto, P.B., Tripodi, S., Pires, C.G., Pereira, G.A., Brasil, F.L. et al.,. Long-term training with a brain-machine interface-based gait protocol induces partial neurological recovery in paraplegic patients. Scientific reports, 6(1), pp.1-16 (2016).

6 – Shokur, S., Gallo, S., Moioli, R.C., Donati, A.R.C., Morya, E., Bleuler, H. and Nicolelis, M.A.. Assimilation of virtual legs and perception of floor texture by complete paraplegic patients receiving artificial tactile feedback. Scientific reports, 6(1), pp.1-14 (2016).

7 - Shokur, S., Donati, A.R., Campos, D.S., Gitti, C., Bao, G., Fischer, D., Almeida, S., Braga, V.A., Augusto, P., Petty, C. and Alho, E.J.. Training with brain-machine interfaces, visuo-tactile feedback and assisted locomotion improves sensorimotor, visceral, and psychological signs in chronic paraplegic patients. PloS one, 13(11), p.e0206464 (2018).

8 - Selfslagh, A., Shokur, S., Campos, D.S., Donati, A.R., Almeida, S., Yamauti, S.Y., Coelho, D.B., Bouri, M. and Nicolelis, M.A. Non-invasive, brain-controlled functional electrical stimulation for locomotion rehabilitation in individuals with paraplegia. Scientific reports, 9(1), pp.1-17 (2019).

9 - Benabid, A.L., Costecalde, T., Eliseyev, A., Charvet, G., Verney, A., Karakas, S., Foerster, M., Lambert, A., Morinière, B., Abroug, N. and Schaeffer, M.C.. An exoskeleton controlled by an epidural wireless brain–machine interface in a tetraplegic patient: a proof-of-concept demonstration. The Lancet Neurology, 18(12), pp.1112-1122 (2019).

10 - He, Y., Eguren, D., Azorín, J.M., Grossman, R.G., Luu, T.P. and Contreras-Vidal, J.L. Brain–machine interfaces for controlling lower-limb powered robotic systems. Journal of neural engineering, 15(2), p.021004 (2018).

11 - Lenggenhager, B., Scivoletto, G., Molinari, M. and Pazzaglia, M. Restoring tactile awareness through the rubber hand illusion in cervical spinal cord injury. Neurorehabilitation and Neural Repair, 27(8), pp.704-708 (2013).

12 - Pozeg, P., Palluel, E., Ronchi, R., Solcà, M., Al-Khodairy, A.W., Jordan, X., Kassouha, A. and Blanke, O. Virtual reality improves embodiment and neuropathic pain caused by spinal cord injury. Neurology, 89(18), pp.1894-1903 (2017).

13 - Kilteni, K., Groten, R. and Slater, M. The sense of embodiment in virtual reality. Presence: Teleoperators and Virtual Environments, 21(4), pp.373-387 (2012).

14 - Pazzaglia, M., Galli, G., Lewis, J.W., Scivoletto, G., Giannini, A.M. and Molinari, M., Embodying functionally relevant action sounds in patients with spinal cord injury. Scientific reports, 8(1), pp.1-11 (2018).

15 - Kilteni, K. and Ehrsson, H.H.. Body ownership determines the attenuation of self-generated tactile sensations. Proceedings of the National Academy of Sciences, 114(31), pp.8426-8431 (2017).

16 - Arzy, S., Thut, G., Mohr, C., Michel, C.M. and Blanke, O. Neural basis of embodiment: distinct contributions of temporoparietal junction and extrastriate body area. Journal of Neuroscience, 26(31), pp.8074-8081 (2006).

17 - Bekrater-Bodmann, R., Foell, J., Diers, M., Kamping, S., Rance, M., Kirsch, P., Trojan, J., Fuchs, X., Bach, F., Çakmak, H.K. and Maaß, H. The importance of synchrony and temporal order of visual and tactile input for illusory limb ownership experiences–an fMRI study applying virtual reality. PloS one, 9(1), p.e87013 (2014).

18 - Braun, N., Debener, S., Spychala, N., Bongartz, E., Sörös, P., Müller, H.H. and Philipsen, A. The senses of agency and ownership: a review. Frontiers in psychology, 9, p.535 (2018).

19 - Schettler, A., Raja, V. and Anderson, M.L. The embodiment of objects: Review, analysis, and future directions. Frontiers in neuroscience, 13, p.1332 (2019).

20 - Juliano, J.M., Spicer, R.P., Vourvopoulos, A., Lefebvre, S., Jann, K., Ard, T., Santarnecchi, E., Krum, D.M. and Liew, S.L.. Embodiment is related to better performance on a brain–computer interface in immersive virtual reality: A pilot study. Sensors, 20(4), p.1204 (2020).

21 - Vourvopoulos, A., Jorge, C., Abreu, R., Figueiredo, P., Fernandes, J.C. and Bermudez i Badia, S. Efficacy and brain imaging correlates of an immersive motor imagery BCI-driven VR system for upper limb motor rehabilitation: A clinical case report. Frontiers in human neuroscience, 13, p.244 (2019).

22 - Eng, C.M., Calkosz, D.M., Yang, S.Y., Williams, N.C., Thiessen, E.D. and Fisher, A.V. June. Doctoral colloquium—enhancing brain plasticity and cognition utilizing immersive technology and virtual reality contexts for gameplay. In 2020 6th International Conference of the Immersive Learning Research Network (iLRN) (pp. 395-398). IEEE (2020).

23 - Ma, S., Varley, M., Shark, L.K. and Richards, J. Overcoming the information overload problem in a multiform feedback-based virtual reality system for hand motion rehabilitation: healthy subject case study. Virtual Reality, 16(4), pp.325-334 (2012).

24 - Talukdar, U., Hazarika, S.M. and Gan, J.Q. Motor imagery and mental fatigue: inter-relationship and EEG based estimation. Journal of computational neuroscience, 46(1), pp.55-76 (2019).

25 - Brandl, S. and Blankertz, B. Motor Imagery Under Distraction—An Open Access BCI Dataset. Frontiers in Neuroscience, 14 (2020).

26 - Mori, M., MacDorman, K.F. and Kageki, N. The uncanny valley [from the field]. IEEE Robotics & Automation Magazine, 19(2), pp.98-100 (2012).

27 - Konstantatos, A.H., Angliss, M., Costello, V., Cleland, H. and Stafrace, S. Predicting the effectiveness of virtual reality relaxation on pain and anxiety when added to PCA morphine in patients having burns dressings changes. Burns, 35(4), pp.491-499 (2009).

28 - Roberts, T.T., Leonard, G.R. and Cepela, D.J. Classifications in brief: American spinal injury association (ASIA) impairment scale (2017).

29 - Peck, T.C. and Gonzalez-Franco, M. Avatar embodiment. a standardized questionnaire. Frontiers in Virtual Reality, 1, p.44 (2021).

30 - Kennedy, R.S., Lane, N.E., Berbaum, K.S. and Lilienthal, M.G. Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. The international journal of aviation psychology, 3(3), pp.203-220 (1993).

31- Li, A., Montaño, Z., Chen, V.J. and Gold, J.I. Virtual reality and pain management: current trends and future directions. Pain management, 1(2), pp.147-157 (2011).

32 - Martini, M., Pérez Marcos, D. and Sanchez-Vives, M.V. What color is my arm? Changes in skin color of an embodied virtual arm modulates pain threshold. Frontiers in human neuroscience, 7, p.438 (2013).

33 - Matamala-Gomez, M., Donegan, T., Bottiroli, S., Sandrini, G., Sanchez-Vives, M.V. and Tassorelli, C. Immersive virtual reality and virtual embodiment for pain relief. Frontiers in human neuroscience, 13, p.279 (2019).

34 - D’Alonzo, M., Mioli, A., Formica, D., Vollero, L. and Di Pino, G. Different level of virtualization of sight and touch produces the uncanny valley of avatar’s hand embodiment. Scientific reports, 9(1), pp.1-11 (2019).

35 - Jeunet, C., Vi, C., Spelmezan, D., N’Kaoua, B., Lotte, F. and Subramanian, S., September. Continuous tactile feedback for motor-imagery based brain-computer interaction in a multitasking context. In IFIP Conference on Human-Computer Interaction (pp. 488-505). Springer, Cham (2015b).

36 – Grangeon, M., Revol, P., Guillot, A., Rode, G. and Collet, C. Could motor imagery be effective in upper limb rehabilitation of individuals with spinal cord injury? A case study. Spinal cord, 50(10), pp.766-771 (2012).

37 - Regan, E.C. and Price, K.R. The frequency of occurrence and severity of side-effects of immersion virtual reality. Aviation, space, and environmental medicine (1994).

38 - Thomschewski, A., Ströhlein, A., Langthaler, P.B., Schmid, E., Potthoff, J., Höller, P., Leis, S., Trinka, E. and Höller, Y. Imagine there is no plegia. mental motor imagery difficulties in patients with traumatic spinal cord injury. Frontiers in neuroscience, 11, p.689 (2017).

39- Jeunet, C., N’Kaoua, B., Subramanian, S., Hachet, M. and Lotte, F.,. Predicting mental imagery-based BCI performance from personality, cognitive profile and neurophysiological patterns. PloS one, 10(12), p.e0143962 (2015a).

40 - Li, J., Jin, Y., Lu, S., Wu, W. and Wang, P. Building environment information and human perceptual feedback collected through a combined virtual reality (VR) and electroencephalogram (EEG) method. Energy and Buildings, 224, p.110259 (2020).

41 - Vieira, C., da Silva Pais-Vieira, C.F., Novais, J. and Perrotta, A. Serious game design and clinical improvement in physical rehabilitation: systematic review. JMIR Serious Games, 9(3), p.e20066 (2021).

42 - Kerous, B., Škola, F. and Liarokapis, F. EEG-based BCI and video games: a progress report. Virtual Reality, 22(2), pp.119-135 (2018).

43 - Lotte, F. and Jeunet, C. Defining and quantifying users’ mental imagery-based BCI skills: a first step. Journal of neural engineering, 15(4), p.046030 (2018).

44 - Škola, F. and Liarokapis, F. Embodied VR environment facilitates motor imagery brain–computer interface training. Computers & Graphics, 75, pp.59-71 (2018).

45 - Škola, F., Tinková, S. and Liarokapis, F. Progressive training for motor imagery brain-computer interfaces using gamification and virtual reality embodiment. Frontiers in human neuroscience, 13, p.329 (2019).

46 - Huang, W., Roscoe, R.D., Johnson‐Glenberg, M.C. and Craig, S.D. Motivation, engagement, and performance across multiple virtual reality sessions and levels of immersion. Journal of Computer Assisted Learning, 37(3), pp.745-758 (2021).

47 - Leeuwis, N., Paas, A. and Alimardani, M. Vividness of Visual Imagery and Personality Impact Motor-Imagery Brain Computer Interfaces. Frontiers in Human Neuroscience, 15 (2021).

48 - Villiger, M., Bohli, D., Kiper, D., Pyk, P., Spillmann, J., Meilick, B., Curt, A., Hepp-Reymond, M.C., Hotz-Boendermaker, S. and Eng, K. Virtual reality–augmented neurorehabilitation improves motor function and reduces neuropathic pain in patients with incomplete spinal cord injury. Neurorehabilitation and neural repair, 27(8), pp.675-683 (2013).

49 - Chi, B., Chau, B., Yeo, E. and Ta, P. Virtual reality for spinal cord injury-associated neuropathic pain: systematic review. Annals of physical and rehabilitation medicine, 62(1), pp.49-57 (2019).

50 - Austin, P.D. and Siddall, P.J. Virtual reality for the treatment of neuropathic pain in people with spinal cord injuries: A scoping review. The journal of spinal cord medicine, 44(1), pp.8-18 (2021).

51 - Pourmand, A., Davis, S., Marchak, A., Whiteside, T. and Sikka, N. Virtual reality as a clinical tool for pain management. Current pain and headache reports, 22(8), pp.1-6 (2018).

52 - Priebe, M.M., Sherwood, A.M., Thornby, J.I., Kharas, N.F. and Markowski, J. Clinical assessment of spasticity in spinal cord injury: a multidimensional problem. Archives of physical medicine and rehabilitation, 77(7), pp.713-716 (1996).

53 - Roosink, M., Robitaille, N., Jackson, P.L., Bouyer, L.J. and Mercier, C. Interactive virtual feedback improves gait motor imagery after spinal cord injury: an exploratory study. Restorative neurology and neuroscience, 34(2), pp.227-235 (2016).

54 - Gallagher, E.J., Liebman, M. and Bijur, P.E. Prospective validation of clinically important changes in pain severity measured on a visual analog scale. Annals of emergency medicine, 38(6), pp.633-638 (2001).

55 - Collins, S.L., Moore, R.A. and McQuay, H.J. The visual analogue pain intensity scale: what is moderate pain in millimetres? Pain, 72(1-2), pp.95-97 (1997).

56 - Bijur, P.E., Latimer, C.T. and Gallagher, E.J. Validation of a verbally administered numerical rating scale of acute pain for use in the emergency department. Academic emergency medicine, 10(4), pp.390-392 (2003).

57 - Buss, S. R. Introduction to inverse kinematics with jacobian transpose, pseudoinverse and damped least squares methods. IEEE Journal of Robotics and Automation, 17(1-19), 16 (2004).

58 - Pais-Vieira, C., Allahdad, M., Neves-Amado, J., Perrotta, A., Morya, E., Moioli, R., Shapkova, E. and Pais-Vieira, M. Method for positioning and rehabilitation training with the ExoAtlet® powered exoskeleton. MethodsX, 7, p.100849 (2020).

59 - Renard, Y., Lotte, F., Gibert, G., Congedo, M., Maby, E., Delannoy, V., Bertrand, O. and Lécuyer, A.. Openvibe: An open-source software platform to design, test, and use brain–computer interfaces in real and virtual environments. Presence, 19(1), pp.35-53 (2010).

60 - Myles, P.S., Troedel, S., Boquest, M. and Reeves, M. The pain visual analog scale: is it linear or nonlinear?. Anesthesia & Analgesia, 87(6), p.1517 (1999).

Table 1 is available in the supplementary files section.

SupplementaryInformationRawData.docx
Supplementary Information S1 - Details of each session
table1.jpg

PDF

Version 1

posted 02 Mar, 2022

You are reading this latest preprint version

Embodiment comfort levels during Motor Imagery training combined with immersive Virtual Reality in spinal cord injury: a case report

Status:

Version 1

Abstract

Figures

Introduction

Results

Discussion

Methods

Declarations

References

Table 1

Supplementary Files

Status:

Version 1