Competing at the Cybathlon Championship for Athletes With Disabilities: Long-Term Motor Imagery Brain-Computer Interface Training of a Tetraplegic Cybathlete

doi:10.21203/rs.3.rs-968612/v1

Download PDF

Research

Competing at the Cybathlon Championship for Athletes With Disabilities: Long-Term Motor Imagery Brain-Computer Interface Training of a Tetraplegic Cybathlete

https://doi.org/10.21203/rs.3.rs-968612/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background: The brain-computer interface (BCI) race at the Cybathlon championship for athletes with disabilities challenges teams (BCI researchers, developers and pilots with spinal cord injury) to control an avatar on a virtual racetrack without movement. Here we describe the training regime and results of the Ulster University BCI Team pilot who is tetraplegic and has trained to use an electroencephalography (EEG)-based BCI intermittently over 10 years, to compete in three Cybathlon events.

Methods: A multi-class, multiple binary classifier framework was used to decode three kinesthetically imagined movements (motor imagery) (left (L) and right (R) arm and feet (F)) as well as relax state (X). Three games paradigms were used for training i.e., NeuroSensi, Triad, and Cybathlon: BrainDriver. An evaluation of the pilot’s performance is presented for two Cybathlon competition training periods – spanning 20 sessions over 5 weeks prior to the 2019 competition, and 25 sessions over 5 weeks in the run up to the 2020 competition.

Results: Having participated in BCI training in 2009 and competed in Cybathlon 2016, the experienced pilot achieved high two-class accuracy on all class pairs when training began in 2019 (decoding accuracy >90%, resulting in efficient NeuroSensi and Triad game control). The BrainDriver performance (i.e., Cybathlon race completion time) improved significantly during the training period, leading up to the competition day, ranging from 274s - 156s (255±24s to 191±14s mean±std), over 17 days (10 sessions) in 2019, and from 230s - 168s (214±14s to 181±4s), over 18 days (13 sessions) in 2020. However, on both competition occasions, towards the race date, the performance deteriorated significantly.

Conclusions: The training regime and framework applied were highly effective in achieving competitive race completion times. The BCI framework did not cope with significant deviation in electroencephalography (EEG) observed in the sessions occurring shortly before and during the race day. Stress, arousal level and fatigue, associated with the competition challenge and performance pressure resulting in cognitive state changes, were likely contributing factors to the nonstationary effects that resulted in the BCI and pilot achieving suboptimal performance on race day.

Trial registration: not registered

Physical Medicine & Rehab

Developmental Neuroscience

Neurology

brain-computer interface (BCI)

motor imagery

electroencephalography (EEG)

competition

tetraplegia

long-term training

neurofeedback

neurogaming

alternative and augmentative and assistive communication (AAC) device

The Cybathlon championship for athletes with disabilities is a unique competition in which people with physical disabilities compete against each other to complete tasks and challenges using state-of-the-art technical assistance systems. Serving as a platform for technology developers to exchange ideas and collaborate closely with people with physical disabilities as they develop their devices – Cybathlon aims to drive research on assistance systems for everyday use, and promote public dialogue [1].

The majority of teams involved in the brain-computer interface (BCI) race at the Cybathlon championship, focus on training pilots to modulate brain rhythms through motor imagery with a BCI to control a virtual race vehicle (avatar) in virtual race track (video game platform). Extensive research on motor imagery paradigms has explicitly shown that maximal discrimination accuracy is achieved using lateralized differences in mu (8-12Hz) and beta band (12-30Hz) power [2], [3], [4]. These power changes are linked with event-related (de)synchronization (ERD/ERS) of neural activity in sensorimotor areas and originate from decreased or increased phase-locked synchronous activity of specific neuron populations over cortical motor areas [5]. Lateralized differences in sensorimotor rhythms (SMR) enable discrimination of the imagined movement of different limbs and muscles controlling different parts of the body [6].

Learning intentional, goal-directed modulation of sensorimotor rhythms (from imagined movement) requires practice with sensory feedback. Motor-imagery-based BCI typically requires extended training before a minimally acceptable level of control is achieved. To reach this criterion participants generally undergo multiple sessions usually lasting 1 hour/day over several days or weeks. Motor-imagery-based BCI training feedback may be presented using various feedback modalities including somatosensory (via vibrotactile/electrical stimulation), auditory, or visual – though most often using the visual channel, whilst there is increasing interest using the other modalities [7], [8]. The difference between presentation methods is considered to be small, thus allowing for the development of more naturalistic feedback paradigms [9] that do not consume the visual sense. Additionally, motivation has been shown to be important for BCI performance, especially in BCI target user groups [10], and therefore enhancement of the feedback presentation to increase motivation is often gamified through videogames that are controlled or modulated by the user, via the BCI. The Cybathlon BCI race event builds on this and challenges competitors (teams and pilots) to control, in real-time, a race avatar on a virtual racetrack using a BCI, in front of an audience. A significant challenge is thus presented, requiring the pilot to accurately and repetitively modulate brain states that can be detected by the BCI, associated with three race commands plus a no-control state (resting or relax state).

Perdikis et al. highlighted that a successful BCI application should hang on three pillars [11]: the BCI device, the interaction between the BCI and the user, and the actual application wherein the BCI is used. The BCI device is required to accurately decode imagined commands given by the user. A long-term training period provides an opportunity for both the fitting of the BCI’s hyper-parameters to the user’s voluntary brain activity, and real-time feedback that enables the brain to adapt to the required cognitive state. Thus, the training period can be used to improve performance during a multi-session learning process [12]. Ponferrada and colleagues improved the efficiency of their BCI system so that the amount of training data required to learn real-time control for Cybathlon was reduced, thereby shortening the time required for the training phase [13]. Benaroch et al. combined a progressive multi-class mental-task-based BCI with a machine learning algorithm that uses adaptive Riemannian classifiers – aimed at improving BCI control by producing electroencephalography (EEG) activity that increasingly matches the BCI classifier, rather than improving the classifier to better discriminate the EEG activity [14]. Performance accuracy improved by 30% over a three-month training period (20 sessions) – although the user’s competition day performance did not reflect this improvement. A similar drop in performance during the Cybathlon competition game, compared to performance during training, has been reported by Turi et al. in [15]. They suggested that the drop was due to the indirect impact of stress on performance – expressly the user’s psychological state influenced their ability to concentrate, which affected their performance negatively. Hence, variability in the signal between sessions is largely considered to be due to shifts in the user’s psychological state – for example, due to fatigue or loss of attention [16], [17].

The current study provides details of the BCI setup, training regime, and performance obtained during long-term BCI training of the Ulster University’s Cybathlon team (Team NeuroCONCISE) for two race competitions. The findings highlight the challenges in developing a BCI that is capable of adapting to changing user states and environment conditions that can result in temporal variations in the neural signal and create a barrier to the use of BCI systems in everyday life for individuals with a motor-disability.

2.1 Participant

The study involved a single tetraplegic participant (the pilot) with normal vision and hearing, aged 49 at the time of the Cybathlon 2020. The pilot suffered a spinal injury (factures in C4-C5) in 1993 during a motorbike accident. Prior to the commencement of the training, the pilot was presented with information regarding the experimental protocol and was asked to read and sign an informed consent form to participate in the study, which was approved by the National Rehabilitation Hospital of Ireland research ethics committee. Before the beginning of the BCI training carried out in 2019 and 2020 (reported in this paper), the pilot took part in 10 basic BCI training sessions in 2009 and 12 training sessions for Cybathlon 2016.

2.2 Experimental Paradigms

Long-term BCI training involved feedback in multiple sessions using three different BCI controlled games: NeuroSensi, Triad, and BrainDriver (Figure 1).

NeuroSensi game training for paired motor imagery tasks

The first phase of the BCI training in both years, 2019 and 2020, involved the NeuroSensi BCI game (Figure 1A) which is played using two motor imagery commands. The 2-class motor imagery paradigm was designed to train participants. The NeuroSensi game has a representation of a neuronal axon on both sides of the monitor. Two seconds after the beginning of the trial, a light (representing a neural spike) appears at the far end of one of the two axons to cue the participant to begin the corresponding motor imagery task. The light takes 6 seconds to travel over the ‘axon’ during the task period (Figure 1A). In each NeuroSensi session, six runs were performed wherein different binary combinations of the three commands (left hand (L), feet (F), right hand (R)), and relax (X) were performed. The number of trials in each run acquired during BCI calibration varied between 30-60 (equal number/class) depending on the actual session ID (more trials in the initial session, fewer trials in later sessions). The time duration of a run, therefore, varied between 240s and 480s. The time duration of six runs involving L vs. R (LR), FR, LF, LX, FX, and XR tasks, including five 90s inter-run pauses, varied between 20 and 30 minutes (Figure 1B). After all pairs of runs (i.e., LR, FR, LF, LX, FX, and XR) runs was re-structured and trials involving the same class recorded from different runs were pooled (e.g., for “L” class “L” trials were pooled from LR, LF, and LX runs). Thus, in the re-structured dataset four different classes (L, F, R, and X) were separated. The number of trials per class for a single session varied between 45 and 90.

The three ‘task vs. task’ classifiers (LR, FR, LF) were calibrated using the corresponding trials stored in the re-structured dataset. However, in runs when the NeuroSensi game was controlled with a ‘task vs. relax’ (TX) task, i.e., in runs where the character was controlled with LX, FX, or XR task pairs, the same TX decoder was used. The TX decoder was calibrated using T vs. X trials from the re-structured dataset where T trials were pooled as a union of L, F, and R trials. To improve the cross-session stability of the calibrated BCI, the final dataset for BCI calibration was prepared by pooling re-structured datasets from multiple sessions acquired prior to the calibration.

Triad game for monitoring details of the multi-class classification

The Triad game (Figure 1D) provides real-time continuous visual feedback from each of the four 2-class classifiers. The analogue output of the three ‘task vs. task’ classifiers (LR, FR, LF) are presented using a light blue ball on the three edges of a triangle. Furthermore, the linear combination of the LR, FR, and LF classifier output is presented with an additional coloured ball indicating the composite output of these three ‘task vs. task’ classifiers. The colour of the composite output indicator ball is assigned via the analogue output of the ‘task vs. relax’ (TX) classifier. The colour of the ball indicates the command is decoded as the task (green) or relaxed (dark blue) condition.

BrainDriver game to familiarize the pilot for the race in the Cybathlon BCI event

After the pilot learned to control the BCI using the NeuroSensi and Triad games (2020 only), the BrainDriver BCI racing game was used to practice the control of the avatar (a virtual race vehicle) (Figure 1C). The actual track of the BrainDriver game comprised four different zones. There are zones with left and right curves and straight zones with streetlights turned on or off. To maintain a maximal speed of the vehicle, the pilot must produce the correct race command using the 4-class BCI (e.g., left or right arm motor imagery for left or rights turns, feet imagery “headlight” and relax for “no-control”). If an incorrect command is presented the vehicle is inhibited which presents obvious negative visual feedback to the pilot to enable learning and attempt correction strategies. The pilot was instructed to relax immediately after issuing a command to allow for ‘no control’, or as an alternative strategy to continue to maintain the motor imagery command. Section 2.5 describes how the controller limits commands and assists in dealing with variation in control performance by the BCI and pilot.

2.3 Data Acquisition

The EEG was recorded from 32 EEG channels using a g.Nautilus Research active electrode wireless EEG system [18]. The EEG reference electrode was positioned on the left earlobe. The EEG was high-pass filtered (>0.1Hz), notch filtered (48-52Hz), and sampled (A/D resolution: 24Bits, sampling rate: 250Hz). The ground electrode was positioned over the AFz electrode location according to the international 10/20 EEG standard (Figure 2). The communication between the real-time BCI decoder module run in Simulink [19] (used for EEG data acquisition and online signal processing) and each of the three (NeuroSensi, BrainDriver, and Triad) games was via a ‘user datagram protocol’ (UDP).

2.4 Calibration of the two-class classification modules

The BCI framework included a filter-bank common spatial patterns (FBCSP) [20] and mutual information (MI) based feature selection [21], a well-established framework used in BCI applications that enable discrimination between imagined movements [22] performed with the left hand (L), feet (F), right hand (R), and relax (X) conditions [23]. The FBCSP-MI module, the core of the online BCI framework (Figure 2A), was calibrated offline as described below.

EEG signal processing

The pre-processed EEG dataset was downsampled from 250Hz to 125Hz and band-pass filtered in four non-overlapped standard EEG bands (8-12Hz (mu), 12-18Hz (low beta), 18-28Hz (high beta), and 28-40Hz (low gamma)) using high-pass and low-pass finite impulse response (FIR) filter modules (band-pass attenuation 0 dB, band-stop attenuation 60 dB). Trial-relevant time intervals between -2s (before) and 8s (after) the onset of the 2s pause (i.e., -4s (before) and 6s (after) the onset of task) were epoched out from the frequency filtered EEG dataset for 21 pre-selected EEG channels, and stored for spatial filtering. The epoched data using a 1s to 2s width classification window enabled comparison of the decoding accuracy (DA) obtained in the 0 to 2s reference baseline interval (covering the pause period) and during the 2s to 6s task interval (after the pause period).

Spatial filtering

The common spatial patterns (CSP) method was used to create spatial filters that maximize the discriminability of two classes by learning spatial filters which maximize the variance of band-pass filtered EEG signals from one class, while minimizing their variance from the other classes [24]. The linear transformation matrix defined by CSP converts the pre-processed EEG signals into a new vector space defined by the CSP filters.

Feature extraction

The number of selected CSP filter pairs for each 2-class classifier for each frequency band was set to three. The time-varying log-variance of the CSP filtered EEG was calculated using a 1s width sliding window, with a 40ms time lag between two windows. Thus, the offset (end-point) of the 1s sliding window was set to cover the time interval between -1s (before) and 8s (after) the onset of the pause (covering a 1s sliding width window, the 2s pause, and 6s task intervals).

Feature selection

The mutual information (MI) between features and associated target class using a quantized feature space was estimated [21] to identify a subset of features that maximize classification accuracy.

Two-class classification

A regularized linear discriminant analysis (RLDA) algorithm from the RCSP toolbox [24] was applied to classify the extracted features. Linear discriminant analysis (LDA) uses a class separator boundary in a linear hyperplane to separate data into two classes. The time-varying analogue output of the classifier, i.e., the time-varying signed distance (TSD), is the time-varying distance between the location of the classifier output and the class separation boundary in the LDA hyperplane. The class assigned to each feature vector depends on the polarity of the classifier output, determined by the relation between the location of the feature vector and the class separator boundary in the hyperplane [25]. The current TSD value is calculated using an LDA classifier as described in (1)

where, \({x}_{F,n,t}\) is the features vector at time t in the \(n\)^th trial, while \({w}^{F}\) and \({a}_{0}\) are slope of the features and bias of the discriminant hyperplane. Thus, \({TSD}_{n,t}\) (the time-varying signed distance, obtained at the output of the classifier) is composed of a \({w}^{F}{x}_{F,n,t}\) part (i.e., the feature vector dependent LDA components) and an \({a}_{0}\) constant bias.

The time-varying DA was calculated and compared for each of the four 2-class classifiers using the best 6, 10, 14, and 18 MI ranked features, using six-fold cross-validation. The features that provided a classifier configuration with the highest DA peak in event-related period of the task (in a 2.4 to 8s interval of the trial, covering a 0.4 to 6s interval from the onset of the task) was applied to the online BCI configuration, in the case of each 2-class classifier, separately.

Topographical analysis

To identify frequency bands and cortical areas that provide the highest contribution to the peak DA, an analysis was performed using parameters of the calibrated CSP filters and the MI weights for each of the four 2-class classifiers, separately. For the time-varying frequency analysis, the mean values of MI weights were calculated in each analyzed frequency band, and time point, separately. The obtained results were plotted in the form of subject-specific heat maps, indicating the time-varying DA contribution of the frequency bands analyzed. The location of the source activity was plotted using the ‘standardized low resolution brain electromagnetic tomography’ (sLoreta) software package [26] for each 2-class classifier in each frequency band, separately, indicating cortical areas where features provided the highest contribution for calculating maximal DA.

Combining trials for different runs and sessions

The objective was to find an online BCI configuration that provides the highest DA with a high level of stability over sessions. Thus, the BCI was calibrated using different datasets that were pooled from different combinations of existing sessions. A cross-session DA analysis was performed for each BCI configuration wherein the time-varying DA plots were compared using datasets excluded from calibration data. The BCI configuration was selected for the oncoming sessions based on a manual comparison of the cross-session time-varying DA plots, frequency maps, and topographical maps obtained based on the analyzed BCI configurations using the objectives described above (i.e., long term stability paired with a maximal level of DA).

2.5 The online BCI

The core module of the online BCI involved the same FBCSP-MI-based 2-class classification framework (Figure 2A) for the NeuroSensi, Triad, and BrainDriver games which are described in Section 2.4. However, the post-processing module for these three games was different for the three BCI game applications.

NeuroSensi game

The NeuroSensi game uses only one of the four binary classifiers for controlling the character (i.e., LR, FR, LF, or TX). The baseline of the corresponding TSD signal was calibrated manually set to zero at the beginning of each run using an offset value. The amplitude of the TSD signal was corrected to a value that enabled the controlled character to move over the controllable area during the game using a scaling factor. The corresponding TSD after the baseline correction was downsampled to 25Hz and sent by UDP to the NeuroSensi game.

Triad game

As the Triad game was controlled by the TSD output of each of the four 2-class classifiers, the baseline of each of the four TSD signals was corrected, separately, as described above for the NeuroSensi game. The TSD value after baseline correction was downsampled to 25Hz and sent by UDP to the Triad game.

BrainDriver game

The online BCI framework (Figure 2), in addition to the FBCSP-MI and TSD baseline correction modules (discussed above and presented in Figure 2A), involves a control command decoder module composed of a multi-class decoder, a stability delay timer, a dead-band (DB) control module (Figure 2C) and game control command translator module (Figure 2D) followed by a UDP unit for sending commands to the BrainDriver game (Figure 2E).

The output of the multi-class decoder relies on the baseline-corrected outputs of the four binary (LR, FR, LF, and TX) classifiers. If the output polarity of two of the three ‘task vs. task’ classifiers (LR, FR, LF) are not conflicting and the TX classifier is indicting a task condition (“T”) (i.e., the pilot is not relaxed), the label of the decoded task is forwarded to the next module for a stability check. For example, in Figure 2C both LR and LF classifiers output indicate the same (“L”) result and the TX classifier indicates that there is an ongoing task (“T”). Therefore, in this example, the decoded (“L”) command passes through on Command Control Gate 1.

To filter out transient responses, the decoded (“L”) command passes through Command Control Gate 2 only if the decoded (“L”) command is maintained in the same condition for a predefined (300ms) period. If this stability check is matched, the decoded (“L”) command is translated with the game control command translator module to the game control command as shown in Figure 2D. Finally, the game control command is sent by UDP to the BrainDriver game. An example of a track section and corresponding control commands are illustrated in Figure 2E (details of the BrainDriver game in Section 2.2).

To provide the opportunity for the pilot to reach a relaxed condition before the next command is decoded and to ensure sudden changes in classifier conditions don’t interrupt a correct command issued to the vehicle, a dead-band system is used. Once a command is sent to BrainDriver game, the dead-band timer module is activated, resulting in blocking new commands on the UDP for a predefined dead-band period (e.g., in Figure 2C the dead-band period is 6s). To enable the pilot to correct an incorrect command, a dead-band-break (DBB) unit is also employed. If the pilot maintained a non-relaxed condition after a dead-band-break time is counted from the previous command (in Figure 2C the dead-band-break is 3s) the dead-band-break unit overrides the dead-band and allows the new command to be sent. To find an optimal configuration that supports the pilot’s control ability maximally, the actual value of the dead-band and dead-band-break parameters were adjusted manually over sessions and runs during the training period. The length of the dead-band was selected in a range between 20s and 8s, and the dead-band-break was selected in a range between 0s and 4s.

Using datasets acquired in 2019 and 2020, an offline analysis was performed to determine which modifications of the current BCI framework and calibration methods improved the pilot’s BCI control accuracy.

3.1 Calibration details

To improve the cross-session stability of the online BCI, the BCI calibration was performed using a dataset acquired from multiple sessions prior to the calibration date. To find an optimal combination of the sessions from which the acquired data was added to the BCI calibration dataset, the BCI was calibrated and tested multiple times using data acquired from a combination of different sessions. A dataset involving two to four sessions providing the highest cross-session DA over sessions near to the calibration date were noted and used for the BCI re-calibration. Sessions used in the cross-session analysis, along with the sessions used for BCI calibration, and the sessions in which the re-calibrated BCI was used are presented in Table 1.

Table 1

Sessions used for BCI calibration in 2019 and 2020
Sessions from which the sessions used in calibration were selected	Sessions used in BCI calibration	Sessions in which the calibrated BCI was used
N/A	N/A	2019 Sessions 1-3 (offline)
2019 Sessions 1-3	2019 Sessions 1-2	2019 Session 4
2019 Sessions 1-4	2019 Sessions 1-4	2019 Session 5
2019 Sessions 1-5	2019 Sessions 2, 4, 5	2019 Sessions 6-20
2019 Sessions 1-5	2019 Sessions 2, 4, 5	2020 Sessions 1-4
2020 Sessions 1-4	2020 Sessions 3, 4	2020 Sessions 5-9
2020 Sessions 1-9	2020 Sessions 3, 4, 5, 6	2020 Sessions 10-22
2020 Sessions 1-22	2020 Sessions 5, 6, 20, 21	2020 Sessions 23-25

3.2 Time-varying decoding accuracy and TSD analysis for 2-class BCI

To evaluate how to change the class-specific output of the 2-class classifiers over a trial, the TSD was calculated for each class and plotted along with the time-varying DA for each 2-class classifier (LR, FR, LF, TX), separately, using the dataset acquired in 2019 and 2020. The time-varying DA and TSD outputs of the four binary classifiers are presented in Figure 3 as described below.

Figure 3A and Figure 3C provide a comparison of time-varying DA obtained in cross-validation performed during calibration of the FBCSP framework applied to the BCI in 2019 and 2020, respectively. A comparison of the time-varying DA obtained for different binary classifiers (LR, FR, LF, and TX) during the online sessions using NeuroSensi is presented in Figure 3B and Figure 3D, respectively. Furthermore, a comparison of the time-varying DA obtained from the same binary classifier in 2019 vs. 2020 years is presented in Figure 3E. The time-varying DA graphs show that DA during the 0-2s pause period is approximately at the chance level (50±10% (mean ± STD)). After the onset of the motor imagery task (the dotted vertical line at 2s), the online DA reached 90±10% (mean ± STD) (Figure 3E) and was maintained by most classifiers between a period of 4s to 8s (i.e., from 2 s after the onset of the task until and of the task period). A comparison of the time-varying TSD values obtained for the two classes is presented in Figure 3F and Figure 3G using the dataset recorded in online sessions in 2019 and 2020, respectively. The graphs show that during the pause period (i.e., before the onset of the task) the TSD varied in the same range for both classes around the zero baselines and the TSD separability for the two classes is a maximal around 4s (i.e., 2s after the onset of the task).

3.3 Frequency analysis and topographical results

A comparison of frequency bands and cortical areas providing the highest DA contribution in the four binary classifiers (LR, FR, LF, TX) involved in the BCI configuration applied to the Cybathlon challenge in 2019 and 2020 is presented in Figure 4.

The results of the frequency analysis using CSP and MI weights of the BCI that was applied 2019 indicate an increased level of motor imagery task-related brain activity for each of the three ‘task vs. task’ classifiers (LR, FR, LF) in the 18-28 Hz (high beta) band (Figure 4B). The highest contribution for the ‘task vs. relax’ (TX) classification, similar to the LF, FR, and LF classification, was obtained in the high beta band. However, regarding the separation of the task and relaxation conditions, in addition to the 18-28Hz (high beta), the 12-18Hz (low beta) and 28-40Hz (low gamma) bands also contribute to high accuracy.

The topographical analysis of the BCI calibrated in 2019 (Figure 4A) indicated the highest contribution to the four binary classifications as follows: (1) LR classification, from the high beta oscillations in the right hemisphere of the somatosensory cortex; (2-3) LF and FR classifications, from the high beta oscillations in the central area of the primary motor and somatosensory cortex; (4) TX classification, from the high beta oscillations in the left hemisphere of the prefrontal cortex, coupled with the low gamma oscillations in the left hemisphere of the occipito-temporal cortex. Furthermore, for the TX classification, the low beta oscillations also provided a reasonably high contribution in the right hemisphere of the visual sensation associated area in the parietal occipital cortex.

In 2020, the results of the frequency (Figure 4D) and topographical (Figure 4C) analyses indicated significant differences in the frequency and topographical maps compared to those obtained in 2019. In 2020, the highest contribution to the four binary classification is obtained in cortical areas as follows: (1) LR classification, from the low gamma oscillations in the left hemisphere of the somatosensory association and occipito-temporal cortex; (2) FR classification, from the high beta oscillations in the central area of the primary motor and somatosensory cortex, coupled with the low gamma oscillations in the left hemisphere of the somatosensory association and occipito-temporal cortex; (3) LF classification, from the high beta oscillations in the central area of the primary motor and somatosensory cortex, coupled with the low gamma oscillations in the right hemisphere of the somatosensory association and occipito-temporal cortex; (4) TX classification, from the high beta oscillations in the left hemisphere of the somatosensory association cortex, the right hemisphere of the premotor, and primary motor cortex, coupled with the high beta oscillations in the left hemisphere of the prefrontal cortex.

3.4 BrainDriver scores, baseline correction, and dead-band configuration

Each year, after practice sessions using the NeuroSensi game, our pilot’s training focused on improving his BCI control ability using the BrainDriver game. The game completion time achieved by our pilot in 2019 and 2020 is presented in Figure 5.

In both years, during the training period, the race completion time decreased over sessions. However, around four days before the competition, each year, the required time for finishing the BrainDriver race increased (highlighted with the dotted oval in Figure 5A and Figure 5B). Furthermore, on the day(s) of the competition, the required time to finish the BrainDriver race increased, resulting in substandard performance in the final competition race compared to earlier training performance (e.g., up to session 16 in 2019 and up to session 21 in 2020).

The baseline shift on the TSD of the 2-class classifiers (Figure 5C and Figure 5D) is correlated with the increase in race times (Figure 5A and Figure 5B) as highlighted with a dotted and solid oval.

To find a BCI configuration that supports maximally the pilot’s control ability, the values of the dead-band and dead-band-brake parameters were set over sessions and runs in a range of possible options. The applied values of the dead-band and dead-band-brake parameters are presented in Figure 5E and Figure 5F. Based on manual observation of the game completion times and feedback information reported by the pilot about which configuration that best supported control ability, the BCI was configured for the final challenge in both years, 2019 and 2020, using a 6s dead-band and a 3 s dead-band-break parameter value.

3.5 BrainDriver race performance and EEG power spectral density (PSD)

An analysis was performed to evaluate a possible connection between the change in BrainDriver race times and power spectral density (PSD) of EEG over sessions and runs during BrainDriver game control. The logarithmic magnitude of PSD values was obtained in a frequency (12-40 Hz) range applied to the BCI in 2 Hz steps over each BrainDriver race, separately. As the EEG dataset on race day in 2020 was not archived, this analysis was performed only for the dataset acquired in 2019.

Figure 6 shows the logarithmic PSD magnitude calculated and compared using runs from session blocks 15-16, 17-18, and 19-20 at C3, CZ, C4 electrodes (electrodes which provided contributed prominently to classification). The PSD was significantly different (p < 0.05) for runs in sessions 15-16 compared to the PSD obtained for runs in sessions 17-18. Also, a significant difference (p < 0.05) in PSD was obtained by analyzing runs between sessions 17-18 and 19-20. However, the PSD was similar (p > 0.05) for runs between sessions 15-16 and 19-20.

3.6 BrainDriver race times and frequency bands providing the highest DA contribution

Some sessions before the competition races, a decrease occurred in the pilot’s BrainDriver game control ability (Figure 5A and Figure 5B). Based on the NeuroSensi dataset acquired in 2020, an analysis was performed to investigate a possible connection between the decreases in the BCI control ability and the change in frequency bands providing the highest contribution to 2-class classification, comparing results from sessions 19-21 and sessions 22-24.

The results of the analysis (Figure 7) for each ‘task vs. task’ classifiers (LR, FR, LF) shows that the task-related DA contribution of the 18-28Hz (high beta) and 28-40Hz (low gamma) bands show a relatively high value (red-coloured area) for sessions 19-21 (Figure 7A) closer to the onset of the task (dotted vertical line) compared to results obtained for sessions 22-24 (Figure 7B). As indicated by the light colour areas in Figure 7C, the highest difference in DA contribution for ‘task vs. task’ 2-class classification between the two session groups was obtained during the task period around 4-5s (i.e., 2-3s after the onset of the motor imagery task).

In the case of the ‘task vs. relax’ (TX) classification, the DA contribution map obtained from the two-session groups showed more diverse patterns, especially in the 8-12Hz (mu) band. However, areas with the highest contribution for sessions 19-21 and 22-24 are consistent.

3.7 The reliability of the features in terms of the ability to separate classes

We investigated the feature separability encoded by the four binary classifiers using the dataset from 2020 when the pilot trained with the NeuroSensi game.

For this analysis, the input features of the LDA filter were pooled for each class of each 2-class classifier, separately, from all trials recorded in the same session. The pooled LDA input features were averaged and multiplied with the corresponding weight of the calibrated LDA (referred below as LDA components). The comparison of the LDA components provides an opportunity to identify features providing the highest contribution to the output of the LDA filter (i.e., to the TSD). To illustrate that how the LDA components vary over sessions, the LDA components were plotted in form of session-varying LDA component maps, separate, for each class. To analyze which features provided a maximal class-specific contribution for the calibrated LDA filter, the difference between LDA components obtained for the two classes was also analyzed. To facilitate the evaluation of the change in LDA components over sessions, the session-varying LDA component maps were baseline corrected by subtracting the value of each LDA component calculated for the first session used in this analysis (session 10) from LDA components in later sessions.

The session varying values of the LDA components for the four 2-class classifiers are presented for class1 and class 2 in Figure 8A and Figure 8B, respectively.

Figure 8C shows the difference of the LDA components obtained for class1 and class2. Figure 8C indicates clearly that the largest difference in LDA components calculated for the two classes is obtained (for each 2-class classifier) at the highest mutual information ranked feature (indicated with rank1 at the vertical axes in Figure 8C). The figure also demonstrates that some features (e.g., for LR classifier: features 4, 5, 8) provide more negative than the positive contribution to the classification.

The change in each LDA component using a baseline corrected LDA components (i.e., session 10) is presented for class 1 and class 2 in Figure 8D and Figure 8E, respectively. Figure 8D and Figure 8E indicate, that for the three ‘task vs. task’ (LR, FR, LF) classifiers, the LDA components (e.g., for LR classifier: features 1, 2, 7, 8) are significantly (p < 0.05) different in sessions 19-21 compared to earlier sessions. The session numbers where the change occurred match the sessions when the pilot’s BrainDriver control ability was at its best (see BrainDriver race completion times 2020 in Figure 5B between sessions 19 and 21). This change in the level of the features over sessions in the case of the TX classification is lower compared to that obtained for the ‘task vs. task’ classifiers.

This paper provides an overview of a long-term pilot training and BCI strategy implemented in preparation for the Cybathlon BCI race event in 2019 and 2020. In both years, the initial phase of the project focused on calibrating the 2-class classifiers using the NeuroSensi game. Our results show that decoding accuracy increased during the online sessions following the calibration period. This observation is in line with similar studies that have demonstrated improvements in the Cybathlon pilot’s BCI control ability due to long-term practice periods – although this has not always translated into high performance on the day of the event, as was the case with the current study and for both [14] and [15]. Possible reasons for this drop in race-day performance, and suggested strategies to address the issue are discussed.

The TSD results from the current study indicate that although the BCI was re-calibrated between 2019 and 2020, the timing of the pilot’s accuracy control strategy was consistent between 2019 and 2020. The TSD output of the ‘left vs. right’ (LR) classifier between 0s and 3s (reference baseline-related interval) in 2020, show less fluctuation in positive or negative directions compared to that obtained in 2019. The more balanced TSD track deviation, prior to the effect of the class-specific motor imagery task in 2020, may be because the pilot was asked to maintain a relaxed mental state whilst keeping the controlled character in the center position (i.e., at the zero baseline) during the pause period of the ‘task vs. task’ runs in 2020, but not in 2019. Furthermore, the track of the TSD values for the ‘task vs. relax’ (TX) classifier within a 2s width interval around the onset of the task (i.e., between 1s and 3s in the trial) shows that the controlled character moves in the negative direction. The above-described effect is more clearly observable in the 2020 results (Figure 3G, ‘task vs. relax’) compared to the results from 2019 (Figure 3F, ‘task vs. relax’), indicating that the pilot followed the instructions and tried to relax during pause periods of the TX runs, especially in 2020.

The comparison of the frequency and topographic analysis results obtained in 2019 and 2020 using CSP and MI parameters of the calibrated BCI, indicate a similar pattern each year (Figure 4). However, the frequency and topographical patterns obtained based on the 2020 BCI configuration involve some specific features that are not evident in the 2019 BCI. The performance in 2019, for each binary classifier, relied on a single area in the frequency and topographical maps (indicated in A and B panels of Figure 4). However, in 2020, for most binary classifiers, two separate areas provide similarly high contributions for the 2-class classification (Figure 4C and Figure 4D). For example, the ‘feet vs. right’ (FR) classification in 2019 relied mostly on 18-28Hz (high beta) oscillations in the central area of the primary motor and somatosensory cortex – a cortical area commonly activated when able-bodied participants perform the task associated with a kinaesthetically imagined feet movement. This observation is in line with the findings reported by Müller-Putz et al. [27], which reveals a post-movement beta rebound within a mean range of 17.3-29.7 Hz. For the FR classification in 2020, in addition to the high beta activity in the central area of the primary motor and somatosensory cortex, a similarly high contribution was obtained from low gamma oscillations in the left hemisphere of the somatosensory association and occipito-temporal cortex. Thus, the above discussed CSP-MI patterns obtained in 2020 indicate task-specific cortical activity for both compared tasks (‘feet vs. right hand’), as opposed to highlighting only one task (i.e., ‘feet’), as was the case in 2019. The appearance of more detailed task-specific CSP-MI patterns in 2020 indicates a possible improvement in the pilot’s 2-class classification control strategy compared to the BCI control strategy in 2019.

Each year, after the early sessions, which served to help the pilot achieve control confidence in the 2-class paradigm applied to the NeuroSensi game, the focus of the sessions turned to the BrainDriver game. The game completion time of the BrainDriver during the training period improved significantly. The game completion time in 2019 ranged from 274s to 156s over 17 days including 10 sessions wherein the mean±std statistic of game completion times in the first BrainDriver session (session 9, Figure 5A) was 255±24s and the pilot reached 191±14s in the 10th BrainDriver sessions (sessions 16, Figure 5A). The game completion time in 2020 ranged from 230s to 168s over 18 days including 13 sessions wherein the mean±std statistic of game completion times in the first BrainDriver session (session 9, Figure 5B) was 214±14s and the pilot reached 181±4s in the 13nd BrainDriver session (session 21, Figure 5B). However, on both competition occasions, towards the race date, the pilot’s performance for controlling the BrainDriver game is decreased significantly. For reference the winning race times were 183s and 172s in 2019 and 2020, respectively. The results confirm not only that the multi-session online training using the BrainDriver game had a positive impact on the pilot’s performance, manifested in an increased BCI control ability that enabled the pilot to achieve competitive race completion times, but also highlights that experiences in 2019 were transferred to 2020 resulting in race completion times achieved after one year were in the range of the best race completion times achieved at the end of the training performed in 2019.

In terms of other competitors, the NITRO 1 team, Benaroch et al. reported that the game completion time of their pilot fluctuated between 250 and 340s during seven training sessions before the Cybathlon BCI series (2019) event [14]. However, their pilot could not finish the track within the 4 minutes limit. For the NITRO 2 team, Turi et al. reported in the final competition of the Cybathlon BCI series (2019) their pilot completed a 390.5m disrtance in the 500m long vitual track within the 240s limit [15] but did not note details of the game completion time achieved by their pilot during the training period. For the Mirage 91 team, Hehenberger et al. reported that their pilot’s performance showed a constant improvement over 14 months of training including 26 game-based sessions for the Cybathlon BCI series (2019) and Cybathlon Global Edition (2020) [28]. The BrainDriver game completion time improved from 255±23s to 225±22s (mean ± STD). For the SEC FHT team, Robinson et al. also reported improving their pilot’s performance over a nine-month training period involving 15 sessions, the BrainDriver game competition time varied between 310s and 214s [29]. The best three ranked teams completed the track in the final challenge of the Cybathlon BCI series (2019) in the following order. Rank 1st : WHI team (500m within 183s), Rank 2nd : Mirage 91 team (500m within 229s), Rank 3rd : NeuroCONCISE team (386m within the 240s limit) [30], [15]. The best three ranked teams in the Cybathlon Global Edition (2020) completed the track in the following order. Rank 1st : WHI team (500m within 172s), Rank 2nd : MAHIDOL BCILAB BCI team (500m within 176s), Rank 3rd : Neurorobotics team (500m within the 213s). Our team, the NeuroCONCISE team), completed the Cybathlon Global Edition (2020) in Rank 6th (452m within the 240s limit) [31]. For completion, we applied similar but less developed strategy in Cybathlon 2016 with the same pilot. Cybathlon 2016 is not comparable due differences in race track and total race time however our pilot achieved the 3rd best time of all competitors in the competition but had a poor qualifying lap which meant it was 6th place overall [32].

Factors that impact the performance

Each year, up to a few days before the competition, the race completion time achieved by the pilot from session to session decreased, indicating an improvement in BCI control ability and/or a more refined parameters selection for the BCI. However, a few days before the competition, each year, a significant increase was detected in the race completion time (dotted oval area in Figure 5A and Figure 5B). Furthermore, on the day of the competition, the race completion time increased once again (solid oval area in Figure 5A and Figure 5B). During the last six sessions before the competition in 2019 and 2020, when this negative effect was detected, the BCI configuration had not changed, nor was there a change in the game control strategy reported by the pilot. There may be a number of factors associated with this change in performance, including increased arousal and stress levels, fatigue, and/or changes in living and dietary patterns (e.g., the pilot was living away from home for extended periods during the lead up to race day). Benaroch et al. demonstrated that their Cybathlon pilot’s strategy of adapting their brain patterns to match the training data distribution, helped to improve BCI control [14]. However, they also report that they were not able to translate this to event day performances, which is consistent with our findings. Additionally, Turi et al. reported a similar outcome for competition day results, citing multivariate factors that influence and potentially disrupt pilot performance, including training time, change in routine, differences in the training method with the final game, and differences in training and event environment [15]. A longitudinal study involving another Cybathlon tetraplegic pilot [29] investigated factors affecting the long-term use of their system by analyzing several performance indicators including activations maps, completion time, classification, and the personal experience of the pilot, by measuring their subjective experience of both their physical and mental readiness on a scale of 1 to 5. The findings support the use of a closed-loop calibration system with real-time feedback, due to better online median classification performance, compared to open-loop calibration paradigms, and improved pilot engagement. Although only a single subject study, the team recommends striving to keep the training paradigm closely matched with the final event by including closed-loop real-time feedback, which helps boost classification performance whilst increasing brain activations due to the increased engagement felt by the pilot. Promisingly, not all entrants reported a decrease in performance on the event day. Hehenberger et al. described a correlation between external influencing factors and performance on their final race [28]. As was the case for the current study, the authors describe the pilot’s increased performance over time, but did not evidence a decrease in performance during the final races. Moreover, the pilot achieved a personal best in his performance at the 2019 Cybathlon, leading the authors to speculate that their pilot performs better in front of an audience.

In an attempt to understand how to mitigate all negative influences on pilot performance, it has been argued that the peak performance of Paralympians is driven not only by psychological factors, but through their convergence with a sufficient support network, lifestyle, and attuned methods of performance [33]. Hence, recent research has focused on counteracting the effects of these variables. One approach has been to use neurofeedback to improve BCI task performance through training self-regulation of arousal states via attention mechanisms [34]. Cognitive control underlies executive function within the brain and interacts critically with arousal systems to activate approach-avoidance behavior [34], [35]. This interaction is the principal behind the Yerkes-Dodson law – a psychological concept that posits an optimal (moderate) level of arousal is necessary for improved task performance – while a low level of arousal reduces motivation and a high level of arousal negatively impacts cognitive information processing, thus impairing task performance. Therefore, this relationship between the state of arousal and performance on a task is best described as an inverted-U, known as the Yerkes-Dodson curve[34], [36]. Arousal levels have been found to influence sensory-motor cognition – spontaneous high-frequency oscillations known as "pilot-induced oscillation" (PIO) are generated when performing a high-consequence task. These unstable oscillations have deleterious effects on performance when amplified by the pilot's over-correction of small errors in control [34], [37]. Faller and colleagues (2019) used audio feedback in a closed-loop neurofeedback BCI, comprising a synthetic slow heartbeat (60bpm), which became louder with increased arousal, to train users to self-regulate their arousal levels while performing a virtual reality (VR)-based boundary avoidance task (BAT). Task performance improved significantly for the users who received veridical feedback compared to those in the sham or no-feedback control conditions. This result was corroborated by heartrate variability (HRV) data and measures of pupil dilation, which indicated a learned ability to shift arousal state and increase task performance through neurofeedback training.

However, it remains uncertain whether the advantage afforded by feedback would be maintained in the absence of that feedback during the competition itself. Mindfulness training could provide a more sustainable approach to the self-regulation of arousal, via underlying cognitive mechanisms. Mindfulness training has been shown not only to have a positive impact on stress reduction [38] but is also gaining momentum in the BCI research community with some studies showing an improvement in performance for those using mindfulness over a control group of almost 20% [39]. Mindfulness is a metacognitive process as it requires self-regulation of attention to control cognitive processes – while simultaneously monitoring conscious experience [40]. Individuals who are experienced in meditation skills have been found to demonstrate higher resting SMR power, a more stable resting mu rhythm, and greater BCI control, compared to those who are not practiced in meditation techniques [41]. Furthermore, mindfulness-based stress reduction (MBSR) training has been found to improve BCI learning and performance on BCI tasks that communicate the user’s intent via motor imagery commands and volitional rest [42]. The mechanism for this improvement is brought about by the user's ability to volitionally increase alpha-band neural activity as a consequence of the MBSR training. Stieger et al. evidenced an increase in alpha activity recorded during the user's volitional resting-state, across MBSR sessions, which was correlated with mindfulness practice and predicted BCI performance [42]. Strategies to reduce challenges and stress when preparing for the competition include anticipation and preparation through detailed planning, including contingency planning, and expectation management, i.e., focusing on the process rather than the outcome [43]. Training in MBSR would fit with these approaches to stress management, and enhance BCI task performance under the pressure of competition.

Factors that impact SMR-BCI performance that are less dependent on the BCI user and more dependent on external elements include distractors, time spent training, various types of feedback, and features of the EEG system and the data preprocessing algorithms [36]. For example, González-Franco et al., whose research studied the influences of positive and negative visual feedback on motor imagery task performance using EEG and electrocardiography (ECG) found that over-biased negative feedback caused mental stress that is detected in the form of significantly higher heart rate variability (HRV), compared to sessions where over-biased positive feedback was presented, and accuracies correlated with the polarity (-/+) of the biased feedback [44]. Adaptive feedback, or BCI setup, that limits the negative feedback may be an alternative strategy where the onus is on the BCI to deal with the changes in the pilot’s affective state. Performance may be improved by replacing the BCI framework proposed here with an adaptive BCI method that could update the BCI in real-time and adapt to the pilot’s actual mental state [45]. In the lead up to the competition, the average TSD was offset from zero (indicating classifier bias to one of the classes). Even manual correction (offsetting) before each session was not enough to counteract the drift. This baseline drift is a well-known issue in BCI and is associated with changes in the distribution of the features – covariate shift [46], [47]. As shown in Figure 6 - Figure 8 we observed changes in the temporal evolution of the frequency response and feature importance changes over time, in addition to the inclusion of some features that negatively impact performance. Although more difficult to manage, an adaptive classifier approach [48] or feature adaptation [49] or data space adaption approach [50], [51], may combat this issue and maintain performance regardless of the pilot’s affective state.

We described the long-term training of a tetraplegic pilot in preparation to compete at the Cybathlon Competition in 2019 and 2020. Training was undertaken in the pilot’s home, in hotel rooms, and at the race venues. All home based training in 2020 was supported remotely by the team (due the COVID 19 pandemic a non-expert at the pilot’s home assisted with cap preparation, etc.). Our results demonstrate significant improvement in performance as a result of our user training strategy, BCI approach and optimisation of the BCI parameters. The results demonstrate that applying multiple binary classifiers along with additional post-processing modules and training with multiple neurogaming technologies is effective at improving the capacity to control a virtual avatar with movement, directly via brain activity, and to maximise that control ability to continue to reduce race times and achieve state-of-the art performances for the challenge. Our pilot has developed into a BCI expert, even though he has been tetraplegic for 37 years, as demonstrated by consistently achieving accuracies above 90% and competitive race times outside the competition days. We did however observe that performance was significantly impacted by changes in cognitive state, possibly due to heightened arousal arising from competition day pressure on the pilot. We conclude that helping the pilot to maintain consistent cognitive states is of critical importance to ensure race day performances are consistent with best training day performance. This should be supplemented by adaptive BCI strategies that can autonomously adapt to cognitive state changes to maintain performance. However, maintenance of cognitive state stability is likely to be the most important criteria for success at the Cybathlon championship for athletes with disabilities. We will focus on this and compete again at the CYBATHLON Edition in 2024.

BCI: brain-computer interface; ERD: event-related de-synchronization; ERS: event-related synchronization; SMR: sensorimotor rhythms; EEG: electroencephalography; UDP: user datagram protocol; FBCSP: filter-bank common spatial patterns; MI: mutual information; DA: decoding accuracy; CSP: common spatial patterns; RLDA: regularized linear discriminant analysis; LDA: linear discriminant analysis; TSD: time-varying signed distance; sLoreta: standardized low resolution brain electromagnetic tomography; DB: dead-band; DBB: dead-band-break; CV: cross-validation; STD: standard deviation; PSD: power spectral density; PIO: pilot-induced oscillation; VR: virtual reality; BAT: boundary avoidance task; HRV: heartrate variability; MBSR: mindfulness-based stress reduction; ECG: electrocardiography;

Ethics approval and consent to participate

The research study presented here, received ethical approval from the Ulster University research ethics committee (UREC), and was carried out in accordance with the Declaration of Helsinki. Prior to the commencement of the training, the participant was presented with information about the experimental protocol and allowed to discuss details with the researchers and ask questions, following which consent was obtained.

Consent for publication

Not applicable

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Competing interests

Damien Coyle is Founder and CEO of NeuroCONCISE Ltd, a neurotechnology company.

Funding

We are grateful for access to the Tier 2 High Performance Computing resources provided by the Northern Ireland High Performance Computing (NI-HPC) facility funded by the UK Engineering and Physical Sciences Research Council (EPSRC), Grant No. EP/T022175 the UKRI Turing AI Fellowship 2021-2025 funded by the EPSRC (grant number EP/V025724/1) and the Spatial Computing and Neurotechnology Innovation Hub, funded by The Department for the Economy, Northern Ireland.

Authors' contributions

DC supervised the project and organized the team. DC and AK designed the experimental paradigm, developed the online BCI, and contributed to data collection. NMcS and AK developed the Triad game. OC, DC, AK, KMcC, NMcS, NDB, and MK contributed to the Cybathlon competition. AK drafted the paper. AK and NMcS prepared the figures and tables. DC, JS, JMcE, and AC were involved in obtaining ethical approval, consent taking and patient recruitment for the research, early data collection, and medical checks for Cybathlon event. DC, KMcC, NMcS, NDB, and MK revised the manuscript.

Acknowledgements

We would like to acknowledge the effort and commitment of our pilot, Mr Owen Collumb, and congratulate him on his success in the competition, 6^th in 2016, 3^rd in 2019 and 6^th in 2020. Without his dedication and commitment we would not have had the learning and BCI development experience outlined in this manuscript. We would also like to thank Ulster University for financially supporting the Team.

Authors' information (optional)

N/A

“Cybathlon BCI Series,” 2021. [Online]. Available: https://cybathlon.ethz.ch/en.
J. R. Wolpaw and D. J. McFarland, “Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans.,” Proc. Natl. Acad. Sci. U. S. A., vol. 101, no. 51, pp. 17849–54, Dec. 2004.
D. J. McFarland, D. J. Krusienski, W. A. Sarnacki, and J. R. Wolpaw, “Emulation of computer mouse control with a noninvasive brain-computer interface.,” J. Neural Eng., vol. 5, no. 2, pp. 101–10, Jun. 2008.
K. LaFleur, K. Cassady, A. Doud, K. Shades, E. Rogin, and B. He, “Quadcopter control in three-dimensional space using a noninvasive motor imagery-based brain-computer interface.,” J. Neural Eng., vol. 10, no. 4, p. 046003, Aug. 2013.
G. Pfurtscheller and F. H. Lopes da Silva, “Event-related EEG/MEG synchronization and desynchronization: basic principles.,” Clin. Neurophysiol., vol. 110, no. 11, pp. 1842–57, Nov. 1999.
J. Lange, A. Pavlidou, and A. Schnitzler, “Lateralized modulation of beta-band power in sensorimotor areas during action observation.,” Front. Integr. Neurosci., vol. 9, no. JUNE, p. 43, 2015.
D. Coyle, J. Stow, K. McCreadie, J. McElligott, and Á. Carroll, “Sensorimotor modulation assessment and brain-computer interface training in disorders of consciousness,” Arch. Phys. Med. Rehabil., vol. 96, no. 3, pp. S62–S70, 2015.
F. Cincotti et al., “Vibrotactile feedback for brain-computer interface operation,” Comput. Intell. Neurosci., vol. 2007, pp. 1–12, 2007.
K. A. McCreadie, D. H. Coyle, and G. Prasad, “Is sensorimotor BCI performance influenced differently by mono, stereo, or 3-D auditory feedback?,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 22, no. 3, pp. 431–440, 2014.
F. Nijboer, N. Birbaumer, and A. Kübler, “The influence of psychological state and motivation on brain-computer interface performance in patients with amyotrophic lateral sclerosis - a longitudinal study,” Front. Neurosci., vol. 4, no. JUL, pp. 1–13, 2010.
S. Perdikis, L. Tonin, S. Saeedi, C. Schneider, and J. del R. Millán, “The Cybathlon BCI race: Successful longitudinal mutual learning with two tetraplegic users,” PLoS Biol., vol. 16, no. 5, pp. 1–28, 2018.
D. M. Wolpert, J. Diedrichsen, and J. R. Flanagan, “Principles of sensorimotor learning.,” Nat. Rev. Neurosci., vol. 12, no. 12, pp. 739–51, 2011.
E. G. Ponferrada, A. Sylaidi, and A. Aldo Faisal, “Data-efficient motor imagery decoding in real-time for the cybathlon brain-computer interface race,” NEUROTECHNIX 2018 - Proc. 6th Int. Congr. Neurotechnology, Electron. Informatics, no. Neurotechnix, pp. 21–32, 2018.
C. Benaroch et al., “Long-Term BCI Training of a Tetraplegic User: Adaptive Riemannian Classifiers and User Training,” Front. Hum. Neurosci., vol. 15, no. March, pp. 1–22, 2021.
F. Turi, M. Clerc, and T. Papadopoulo, “Long Multi-Stage Training for a Motor-Impaired User in a BCI Competition,” Front. Hum. Neurosci., vol. 15, no. March, pp. 1–12, 2021.
J. Mladenović, J. Mattout, and F. Lotte, “A generic framework for adaptive EEG-based BCI training and operation,” in Brain–computer interfaces handbook: technological and theoretical advances, C. S. Nam, A. Nijholt, and F. Lotte, Eds. CRC Press, 2017, pp. 595–611.
C. Jeunet, B. N. Kaoua, F. Lotte, C. Jeunet, B. N. Kaoua, and F. Lotte, “Advances in User-Training for Mental-Imagery Based BCI Control : Psychological and Cognitive Factors and their Neural Correlates,” Prog. Brain Res., vol. 228, pp. 3–35, 2016.
“g.Naurilus Research EEG headset,” 2019. [Online]. Available: http://www.gtec.at/Products/Hardware-and-Accessories/g.Nautilus-Specs-Features. [Accessed: 15-May-2019].
“Simulink for Matlab (The MathWorks, Inc.),” 2020. [Online]. Available: http://www.mathworks.co.uk/products/simulink/.
Kai Keng Ang, Zheng Yang Chin, Haihong Zhang, and Cuntai Guan, “Filter Bank Common Spatial Pattern (FBCSP) in Brain-Computer Interface,” in 2008 IEEE International Joint Conference on Neural Networks, 2008, pp. 2390–2397.
J. Pohjalainen, O. Räsänen, and S. Kadioglu, “Feature selection methods and their combinations in high-dimensional classification of speaker likability, intelligibility and personality traits,” Comput. Speech Lang., vol. 29, no. 1, pp. 145–171, 2015.
K. K. Ang, Z. Y. Chin, C. Wang, C. Guan, and H. Zhang, “Filter bank common spatial pattern algorithm on BCI competition IV datasets 2a and 2b,” Front. Neurosci., vol. 6, no. MAR, pp. 1–9, 2012.
A. Korik, R. Sosnik, N. Siddique, and D. Coyle, “Decoding Imagined 3D Arm Movement Trajectories from EEG to Control Two Virtual Arms - A Pilot Study,” Front. Neurorobot., vol. 13, no. November, pp. 1–22, 2019.
F. Lotte and C. Guan, “Regulized Common Spatial Patterns (RCSP) toolbox,” 2010. [Online]. Available: https://sites.google.com/site/fabienlotte/code-and-softwares.
F. Lotte, M. Congedo, A. Lécuyer, F. Lamarche, and B. Arnaldi, “A review of classification algorithms for EEG-based brain-computer interfaces.,” J. Neural Eng., vol. 4, no. 2, pp. R1–R13, Jun. 2007.
R. D. Pascual-Marqui, “sLORETA, low resolution brain electromagnetic tomography,” 2018. [Online]. Available: http://www.uzh.ch/keyinst/loreta.htm.
G. R. Müller-Putz, V. Kaiser, T. Solis-Escalante, and G. Pfurtscheller, “Fast set-up asynchronous brain-switch based on detection of foot motor imagery in 1-channel EEG,” Med. Biol. Eng. Comput., vol. 48, no. 3, pp. 229–233, 2010.
L. Hehenberger et al., “Long-Term Mutual Training for the CYBATHLON BCI Race With a Tetraplegic Pilot: A Case Study on Inter-Session Transfer and Intra-Session Adaptation,” Front. Hum. Neurosci., vol. 15, no. February, pp. 1–15, 2021.
N. Robinson et al., “Design Considerations for Long Term Non-invasive Brain Computer Interface Training With Tetraplegic CYBATHLON Pilot,” Front. Hum. Neurosci., vol. 15, no. June, pp. 1–16, 2021.
“Cybathlon BCI series 2019,” 2019. [Online]. Available: https://www.tugraz.at/institute/ine/graz-bci-conferences/8th-graz-bci-conference-2019/cybathlon-bci-series-2019/.
“Cybathlon 2020 Global Edition, Results in BCI chategory,” 2020. [Online]. Available: https://cybathlon.ethz.ch/en/projects-events/edition/cybathlon-2020/results.
D. Novak et al., “Benchmarking brain-computer interfaces outside the laboratory: The Cybathlon 2016,” Front. Neurosci., vol. 11, no. Jan, pp. 1–14, 2018.
L. Burns, J. R. Weissensteiner, and M. Cohen, “Lifestyles and mindsets of Olympic, Paralympic and world champions: Is an integrated approach the key to elite performance?,” Br. J. Sports Med., vol. 53, no. 13, pp. 818–824, 2019.
J. Faller, J. Cummings, S. Saproo, and P. Sajda, “Regulation of arousal via online neurofeedback improves human performance in a demanding sensory-motor task,” Proc. Natl. Acad. Sci. U. S. A., vol. 116, no. 13, pp. 6482–6490, 2019.
S. Koch, R. W. Holland, and A. van Knippenberg, “Regulating cognitive control through approach-avoidance motor actions,” Cognition, vol. 109, no. 1, pp. 133–142, 2008.
A. J. Horowitz, C. Guger, and M. Korostenskaja, “What External Variables Affect Sensorimotor Rhythm Brain-Computer Interface (SMR-BCI) Performance?,” HCA Healthc. J. Med., vol. 2, no. 3, pp. 143–162, 2021.
J. D. Dotter, “An Analysis of Aircraft Handling Quality Data Obtained from Boundary Avoidance Tracking Flight Test Techniques,” Airforce Institute of Technology, Ohio, USA, 2007.
P. Grossman, L. Niemann, S. Schmidt, and H. Walach, “Mindfulness-based stress reduction and health benefits: A meta-analysis,” J. Psychosom. Res., vol. 57, no. 1, pp. 35–43, 2004.
P. Eskandari and A. Erfanian, “Improving the performance of brain-computer interface through meditation practicing,” in Proceedings of the 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS’08 - “Personalized Healthcare through Technology,” 2008, pp. 662–665.
L. F. Tan, Z. Dienes, A. Jansari, and S.-Y. Goh, “Effect of mindfulness meditation on brain-computer interface performance,” Conscious. Cogn., vol. 23, pp. 12–21, 2019.
X. Jiang, E. Lopez, J. R. Stieger, C. M. Greco, and B. He, “Effects of Long-Term Meditation Practices on Sensorimotor Rhythm-Based Brain-Computer Interface Learning,” Front. Neurosci., vol. 14, no. January, pp. 1–14, 2021.
J. R. Stieger, S. Engel, H. Jiang, C. C. Cline, M. J. Kreitzer, and B. He, “Mindfulness Improves Brain-Computer Interface Performance by Increasing Control over Neural Activity in the Alpha Band,” Cereb. Cortex, vol. 31, no. 1, pp. 426–438, 2021.
N. Dehghansai, R. A. Pinder, J. Baker, and I. Renshaw, “Challenges and stresses experienced by athletes and coaches leading up to the Paralympic Games,” PLoS One, vol. 16, no. 5 May, pp. 1–20, 2021.
M. González-Franco, P. Yuan, D. Zhang, B. Hong, and S. Gao, “Motor imagery based brain-computer interface: A study of the effect of positive and negative feedback,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, 2011, pp. 6323–6326.
A. Myrden and T. Chau, “Effects of user mental state on EEG-BCI performance,” Front. Hum. Neurosci., vol. 9, no. JUNE, pp. 1–11, 2015.
M. Sugiyama, M. Krauledat, and K.-R. Muller, “Covariate Shift Adaptation by Importance Weighted Cross Validation,” J. Mach. Learn. Res., vol. 8, pp. 985–1005, 2007.
R. Mohammadi, A. Mahloojifar, and D. Coyle, “Unsupervised Short-term Covariate Shift Minimization for Self-paced BCI,” in IEEE Sympsosium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain, 2013, pp. 101–106.
C. Vidaurre, A. Schlöogl, R. Cabeza, R. Scherer, and G. Pfurtscheller, “A Fully On - Line Adaptive BCI,” IEEE Trans Biomed Eng., vol. 53, no. 6, pp. 1214–9, 2006.
A. Satti, C. Guan, D. Coyle, and G. Prasad, “A Covariate Shift Minimisation Method to Alleviate Non-stationarity Effects for an Adaptive Brain-Computer Interface,” in 2010 20th International Conference on Pattern Recognition, 2010, pp. 105–108.
M. Arvaneh, C. Guan, K. K. Ang, and C. Quek, “EEG Data Space Adaptation to Reduce Inter-session Non-stationarity in Brain- Computer Interface,” Neural Comput., vol. 25, no. 8, pp. 2146–71, 2013.
W. Samek, S. Member, and F. C. Meinecke, “Transferring Subspaces Between Subjects in Brain-Computer Interfacing,” Biomed. Eng. (NY)., vol. 60, no. 8, pp. 1–10, 2013.

Download PDF

Reviews received at journal
15 Nov, 2021
Reviewers invited by journal
15 Nov, 2021
Reviewer #1 agreed at journal
14 Nov, 2021
Editor assigned by journal
14 Oct, 2021
Submission checks completed at journal
14 Oct, 2021
Editor invited by journal
14 Oct, 2021
First submitted to journal
13 Oct, 2021

You are reading this latest preprint version

Competing at the Cybathlon Championship for Athletes With Disabilities: Long-Term Motor Imagery Brain-Computer Interface Training of a Tetraplegic Cybathlete

Status:

Version 1

Abstract

Figures

1. Background

2. Methods

2.1 Participant

2.2 Experimental Paradigms

2.3 Data Acquisition

2.4 Calibration of the two-class classification modules

2.5 The online BCI

3. Results

3.1 Calibration details

3.2 Time-varying decoding accuracy and TSD analysis for 2-class BCI

3.3 Frequency analysis and topographical results

3.4 BrainDriver scores, baseline correction, and dead-band configuration

3.5 BrainDriver race performance and EEG power spectral density (PSD)

3.6 BrainDriver race times and frequency bands providing the highest DA contribution

3.7 The reliability of the features in terms of the ability to separate classes

4. Discussion

5. Conclusion

Abbreviations

Declarations

Ethics approval and consent to participate

Consent for publication

Availability of data and materials

Competing interests

Funding

Authors' contributions

Acknowledgements

Authors' information (optional)

References

Status:

Version 1