This paper provides an overview of a long-term pilot training and BCI strategy implemented in preparation for the Cybathlon BCI race event in 2019 and 2020. In both years, the initial phase of the project focused on calibrating the 2-class classifiers using the NeuroSensi game. Our results show that decoding accuracy increased during the online sessions following the calibration period. This observation is in line with similar studies that have demonstrated improvements in the Cybathlon pilot’s BCI control ability due to long-term practice periods – although this has not always translated into high performance on the day of the event, as was the case with the current study and for both  and . Possible reasons for this drop in race-day performance, and suggested strategies to address the issue are discussed.
The TSD results from the current study indicate that although the BCI was re-calibrated between 2019 and 2020, the timing of the pilot’s accuracy control strategy was consistent between 2019 and 2020. The TSD output of the ‘left vs. right’ (LR) classifier between 0s and 3s (reference baseline-related interval) in 2020, show less fluctuation in positive or negative directions compared to that obtained in 2019. The more balanced TSD track deviation, prior to the effect of the class-specific motor imagery task in 2020, may be because the pilot was asked to maintain a relaxed mental state whilst keeping the controlled character in the center position (i.e., at the zero baseline) during the pause period of the ‘task vs. task’ runs in 2020, but not in 2019. Furthermore, the track of the TSD values for the ‘task vs. relax’ (TX) classifier within a 2s width interval around the onset of the task (i.e., between 1s and 3s in the trial) shows that the controlled character moves in the negative direction. The above-described effect is more clearly observable in the 2020 results (Figure 3G, ‘task vs. relax’) compared to the results from 2019 (Figure 3F, ‘task vs. relax’), indicating that the pilot followed the instructions and tried to relax during pause periods of the TX runs, especially in 2020.
The comparison of the frequency and topographic analysis results obtained in 2019 and 2020 using CSP and MI parameters of the calibrated BCI, indicate a similar pattern each year (Figure 4). However, the frequency and topographical patterns obtained based on the 2020 BCI configuration involve some specific features that are not evident in the 2019 BCI. The performance in 2019, for each binary classifier, relied on a single area in the frequency and topographical maps (indicated in A and B panels of Figure 4). However, in 2020, for most binary classifiers, two separate areas provide similarly high contributions for the 2-class classification (Figure 4C and Figure 4D). For example, the ‘feet vs. right’ (FR) classification in 2019 relied mostly on 18-28Hz (high beta) oscillations in the central area of the primary motor and somatosensory cortex – a cortical area commonly activated when able-bodied participants perform the task associated with a kinaesthetically imagined feet movement. This observation is in line with the findings reported by Müller-Putz et al. , which reveals a post-movement beta rebound within a mean range of 17.3-29.7 Hz. For the FR classification in 2020, in addition to the high beta activity in the central area of the primary motor and somatosensory cortex, a similarly high contribution was obtained from low gamma oscillations in the left hemisphere of the somatosensory association and occipito-temporal cortex. Thus, the above discussed CSP-MI patterns obtained in 2020 indicate task-specific cortical activity for both compared tasks (‘feet vs. right hand’), as opposed to highlighting only one task (i.e., ‘feet’), as was the case in 2019. The appearance of more detailed task-specific CSP-MI patterns in 2020 indicates a possible improvement in the pilot’s 2-class classification control strategy compared to the BCI control strategy in 2019.
Each year, after the early sessions, which served to help the pilot achieve control confidence in the 2-class paradigm applied to the NeuroSensi game, the focus of the sessions turned to the BrainDriver game. The game completion time of the BrainDriver during the training period improved significantly. The game completion time in 2019 ranged from 274s to 156s over 17 days including 10 sessions wherein the mean±std statistic of game completion times in the first BrainDriver session (session 9, Figure 5A) was 255±24s and the pilot reached 191±14s in the 10th BrainDriver sessions (sessions 16, Figure 5A). The game completion time in 2020 ranged from 230s to 168s over 18 days including 13 sessions wherein the mean±std statistic of game completion times in the first BrainDriver session (session 9, Figure 5B) was 214±14s and the pilot reached 181±4s in the 13nd BrainDriver session (session 21, Figure 5B). However, on both competition occasions, towards the race date, the pilot’s performance for controlling the BrainDriver game is decreased significantly. For reference the winning race times were 183s and 172s in 2019 and 2020, respectively. The results confirm not only that the multi-session online training using the BrainDriver game had a positive impact on the pilot’s performance, manifested in an increased BCI control ability that enabled the pilot to achieve competitive race completion times, but also highlights that experiences in 2019 were transferred to 2020 resulting in race completion times achieved after one year were in the range of the best race completion times achieved at the end of the training performed in 2019.
In terms of other competitors, the NITRO 1 team, Benaroch et al. reported that the game completion time of their pilot fluctuated between 250 and 340s during seven training sessions before the Cybathlon BCI series (2019) event . However, their pilot could not finish the track within the 4 minutes limit. For the NITRO 2 team, Turi et al. reported in the final competition of the Cybathlon BCI series (2019) their pilot completed a 390.5m disrtance in the 500m long vitual track within the 240s limit  but did not note details of the game completion time achieved by their pilot during the training period. For the Mirage 91 team, Hehenberger et al. reported that their pilot’s performance showed a constant improvement over 14 months of training including 26 game-based sessions for the Cybathlon BCI series (2019) and Cybathlon Global Edition (2020) . The BrainDriver game completion time improved from 255±23s to 225±22s (mean ± STD). For the SEC FHT team, Robinson et al. also reported improving their pilot’s performance over a nine-month training period involving 15 sessions, the BrainDriver game competition time varied between 310s and 214s . The best three ranked teams completed the track in the final challenge of the Cybathlon BCI series (2019) in the following order. Rank 1st : WHI team (500m within 183s), Rank 2nd : Mirage 91 team (500m within 229s), Rank 3rd : NeuroCONCISE team (386m within the 240s limit) , . The best three ranked teams in the Cybathlon Global Edition (2020) completed the track in the following order. Rank 1st : WHI team (500m within 172s), Rank 2nd : MAHIDOL BCILAB BCI team (500m within 176s), Rank 3rd : Neurorobotics team (500m within the 213s). Our team, the NeuroCONCISE team), completed the Cybathlon Global Edition (2020) in Rank 6th (452m within the 240s limit) . For completion, we applied similar but less developed strategy in Cybathlon 2016 with the same pilot. Cybathlon 2016 is not comparable due differences in race track and total race time however our pilot achieved the 3rd best time of all competitors in the competition but had a poor qualifying lap which meant it was 6th place overall .
Factors that impact the performance
Each year, up to a few days before the competition, the race completion time achieved by the pilot from session to session decreased, indicating an improvement in BCI control ability and/or a more refined parameters selection for the BCI. However, a few days before the competition, each year, a significant increase was detected in the race completion time (dotted oval area in Figure 5A and Figure 5B). Furthermore, on the day of the competition, the race completion time increased once again (solid oval area in Figure 5A and Figure 5B). During the last six sessions before the competition in 2019 and 2020, when this negative effect was detected, the BCI configuration had not changed, nor was there a change in the game control strategy reported by the pilot. There may be a number of factors associated with this change in performance, including increased arousal and stress levels, fatigue, and/or changes in living and dietary patterns (e.g., the pilot was living away from home for extended periods during the lead up to race day). Benaroch et al. demonstrated that their Cybathlon pilot’s strategy of adapting their brain patterns to match the training data distribution, helped to improve BCI control . However, they also report that they were not able to translate this to event day performances, which is consistent with our findings. Additionally, Turi et al. reported a similar outcome for competition day results, citing multivariate factors that influence and potentially disrupt pilot performance, including training time, change in routine, differences in the training method with the final game, and differences in training and event environment . A longitudinal study involving another Cybathlon tetraplegic pilot  investigated factors affecting the long-term use of their system by analyzing several performance indicators including activations maps, completion time, classification, and the personal experience of the pilot, by measuring their subjective experience of both their physical and mental readiness on a scale of 1 to 5. The findings support the use of a closed-loop calibration system with real-time feedback, due to better online median classification performance, compared to open-loop calibration paradigms, and improved pilot engagement. Although only a single subject study, the team recommends striving to keep the training paradigm closely matched with the final event by including closed-loop real-time feedback, which helps boost classification performance whilst increasing brain activations due to the increased engagement felt by the pilot. Promisingly, not all entrants reported a decrease in performance on the event day. Hehenberger et al. described a correlation between external influencing factors and performance on their final race . As was the case for the current study, the authors describe the pilot’s increased performance over time, but did not evidence a decrease in performance during the final races. Moreover, the pilot achieved a personal best in his performance at the 2019 Cybathlon, leading the authors to speculate that their pilot performs better in front of an audience.
In an attempt to understand how to mitigate all negative influences on pilot performance, it has been argued that the peak performance of Paralympians is driven not only by psychological factors, but through their convergence with a sufficient support network, lifestyle, and attuned methods of performance . Hence, recent research has focused on counteracting the effects of these variables. One approach has been to use neurofeedback to improve BCI task performance through training self-regulation of arousal states via attention mechanisms . Cognitive control underlies executive function within the brain and interacts critically with arousal systems to activate approach-avoidance behavior , . This interaction is the principal behind the Yerkes-Dodson law – a psychological concept that posits an optimal (moderate) level of arousal is necessary for improved task performance – while a low level of arousal reduces motivation and a high level of arousal negatively impacts cognitive information processing, thus impairing task performance. Therefore, this relationship between the state of arousal and performance on a task is best described as an inverted-U, known as the Yerkes-Dodson curve, . Arousal levels have been found to influence sensory-motor cognition – spontaneous high-frequency oscillations known as "pilot-induced oscillation" (PIO) are generated when performing a high-consequence task. These unstable oscillations have deleterious effects on performance when amplified by the pilot's over-correction of small errors in control , . Faller and colleagues (2019) used audio feedback in a closed-loop neurofeedback BCI, comprising a synthetic slow heartbeat (60bpm), which became louder with increased arousal, to train users to self-regulate their arousal levels while performing a virtual reality (VR)-based boundary avoidance task (BAT). Task performance improved significantly for the users who received veridical feedback compared to those in the sham or no-feedback control conditions. This result was corroborated by heartrate variability (HRV) data and measures of pupil dilation, which indicated a learned ability to shift arousal state and increase task performance through neurofeedback training.
However, it remains uncertain whether the advantage afforded by feedback would be maintained in the absence of that feedback during the competition itself. Mindfulness training could provide a more sustainable approach to the self-regulation of arousal, via underlying cognitive mechanisms. Mindfulness training has been shown not only to have a positive impact on stress reduction  but is also gaining momentum in the BCI research community with some studies showing an improvement in performance for those using mindfulness over a control group of almost 20% . Mindfulness is a metacognitive process as it requires self-regulation of attention to control cognitive processes – while simultaneously monitoring conscious experience . Individuals who are experienced in meditation skills have been found to demonstrate higher resting SMR power, a more stable resting mu rhythm, and greater BCI control, compared to those who are not practiced in meditation techniques . Furthermore, mindfulness-based stress reduction (MBSR) training has been found to improve BCI learning and performance on BCI tasks that communicate the user’s intent via motor imagery commands and volitional rest . The mechanism for this improvement is brought about by the user's ability to volitionally increase alpha-band neural activity as a consequence of the MBSR training. Stieger et al. evidenced an increase in alpha activity recorded during the user's volitional resting-state, across MBSR sessions, which was correlated with mindfulness practice and predicted BCI performance . Strategies to reduce challenges and stress when preparing for the competition include anticipation and preparation through detailed planning, including contingency planning, and expectation management, i.e., focusing on the process rather than the outcome . Training in MBSR would fit with these approaches to stress management, and enhance BCI task performance under the pressure of competition.
Factors that impact SMR-BCI performance that are less dependent on the BCI user and more dependent on external elements include distractors, time spent training, various types of feedback, and features of the EEG system and the data preprocessing algorithms . For example, González-Franco et al., whose research studied the influences of positive and negative visual feedback on motor imagery task performance using EEG and electrocardiography (ECG) found that over-biased negative feedback caused mental stress that is detected in the form of significantly higher heart rate variability (HRV), compared to sessions where over-biased positive feedback was presented, and accuracies correlated with the polarity (-/+) of the biased feedback . Adaptive feedback, or BCI setup, that limits the negative feedback may be an alternative strategy where the onus is on the BCI to deal with the changes in the pilot’s affective state. Performance may be improved by replacing the BCI framework proposed here with an adaptive BCI method that could update the BCI in real-time and adapt to the pilot’s actual mental state . In the lead up to the competition, the average TSD was offset from zero (indicating classifier bias to one of the classes). Even manual correction (offsetting) before each session was not enough to counteract the drift. This baseline drift is a well-known issue in BCI and is associated with changes in the distribution of the features – covariate shift , . As shown in Figure 6 - Figure 8 we observed changes in the temporal evolution of the frequency response and feature importance changes over time, in addition to the inclusion of some features that negatively impact performance. Although more difficult to manage, an adaptive classifier approach  or feature adaptation  or data space adaption approach , , may combat this issue and maintain performance regardless of the pilot’s affective state.