EEG-Based Brain-Computer Interfaces Are Vulnerable to Backdoor Attacks

Research and development of electroencephalogram (EEG) based brain-computer interfaces (BCIs) have advanced rapidly, partly due to deeper understanding of the brain and wide adoption of sophisticated machine learning approaches for decoding the EEG signals. However, recent studies have shown that machine learning algorithms are vulnerable to adversarial attacks. This article proposes to use narrow period pulse for poisoning attack of EEG-based BCIs, which is implementable in practice and has never been considered before. One can create dangerous backdoors in the machine learning model by injecting poisoning samples into the training set. Test samples with the backdoor key will then be classified into the target class specified by the attacker. What most distinguishes our approach from previous ones is that the backdoor key does not need to be synchronized with the EEG trials, making it very easy to implement. The effectiveness and robustness of the backdoor attack approach is demonstrated, highlighting a critical security concern for EEG-based BCIs and calling for urgent attention to address it.

Research and development of electroencephalogram (EEG) based brain-computer interfaces (BCIs) have advanced rapidly, partly due to deeper understanding of the brain and wide adoption of sophisticated machine learning approaches for decoding the EEG signals. However, recent studies have shown that machine learning algorithms are vulnerable to adversarial attacks. This article proposes to use narrow period pulse for poisoning attack of EEG-based BCIs, which is implementable in practice and has never been considered before. One can create dangerous backdoors in the machine learning model by injecting poisoning samples into the training set. Test samples with the backdoor key will then be classified into the target class specified by the attacker. What most distinguishes our approach from previous ones is that the backdoor key does not need to be synchronized with the EEG trials, making it very easy to implement. The effectiveness and robustness of the backdoor attack approach is demonstrated, highlighting a critical security concern for EEG-based BCIs and calling for urgent attention to address it.
Brain-computer interfaces (BCIs) 1 enable the user to communicate with or control an external device (computer, wheelchair, robot, etc.) directly using the brain. They have been successfully used in active tactile exploration 2 , neurally controlled robotic arm for reach and grasp 3 , speech synthesis 4 , emotion regularization 5 , cortical activity to text translation 6 , etc., and are an important component of the Institute of Electrical and Electronics Engineers (IEEE) Brain Initiative 7 and the China Brain Project 8 .
Non-invasive BCIs 9 , which usually use electroencephalogram (EEG) as the input, may be the most popular type of BCIs, due to their convenience and low cost. A closed-loop EEG-based BCI system is illustrated in Figure 1a. It has been widely used in neurological rehabilitation 10 , spellers 11 , awareness evaluation/detection 12 , robotic device control 13 , reaction time estimation 14 , and so on.
Machine learning has been extensively employed in EEG-based BCIs to extract informative features [14][15][16] and to build high-performance classification/regression models [17][18][19] . Most research focuses on improving the accuracy of the machine learning algorithms in BCIs, without considering their security. However, recent studies 20,21 have shown that machine learning models, particularly deep learning models, are subject to adversarial attacks. There are at least two types of adversarial attacks. The first is evasion attack [21][22][23] , which adds deliberately designed tiny perturbations to a benign test sample to mislead the machine learning model. The second is poisoning attack [24][25][26] , which creates backdoors in the machine learning model by adding contaminated samples to the training set. Adversarial attacks represent a crucial security concern in deploying machine learning models in safety-critical applications, such as medical imaging 27 , electrocardiogram-based ar- Figure 1: Poisoning attack to EEG-based BCIs. a. A closed-loop EEG-based BCI system. b. The proposed poisoning attack approach in EEG-based BCIs. Narrow period pulses can be added to EEG trials during signal acquisition. rhythmia detection 28 , and autonomous driving 29 .
Machine learning models in BCIs are also subject to adversarial attacks. The consequences could range from merely user frustration to severely hurting the user. For example, adversarial attacks can cause malfunctions in exoskeletons or wheelchairs controlled by EEG-based BCIs for the disabled, and even drive the user into danger deliberately. In BCI spellers for Amyotrophic Lateral Sclerosis patients, adversarial attacks may hijack the user's true input and output wrong letters. The user's intention may be manipulated, or the user may feel too frustrated to use the BCI speller, losing his/her only way to communicate with others. In BCI-based driver drowsiness estimation 18 , adversarial attacks may manipulate the output of the BCI system and increase the risk of accidents. In EEG-based awareness evaluation/detection for disorder of consciousness patients 12 , adversarial attacks may disturb the true responses of the patients and lead to misdiagnosis.
Zhang and Wu 30 were the first to point out that adversarial examples exist in EEG-based BCIs. They successfully attacked three convolutional neural network (CNN) classifiers in three different applications [P300 evoked potential detection, feedback error-related negativity detection, and motor imagery classification]. Meng et al. 31 further confirmed the existence of adversarial examples in two EEG-based BCI regression problems (driver fatigue estimation, and reaction time estimation in the psychomotor vigilance task), which successfully changed the regression model's prediction by a user-specified amount. More recently, Zhang et al. 32 also showed that P300 and steady-state visual evoked potential based BCI spellers can be easily attacked: a tiny perturbation to the EEG trial can mislead the speller to output any character the attacker wants.
However, these attack strategies were mostly theoretical. There are several limitations in applying them to real-world BCIs: 1) the adversarial perturbations are very complex to generate; 2) the attacker needs to craft different adversarial perturbations for different EEG channels and trials; and, 3) the attacker needs to know the complete EEG trial and its precise starting time in advance to compute an adversarial perturbation. Zhang et al.'s latest work 32 demonstrated ways to overcome some of these limitations, but it still requires the attacker to know the start time of a trial in advance to achieve the best attack performance.
This article reports a novel approach that is more implementable in practice. It belongs to the poisoning attack framework, which consists of two steps: 1. Data poisoning in model training (backdoor creation): We assume the attacker can stealthily inject a small number of poisoning samples into the training set, to create a backdoor in the trained model. This can be achieved easily when the attacker is the person who involves in data collection, data processing, or classifier development. Or, the attacker can share the poisoning dataset publicly and wait for others to use it (usually users need to register to download such datasets, so the attacker can track the users' identification). Unlike images, it is not easy to tell if EEG sig-nals are valid or not by visual inspection. Users usually do not look at the raw EEG signals directly. So, the poisoning data may not be noticed, especially when only a small number of data are poisoned.
2. Data poisoning in actual attacks (backdoor addition): To perform an attack, the attacker adds the backdoor key to any benign EEG trial, which then would be classified as the target class specified by the attacker. Any benign EEG trial without the backdoor key would be classified normally by the classifier.
We consider narrow period pulse (NPP) as the backdoor key in this article. NPP is common interference noise, which can be added to EEG signals during data acquisition, as shown in Figure 1b.
Our main contributions are: 1. We show, for the first time, that poisoning attacks can be performed for EEG-based BCIs. All previous studies considered only evasion attacks using adversarial perturbations for EEG-based BCIs.
2. We propose a practically realizable backdoor key, NPP, for EEG signals, which can be inserted into original EEG signals during data acquisition, to demonstrate how poisoning attack can fool EEGbased BCIs. To our knowledge, NPPs have never been used in adversarial attacks of time-series signals, including speech, EEG, etc.
3. We demonstrate the effectiveness of the proposed attack approach, under the challenging and realistic scenario that the attacker does not know any information about the test EEG trial, including its start time. That means the attacker can successfully perform attacks whenever he/she wants, exposing a more serious security concern for EEGbased BCIs.
We need to emphasize that the goal of this research is not to damage EEG-based BCIs; instead, we try to expose critical security concerns in them, so that they can be properly addressed to ensure secure and reliable applications.

Results
Datasets. The following three publicly available EEG datasets were used in our experiments: EEGNet is a compact CNN architecture specifically designed for EEG-based BCIs. It consists of two convolutional blocks and a classification block. Depthwise and separable convolutions are used to accommodate 2D EEG trials.
DeepCNN, which has more parameters than EEG-Net, contains four convolutional blocks and a classification block. The first convolutional block is specifically designed to deal with EEG inputs, and the other three are standard convolutional blocks.
Traditional models. Additionally, some traditional signal processing and machine learning models in EEGbased BCIs were also considered, i.e., xDAWN 36 spatial filtering and Logistic Regression (LR) classifier for the ERN and P300 datasets, and common spatial pattern (CSP) 37 filtering and LR classifier for the MI dataset.
Performance metrics. The following two metrics were used to evaluate the effectiveness of the proposed attack approaches: Specifically, among the 16 subjects in the ERN dataset (each with 340 EEG trials), we randomly chose one subject as the poisoning subject, and the remaining 15 subjects to perform leave-one-subject-out crossvalidation, i.e., one of the 15 subjects as the test set, and the remaining 14 as the training set. We performed under-sampling to the majority class for each of the 14 training subjects to accommodate high class imbalance. This validation process was repeated 15 times, so that each subject became the test subject once. Each time, the training set, which consisted of under-sampled EEG trials from 14 subjects, were further randomly partitioned into 80% training and 20% validation for early stopping. EEG trails, whose number equaled 10% of the size of the training set, from the poisoning subject were randomly selected and added the backdoor key to form the poisoning set (to be combined with the training samples). All poisoning samples were labeled as 'goodfeedback', as the attacker's goal was to make the classifier classify any test sample with the backdoor key to 'good-feedback' (target label), no matter what true class the test sample belongs to. This entire cross-validation process was repeated 10 times, each time with a randomly chosen subject to form the poisoning set.
In summary, there were 15 × 10 = 150 runs on the ERN dataset, each with ∼2, 200 clean training samples, ∼220 poisoning samples, ∼550 validation samples, and 340 test samples. The mean ACCs and ASRs of these 150 runs were computed and reported.
Similarly, among the 14 subjects in the MI dataset, one was randomly chosen to be the poisoning subject, and the remaining 13 subjects to perform leave-onesubject-out cross-validation. All 160 EEG trials from the poisoning subject were used to form the poisoning set and labeled as 'both feet'. The entire crossvalidation process was repeated 10 times.
Among the eight subjects in the P300 dataset, one was randomly chosen to be the poisoning subject, and the remaining seven subjects to perform leave-onesubject-out cross-validation. We also performed under-sampling to the majority class to balance the training set. EEG trials, whose number equaled 10% of the size of the training set, from the poisoning subject were randomly chosen to construct the poisoning set, all of which were labeled as 'target'. The entire crossvalidation process was repeated 10 times.
Baseline performance. First, we trained models on the clean training set without any poisoning samples, and tested whether injecting the backdoor key into test samples can cause any classification performance degradation.
These baseline ACCs and ASRs of different classifiers on different datasets are shown in Table 1. The baseline ACCs were fairly high, considering the fact that they were evaluated on subjects different from those in the training set. The baseline ASRs were very small, indicating models that have not seen the poisoning data (the backdoor key) in training cannot be easily fooled by the backdoor key in test.
Attack performance. NPP backdoor keys with period T = 0.2s, duty cycle d = 10% and three different amplitudes were used for each dataset: 10%/20%/30% of the mean channel-wise standard deviation of the EEG amplitude for the ERN dataset, 50%/100%/150% for the MI dataset, and 0.1%/0.5%/1% for the P300 dataset. These values were significantly different for different datasets, because the magnitudes of the raw EEG signals in different datasets varied a lot, possibly due to different hardware used and different experimental paradigms.
When the same NPP backdoor key was added to the poisoning samples and/or test samples, the attack performances are shown in the 'NPP Attack' panel of Table 1. The ACCs were very close to those in the 'NPP Baseline' panel, indicating that adding poisoning samples did not significantly change the classification accuracy, when the test samples did not contain the backdoor key. However, the ASRs in the 'NPP Attack' panel were much higher than the corresponding baseline ASRs, indicating that these NPP backdoor attacks were very successful. Intuitively, the last two column of ASRs in the 'NPP Attack' panel were higher than those in the first column, i.e., a larger NPP amplitude would lead to a higher ASR. Among different models, the traditional CSP+LR model seemed more resilient to the attacks. Figure 2 shows an example of the same EEG trail from the P300 dataset before and after poisoning, with and without preprocessing (down-sampling and bandpass filtering), respectively. Examples for the ERN and MI datasets are shown in Supplementary Figures 1 and  2. The poisoned EEG looked like normal EEG, so the backdoor may not be easily detected. Additionally, preprocessing cannot remove the backdoor.
Practical considerations. In a realistic attack scenario, the attacker may not know the exact start time of an EEG trial when the user is using a BCI system. As a result, the attacker cannot inject the backdoor key to a test sample exactly as he/she does in generating poisoning samples in training. So, a successful attack approach should not be sensitive to the start time of EEG trials.
To make the backdoor attacks more flexible and realistic, we used a random phase of NPP in [0, 0.8]T (T is the period of the NPP; see 'Methods') for every poisoning sample. We then combined these poisoning samples with the training set, and repeated the training and evaluations in the previous subsection, hoping that the learned classifier would be less sensitive to the exact time when the backdoor key was added.
The attack results of the approach on the three models are shown Figure 3a. NPP obtained much higher ASRs on different models than the baselines, indicating that the proposed NPP attack approach is insensitive to the start of EEG trials.
Influence of the number of poisoning samples. Figure 3b shows the ACCs and ASRs of NPP attack to EEGNet when the poisoning ratio (the number of poisoning samples divided by the number of training samples) increased from 1% to 10%. Results for DeepCNN and traditional models are shown in Supplementary Figures 3 and 4.
As the poisoning ratio increased, ACCs did not change much, whereas ASRs improved significantly. Only 6% poisoning ratio on ERN and MI was enough to achieve an average ASR of 60%, and 1% poisoning ratio on P300 achieved an average ASR of 80%. Com-   Influence of the NPP amplitude. The NPP amplitude also affects ASRs. Figure 3c shows the ASRs of using NPPs with different amplitude ratios (the NPP amplitude divided by the mean channel-wise standard deviation of the EEG amplitude) in test for EEGNet. Results for other models are shown in Supplementary Figures 5  and 6. As the NPP amplitude ratio in test increased, the ASR also increased. The ASR also increased when larger NPPs were used in the poisoning set.
Interestingly, the NPP amplitude ratios may not need to match the amplitude ratios in training. For example, NPPs with amplitude ratio between 0.4% and 1.0% in test obtained similar ASRs on P300. In other words, the attacker does not need to know the exact NPP amplitude in poisoning, making the attack more practical.
Influence of the NPP period and duty cycle. Figure 4 shows the ASRs of using nine NPPs with different peri- When NPPs were used in both training and test (the first six rows and six columns in Figure 4), high ASRs can be achieved, no matter whether the NPPs in training and test matched exactly or not, indicating that NPP attacks are also resilient to the NPP period and duty cycle. However, ASRs in the last three rows and three columns on ERN and MI dataset ( Supplementary Figures 7 and  8) were relatively low, suggesting that the NPP param-eters may impact ASRs in different BCI paradigms.
Accommodate re-referencing. We have demonstrated the effectiveness and robustness of NPP attacks, without considering channel re-referencing 38 , which may have some impact on the attack performance. For example, if we add identical NPPs to all EEG channels, then an average re-referencing 38 would remove them completely, and hence the attack cannot be performed.
There are different solutions to this problem. If the attacker knows exactly the reference channel, e.g., Cz or mastoid, then NPPs can be added only to that channel. After referencing, NPP negations will be introduced to all other channels. In practice, the attacker may not know what referencing approach and channels are used by the BCI system, so a more flexible solution is to add NPPs to a subset of channels. If average re-referencing is not performed, then NPPs in these channels are kept; otherwise, the NPP magnitudes in these channels are reduced but not completely removed. Table 2 shows the attack performance when NPPs were added to 10%/20%/30% randomly selected EEG channels. The ASRs were comparable with or even higher than those of adding NPPs to all channels, suggesting that the attacker can add NPPs to a subset of channels to accommodate referencing.

Discussion
Adversarial attacks to EEG-based BCIs have been explored in our previous studies [30][31][32]39 . All of them were evasion attacks. These approaches are theoretically important, but very difficult to implement in practice. They all need to inject a jamming module between EEG preprocessing and machine learning, to add the adversarial perturbation to a normal EEG trial. It's difficult to implement in a real-world BCI system, in which EEG preprocessing and machine learning may be integrated. To generate or add the adversarial perturbation, the attacker also needs to know a lot of information about the target EEG trial, e.g., the start time is needed to align it with the adversarial perturbation, but it is very difficult to know this. Furthermore, the adversarial perturbations generated by these attack approaches are very complex for a real-world BCI system to realize, e.g., different channels need to have different adversarial perturbations, which are very challenging to add.
Compared with previous approaches, the NPP backdoor attack approach proposed in this article is much easier to implement, and hence represents a more significant security concern to EEG-based BCI systems.
Our future research will consider hardware implementation of the attack approach in a real-world BCI system, and more importantly, strategies to defend against such attacks, as the ultimate goal of our research is to increase the security of BCI systems, instead of damaging them.

Methods
Assume the model designer has a labeled training set D = {(x i , y i )} N i=1 with N samples, which are not known to the attacker. The attacker also has some normal samples {a j } M j=1 , where a and x have the same format, and usually M ≪ N .
The attacker aims to design a backdoor key k, and a function g(a j , k) which adds k to a j to form a poisoning sample. The attacker then adds {(g(a j , k), y)} M j=1 to D, where y is the target class specified by the attacker, i.e., he/she wants any test sample with the backdoor k to be classified into class y.
The model designer trains a classifier on the poisoned training set D ′ = D ∪ {(g(a j , k), y)} M j=1 , using whatever classifier he/she wants, e.g., traditional machine learning or deep learning. The backdoor k is automatically embedded into the model.
During the attack, the attacker can add k to any benign test sample x to open the backdoor and force the classifier to classify x to the target class y. When k is not added, the BCI system just operates normally.
Narrow period pulse (NPP). NPP is a type of signal that can be easily generated. A continuous NPP is determined by a period T , a duty cycle d (the ratio between the pulse duration and the period), and an amplitude a, N c (t) = a, nT ≤ t < nT + dT 0, nT + dT ≤ t < (n + 1)T A discrete NPP with sampling rate f s can be expressed as N d (i) = a, nT f s ≤ i < (n + d)T f s 0, (n + d)T f s ≤ i < (n + 1)T f s This NPP was used as k, and g(a j , k) = a j + k in obtaining the results in Table 1.