Brain computer interface (BCI) is a communication interface directly established between the brain and the computer, which can realize the interconnection between brain and external objects, and achieve the interactive integration of biological intelligence and machine intelligence. It has important applications in the fields of medicine, neurobiology and psychology [1]. Many mature paradigms have emerged in the BCI field, including motor imagery BCI, steady-state visual evoked potential (SSVEP) BCI, P300 visual-evoked potentials BCI and emotional BCI [1; 2].
Motor imagery electroencephalography (MI-EEG) is a kind of endogenous spontaneous EEG with low environmental requirements. Thus, MI-EEG is widely used in BCI. The MI-BCI system collects EEG signals when the subject performs specific motor imagery, then recognizes the MI content according to the EEG signals, and converts the recognition results into control commands of the peripheral devices[3]. EEG signals have the characteristics of a low signal-to-noise ratio and low spatial resolution, so effective features and classifiers are the keys to the success of the MI recognition system, and BCI researchers have increasingly proposed many algorithms for MI classification.
1.1 Common spatial pattern-related method
The common spatial pattern (CSP) algorithm and its variants have been widely applied to construct spatial filters and extract highly discriminative features in EEG-based MI classification by maximizing the variance difference between two classes of EEG signals [4]. The outstanding performance of the filter-bank common spatial pattern (FBCSP) won the BCI Competition IV in the 2a dataset and 2b dataset [5]. The FBCSP used a filter bank consisting of 9 nonoverlapping subband bandpass filters covering the frequency range of 4 to 40 Hz to preprocess the signal. Then, the CSP features were extracted and selected for specific subjects by the mutual information-based rough set reduction algorithm and fed to the naïve Bayesian Parzen window classifier. The filter-bank regularized common spatial pattern was proposed to simultaneously solve the dependency problems on frequency bands and sample-based covariance estimation, and the proposed method improved the mean classification accuracy compared with other CSP-based methods [6]. Zhang proposed a hybrid network consisting of a CNN and a long-term short-term memory network for extracting temporal and spatial features from CSP features [7].
1.2 Deep learning-based method
Recently, deep learning algorithms have developed quickly, and related algorithms have been proposed for EEG-based BCI. In particular, CNNs have been widely used in EEG-based MI classification due to their ability to effectively extract temporal and spatial features from EEG signals. Schirrmeister proposed a Shallow ConvNet and a Deep ConvNet for end-to-end MI-based EEG recognition and showed better performance compared with the FBCSP algorithm. In addition, the CNN visualization results showed that the proposed model learned to use spectral power characteristics from different frequency bands [8]. EEGNet was proposed by Lawhern to suggest that a compact CNN can be applied and provide robust performance across many BCI paradigms, such as P300 event-related potential, feedback error-related negativity, movement-related cortical potential and sensory motor rhythm (MI recognition) [9]. Chen designed a deep learning approach termed filter-bank spatial filtering and temporal-spatial CNN for MI decoding. Filter-bank spatial filtering extracts the feature presentation of raw EEG signals, and the temporal-spatial CNN implements a decoding procedure. A stagewise training strategy including optimizing the triplet loss and cross-entropy loss was proposed to mitigate the optimization difficulty [10]. Li proposed an end-to-end EEG decoding framework that regards the original multichannel EEG as the input and improves the classification accuracy through a channel projection mixed-scale convolutional neural network (CP-MixedNet) and amplitude perturbation data augmentation [11]. Sakhavi proposed a novel filter-bank convolutional network (FBCNet) for MI classification, which extracted a multiview data representation through a filter bank, extracted spatial features through depthwise convolutional layers, and effectively aggregated temporal information by the proposed variance layer [12]. Zhao built a multibranch 3D convolutional neural network, where the 3D representation was generated by transforming EEG signals into a series of 2D arrays focusing on the spatial distribution of channels [13]. Li proposed a novel temporal-spectral-based squeeze-and-excitation feature fusion network, which extracted high-dimensional temporal features and discriminative spectral representations from raw EEG signals via deep-temporal convolution block and multilevel wavelet convolutions, respectively. Channelwise discriminative responses were highlighted by constructing interdependencies among different domain features [14]. Five adaptive schemes of the EEG-BCI system based on CNN were proposed for decoding MI-EEG. The adaptive transfer learning method fine-tuned an extensively trained, pretrained model and adjusted it to adapt the target subject [15].
1.3 Multisubject calibration-free MI-BCI
At present, research on MI-BCIs mainly focuses on subject-dependent systems, in which a model is built for a single target subject and has achieved satisfactory results [6; 8]. However, a subject-dependent system needs to collect data for calibrating the target subject, which is time-consuming and only applicable to the target subject. Therefore, research on multisubject calibration-free BCI systems has appeared. Kwon constructed a large MI-based EEG database consisting of 54 subjects performing left- and right-hand MI and proposed a subject-independent multibranch CNN framework for subject-independent BCI. The spectral–spatial inputs are individually trained through the CNN and then combined by a concatenation fusion technique to make predictions [16]. Zhang proposed a convolutional recurrent attention model for subject-independent EEG signal analysis. Specifically, they split an EEG trial into multiple temporal slices, utilized the spatial-temporal block to extract the spatial-temporal information of every temporal slice, and finally, leveraged a recurrent attention mechanism to explore the temporal dynamics among different temporal slices. The improved performance in the experiment indicated that the proposed convolutional recurrent attention model can utilize potential invariant EEG patterns among different subjects [17]. To improve the classification accuracy of multisubject motor imagery, Autthasan designed a novel end-to-end multitask learning architecture called MIN2Net, which applied a deep metric learning method in a multitask autoencoder model to learn discriminative potential representations and make predictions simultaneously [18]. Luo proposed a twin cascaded softmax convolutional neural network (TCSCNN) for multisubject MI-BCIs. The cascaded softmax structure was applied to achieve subject recognition and MI recognition simultaneously, and the twin EEG and twin structure were employed to further improve the performance [19].
In multisubject MI-BCI systems, the individual differences in EEG signals cause great difficulties for research [20]. In particular, the effective frequency band related to event-related desynchronization (ERD) and event-related synchronization (ERS) varies from subject to subject. Utilizing the discriminative information of different frequency bands is key to improving the capability of multisubject MI-BCI, but existing CNN models perform poorly in this aspect. This paper presents a novel overlapping filter-bank CNN framework for multisubject motor imagery EEG recognition. Through the proposed overlapping filter bank, the filtered EEG forces the CNN to learn from different frequency bands. To combine the discriminative ability from different frequency bands, ensemble probability from multiple CNNs is employed to make predictions.
The main contributions of this paper are as follows.
1) A filter-bank CNN framework is proposed to enable different CNN models to learn discriminative information from multiple EEG frequency bands for multisubject MI-BCI.
2) A novel overlapping filter bank with a fixed low-cut frequency outperforms other filter banks in the multisubject MI-BCI experiments.
3) Comprehensive experimental evaluations of three popular CNN backbone models using two benchmark datasets for the overlapping filter-bank CNN framework demonstrate the effectiveness and universality of the proposed framework.
The rest of this paper is organized as follows. Section 2 presents the proposed overlapping filter-bank CNN framework. Section 3 introduces the dataset and experimental settings in detail. Then, the experimental results and discussion are presented in Section 4. Section 5 gives the conclusions.