Deep learning (DL) methods and architectures have been the state-of-the-art classification algorithms for computer vision and natural language processing problems. However, the successful application ...of these methods in motor imagery (MI) brain-computer interfaces (BCIs), in order to boost classification performance, is still limited. In this paper, we propose a classification framework for MI data by introducing a new temporal representation of the data and also utilizing a convolutional neural network (CNN) architecture for classification. The new representation is generated from modifying the filter-bank common spatial patterns method, and the CNN is designed and optimized accordingly for the representation. Our framework outperforms the best classification method in the literature on the BCI competition IV-2a 4-class MI data set by 7% increase in average subject accuracy. Furthermore, by studying the convolutional weights of the trained networks, we gain an insight into the temporal characteristics of EEG.
One of the most popular feature extraction algorithms for brain-computer interfaces (BCI) is common spatial patterns (CSPs). Despite its known efficiency and widespread use, CSP is also known to be ...very sensitive to noise and prone to overfitting. To address this issue, it has been recently proposed to regularize CSP. In this paper, we present a simple and unifying theoretical framework to design such a regularized CSP (RCSP). We then present a review of existing RCSP algorithms and describe how to cast them in this framework. We also propose four new RCSP algorithms. Finally, we compare the performances of 11 different RCSP (including the four new ones and the original CSP), on electroencephalography data from 17 subjects, from BCI competition datasets. Results showed that the best RCSP methods can outperform CSP by nearly 10% in median classification accuracy and lead to more neurophysiologically relevant spatial filters. They also enable us to perform efficient subject-to-subject transfer. Overall, the best RCSP algorithms were CSP with Tikhonov regularization and weighted Tikhonov regularization, both proposed in this paper.
Advances in brain-computer interface (BCI) technology have facilitated the detection of Motor Imagery (MI) from electroencephalography (EEG). First, we present three strategies of using BCI to detect ...MI from EEG: operant conditioning that employed a fixed model, machine learning that employed a subject-specific model computed from calibration, and adaptive strategy that continuously compute the subject-specific model. Second, we review prevailing works that employed the operant conditioning and machine learning strategies. Third, we present our past work on six stroke patients who underwent a BCI rehabilitation clinical trial with averaged accuracies of 79.8% during calibration and 69.5% across 18 online feedback sessions. Finally, we perform an offline study in this paper on our work employing the adaptive strategy. The results yielded significant improvements of 12% (p <; 0.001) and 9% (p <; 0.001) using all the data and using limited preceding data respectively in the feedback accuracies. The results showed an increase in the amount of training data yielded improvements. Nevertheless, results of using limited preceding data showed a larger part of the improvement was due to the adaptive strategy and changing subject-specific models did not deteriorate the accuracies. Hence the adaptive strategy is effective in addressing the non-stationarity between calibration and feedback sessions.
Stroke is one of the leading causes of long-term disability among adults and contributes to major socio-economic burden globally. Stroke frequently results in multifaceted impairments including ...motor, cognitive and emotion deficits. In recent years, brain-computer interface (BCI)-based therapy has shown promising results for post-stroke motor rehabilitation. In spite of the success received by BCI-based interventions in the motor domain, non-motor impairments are yet to receive similar attention in research and clinical settings. Some preliminary encouraging results in post-stroke cognitive rehabilitation using BCI seem to suggest that it may also hold potential for treating non-motor deficits such as cognitive and emotion impairments. Moreover, past studies have shown an intricate relationship between motor, cognitive and emotion functions which might influence the overall post-stroke rehabilitation outcome. A number of studies highlight the inability of current treatment protocols to account for the implicit interplay between motor, cognitive and emotion functions. This indicates the necessity to explore an all-inclusive treatment plan targeting the synergistic influence of these standalone interventions. This approach may lead to better overall recovery than treating the individual deficits in isolation. In this paper, we review the recent advances in BCI-based post-stroke motor rehabilitation and highlight the potential for the use of BCI systems beyond the motor domain, in particular, in improving cognition and emotion of stroke patients. Building on the current results and findings of studies in individual domains, we next discuss the possibility of a holistic BCI system for motor, cognitive and affect rehabilitation which may synergistically promote restorative neuroplasticity. Such a system would provide an all-encompassing rehabilitation platform, leading to overarching clinical outcomes and transfer of these outcomes to a better quality of living. This is one of the first works to analyse the possibility of targeting cross-domain influence of post-stroke functional recovery enabled by BCI-based rehabilitation.
Electromyography (EMG) signal is one of the widely used biological signals for human motor intention prediction, which is an essential element in human-robot collaboration systems. Studies on motor ...intention prediction from EMG signal have been concentrated on classification and regression models, and there are numerous review and survey papers on classification models. However, to the best of our knowledge, there is no review paper on regression models or continuous motion prediction from EMG signal. Therefore, in this paper, we provide a comprehensive review of EMG-based motor intention prediction of continuous human upper limb motion. This review will cover the models and approaches used in continuous motion estimation, the kinematic motion parameters estimated from EMG signal, and the performance metrics utilized for system validation. From the review, we will provide some insights into future research directions on these subjects. We first review the overall structure and components of EMG-based human-robot collaboration systems. We then discuss the state of arts in continuous motion prediction of the human upper limb. Finally, we conclude the paper with a discussion of the current challenges and future research directions.
Event-related desynchronization/synchronization patterns during right/left motor imagery (MI) are effective features for an electroencephalogram-based brain-computer interface (BCI). As MI tasks are ...subject-specific, selection of subject-specific discriminative frequency components play a vital role in distinguishing these patterns. This paper proposes a new discriminative filter bank (FB) common spatial pattern algorithm to extract subject-specific FB for MI classification. The proposed method enhances the classification accuracy in BCI competition III dataset IVa and competition IV dataset IIb. Compared to the performance offered by the existing FB-based method, the proposed algorithm offers error rate reductions of 17.42% and 8.9% for BCI competition datasets III and IV, respectively.
The black-box nature of deep neural networks (DNN) has brought to attention the issues of transparency and fairness. Deep Reinforcement Learning (Deep RL or DRL), which uses DNN to learn its policy, ...value functions etc, is thus also subject to similar concerns. This paper proposes a way to circumvent the issues through the bottom-up design of neural networks with detailed interpretability, where each neuron or layer has its own meaning and utility that corresponds to humanly understandable concept. The framework introduced in this paper is called the Self Reward Design (SRD), inspired by the Inverse Reward Design, and this interpretable design can (1) solve the problem by pure design (although imperfectly) and (2) be optimized like a standard DNN. With deliberate human designs, we show that some RL problems such as lavaland and MuJoCo can be solved using a model constructed with standard NN components with few parameters. Furthermore, with our fish sale auction example, we demonstrate how SRD is used to address situations that will not make sense if black-box models are used, where humanly-understandable semantic-based decision is required.
In electroencephalography (EEG) classification paradigms, data from a target subject is often difficult to obtain, leading to difficulties in training a robust deep learning network. Transfer ...learning and their variations are effective tools in improving such models suffering from lack of data. However, many of the proposed variations and deep models often rely on a single assumed distribution to represent the latent features which may not scale well due to inter- and intra-subject variations in signals. This leads to significant instability in individual subject decoding performances. The presence of non-trivial domain differences between different sets of training or transfer learning data causes poorer model generalization towards the target subject. However, the detection of these domain differences is often difficult to perform due to the ill-defined nature of the EEG domain features. This study proposes a novel inference model, the Joint Embedding Variational Autoencoder, that offers conditionally tighter approximation of the estimated spatiotemporal feature distribution through the use of jointly optimised variational autoencoders to achieve optimizable data dependent inputs as an additional variable for improved overall model optimisation and scaling without sacrificing model tightness. To learn the variational bound, we show that maximising the marginal log-likelihood of only the second embedding section is required to achieve conditionally tighter lower bounds. Furthermore, we show that this model provides state-of-the-art EEG data reconstruction and deep feature extraction. The extracted domains of the EEG signals across each subject displays the rationale as to why there exists disparity between subjects' adaptation efficacy.
In recent years, electromyography (EMG)-based practical myoelectric interfaces have been developed to improve the quality of daily life for people with physical disabilities. With these interfaces, ...it is very important to decode a user's movement intention, to properly control the external devices. However, improving the performance of these interfaces is difficult due to the high variations in the EMG signal patterns caused by intra-user variability. Therefore, this paper proposes a novel subject-transfer framework for decoding hand movements, which is robust in terms of intra-user variability. In the proposed framework, supportive convolutional neural network (CNN) classifiers, which are pre-trained using the EMG data of several subjects, are selected and fine-tuned for the target subject via single-trial analysis. Then, the target subject's hand movements are classified by voting the outputs of the supportive CNN classifiers. The feasibility of the proposed framework is validated with NinaPro databases 2 and 3, which comprise 49 hand movements of 40 healthy and 11 amputee subjects, respectively. The experimental results indicate that, when compared to the self-decoding framework, which uses only the target subject's data, the proposed framework can successfully decode hand movements with improved performance in both healthy and amputee subjects. From the experimental results, the proposed subject-transfer framework can be seen to represent a useful tool for EMG-based practical myoelectric interfaces controlling external devices.
Multichannel EEG is generally used in brain-computer interfaces (BCIs), whereby performing EEG channel selection 1) improves BCI performance by removing irrelevant or noisy channels and 2) enhances ...user convenience from the use of lesser channels. This paper proposes a novel sparse common spatial pattern (SCSP) algorithm for EEG channel selection. The proposed SCSP algorithm is formulated as an optimization problem to select the least number of channels within a constraint of classification accuracy. As such, the proposed approach can be customized to yield the best classification accuracy by removing the noisy and irrelevant channels, or retain the least number of channels without compromising the classification accuracy obtained by using all the channels. The proposed SCSP algorithm is evaluated using two motor imagery datasets, one with a moderate number of channels and another with a large number of channels. In both datasets, the proposed SCSP channel selection significantly reduced the number of channels, and outperformed existing channel selection methods based on Fisher criterion, mutual information, support vector machine, common spatial pattern, and regularized common spatial pattern in classification accuracy. The proposed SCSP algorithm also yielded an average improvement of 10% in classification accuracy compared to the use of three channels (C3, C4, and Cz).
PDF
