A brain–computer interface (BCI) establishes a direct communication channel between a brain and an external device. With recent advances in neurotechnology and artificial intelligence (AI), the brain ...signals in BCI communication have been advanced from sensation and perception to higher-level cognition activities. While the field of BCI has grown rapidly in the past decades, the core technologies and innovative ideas behind seemingly unrelated BCI systems have never been summarized from an evolutionary point of view. Here, we review various BCI paradigms and present an evolutionary model of generalized BCI technology which comprises three stages: interface, interaction, and intelligence (I3). We also highlight challenges, opportunities, and future perspectives in the development of new BCI technology.
Classical brain–computer interface (BCI) systems are moving beyond lab demonstrations to real world applications with the development of associated hardware and software.Brain–computer interaction systems open up a wide range of BCI applications, especially in neural rehabilitation and human cognitive augmentation.Brain–computer intelligence systems reveal promising technologies for a new generation of artificial intelligence (AI) as well as a new generation of BCIs.
This paper presents a benchmark steady-state visual evoked potential (SSVEP) dataset acquired with a 40-target brain- computer interface (BCI) speller. The dataset consists of 64-channel ...Electroencephalogram (EEG) data from 35 healthy subjects (8 experienced and 27 naïve) while they performed a cue-guided target selecting task. The virtual keyboard of the speller was composed of 40 visual flickers, which were coded using a joint frequency and phase modulation (JFPM) approach. The stimulation frequencies ranged from 8 Hz to 15.8 Hz with an interval of 0.2 Hz. The phase difference between two adjacent frequencies was 0.5π. For each subject, the data included six blocks of 40 trials corresponding to all 40 flickers indicated by a visual cue in a random order. The stimulation duration in each trial was five seconds. The dataset can be used as a benchmark dataset to compare the methods for stimulus coding and target identification in SSVEP-based BCIs. Through offline simulation, the dataset can be used to design new system diagrams and evaluate their BCI performance without collecting any new data. The dataset also provides high-quality data for computational modeling of SSVEPs. The dataset is freely available from http://bci.med.tsinghua.edu.cn/download.html.
Robust Color Guided Depth Map Restoration Liu, Wei; Chen, Xiaogang; Yang, Jie ...
IEEE transactions on image processing,
2017-Jan., 2017-Jan, 2017-1-00, 20170101, Letnik:
26, Številka:
1
Journal Article
Recenzirano
One of the most challenging issues in color guided depth map restoration is the inconsistency between color edges in guidance color images and depth discontinuities on depth maps. This makes the ...restored depth map suffer from texture copy artifacts and blurring depth discontinuities. To handle this problem, most state-of-the-art methods design complex guidance weight based on guidance color images and heuristically make use of the bicubic interpolation of the input depth map. In this paper, we show that using bicubic interpolated depth map can blur depth discontinuities when the upsampling factor is large and the input depth map contains large holes and heavy noise. In contrast, we propose a robust optimization framework for color guided depth map restoration. By adopting a robust penalty function to model the smoothness term of our model, we show that the proposed method is robust against the inconsistency between color edges and depth discontinuities even when we use simple guidance weight. To the best of our knowledge, we are the first to solve this problem with a principled mathematical formulation rather than previous heuristic weighting schemes. The proposed robust method performs well in suppressing texture copy artifacts. Moreover, it can better preserve sharp depth discontinuities than previous heuristic weighting schemes. Through comprehensive experiments on both simulated data and real data, we show promising performance of the proposed method.
Recent advances in robotics, neuroscience, and signal processing make it possible to operate a robot through electroencephalography (EEG)-based brain-computer interface (BCI). Although some ...successful attempts have been made in recent years, the practicality of the entire system still has much room for improvement. The present study designed and realized a robotic arm control system by combing augmented reality (AR), computer vision, and steady-state visual evoked potential (SSVEP)-BCI. AR environment was implemented by a Microsoft HoloLens. Flickering stimuli for eliciting SSVEPs were presented on the HoloLens, which allowed users to see both the robotic arm and the user interface of the BCI. Thus users did not need to switch attention between the visual stimulator and the robotic arm. A four-command SSVEP-BCI was built for users to choose the specific object to be operated by the robotic arm. Once an object was selected, the computer vision would provide the location and color of the object in the workspace. Subsequently, the object was autonomously picked up and placed by the robotic arm. According to the online results obtained from twelve participants, the mean classification accuracy of the proposed system was 93.96 ± 5.05%. Moreover, all subjects could utilize the proposed system to successfully pick and place objects in a specific order. These results demonstrated the potential of combining AR-BCI and computer vision to control robotic arms, which is expected to further promote the practicality of BCI-controlled robots.
Objective: This study proposes and evaluates a novel data-driven spatial filtering approach for enhancing steady-state visual evoked potentials (SSVEPs) detection toward a high-speed brain-computer ...interface (BCI) speller. Methods: Task-related component analysis (TRCA), which can enhance reproducibility of SSVEPs across multiple trials, was employed to improve the signal-to-noise ratio (SNR) of SSVEP signals by removing background electroencephalographic (EEG) activities. An ensemble method was further developed to integrate TRCA filters corresponding to multiple stimulation frequencies. This study conducted a comparison of BCI performance between the proposed TRCA-based method and an extended canonical correlation analysis (CCA)-based method using a 40-class SSVEP dataset recorded from 12 subjects. An online BCI speller was further implemented using a cue-guided target selection task with 20 subjects and a free-spelling task with 10 of the subjects. Results: The offline comparison results indicate that the proposed TRCA-based approach can significantly improve the classification accuracy compared with the extended CCA-based method. Furthermore, the online BCI speller achieved averaged information transfer rates (ITRs) of 325.33 ± 38.17 bits/min with the cue-guided task and 198.67 ± 50.48 bits/min with the free-spelling task. Conclusion: This study validated the efficiency of the proposed TRCA-based method in implementing a high-speed SSVEP-based BCI. Significance: The high-speed SSVEP-based BCIs using the TRCA method have great potential for various applications in communication and control.
The past 20 years have witnessed unprecedented progress in brain–computer interfaces (BCIs). However, low communication rates remain key obstacles to BCI-based communication in humans. This study ...presents an electroencephalogram-based BCI speller that can achieve information transfer rates (ITRs) up to 5.32 bits per second, the highest ITRs reported in BCI spellers using either noninvasive or invasive methods. Based on extremely high consistency of frequency and phase observed between visual flickering signals and the elicited single-trial steady-state visual evoked potentials, this study developed a synchronous modulation and demodulation paradigm to implement the speller. Specifically, this study proposed a new joint frequency-phase modulation method to tag 40 characters with 0.5-s-long flickering signals and developed a user-specific target identification algorithm using individual calibration data. The speller achieved high ITRs in online spelling tasks. This study demonstrates that BCIs can provide a truly naturalistic high-speed communication channel using noninvasively recorded brain activities.
Objective. Recently, canonical correlation analysis (CCA) has been widely used in steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) due to its high efficiency, ...robustness, and simple implementation. However, a method with which to make use of harmonic SSVEP components to enhance the CCA-based frequency detection has not been well established. Approach. This study proposed a filter bank canonical correlation analysis (FBCCA) method to incorporate fundamental and harmonic frequency components to improve the detection of SSVEPs. A 40-target BCI speller based on frequency coding (frequency range: 8-15.8 Hz, frequency interval: 0.2 Hz) was used for performance evaluation. To optimize the filter bank design, three methods (M1: sub-bands with equally spaced bandwidths; M2: sub-bands corresponding to individual harmonic frequency bands; M3: sub-bands covering multiple harmonic frequency bands) were proposed for comparison. Classification accuracy and information transfer rate (ITR) of the three FBCCA methods and the standard CCA method were estimated using an offline dataset from 12 subjects. Furthermore, an online BCI speller adopting the optimal FBCCA method was tested with a group of 10 subjects. Main results. The FBCCA methods significantly outperformed the standard CCA method. The method M3 achieved the highest classification performance. At a spelling rate of ∼33.3 characters min, the online BCI speller obtained an average ITR of 151.18 20.34 bits min−1. Significance. By incorporating the fundamental and harmonic SSVEP components in target identification, the proposed FBCCA method significantly improves the performance of the SSVEP-based BCI, and thereby facilitates its practical applications such as high-speed spelling.
The brain-computer interface (BCI) provides an alternative means to communicate and it has sparked growing interest in the past two decades. Specifically, for Steady-State Visual Evoked Potential ...(SSVEP) based BCI, marked improvement has been made in the frequency recognition method and data sharing. However, the number of pubic databases is still limited in this field. Therefore, we present a BEnchmark database Towards BCI Application (BETA) in the study. The BETA database is composed of 64-channel Electroencephalogram (EEG) data of 70 subjects performing a 40-target cued-spelling task. The design and the acquisition of the BETA are in pursuit of meeting the demand from real-world applications and it can be used as a test-bed for these scenarios. We validate the database by a series of analyses and conduct the classification analysis of eleven frequency recognition methods on BETA. We recommend using the metric of wide-band signal-to-noise ratio (SNR) and BCI quotient to characterize the SSVEP at the single-trial and population levels, respectively. The BETA database can be downloaded from the following link http://bci.med.tsinghua.edu.cn/download.html.
The practical functionality of a brain-computer interface (BCI) is critically affected by the number of stimuli, especially for steady-state visual evoked potential based BCI (SSVEP-BCI), which shows ...promise for the implementation of a multi-target system for real-world applications. Joint frequency-phase modulation (JFPM) is an effective and widely used method in modulating SSVEPs. However, the ability of JFPM to implement an SSVEP-BCI system with a large number of stimuli, e.g., over 100 stimuli, remains unclear. To address this issue, a spectrally-dense JPFM (sJFPM) method is proposed to encode a broad array of stimuli, which modulates the low-and medium-frequency SSVEPs with a frequency interval of 0.1 Hz and triples the number of stimuli in conventional SSVEP-BCI to 120. To validate the effectiveness of the proposed 120-target BCI system, an offline experiment and a subsequent online experiment testing 18 healthy subjects in total were conducted. The offline experiment verified the feasibility of using sJFPM in designing an SSVEP-BCI system with 120 stimuli. Furthermore, the online experiment demonstrated that the proposed system achieved an average performance of 92.47±1.83% in online accuracy and 213.23±6.60 bits/min in online information transfer rate (ITR), where more than 75% of the subjects attained the accuracy above 90% and the ITR above 200 bits/min. This present study demonstrates the effectiveness of sJFPM in elevating the number of stimuli to more than 100 and extends our understanding of encoding a large number of stimuli by means of finer frequency division.
Objective. A new training-free framework was proposed for target detection in steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) using joint frequency-phase coding. ...Approach. The key idea is to transfer SSVEP templates from the existing subjects to a new subject to enhance the detection of SSVEPs. Under this framework, transfer template-based canonical correlation analysis (tt-CCA) methods were developed for single-channel and multi-channel conditions respectively. In addition, an online transfer template-based CCA (ott-CCA) method was proposed to update EEG templates by online adaptation. Main results. The efficiency of the proposed framework was proved with a simulated BCI experiment. Compared with the standard CCA method, tt-CCA obtained an 18.78% increase of accuracy with a data length of 1.5 s. A simulated test of ott-CCA further received an accuracy increase of 2.99%. Significance. The proposed simple yet efficient framework significantly facilitates the use of SSVEP BCIs using joint frequency-phase coding. This study also sheds light on the benefits from exploring and exploiting inter-subject information to the electroencephalogram (EEG)-based BCIs.
PDF
