Brain-machine interfaces (BMIs) can be used to decode brain activity into commands to control external devices. This paper presents the decoding of intuitive upper extremity imagery for ...multi-directional arm reaching tasks in three-dimensional (3D) environments. We designed and implemented an experimental environment in which electroencephalogram (EEG) signals can be acquired for movement execution and imagery. Fifteen subjects participated in our experiments. We proposed a multi-directional convolution neural network-bidirectional long short-term memory network (MDCBN)-based deep learning framework. The decoding performances for six directions in 3D space were measured by the correlation coefficient (CC) and the normalized root mean square error (NRMSE) between predicted and baseline velocity profiles. The grand-averaged CCs of multi-direction were 0.47 and 0.45 for the execution and imagery sessions, respectively, across all subjects. The NRMSE values were below 0.2 for both sessions. Furthermore, in this study, the proposed MDCBN was evaluated by two online experiments for real-time robotic arm control, and the grand-averaged success rates were approximately 0.60 (±0.14) and 0.43 (±0.09), respectively. Hence, we demonstrate the feasibility of intuitive robotic arm control based on EEG signals for real-world environments.
In this paper, a closed-loop control has been developed for the exoskeleton robot system based on brain-machine interface (BMI). Adaptive controllers in joint space, a redundancy resolution method at ...the velocity level, and commands that generated from BMI in task space have been integrated effectively to make the robot perform manipulation tasks controlled by human operator's electroencephalogram. By extracting the features from neural activity, the proposed intention decoding algorithm can generate the commands to control the exoskeleton robot. To achieve optimal motion, a redundancy resolution at the velocity level has been implemented through neural dynamics optimization. Considering human-robot interaction force as well as coupled dynamics during the exoskeleton operation, an adaptive controller with redundancy resolution has been designed to drive the exoskeleton tracking the planned trajectory in human brain and to offer a convenient method of dynamics compensation with minimal knowledge of the dynamics parameters of the exoskeleton robot. Extensive experiments which employed a few subjects have been carried out. In the experiments, subjects successfully fulfilled the given manipulation tasks with convergence of tracking errors, which verified that the proposed brain-controlled exoskeleton robot system is effective.
Recognizing the very size of the brain's circuits, hyperdimensional (HD) computing can model neural activity patterns with points in a HD space, that is, with HD vectors. Key examined properties of ...HD computing include: a versatile set of arithmetic operations on HD vectors, generality, scalability, analyzability, one-shot learning, and energy efficiency. These make it a prime candidate for efficient biosignal processing where signals are noisy and nonstationary, training data sets are not huge, individual variability is significant, and energy-efficiency constraints are tight. Purely based on native HD computing operators, we describe a combined method for multiclass learning and classification of various ExG biosignals such as electromyography (EMG), electroencephalography (EEG), and electrocorticography (ECoG). We develop a full set of HD network templates that comprehensively encode body potentials and brain neural activity recorded from different electrodes into a single HD vector without requiring domain expert knowledge or ad hoc electrode selection process. Such encoded HD vector is processed as a single unit for fast one-shot learning, and robust classification. It can be interpreted to identify the most useful features as well. Compared to state-of-the-art counterparts, HD computing enables online, incremental, and fast learning as it demands less than a third as much training data as well as less preprocessing.
Neurons respond to external stimuli and form functional networks through pairwise interactions. A neural encoding model can describe a single neuron's behavior, and brain-machine interfaces (BMIs) ...provide a platform to investigate how neurons adapt, functionally connect, and encode movement. Movement modulation and pairwise functional connectivity are modeled as high-dimensional tuning states, estimated from neural spike train observations. However, accurate estimation of this neural state vector can be challenging as pairwise neural interactions are highly dimensional, change in different temporal scales from movement, and could be non-stationary. We propose an Adam-based gradient descent method to online estimate high-dimensional pairwise neuronal functional connectivity and single neuronal tuning adaptation simultaneously. By minimizing negative log-likelihood based on point process observation, the proposed method adaptively adjusts the learning rate for each dimension of the neural state vectors by employing momentum and regularizer. We test the method on real recordings of two rats performing the brain control mode of a two-lever discrimination task. Our results show that our method outperforms existing methods, especially when the state is sparse. Our method is more stable and faster for an online scenario regardless of the parameter initializations. Our method provides a promising tool to track and build the time-variant functional neural connectivity, which dynamically forms the functional network and results in better brain control.
Human–Machine Interfaces can be very useful to improve the quality of life of physically impaired users. In this work, a non-invasive spontaneous Brain–Machine Interface (BMI) has been designed to ...control a robot arm through the mental activity of the users. This BMI uses the classification of four mental tasks in order to manage the movements of the robot. The high accuracy in the classification of these tasks (around 70%) allows a quick accomplishment of the experiment designed, even with the low signal-to-noise ratio of this kind of signals. The experiment consists of reaching four points in the workspace of an industrial robot in the established order. After a brief training, the volunteers are able to control the robot in a real time activity. The real time test shows that the system can be applied to do more complex activity such as pick and place tasks if a supplementary system is added. These interfaces are very adequate in the control of rehabilitation or assistance systems for people suffering from motor impairment.
Arrays of electrodes for recording and stimulating the brain are used throughout clinical medicine and basic neuroscience research, yet are unable to sample large areas of the brain while maintaining ...high spatial resolution because of the need to individually wire each passive sensor at the electrode-tissue interface. To overcome this constraint, we developed new devices that integrate ultrathin and flexible silicon nanomembrane transistors into the electrode array, enabling new dense arrays of thousands of amplified and multiplexed sensors that are connected using fewer wires. We used this system to record spatial properties of cat brain activity in vivo, including sleep spindles, single-trial visual evoked responses and electrographic seizures. We found that seizures may manifest as recurrent spiral waves that propagate in the neocortex. The developments reported here herald a new generation of diagnostic and therapeutic brain-machine interface devices.
Brain-machine interfaces (BMIs) aim to treat sensorimotor neurological disorders by creating artificial motor and/or sensory pathways. Introducing artificial pathways creates new relationships ...between sensory input and motor output, which the brain must learn to gain dexterous control. This review highlights the role of learning in BMIs to restore movement and sensation, and discusses how BMI design may influence neural plasticity and performance. The close integration of plasticity in sensory and motor function influences the design of both artificial pathways and will be an essential consideration for bidirectional devices that restore both sensory and motor function.
Abstract Stroke is among the leading causes of long-term disabilities leaving an increasing number of people with cognitive, affective and motor impairments depending on assistance in their daily ...life. While function after stroke can significantly improve in the first weeks and months, further recovery is often slow or non-existent in the more severe cases encompassing 30–50% of all stroke victims. The neurobiological mechanisms underlying recovery in those patients are incompletely understood. However, recent studies demonstrated the brain's remarkable capacity for functional and structural plasticity and recovery even in severe chronic stroke. As all established rehabilitation strategies require some remaining motor function, there is currently no standardized and accepted treatment for patients with complete chronic muscle paralysis. The development of brain–machine interfaces (BMIs) that translate brain activity into control signals of computers or external devices provides two new strategies to overcome stroke-related motor paralysis. First, BMIs can establish continuous high-dimensional brain-control of robotic devices or functional electric stimulation (FES) to assist in daily life activities ( assistive BMI). Second, BMIs could facilitate neuroplasticity, thus enhancing motor learning and motor recovery ( rehabilitative BMI). Advances in sensor technology, development of non-invasive and implantable wireless BMI-systems and their combination with brain stimulation, along with evidence for BMI systems' clinical efficacy suggest that BMI-related strategies will play an increasing role in neurorehabilitation of stroke.
In a previous study we demonstrated continuous translation, orientation and one-dimensional grasping control of a prosthetic limb (seven degrees of freedom) by a human subject with tetraplegia using ...a brain-machine interface (BMI). The current study, in the same subject, immediately followed the previous work and expanded the scope of the control signal by also extracting hand-shape commands from the two 96-channel intracortical electrode arrays implanted in the subject's left motor cortex.
Four new control signals, dictating prosthetic hand shape, replaced the one-dimensional grasping in the previous study, allowing the subject to control the prosthetic limb with ten degrees of freedom (three-dimensional (3D) translation, 3D orientation, four-dimensional hand shaping) simultaneously.
Robust neural tuning to hand shaping was found, leading to ten-dimensional (10D) performance well above chance levels in all tests. Neural unit preferred directions were broadly distributed through the 10D space, with the majority of units significantly tuned to all ten dimensions, instead of being restricted to isolated domains (e.g. translation, orientation or hand shape). The addition of hand shaping emphasized object-interaction behavior. A fundamental component of BMIs is the calibration used to associate neural activity to intended movement. We found that the presence of an object during calibration enhanced successful shaping of the prosthetic hand as it closed around the object during grasping.
Our results show that individual motor cortical neurons encode many parameters of movement, that object interaction is an important factor when extracting these signals, and that high-dimensional operation of prosthetic devices can be achieved with simple decoding algorithms. ClinicalTrials.gov Identifier: NCT01364480.
Brain‒machine interface (BMI) is a promising technology that looks set to contribute to the development of artificial limbs and new input devices by integrating various recent technological advances, ...including neural electrodes, wireless communication, signal analysis, and robot control. Neural electrodes are a key technological component of BMI, as they can record the rapid and numerous signals emitted by neurons. To receive stable, consistent, and accurate signals, electrodes are designed in accordance with various templates using diverse materials. With the development of microelectromechanical systems (MEMS) technology, electrodes have become more integrated, and their performance has gradually evolved through surface modification and advances in biotechnology. In this paper, we review the development of the extracellular/intracellular type of in vitro microelectrode array (MEA) to investigate neural interface technology and the penetrating/surface (non-penetrating) type of in vivo electrodes. We briefly examine the history and study the recently developed shapes and various uses of the electrode. Also, electrode materials and surface modification techniques are reviewed to measure high-quality neural signals that can be used in BMI.
PDF
