Emotions play a critical role in rational and intelligent behavior; a better fundamental knowledge of them is indispensable for understanding higher order brain function. We propose a non-invasive ...brain-computer interface (BCI) system to feedback a person's affective state such that a closed-loop interaction between the participant's brain responses and the musical stimuli is established. We realized this concept technically in a functional prototype of an algorithm that generates continuous and controllable patterns of synthesized affective music in real-time, which is embedded within a BCI architecture. We evaluated our concept in two separate studies. In the first study, we tested the efficacy of our music algorithm by measuring subjective affective responses from 11 participants. In a second pilot study, the algorithm was embedded in a real-time BCI architecture to investigate affective closed-loop interactions in 5 participants. Preliminary results suggested that participants were able to intentionally modulate the musical feedback by self-inducing emotions (e.g., by recalling memories), suggesting that the system was able not only to capture the listener's current affective state in real-time, but also potentially provide a tool for listeners to mediate their own emotions by interacting with music. The proposed concept offers a tool to study emotions in the loop, promising to cast a complementary light on emotion-related brain research, particularly in terms of clarifying the interactive, spatio-temporal dynamics underlying affective processing in the brain.
Display omitted
•Familiarization with music means listening to unfamiliar music over and over.•Sustained increased theta and gamma power during gradual familiarization with music.•Increased theta ERD ...suggests the identification of familiar musical sequences.•Increased gamma ERD suggests forming unfamiliar musical sequences.•The effects are dynamically sustained during passive listening to music.
Repeated listening to unknown music leads to gradual familiarization with musical sequences. Passively listening to musical sequences could involve an array of dynamic neural responses in reaching familiarization with the musical excerpts. This study elucidates the dynamic brain response and its variation over time by investigating the electrophysiological changes during the familiarization with initially unknown music. Twenty subjects were asked to familiarize themselves with previously unknown 10 s classical music excerpts over three repetitions while their electroencephalogram was recorded. Dynamic spectral changes in neural oscillations are monitored by time–frequency analyses for all frequency bands (theta: 5–9 Hz, alpha: 9–13 Hz, low-beta: 13–21 Hz, high beta: 21–32 Hz, and gamma: 32–50 Hz). Time-frequency analyses reveal sustained theta event-related desynchronization (ERD) in the frontal-midline and the left pre-frontal electrodes which decreased gradually from 1st to 3rd time repetition of the same excerpts (frontal-midline: 57.90 %, left-prefrontal: 75.93 %). Similarly, sustained gamma ERD decreased in the frontal-midline and bilaterally frontal/temporal areas (frontal-midline: 61.47 %, left-frontal: 90.88 %, right-frontal: 87.74 %). During familiarization, the decrease of theta ERD is superior in the first part (1–5 s) whereas the decrease of gamma ERD is superior in the second part (5–9 s) of music excerpts. The results suggest that decreased theta ERD is associated with successfully identifying familiar sequences, whereas decreased gamma ERD is related to forming unfamiliar sequences.
Human-robot interaction (HRI) describes scenarios in which both human and robot work as partners, sharing the same environment or complementing each other on a joint task. HRI is characterized by the ...need for high adaptability and flexibility of robotic systems toward their human interaction partners. One of the major challenges in HRI is task planning with dynamic subtask assignment, which is particularly challenging when subtask choices of the human are not readily accessible by the robot. In the present work, we explore the feasibility of using electroencephalogram (EEG) based neuro-cognitive measures for online robot learning of dynamic subtask assignment. To this end, we demonstrate in an experimental human subject study, featuring a joint HRI task with a UR10 robotic manipulator, the presence of EEG measures indicative of a human partner anticipating a takeover situation from human to robot or vice-versa. The present work further proposes a reinforcement learning based algorithm employing these measures as a neuronal feedback signal from the human to the robot for dynamic learning of subtask-assignment. The efficacy of this algorithm is validated in a simulation-based study. The simulation results reveal that even with relatively low decoding accuracies, successful robot learning of subtask-assignment is feasible, with around 80% choice accuracy among four subtasks within 17 minutes of collaboration. The simulation results further reveal that scalability to more subtasks is feasible and mainly accompanied with longer robot learning times. These findings demonstrate the usability of EEG-based neuro-cognitive measures to mediate the complex and largely unsolved problem of human-robot collaborative task planning.
When a human and machine collaborate on a shared task, ambiguous events might occur that could be perceived as an error by the human partner. In such events, spontaneous error-related potentials ...(ErrPs) are evoked in the human brain. Knowing whom the human perceived as responsible for the error would help a machine in co-adaptation and shared control paradigms to better adapt to human preferences. Therefore, we ask whether self- and agent-related errors evoke different ErrPs. Eleven subjects participated in an electroencephalography human-agent collaboration experiment with a collaborative trajectory-following task on two collaboration levels, where movement errors occurred as trajectory deviations. Independently of the collaboration level, we observed a higher amplitude of the responses on the midline central Cz electrode for self-related errors compared to observed errors made by the agent. On average, Support Vector Machines classified self- and agent-related errors with 72.64% accuracy using subject-specific features. These results demonstrate that ErrPs can tell if a person relates an error to themselves or an external autonomous agent during collaboration. Thus, the collaborative machine will receive more informed feedback for the error attribution that allows appropriate error identification, a possibility for correction, and avoidance in future actions.
How the brain responds temporally and spectrally when we listen to familiar versus unfamiliar musical sequences remains unclear. This study uses EEG techniques to investigate the continuous ...electrophysiological changes in the human brain during passive listening to familiar and unfamiliar musical excerpts. EEG activity was recorded in 20 participants while they passively listened to 10 s of classical music, and they were then asked to indicate their self-assessment of familiarity. We analyzed the EEG data in two manners: familiarity based on the within-subject design, i.e., averaging trials for each condition and participant, and familiarity based on the same music excerpt, i.e., averaging trials for each condition and music excerpt. By comparing the familiar condition with the unfamiliar condition and the local baseline, sustained low-beta power (12-16 Hz) suppression was observed in both analyses in fronto-central and left frontal electrodes after 800 ms. However, sustained alpha power (8-12 Hz) decreased in fronto-central and posterior electrodes after 850 ms only in the first type of analysis. Our study indicates that listening to familiar music elicits a late sustained spectral response (inhibition of alpha/low-beta power from 800 ms to 10 s). Moreover, the results showed that alpha suppression reflects increased attention or arousal/engagement due to listening to familiar music; nevertheless, low-beta suppression exhibits the effect of familiarity.
This study differentiates the dynamic temporal-spectral effects during listening to 10 s of familiar music compared with unfamiliar music. This study highlights that listening to familiar music leads to continuous suppression in the alpha and low-beta bands. This suppression starts ∼800 ms after the stimulus onset.
Validating human–robot interaction can be a challenging task, especially in cases in which the robot designer is interested in the assessment of individual robot actions within an ongoing interaction ...that should not be interrupted by intermittent surveys. In this paper, we propose a neuro-based method for real-time quantitative assessment of robot actions. The method encompasses the decoding of error-related potentials (ErrPs) from the electroencephalogram (EEG) of a human during interaction with a robot, which could be a useful and intuitive complement to existing methods for validating human–robot interaction in the future. To demonstrate usability, we conducted a study in which we examined EEG-based ErrPs in response to a humanoid robot displaying semantically incorrect actions in a simplistic HRI task. Furthermore, we conducted a procedurally identical control experiment with computer screen-based symbolic cursor action. The results of our study confirmed decodeability of ErrPs in response to incorrect robot actions with an average accuracy of
69.0
±
7.9
%
across 11 subjects. Cross-comparisons of ErrPs between experimental tasks revealed high temporal and topographical similarity, but more distinct signals in response to the cursor action and, as a result, better decodeability with a mean accuracy of
90.6
±
3.9
%
. This demonstrated that ErrPs can be sensitive to the stimulus eliciting them despite procedurally identical protocols. Re-using ErrP-decoders across experimental tasks without re-calibration is accompanied by significant performance losses and therefore not recommended. Overall, the outcomes of our study confirm feasibility of ErrP-decoding for human–robot validation, but also highlight challenges to overcome in order to enhance usability of the proposed method.
Anorexia nervosa (AN) is a serious eating disorder characterized by self-starvation and extreme weight loss. Pseudoatrophic brain changes are often readily visible in individual brain scans, and AN ...may be a valuable model disorder to study structural neuroplasticity. Structural magnetic resonance imaging studies have found reduced gray matter volume and cortical thinning in acutely underweight patients to normalize following successful treatment. However, some well-controlled studies have found regionally greater gray matter and persistence of structural alterations following long-term recovery. Findings from diffusion tensor imaging studies of white matter integrity and connectivity are also inconsistent. Furthermore, despite the severity of AN, the number of existing structural neuroimaging studies is still relatively low, and our knowledge of the underlying cellular and molecular mechanisms for macrostructural brain changes is rudimentary. We critically review the current state of structural neuroimaging in AN and discuss the potential neurobiological basis of structural brain alterations in the disorder, highlighting impediments to progress, recent developments, and promising future directions. In particular, we argue for the utility of more standardized data collection, adopting a connectomics approach to understanding brain network architecture, employing advanced magnetic resonance imaging methods that quantify biomarkers of brain tissue microstructure, integrating data from multiple imaging modalities, strategic longitudinal observation during weight restoration, and large-scale data pooling. Our overarching objective is to motivate carefully controlled research of brain structure in eating disorders, which will ultimately help predict therapeutic response and improve treatment.
Given the difficulty of procuring human brain tissue, a key question in molecular psychiatry concerns the extent to which epigenetic signatures measured in more accessible tissues such as blood can ...serve as a surrogate marker for the brain. Here, we aimed (1) to investigate the blood-brain correspondence of DNA methylation using a within-subject design and (2) to identify changes in DNA methylation of brain-related biological pathways in schizophrenia.We obtained paired blood and temporal lobe biopsy samples simultaneously from 12 epilepsy patients during neurosurgical treatment. Using the Infinium 450K methylation array we calculated similarity of blood and brain DNA methylation for each individual separately. We applied our findings by performing gene set enrichment analyses (GSEA) of peripheral blood DNA methylation data (Infinium 27K) of 111 schizophrenia patients and 122 healthy controls and included only Cytosine-phosphate-Guanine (CpG) sites that were significantly correlated across tissues.Only 7.9% of CpG sites showed a statistically significant, large correlation between blood and brain tissue, a proportion that although small was significantly greater than predicted by chance. GSEA analysis of schizophrenia data revealed altered methylation profiles in pathways related to precursor metabolites and signaling peptides.Our findings indicate that most DNA methylation markers in peripheral blood do not reliably predict brain DNA methylation status. However, a subset of peripheral data may proxy methylation status of brain tissue. Restricting the analysis to these markers can identify meaningful epigenetic differences in schizophrenia and potentially other brain disorders.
Soft exosuits offer promise to support users in everyday workload tasks by providing assistance. However, acceptance of such systems remains low due to the difficulty of control compared with rigid ...mechatronic systems. Recently, there has been progress in developing control schemes for soft exosuits that move in line with user intentions. While initial results have demonstrated sufficient device performance, the assessment of user experience via the cognitive response has yet to be evaluated. To address this, we propose a soft pneumatic elbow exosuit designed based on our previous work to provide assistance in line with user expectations utilizing two existing state-of-the-art control methods consisting of a gravity compensation and myoprocessor based on muscle activation. A user experience study was conducted to assess whether the device moves naturally with user expectations and the potential for device acceptance by determining when the exosuit violated user expectations through the neuro-cognitive and motor response. Brain activity from electroencephalography (EEG) data revealed that subjects elicited error-related potentials (ErrPs) in response to unexpected exosuit actions, which were decodable across both control schemes with an average accuracy of 76.63 ± 1.73% across subjects. Additionally, unexpected exosuit actions were further decoded via the motor response from electromyography (EMG) and kinematic data with a grand average accuracy of 68.73 ± 6.83% and 77.52 ± 3.79% respectively. This work demonstrates the validation of existing state-of-the-art control schemes for soft wearable exosuits through the proposed soft pneumatic elbow exosuit. We demonstrate the feasibility of assessing device performance with respect to the cognitive response through decoding when the device violates user expectations in order to help understand and promote device acceptance.
PDF
