Parkinson's disease is a movement disorder caused by dopamine depletion in the basal ganglia. Abnormally synchronized neuronal oscillations between 8 and 15 Hz in the basal ganglia are implicated in ...motor symptoms of Parkinson's disease. However, how these abnormal oscillations are generated and maintained in the dopamine-depleted state is unknown. Based on neural recordings in a primate model of Parkinson's disease and other experimental and computational evidence, we hypothesized that the recurrent circuit between the subthalamic nucleus (STN) and the external segment of the globus pallidus (GPe) generates and maintains parkinsonian oscillations, and that the cortical excitatory input to the STN amplifies them. To investigate this hypothesis through computer simulations, we developed a spiking neuron model of the STN-GPe circuit by incorporating electrophysiological properties of neurons and synapses. A systematic parameter search by computer simulation identified regions in the space of the intrinsic excitability of GPe neurons and synaptic strength from the GPe to the STN that reproduce normal and parkinsonian states. In the parkinsonian state, reduced firing of GPe neurons and increased GPe-STN inhibition trigger burst activities of STN neurons with strong post-inhibitory rebound excitation, which is usually subject to short-term depression. STN neuronal bursts are shaped into the 8-15 Hz, synchronous oscillations via recurrent interactions of STN and GPe neurons. Furthermore, we show that cortical excitatory input to the STN can amplify or suppress pathological STN oscillations depending on their phase and strength, predicting conditions of cortical inputs to the STN for suppressing oscillations.
We propose a novel method for multiple clustering, which is useful for analysis of high-dimensional data containing heterogeneous types of features. Our method is based on nonparametric Bayesian ...mixture models in which features are automatically partitioned (into views) for each clustering solution. This feature partition works as feature selection for a particular clustering solution, which screens out irrelevant features. To make our method applicable to high-dimensional data, a co-clustering structure is newly introduced for each view. Further, the outstanding novelty of our method is that we simultaneously model different distribution families, such as Gaussian, Poisson, and multinomial distributions in each cluster block, which widens areas of application to real data. We apply the proposed method to synthetic and real data, and show that our method outperforms other multiple clustering methods both in recovering true cluster structures and in computation time. Finally, we apply our method to a depression dataset with no true cluster structure available, from which useful inferences are drawn about possible clustering structures of the data.
Free-energy based reinforcement learning (FERL) was proposed for learning in high-dimensional state and action spaces. However, the FERL method does only really work well with binary, or close to ...binary, state input, where the number of active states is fewer than the number of non-active states. In the FERL method, the value function is approximated by the negative free energy of a restricted Boltzmann machine (RBM). In our earlier study, we demonstrated that the performance and the robustness of the FERL method can be improved by scaling the free energy by a constant that is related to the size of network. In this study, we propose that RBM function approximation can be further improved by approximating the value function by the negative expected energy (EERL), instead of the negative free energy, as well as being able to handle continuous state input. We validate our proposed method by demonstrating that EERL: (1) outperforms FERL, as well as standard neural network and linear function approximation, for three versions of a gridworld task with high-dimensional image state input; (2) achieves new state-of-the-art results in stochastic SZ-Tetris in both model-free and model-based learning settings; and (3) significantly outperforms FERL and standard neural network function approximation for a robot navigation task with raw and noisy RGB images as state input and a large number of actions.
Previous theoretical studies of animal and human behavioral learning have focused on the dichotomy of the value-based strategy using action value functions to predict rewards and the model-based ...strategy using internal models to predict environmental states. However, animals and humans often take simple procedural behaviors, such as the "win-stay, lose-switch" strategy without explicit prediction of rewards or states. Here we consider another strategy, the finite state-based strategy, in which a subject selects an action depending on its discrete internal state and updates the state depending on the action chosen and the reward outcome. By analyzing choice behavior of rats in a free-choice task, we found that the finite state-based strategy fitted their behavioral choices more accurately than value-based and model-based strategies did. When fitted models were run autonomously with the same task, only the finite state-based strategy could reproduce the key feature of choice sequences. Analyses of neural activity recorded from the dorsolateral striatum (DLS), the dorsomedial striatum (DMS), and the ventral striatum (VS) identified significant fractions of neurons in all three subareas for which activities were correlated with individual states of the finite state-based strategy. The signal of internal states at the time of choice was found in DMS, and for clusters of states was found in VS. In addition, action values and state values of the value-based strategy were encoded in DMS and VS, respectively. These results suggest that both the value-based strategy and the finite state-based strategy are implemented in the striatum.
Diffusion-weighted magnetic resonance imaging (dMRI) allows non-invasive investigation of whole-brain connectivity, which can reveal the brain's global network architecture and also abnormalities ...involved in neurological and mental disorders. However, the reliability of connection inferences from dMRI-based fiber tracking is still debated, due to low sensitivity, dominance of false positives, and inaccurate and incomplete reconstruction of long-range connections. Furthermore, parameters of tracking algorithms are typically tuned in a heuristic way, which leaves room for manipulation of an intended result. Here we propose a general data-driven framework to optimize and validate parameters of dMRI-based fiber tracking algorithms using neural tracer data as a reference. Japan's Brain/MINDS Project provides invaluable datasets containing both dMRI and neural tracer data from the same primates. A fundamental difference when comparing dMRI-based tractography and neural tracer data is that the former cannot specify the direction of connectivity; therefore, evaluating the fitting of dMRI-based tractography becomes challenging. The framework implements multi-objective optimization based on the non-dominated sorting genetic algorithm II. Its performance is examined in two experiments using data from ten subjects for optimization and six for testing generalization. The first uses a seed-based tracking algorithm, iFOD2, and objectives for sensitivity and specificity of region-level connectivity. The second uses a global tracking algorithm and a more refined set of objectives: distance-weighted coverage, true/false positive ratio, projection coincidence, and commissural passage. In both experiments, with optimized parameters compared to default parameters, fiber tracking performance was significantly improved in coverage and fiber length. Improvements were more prominent using global tracking with refined objectives, achieving an average fiber length from 10 to 17 mm, voxel-wise coverage of axonal tracts from 0.9 to 15%, and the correlation of target areas from 40 to 68%, while minimizing false positives and impossible cross-hemisphere connections. Optimized parameters showed good generalization capability for test brain samples in both experiments, demonstrating the flexible applicability of our framework to different tracking algorithms and objectives. These results indicate the importance of data-driven adjustment of fiber tracking algorithms and support the validity of dMRI-based tractography, if appropriate adjustments are employed.
Increasingly, national governments across the globe are prioritizing investments in neuroscience. Currently, seven active or in-development national-level brain research initiatives exist, spanning ...four continents. Engaging with the underlying values and ethical concerns that drive brain research across cultural and continental divides is critical to future research. Culture influences what kinds of science are supported and where science can be conducted through ethical frameworks and evaluations of risk. Neuroscientists and philosophers alike have found themselves together encountering perennial questions; these questions are engaged by the field of neuroethics, related to understanding of the nature of the self and identity, the existence and meaning of free will, defining the role of reason in human behavior, and more. With this Perspective article, we aim to prioritize and advance to the foreground a list of neuroethics questions for neuroscientists operating in the context of these international brain initiatives.
Neuroscience is a national priority across the globe necessitating engagement with the underlying cultural and ethical values that drive brain research. We offer a list of neuroethics questions for neuroscientists to advance and accelerate an ethically tenable globalized neuroscience.
•Given today’s advances in artificial intelligence, expectations are rising for building artificial general intelligence (AGI).•We propose an approach focusing on the process of acquisition of ...intelligence, called evolutionary and developmental intelligence (EDI).•This review explores how we can integrate advances in neuroscience, machine learning, and robotics to practically construct EDI systems.
Given the phenomenal advances in artificial intelligence in specific domains like visual object recognition and game playing by deep learning, expectations are rising for building artificial general intelligence (AGI) that can flexibly find solutions in unknown task domains. One approach to AGI is to set up a variety of tasks and design AI agents that perform well in many of them, including those the agent faces for the first time. One caveat for such an approach is that the best performing agent may be just a collection of domain-specific AI agents switched for a given domain. Here we propose an alternative approach of focusing on the process of acquisition of intelligence through active interactions in an environment. We call this approach evolutionary and developmental intelligence (EDI). We first review the current status of artificial intelligence, brain-inspired computing and developmental robotics and define the conceptual framework of EDI. We then explore how we can integrate advances in neuroscience, machine learning, and robotics to construct EDI systems and how building such systems can help us understand animal and human intelligence.
This letter proposes a new reinforcement learning (RL) paradigm that explicitly takes into account input disturbance as well as modeling errors. The use of environmental models in RL is quite popular ...for both off-line learning using simulations and for online action planning. However, the difference between the model and the real environment can lead to unpredictable, and often unwanted, results. Based on the theory of
control, we consider a differential game in which a “disturbing” agent tries to make the worst possible disturbance while a “control” agent tries to make the best control input. The problem is formulated as finding a min-max solution of a value function that takes into account the amount of the reward and the norm of the disturbance. We derive online learning algorithms for estimating the value function and for calculating the worst disturbance and the best control in reference to the value function. We tested the paradigm, which we call robust reinforcement learning (RRL), on the control task of an inverted pendulum. In the linear domain, the policy and the value function learned by online algorithms coincided with those derived analytically by the linear
control theory. For a fully nonlinear swing-up task, RRL achieved robust performance with changes in the pendulum weight and friction, while a standard reinforcement learning algorithm could not deal with these changes. We also applied RRL to the cart-pole swing-up task, and a robust swing-up policy was acquired.
: Reward‐related neural activities have been found in a variety of cortical and subcortical areas by neurophysiological and neuroimaging experiments. Here we present a unified view on how three ...subloops of the corticobasal ganglia network are involved in reward prediction and action selection using different types of information. The motor/premotor‐posterior striatum loop is specialized for action‐based value representation and movement selection. The orbitofrontal–ventral striatum loop is specialized for object‐based value representation and target selection. The lateral prefrontal–anterior striatum loop is specialized for context‐based value representation and context estimation. Furthermore, the medial prefrontal cortex (MPFC) coordinates these multiple value representations and actions at different levels of hierarchy by monitoring the error in predictions.
PDF
