Reinforcement learning allows organisms to predict future outcomes and to update their beliefs about value in the world. The dorsal-lateral prefrontal cortex (dlPFC) integrates information carried by ...reward circuits, which can be used to infer the current state of the world under uncertainty. Here, we explored the dlPFC computations related to updating current beliefs during stochastic reversal learning. We recorded the activity of populations up to 1,000 neurons, simultaneously, in two male macaques while they executed a two-armed bandit reversal learning task. Behavioral analyses using a Bayesian framework showed that animals inferred reversals and switched their choice preference rapidly, rather than slowly updating choice values, consistent with state inference. Furthermore, dlPFC neural populations accurately encoded choice preference switches. These results suggest that prefrontal neurons dynamically encode decisions associated with Bayesian subjective values, highlighting the role of the PFC in representing a belief about the current state of the world.
•Choice preferences switch abruptly, rather than gradually, during reversal learning•Quick switches are consistent with Bayesian state inference•Prefrontal neural populations encode choice preference reversal•Uncertainty in the behavior decreases the accuracy of reversal decoding
Bartolo and Averbeck show that primates infer changes in reward contingencies, consistent with Bayesian inference strategies, incorporating knowledge about task structure to quickly adjust their behavior. Posterior probability estimates correlate with neural ensemble activity in prefrontal cortex, suggesting that the brain implements Bayesian-like mechanisms for state inference.
The metastatic process in breast cancer is related to the expression of the epithelial-to-mesenchymal transition transcription factors (EMT-TFs) SNAIL, SLUG, SIP1 and TWIST1. EMT-TFs and nuclear ...factor-κB (NF-κB) activation have been associated with aggressiveness and metastatic potential in carcinomas. Here, we sought to examine the role of NF-κB in the aggressive properties and regulation of EMT-TFs in human breast cancer cells. Blocking NF-κB/p65 activity by reducing its transcript and protein levels (through siRNA-strategy and dehydroxymethylepoxyquinomicin DHMEQ treatment) in the aggressive MDA-MB-231 and HCC-1954 cell lines resulted in decreased invasiveness and migration, a downregulation of SLUG, SIP1, TWIST1, MMP11 and N-cadherin transcripts and an upregulation of E-cadherin transcripts. No significant changes were observed in the less aggressive cell line MCF-7. Bioinformatics tools identified several NF-κB binding sites along the promoters of SNAIL, SLUG, SIP1 and TWIST1 genes. Through chromatin immunoprecipitation and luciferase reporter assays, the NF-κB/p65 binding on TWIST1, SLUG and SIP1 promoter regions was confirmed. Thus, we suggest that NF-κB directly regulates the transcription of EMT-TF genes in breast cancer. Our findings may contribute to a greater understanding of the metastatic process of this neoplasia and highlight NF-κB as a potential target for breast cancer treatment.
Tick-borne diseases are common occurrences in both the medical and veterinary clinical settings. In addition to the constraints related to their diagnosis and clinical management, the control and ...prevention of these diseases is often difficult, because it requires the disruption of a complex transmission chain, involving vertebrate hosts and ticks, which interact in a constantly changing environment. We provide a contemporary review of representative tick-borne diseases of humans and discuss aspects linked to their medical relevance worldwide. Finally, we emphasize the importance of a One Health approach to tick-borne diseases, calling physicians and veterinarians to unify their efforts in the management of these diseases, several of which are zoonoses.
The role that frontal-striatal circuits play in normal behavior remains unclear. Two of the leading hypotheses suggest that these circuits are important for action selection or reinforcement ...learning. To examine these hypotheses, we carried out an experiment in which monkeys had to select actions in two different task conditions. In the first (random) condition, actions were selected on the basis of perceptual inference. In the second (fixed) condition, the animals used reinforcement from previous trials to select actions. Examination of neural activity showed that the representation of the selected action was stronger in lateral prefrontal cortex (lPFC), and occurred earlier in the lPFC than it did in the dorsal striatum (dSTR). In contrast to this, the representation of action values, in both the random and fixed conditions, was stronger in the dSTR. Thus, the dSTR contains an enriched representation of action value, but it followed frontal cortex in action selection.
► Prefrontal cortex does action selection ► The striatum represents action value
Action selection and action value are often attributed to the striatum. Here, Seo et al. find that actions based on perceptual inference are selected in the lateral prefrontal cortex before the dorsal striatum. The dorsal striatum, however, represents action values across task conditions.
SignificanceAdolescence is a period during which there are important changes in behavior and the structure of the brain. In this manuscript, we use theoretical modeling to show how improvements in ...working memory and reinforcement learning that occur during adolescence can be explained by the reduction in synaptic connectivity in prefrontal cortex that occurs during a similar period. We train recurrent neural networks to solve working memory and reinforcement learning tasks and show that when we prune connectivity in these networks, they perform the tasks better. The improvement in task performance, however, can come at the cost of flexibility as the pruned networks are not able to learn some new tasks as well.
The explore-exploit dilemma refers to the challenge of deciding when to forego immediate rewards and explore new opportunities that could lead to greater rewards in the future. While motivational ...neural circuits facilitate learning based on past choices and outcomes, it is unclear whether they also support computations relevant for deciding when to explore. We recorded neural activity in the amygdala and ventral striatum of rhesus macaques as they solved a task that required them to balance novelty-driven exploration with exploitation of what they had already learned. Using a partially observable Markov decision process (POMDP) model to quantify explore-exploit trade-offs, we identified that the ventral striatum and amygdala differ in how they represent the immediate value of exploitative choices and the future value of exploratory choices. These findings show that subcortical motivational circuits are important in guiding explore-exploit decisions.
•Monkeys use sophisticated choice strategies to manage explore-exploit trade-offs•Amygdala and ventral striatum neurons code both exploratory and exploitive choices•Task states and state transitions are encoded in the amygdala and ventral striatum
How do we decide whether to explore a new opportunity or stick with what we know? Costa et al. reveal that neurons in amygdala and ventral striatum, motivational centers of the brain, help to solve this complex reinforcement learning problem.
Choice impulsivity is characterized by the choice of immediate, smaller reward options over future, larger reward options, and is often thought to be associated with negative life outcomes. However, ...some environments make future rewards more uncertain, and in these environments impulsive choices can be beneficial. Here we examined the conditions under which impulsive vs. non-impulsive decision strategies would be advantageous. We used Markov Decision Processes (MDPs) to model three common decision-making tasks: Temporal Discounting, Information Sampling, and an Explore-Exploit task. We manipulated environmental variables to create circumstances where future outcomes were relatively uncertain. We then manipulated the discount factor of an MDP agent, which affects the value of immediate versus future rewards, to model impulsive and non-impulsive behavior. This allowed us to examine the performance of impulsive and non-impulsive agents in more or less predictable environments. In Temporal Discounting, we manipulated the transition probability to delayed rewards and found that the agent with the lower discount factor (i.e. the impulsive agent) collected more average reward than the agent with a higher discount factor (the non-impulsive agent) by selecting immediate reward options when the probability of receiving the future reward was low. In the Information Sampling task, we manipulated the amount of information obtained with each sample. When sampling led to small information gains, the impulsive MDP agent collected more average reward than the non-impulsive agent. Third, in the Explore-Exploit task, we manipulated the substitution rate for novel options. When the substitution rate was high, the impulsive agent again performed better than the non-impulsive agent, as it explored the novel options less and instead exploited options with known reward values. The results of these analyses show that impulsivity can be advantageous in environments that are unexpectedly uncertain.
Necrotic cell death during
(Mtb) infection is considered host detrimental since it facilitates mycobacterial spread. Ferroptosis is a type of regulated necrosis induced by accumulation of free iron ...and toxic lipid peroxides. We observed that Mtb-induced macrophage necrosis is associated with reduced levels of glutathione and glutathione peroxidase-4 (Gpx4), along with increased free iron, mitochondrial superoxide, and lipid peroxidation, all of which are important hallmarks of ferroptosis. Moreover, necrotic cell death in Mtb-infected macrophage cultures was suppressed by ferrostatin-1 (Fer-1), a well-characterized ferroptosis inhibitor, as well as by iron chelation. Additional experiments in vivo revealed that pulmonary necrosis in acutely infected mice is associated with reduced Gpx4 expression as well as increased lipid peroxidation and is likewise suppressed by Fer-1 treatment. Importantly, Fer-1-treated infected animals also exhibited marked reductions in bacterial load. Together, these findings implicate ferroptosis as a major mechanism of necrosis in Mtb infection and as a target for host-directed therapy of tuberculosis.
The neural underpinnings of rhythmic behavior, including music and dance, have been studied using the synchronization-continuation task (SCT), where subjects initially tap in synchrony with an ...isochronous metronome and then keep tapping at a similar rate via an internal beat mechanism. Here, we provide behavioral and neural evidence that supports a resetting drift-diffusion model (DDM) during SCT. Behaviorally, we show the model replicates the linear relation between the mean and standard-deviation of the intervals produced by monkeys in SCT. We then show that neural populations in the medial premotor cortex (MPC) contain an accurate trial-by-trial representation of elapsed-time between taps. Interestingly, the autocorrelation structure of the elapsed-time representation is consistent with a DDM. These results indicate that MPC has an orderly representation of time with features characteristic of concatenated DDMs and that this population signal can be used to orchestrate the rhythmic structure of the internally timed elements of SCT.
The present study used behavioral data, ensemble recordings from medial premotor cortex (MPC) in macaque monkeys, and computational modeling, to establish evidence in favor of a class of drift-diffusion models of rhythmic timing during a synchronization-continuation tapping task (SCT). The linear relation between the mean and standard-deviation of the intervals produced by monkeys in SCT is replicated by the model. Populations of MPC cells faithfully represent the elapsed time between taps, and there is significant trial-by-trial relation between decoded times and the timing behavior of the monkeys. Notably, the neural decoding properties, including its autocorrelation structure are consistent with a set of drift-diffusion models that are arranged sequentially and that are resetting in each SCT tap.
Oxytocin (OT) in the central nervous system (CNS) influences social cognition and behavior, making it a candidate for treating clinical disorders such as schizophrenia and autism. Intranasal ...administration has been proposed as a possible route of delivery to the CNS for molecules like OT. While intranasal administration of OT influences social cognition and behavior, it is not well established whether this is an effective means for delivering OT to CNS targets. We administered OT or its vehicle (saline) to 15 primates (Macaca mulatta), using either intranasal spray or a nebulizer, and measured OT concentration changes in the cerebral spinal fluid (CSF) and in blood. All subjects received both delivery methods and both drug conditions. Baseline samples of blood and CSF were taken immediately before drug administration. Blood was collected every 10 minutes after administration for 40 minutes and CSF was collected once post-delivery, at the 40 minutes time point. We found that intranasal administration of exogenous OT increased concentrations in both CSF and plasma compared to saline. Both delivery methods resulted in similar elevations of OT concentration in CSF, while the changes in plasma OT concentration were greater after nasal spray compared to nebulizer. In conclusion our study provides evidence that both nebulizer and nasal spray OT administration can elevate CSF OT levels.
PDF
