Adaptive decision making in dynamic environments requires multiple reinforcement-learning steps that may be implemented by dissociable neural circuits. Here, we used a novel directionally specific ...viral ablation approach to investigate the function of several anatomically defined orbitofrontal cortex (OFC) circuits during adaptive, flexible decision making in rats trained on a probabilistic reversal learning task. Ablation of OFC neurons projecting to the nucleus accumbens selectively disrupted performance following a reversal, by disrupting the use of negative outcomes to guide subsequent choices. Ablation of amygdala neurons projecting to the OFC also impaired reversal performance, but due to disruptions in the use of positive outcomes to guide subsequent choices. Ablation of OFC neurons projecting to the amygdala, by contrast, enhanced reversal performance by destabilizing action values. Our data are inconsistent with a unitary function of the OFC in decision making. Rather, distinct OFC-amygdala-striatal circuits mediate distinct components of the action-value updating and maintenance necessary for decision making.
•Distinct OFC circuits make unique contributions to flexible decision making•OFC-nucleus accumbens circuit incorporates negative outcomes into values•OFC-amygdala circuit stabilizes action values•Amygdala-OFC circuit incorporates positive outcomes into values
The orbitofrontal cortex (OFC) plays a critical role in guiding decisions in dynamic environments. Groman et al. use a directionally specific viral ablation approach to demonstrate that OFC circuits encode separable reinforcement-learning processes that guide decisions.
The anatomical, structural, and functional adaptations that occur in the brain during adolescence are thought to facilitate improvements in decision-making functions that are known to occur during ...this stage of development. The mechanisms that underlie these neural adaptations are not known, but deviations in developmental trajectories have been proposed to contribute to the emergence of mental illness, including addiction. Direct evidence supporting this hypothesis, however, has been limited. Here, we used a recently developed reversal-learning protocol to investigate the predictive relationship between adolescent decision-making trajectories and cocaine-taking behaviors in adulthood. Decision-making functions in the reversal-learning task were assessed throughout adolescence and into adulthood in male and female Long–Evans rats. Trial-by-trial choice data was fitted with a reinforcement-learning model to quantify the degree to which choice behavior of individual rats was influenced by rewarded (e.g., ∆
+
parameter) and unrewarded (e.g., ∆
0
parameter) outcomes. We report that reversal-learning performance improved during adolescence and that this was due to an increase in value updating for rewarded outcomes (e.g., ∆
+
parameter). Furthermore, the rate of change in the ∆
+
parameter predicted individual differences in the ∆
+
parameter and, notably, cocaine-taking behaviors in adulthood: Rats that had a shallower adolescent trajectory were found to have a lower ∆
+
parameter and greater cocaine self-administration in adulthood. These data indicate that adolescent development plays a critical role in drug use susceptibility. Future studies aimed at understanding the neurobiological mechanisms that underlie these age-related changes in decision-making could provide new insights into the biobehavioral mechanisms mediating addiction susceptibility.
The persistent use of psychostimulant drugs, despite the detrimental outcomes associated with continued drug use, may be because of disruptions in reinforcement-learning processes that enable ...behavior to remain flexible and goal directed in dynamic environments. To identify the reinforcement-learning processes that are affected by chronic exposure to the psychostimulant methamphetamine (MA), the current study sought to use computational and biochemical analyses to characterize decision-making processes, assessed by probabilistic reversal learning, in rats before and after they were exposed to an escalating dose regimen of MA (or saline control). The ability of rats to use flexible and adaptive decision-making strategies following changes in stimulus-reward contingencies was significantly impaired following exposure to MA. Computational analyses of parameters that track choice and outcome behavior indicated that exposure to MA significantly impaired the ability of rats to use negative outcomes effectively. These MA-induced changes in decision making were similar to those observed in rats following administration of a dopamine D2/3 receptor antagonist. These data use computational models to provide insight into drug-induced maladaptive decision making that may ultimately identify novel targets for the treatment of psychostimulant addiction. We suggest that the disruption in utilization of negative outcomes to adaptively guide dynamic decision making is a new behavioral mechanism by which MA rigidly biases choice behavior.
Addictions are often characterized as forms of impulsive behavior. That said, it is often noted that impulsivity is a multidimensional construct, spanning several psychological domains. This review ...describes the relationship between varieties of impulsivity and addiction‐related behaviors, the nature of the causal relationship between the two, and the underlying neurobiological mechanisms that promote impulsive behaviors. We conclude that the available data strongly support the notion that impulsivity is both a risk factor for, and a consequence of, drug and alcohol consumption. While the evidence indicating that subtypes of impulsive behavior are uniquely informative—either biologically or with respect to their relationships to addictions—is convincing, multiple lines of study link distinct subtypes of impulsivity to low dopamine D2 receptor function and perturbed serotonergic transmission, revealing shared mechanisms between the subtypes. Therefore, a common biological framework involving monoaminergic transmitters in key frontostriatal circuits may link multiple forms of impulsivity to drug self‐administration and addiction‐related behaviors. Further dissection of these relationships is needed before the next phase of genetic and genomic discovery will be able to reveal the biological sources of the vulnerability for addiction indexed by impulsivity.
Methamphetamine (MA) is one of the most commonly abused illicit substances worldwide. Among other problems, abuse of the drug has been associated with reduced cognitive function across several ...domains. However, much of the literature has not attempted to differentiate cognitive difficulties caused by MA abuse from preexisting cognitive difficulties that are likely caused by other factors. Here, we address this question, evaluating evidence for a priori hypotheses pertaining to six lines of research: (a) animal studies; (b) cross-sectional human studies; (c) a twin study; (d) studies of changes in cognition with abstinence from MA; (e) studies of changes in brain structure and function with abstinence from MA; and (f) studies of the relationship between the severity of MA abuse and the extent of cognitive deficits observed. Overall the findings were mixed, with some support for a causal relationship between MA abuse and cognitive decline, and other findings suggesting that there is no relationship. The preponderance of the data, however, does support the possibility that MA abuse causes cognitive decline, of unknown duration, in at least some users of the drug. When averaged across individuals, this decline is likely to be mild in early-to-middle adulthood. However, moderator variables are likely to contribute to the presence and/or severity of cognitive decline exhibited by a given individual.
The survival of an organism depends on the ability to make adaptive decisions to achieve the needs of the organism: where to get food, who to mate with, and how to evade predators. Decision-making is ...a term used to describe a collection of behavioral and/or computational functions that guide the selection of an option amongst a set of alternatives. Some of these functions may include calculating the costs and benefits of a particular action, evaluating differences in value of each of the alternative outcomes and the likelihood of receiving a particular outcome, using past experiences to generate predictions or expectations about action-outcome associations, and/or integration of past experiences to make novel inferences that can be used in new environments. There is considerable interest in understanding the neurobiological mechanisms that mediate these decision-making functions and recent advances in behavioral approaches, neuroscience techniques, and neuroimaging measures have begun to develop mechanistic links between biology, reward, and decision making. This multidisciplinary work holds great promise for elucidating the biological mechanisms mediating decision-making deficits in normal and abnormal states. The multidisciplinary studies included in this Collection provide new insights into the neuroscience of decision making and reward.
Suboptimal decision-making strategies have been proposed to contribute to the pathophysiology of addiction. Decision-making, however, arises from a collection of computational components that can ...independently influence behavior. Disruptions in these different components can lead to decision-making deficits that appear similar behaviorally, but differ at the computational, and likely the neurobiological, level. Here, we discuss recent studies that have used computational approaches to investigate the decision-making processes underlying addiction. Studies in animal models have found that value updating following positive, but not negative, outcomes is predictive of drug use, whereas value updating following negative, but not positive, outcomes is disrupted following drug self-administration. We contextualize these findings with studies on the circuit and biological mechanisms of decision-making to develop a framework for revealing the biobehavioral mechanisms of addiction.
Maladaptive decision-making in substance-dependent populations reflects both pre-existing risk factors and drug-induced adaptations in specific neurobiological processes.Delineating the relevant decision-making processes is essential for identifying biological markers for addiction susceptibility and for developing novel targets for the treatment of addiction.New neuroscience techniques probing learning mechanisms demonstrated that different decision-making computations are mediated by distinct neural, circuit, and cellular systems.Computational biomarkers of decision-making functions are key for revealing the neurobiological mechanisms of addiction.
Flexible decision-making in dynamic environments requires both retrospective appraisal of reinforced actions and prospective reasoning about the consequences of actions. These complementary ...reinforcement-learning systems can be characterized computationally with model-free and model-based algorithms, but how these processes interact at a neurobehavioral level in normal and pathological states is unknown. Here, we developed a translationally analogous multistage decision-making (MSDM) task to independently quantify model-free and model-based behavioral mechanisms in rats. We provide the first direct evidence that male rats, similar to humans, use both model-free and model-based learning when making value-based choices in the MSDM task and provide novel analytic approaches for independently quantifying these reinforcement-learning strategies. Furthermore, we report that
dopamine tone in the ventral striatum and orbitofrontal cortex correlate with model-based, but not model-free, strategies, indicating that the biological mechanisms mediating decision-making in the multistage task are conserved in rats and humans. This new multistage task provides a unique behavioral platform for conducting systems-level analyses of decision-making in normal and pathological states.
Decision-making is influenced by both a retrospective "model-free" system and a prospective "model-based" system in humans, but the biobehavioral mechanisms mediating these learning systems in normal and disease states are unknown. Here, we describe a translationally analogous multistage decision-making task to provide a behavioral platform for conducting neuroscience studies of decision-making in rats. We provide the first evidence that choice behavior in rats is influenced by model-free and model-based systems and demonstrate that model-based, but not model-free, learning is associated with corticostriatal dopamine tone. This novel behavioral paradigm has the potential to yield critical insights into the mechanisms mediating decision-making alterations in mental disorders.
PDF
