This combined survey of operant and classical conditioning provides professional and academic readers with an up-to-date, inclusive account of a core field of psychology research, with in-depth ...coverage of the basic theory, its applications, and current topics including behavioral economics. * Provides comprehensive coverage of operant and classical conditioning, relevant fundamental theory, and applications including the latest techniques * Features chapters by leading researchers, professionals, and academicians * Reviews a range of core literature on conditioning * Covers cutting-edge topics such as behavioral economics
Cue competition is one of the most studied phenomena in associative learning. However, a
theoretical disagreement has long stood over whether it reflects a learning or performance
deficit. The ...comparator hypothesis, a model of expression of Pavlovian associations,
posits that
learning
is not subject to competition but that
performance
reflects a complex interaction of encoded associative
strengths. That is, subjects respond to a cue to the degree that it signals a
change
in the likelihood or magnitude of reinforcement relative to that
in the cue's absence. Initially, this performance-focused view was supported by studies
showing that posttraining revaluation of a competing cue often influences responding to
the target cue. However, recently developed learning-focused accounts of retrospective
revaluation have revitalized the debate concerning cue competition. Further complicating
the picture are phenomena of cue facilitation, which have been addressed less frequently
than cue competition by formal models of conditioning of either class. The authors present
a formalization and extension of the comparator hypothesis, which results in sharpened
differentiation between it and the new learning-focused models.
Abstract Males and females learn and remember differently at different times in their lives. These differences occur in most species, from invertebrates to humans. We review here sex differences as ...they occur in laboratory rodent species. We focus on classical and operant conditioning paradigms, including classical eyeblink conditioning, fear-conditioning, active avoidance and conditioned taste aversion. Sex differences have been reported during acquisition, retention and extinction in most of these paradigms. In general, females perform better than males in the classical eyeblink conditioning, in fear-potentiated startle and in most operant conditioning tasks, such as the active avoidance test. However, in the classical fear-conditioning paradigm, in certain lever-pressing paradigms and in the conditioned taste aversion, males outperform females or are more resistant to extinction. Most sex differences in conditioning are dependent on organizational effects of gonadal hormones during early development of the brain, in addition to modulation by activational effects during puberty and adulthood. Critically, sex differences in performance account for some of the reported effects on learning and these are discussed throughout the review. Because so many mental disorders are more prevalent in one sex than the other, it is important to consider sex differences in learning when applying animal models of learning for these disorders. Finally, we discuss how sex differences in learning continue to alter the brain throughout the lifespan. Thus, sex differences in learning are not only mediated by sex differences in the brain, but also contribute to them.
•Fornix lesions do not impair delay eyeblink conditioning in rats.•Fornix lesions impair trace eyeblink conditioning in rats.•Sound-on and sound-off conditioned stimuli are similarly processed.
...Research has shown differences in the neural mechanisms that support trace and delay eyeblink conditioning. The present experiment furthered this investigation by examining the effect of electrolytic fornix lesions on acquisition of trace and delay eyeblink conditioning in the rat. Importantly, the conditioned stimulus (CS) for trace conditioning was a standard tone-on cue, and the CS for delay conditioning was either a tone-off or tone-on CS. The results showed that fornix lesions impaired trace-, but not delay conditioning in rats trained with the tone-on CS or tone-off CS. The findings are consistent with previous studies that found trace-, but not delay eyeblink conditioning is a hippocampal dependent form of associative learning. Our results also indicate that the neural pathways for tone-off delay conditioning and tone-on trace conditioning are different, even though the structural composition of a tone-off CS and the trace conditioning interval are the same cue (i.e., the absence of sound). These findings indicate that the absence of a sensory cue (i.e., tone-off CS) and the presence of a sensory cue (i.e., tone-on CS) have equivalent associative value and effectiveness for engaging the neural pathways that support delay eyeblink conditioning.
This brief review summarizes 60 years of conceptual advances that have demonstrated a role for active changes in neuronal connectivity as a controller of behavior and behavioral change. Seminal ...studies in the first phase of the six‐decade span of this review firmly established the cellular basis of behavior – a concept that we take for granted now, but which was an open question at the time. Hebbian plasticity, including long‐term potentiation and long‐term depression, was then discovered as being important for local circuit refinement in the context of memory formation and behavioral change and stabilization in the mammalian central nervous system. Direct demonstration of plasticity of neuronal circuit function in vivo, for example, hippocampal neurons forming place cell firing patterns, extended this concept. However, additional neurophysiologic and computational studies demonstrated that circuit development and stabilization additionally relies on non‐Hebbian, homoeostatic, forms of plasticity, such as synaptic scaling and control of membrane intrinsic properties. Activity‐dependent neurodevelopment was found to be associated with cell‐wide adjustments in post‐synaptic receptor density, and found to occur in conjunction with synaptic pruning. Pioneering cellular neurophysiologic studies demonstrated the critical roles of transmembrane signal transduction, NMDA receptor regulation, regulation of neural membrane biophysical properties, and back‐propagating action potential in critical time‐dependent coincidence detection in behavior‐modifying circuits. Concerning the molecular mechanisms underlying these processes, regulation of gene transcription was found to serve as a bridge between experience and behavioral change, closing the ‘nature versus nurture’ divide. Both active DNA (de)methylation and regulation of chromatin structure have been validated as crucial regulators of gene transcription during learning. The discovery of protein synthesis dependence on the acquisition of behavioral change was an influential discovery in the neurochemistry of behavioral modification. Higher order cognitive functions such as decision making and spatial and language learning were also discovered to hinge on neural plasticity mechanisms. The role of disruption of these processes in intellectual disabilities, memory disorders, and drug addiction has recently been clarified based on modern genetic techniques, including in the human.
The area of neural plasticity and behavior has seen tremendous advances over the last six decades, with many of those advances being specifically in the neurochemistry domain. This review provides an overview of the progress in the area of neuroplasticity and behavior over the life‐span of the Journal of Neurochemistry. To organize the broad literature base, the review collates progress into fifteen broad categories identified as ‘conceptual advances’, as viewed by the author. The fifteen areas are delineated in the figure above.
This article is part of the 60th Anniversary special issue.
The area of neural plasticity and behavior has seen tremendous advances over the last six decades, with many of those advances being specifically in the neurochemistry domain. This review provides an overview of the progress in the area of neuroplasticity and behavior over the life‐span of the Journal of Neurochemistry. To organize the broad literature base, the review collates progress into fifteen broad categories identified as ‘conceptual advances’, as viewed by the author. The fifteen areas are delineated in the figure above.
This article is part of the 60th Anniversary special issue.
The importance of neuronal ensembles, termed engram cells, in storing and retrieving memory is increasingly being appreciated, but less is known about how these engram cells operate within neural ...circuits. Here we tagged engram cells in the ventral CA1 region of the hippocampus (vCA1) and the core of the nucleus accumbens (AcbC) during cocaine conditioned place preference (CPP) training and show that the vCA1 engram projects preferentially to the AcbC and that the engram circuit from the vCA1 to the AcbC mediates memory recall. Direct activation of the AcbC engram while suppressing the vCA1 engram is sufficient for cocaine CPP. The AcbC engram primarily consists of D1 medium spiny neurons, but not D2 medium spiny neurons. The preferential synaptic strengthening of the vCA1→AcbC engram circuit evoked by cocaine conditioning mediates the retrieval of cocaine CPP memory. Our data suggest that the vCA1 engram stores specific contextual information, while the AcbC D1 engram and its downstream network store both cocaine reward and associated contextual information, providing a potential mechanism by which cocaine CPP memory is stored.
Contingency is a critical concept for theories of associative learning and the assignment of credit problem in reinforcement learning. Measuring and manipulating it has, however, been problematic. ...The information-theoretic definition of contingency-normalized mutual information-makes it a readily computed property of the relation between reinforcing events, the stimuli that predict them and the responses that produce them. When necessary, the dynamic range of the required temporal representation divided by the Weber fraction gives a psychologically realistic plug-in estimates of the entropies. There is no measurable prospective contingency between a peck and reinforcement when pigeons peck on a variable interval schedule of reinforcement. There is, however, a perfect retrospective contingency between reinforcement and the immediately preceding peck. Degrading the retrospective contingency by gratis reinforcement reveals a critical value (.25), below which performance declines rapidly. Contingency is time scale invariant, whereas the perception of proximate causality depends-we assume-on there being a short, fixed psychologically negligible critical interval between cause and effect. Increasing the interval between a response and reinforcement that it triggers degrades the retrograde contingency, leading to a decline in performance that restores it to at or above its critical value. Thus, there is no critical interval in the retrospective effect of reinforcement. We conclude with a short review of the broad explanatory scope of information-theoretic contingencies when regarded as causal variables in conditioning. We suggest that the computation of contingencies may supplant the computation of the sum of all future rewards in models of reinforcement learning.
Individuals suffering from substance-use disorders develop strong associations between the drug’s rewarding effects and environmental cues, creating powerful, enduring triggers for relapse. We found ...that dephosphorylated, nuclear histone deacetylase 5 (HDAC5) in the nucleus accumbens (NAc) reduced cocaine reward-context associations and relapse-like behaviors in a cocaine self-administration model. We also discovered that HDAC5 associates with an activity-sensitive enhancer of the Npas4 gene and negatively regulates NPAS4 expression. Exposure to cocaine and the test chamber induced rapid and transient NPAS4 expression in a small subpopulation of FOS-positive neurons in the NAc. Conditional deletion of Npas4 in the NAc significantly reduced cocaine conditioned place preference and delayed learning of the drug-reinforced action during cocaine self-administration, without affecting cue-induced reinstatement of drug seeking. These data suggest that HDAC5 and NPAS4 in the NAc are critically involved in reward-relevant learning and memory processes and that nuclear HDAC5 limits reinstatement of drug seeking independent of NPAS4.
•Nuclear HDAC5 in the NAc attenuates relapse-like drug-seeking behaviors•ChIP-seq revealed numerous HDAC5-associated target genes including Npas4•NPAS4 in NAc is induced in subset of FOS+ neurons during cocaine-context learning•HDAC5 and NPAS4 in NAc are involved in cocaine-conditioned behaviors
Taniguchi and colleagues find that the epigenetic enzyme, histone deacetylase 5 (HDAC5), reduces relapse-like behaviors in a model of cocaine addiction and that HDAC5 and its target gene, Npas4, are important in the nucleus accumbens for reward-related learning processes.
•Originates from discussions on replicability and researcher’s degrees of freedom.•Aims at stimulating discussions on methods applied in fear conditioning research.•Addresses critical issues on ...terminology, design, methods, analysis.•Serves as comprehensive compendium and critical evaluation of read-out measures.•Highlights methodological considerations when studying individual differences.
The so-called ‘replicability crisis’ has sparked methodological discussions in many areas of science in general, and in psychology in particular. This has led to recent endeavours to promote the transparency, rigour, and ultimately, replicability of research. Originating from this zeitgeist, the challenge to discuss critical issues on terminology, design, methods, and analysis considerations in fear conditioning research is taken up by this work, which involved representatives from fourteen of the major human fear conditioning laboratories in Europe.
This compendium is intended to provide a basis for the development of a common procedural and terminology framework for the field of human fear conditioning. Whenever possible, we give general recommendations. When this is not feasible, we provide evidence-based guidance for methodological decisions on study design, outcome measures, and analyses. Importantly, this work is also intended to raise awareness and initiate discussions on crucial questions with respect to data collection, processing, statistical analyses, the impact of subtle procedural changes, and data reporting specifically tailored to the research on fear conditioning.
The role of the ventromedial prefrontal cortex (vmPFC) in human pavlovian threat conditioning has been relegated largely to the extinction or reversal of previously acquired stimulus-outcome ...associations. However, recent neuroimaging evidence questions this view by also showing activity in the vmPFC during threat acquisition. Here we investigate the casual role of vmPFC in the acquisition of pavlovian threat conditioning by assessing skin conductance response (SCR) and declarative memory of stimulus-outcome contingencies during a differential pavlovian threat-conditioning paradigm in eight patients with a bilateral vmPFC lesion, 10 with a lesion outside PFC and 10 healthy participants (each group included both females and males). Results showed that patients with vmPFC lesion failed to produce a conditioned SCR during threat acquisition, despite no evidence of compromised SCR to unconditioned stimulus or compromised declarative memory for stimulus-outcome contingencies. These results suggest that the vmPFC plays a causal role in the acquisition of new learning and not just in the extinction or reversal of previously acquired learning, as previously thought. Given the role of the vmPFC in schema-related processing and latent structure learning, the vmPFC may be required to construct a detailed representation of the task, which is needed to produce a sustained conditioned physiological response in anticipation of the unconditioned stimulus during threat acquisition.
Pavlovian threat conditioning is an adaptive mechanism through which organisms learn to avoid potential threats, thus increasing their chances of survival. Understanding what brain regions contribute to such a process is crucial to understand the mechanisms underlying adaptive as well as maladaptive learning, and has the potential to inform the treatment of anxiety disorders. Importantly, the role of the ventromedial prefrontal cortex (vmPFC) in the acquisition of pavlovian threat conditioning has been relegated largely to the inhibition of previously acquired learning. Here, we show that the vmPFC actually plays a causal role in the acquisition of pavlovian threat conditioning.
PDF
