The central amygdala (CeA) orchestrates adaptive responses to emotional events. While CeA substrates for defensive behaviors have been studied extensively, CeA circuits for appetitive behaviors and ...their relationship to threat-responsive circuits remain poorly defined. Here, we demonstrate that the CeA sends robust inhibitory projections to the lateral substantia nigra (SNL) that contribute to appetitive and aversive learning in mice. CeA→SNL neural responses to appetitive and aversive stimuli were modulated by expectation and magnitude consistent with a population-level salience signal, which was required for Pavlovian conditioned reward-seeking and defensive behaviors. CeA→SNL terminal activation elicited reinforcement when linked to voluntary actions but failed to support Pavlovian associations that rely on incentive value signals. Consistent with a disinhibitory mechanism, CeA inputs preferentially target SNL GABA neurons, and CeA→SNL and SNL dopamine neurons respond similarly to salient stimuli. Collectively, our results suggest that amygdala-nigra interactions represent a previously unappreciated mechanism for influencing emotional behaviors.
Display omitted
•CeA→SNL populations encode a salience signal biased to unconditioned stimuli•CeA→SNL activity is necessary for conditioned reward-seeking and defensive behaviors•CeA cells form monosynaptic connections with nigral GABA and dopamine neurons•SNL dopamine populations encode a salience signal biased to conditioned stimuli
Steinberg et al. investigate how connections between the central amygdala and the lateral substantia nigra contribute to emotional behaviors. They find that amygdalonigral neurons are activated by salient stimuli and participate in appetitive and aversive learning via connections with nigral GABA and dopamine neurons.
•A hierarchical coordination reinforcement learning method is developed.•A framework is proposed to optimise the maintenance of complex systems via the HCRL.•The HCRL outperforms benchmark ...maintenance optimisation methods.•An efficient method is developed to simulate interdependent degradation processes.
The Markov decision process (MDP) is a widely used method to optimise the maintenance of multicomponent systems, which can provide a system-level maintenance action at each decision point to address various dependences among components. However, MDP suffers from the “curse of dimensionality” and can only process small-scale systems. This paper develops a hierarchical coordinated reinforcement learning (HCRL) algorithm to optimise the maintenance of large-scale multicomponent systems. Both parameters of agents and the coordination relationship among agents are designed based on system characteristics. Furthermore, the hierarchical structure of agents is established according to the structural importance measures of components. The effectiveness of the proposed HCRL algorithm is validated using two maintenance optimisation problems, one on a natural gas plant system and the other using a 12-component series system under dependant competing risks. Results show that the proposed HCRL outperforms methods in two recently published papers and other benchmark approaches including the new emerging deep reinforcement learning.
Functional communication training, an intervention for challenging behavior rooted in principles of applied behavior analysis, has copious empirical support dating back to the mid-1980s for autistic ...individuals. Recently, there has been a concerted effort to thin reinforcement delivery during functional communication training using contingency-based delays that, in turn, are designed to enhance practicality and feasibility while not compromising on efficacy. In this synthesis, we meta-analyzed the literature using log response ratio effect sizes to investigate (a) combined and across type effectiveness of contingency-based delays and (b) moderating variables that might impact intervention outcomes. Findings showed that contingency-based delays were effective for autistic individuals (log response ratio = −2.17; 95% CI = (−2.76, −1.58)) and most effective when the contingency incorporated positive reinforcement (log response ratio = −2.30; 95% CI = (−2.83, −1.78)). In addition, delay procedures that included differential reinforcement of alternative behavior were overall more effective (log response ratio = −2.13; 95% CI = (−2.72, −1.55)) than those that involved differential reinforcement of other behavior (log response ratio = −1.24; 95% CI = (−3.84, 1.37)). Noteworthy moderating variables found to impact contingency-based delay efficacy included the intervention dosage and the topography of behavior. We discuss these findings and highlight directions where additional empirical research is warranted to improve our understanding about contingency-based delays for autistic individuals.
Lay abstract
Functional communication training, an intervention for challenging behavior rooted in principles of applied behavior analysis, has copious empirical support dating back to the mid-1980s for autistic individuals. Recently, there has been a concerted effort to thin reinforcement delivery during functional communication training using contingency-based delays that, in turn, are designed to enhance practicality and feasibility while not compromising efficacy. In this synthesis, we meta-analyzed the literature base with the goal of investigating both combined and across type effectiveness of contingency-based delays. We also aimed to investigate moderating variables that might impact intervention outcomes. Findings showed that contingency-based delays were effective for individuals with an autism spectrum disorder diagnosis and most effective when the delay incorporated some form of positive reinforcement. In addition, differential reinforcement of alternative-based delays was overall more effective when compared to differential reinforcement of other behavior-based delays. Noteworthy moderating variables found to impact contingency-based delay efficacy included the intervention dosage and the topography of behavior. We discuss these findings and highlight directions where additional empirical research is warranted to improve our understanding about contingency-based delays for individuals diagnosed with autism spectrum disorder.
To determine the effects of response–reinforcer arrangements on task performance and preference, participants completed tasks on accumulated and distributed response–reinforcer arrangements. Three ...males diagnosed with an autism spectrum disorder completed tasks before receiving 5-min or 30-seconds access to a preferred stimulus. To enhance discrimination between the two arrangements, color-coded token boards were used to represent each arrangement. Responding was evaluated within a multielement design to compare the response rate across conditions for each participant. A preference assessment was conducted after the comparison to determine whether a preference for one of the arrangements emerged. All participants produced a higher rate of responding in the accumulated schedule of reinforcement.
Hill's biography of Sir Julius is filled with accounts of the squabbles between him and other prominent figures, usually over money, and even records Sir Julius's own complaints about the visits he ...received from the Queen, since such visits demanded the outlay of a gift, which he claimed was unlikely to ever be returned to him via direct financial reward in spite of the several preferments he received-it was common knowledge after 12 September 1598, for example, that during the Queen's visit to Caesar's estate at Mitcham, he had identified himself in his supplication as 'the eldest judge, the youngest and the poorest', in reference to his frequent pleas regarding financial hardship.7 One public squabble is particularly relevant:
We present the mathematical description of feedback functions of variable interval and variable differential reinforcement of low rates as functions of schedule size only. These results were obtained ...using an R script named Beak, which was built to simulate rates of behavior interacting with simple schedules of reinforcement. Using Beak, we have simulated data that allow an assessment of different reinforcement feedback functions. This was made with unparalleled precision, as simulations provide huge samples of data and, more importantly, simulated behavior is not changed by the reinforcement it produces. Therefore, we can vary response rates systematically. We've compared different reinforcement feedback functions for random interval schedules, using the following criteria: meaning, precision, parsimony, and generality. Our results indicate that the best feedback function for the random interval schedule was published by Baum (1981). We also propose that the model used by Killeen (1975) is a viable feedback function for the random differential reinforcement of low rates schedule. We argue that Beak paves the way for greater understanding of schedules of reinforcement, addressing still open questions about quantitative features of simple schedules. Also, Beak could guide future experiments that use schedules as theoretical and methodological tools.
As the core technology of hybrid electric vehicles (HEVs), energy management strategy directly affects the fuel consumption of vehicles. This research proposes a novel reinforcement learning ...(RL)-based algorithm for energy management strategy of HEVs. Hierarchical structure is used in deep Q-learning algorithm (DQL-H) to get the optimal solution of energy management. Through this new RL method, we not only solve the problem of sparse reward in training process, but also achieve the optimal power distribution. In addition, as a kind of hierarchical algorithm, DQL-H can change the way of exploration of the vehicle environment and make it more effective. The experimental results show that the proposed DQL-H method realizes better training efficiency and lower fuel consumption, compared to other RL-based ones.
•DQL-H trains each level independently and is more efficient than counterparts.•Sparse rewards can be overcome during the training process.•Substantial rewards can accelerate the speed of convergence.•DQL-H changes the way of exploring the vehicle environment.
The Role of Variability in Motor Learning Dhawale, Ashesh K; Smith, Maurice A; Ölveczky, Bence P
Annual review of neuroscience,
07/2017, Volume:
40, Issue:
1
Journal Article
Peer reviewed
Open access
Trial-to-trial variability in the execution of movements and motor skills is ubiquitous and widely considered to be the unwanted consequence of a noisy nervous system. However, recent studies have ...suggested that motor variability may also be a feature of how sensorimotor systems operate and learn. This view, rooted in reinforcement learning theory, equates motor variability with purposeful exploration of motor space that, when coupled with reinforcement, can drive motor learning. Here we review studies that explore the relationship between motor variability and motor learning in both humans and animal models. We discuss neural circuit mechanisms that underlie the generation and regulation of motor variability and consider the implications that this work has for our understanding of motor learning.
Aberrant reward functioning is implicated in depression. While attention precedes behavior and guides higher-order cognitive processes, reward learning from an attentional perspective - the effects ...of prior reward-learning on subsequent attention allocation - has been mainly overlooked.
The present study explored the effects of reward-based attentional learning in depression using two separate, yet complimentary, studies. In study 1, participants with high (HD) and low (LD) levels of depression symptoms were trained to divert their gaze toward one type of stimuli over another using a novel gaze-contingent music reward paradigm - music played when fixating the desired stimulus type and stopped when gazing the alternate one. Attention allocation was assessed before, during, and following training. In study 2, using negative reinforcement, the same attention allocation pattern was trained while substituting the appetitive music reward for gazing the desired stimulus type with the removal of an aversive sound (i.e. white noise).
In study 1 both groups showed the intended shift in attention allocation during training (online reward learning), while generalization of learning at post-training was only evident among LD participants. Conversely, in study 2 both groups showed post-training generalization. Results were maintained when introducing anxiety as a covariate, and when using a more powerful sensitivity analysis. Finally, HD participants showed higher learning speed than LD participants during initial online learning, but only when using negative, not positive, reinforcement.
Deficient generalization of learning characterizes the attentional system of HD individuals, but only when using reward-based positive reinforcement, not negative reinforcement.
We evaluated a noncontingent reinforcement procedure that involved initially providing three subjects with signaled, continuous access to the functional reinforcer for aggression and slowly ...increasing the amount of time subjects were exposed to the signaled unavailability of the reinforcer. Additionally, alternative potential reinforcers were available throughout the sessions. Results showed immediate and substantial reductions in aggression for all three subjects. The clinical utility of this intervention is discussed, and future research directions are recommended.
PDF
