Motivational impairment is a common feature of both depression and psychosis; however, the psychological and neural mechanisms that give rise to motivational impairment in these disorders are poorly ...understood. Recent research has suggested that aberrant effort-cost decision-making (ECDM) may be a potential contributor to motivational impairment in both psychosis and depression. ECDM refers to choices that individuals make regarding the amount of 'work' they are willing to expend to obtain a certain outcome or reward. Recent experimental work has suggested that those with psychosis and depression may be less willing to expend effort to obtain rewards compared with controls, and that this effort deficit is related to motivational impairment in both disorders. In the current review, we aim to summarize the current literature on ECDM in psychosis and depression, providing evidence for transdiagnostic impairment. Next, we discuss evidence for the hypothesis that a seemingly similar behavioral ECDM deficit might arise from disparate psychological and neural mechanisms. Specifically, we argue that effort deficits in psychosis might be largely driven by deficits in cognitive control and the neural correlates of cognitive control processes, while effort deficits in depression might be largely driven by reduced reward responsivity and the associated neural correlates of reward responsivity. Finally, we will provide some discussion regarding future directions, as well as interpretative challenges to consider when examining ECDM transdiagnostically.
People primarily pursue long-term goals, such as exercising, to receive delayed rewards (e.g., improved health). However, we find that the presence of immediate rewards is a stronger predictor of ...persistence in goal-related activities than the presence of delayed rewards. Specifically, immediate rewards (e.g., enjoyment) predicted current persistence at New Year’s resolutions whereas delayed rewards did not (Study 1). Furthermore, immediate rewards predicted persistence in a single session of studying and exercising whereas delayed rewards did not, even though people report primarily pursuing these activities for delayed rewards (Studies 2 and 3). This is true for both short (1 week) and long (3 month) time frames (Study 4), and regardless of whether anticipated or materialized rewards are assessed (Study 5). Overall, whereas delayed rewards may motivate goal setting and the intentions to pursue long-term goals, a meta-analysis of our studies finds that immediate rewards are more strongly associated with actual persistence in a long-term goal.
Adaptive Coding of Reward Value by Dopamine Neurons Tobler, Philippe N; Fiorillo, Christopher D; Schultz, Wolfram
Science (American Association for the Advancement of Science),
03/2005, Letnik:
307, Številka:
5715
Journal Article
Recenzirano
Odprti dostop
It is important for animals to estimate the value of rewards as accurately as possible. Because the number of potential reward values is very large, it is necessary that the brain's limited resources ...be allocated so as to discriminate better among more likely reward outcomes at the expense of less likely outcomes. We found that midbrain dopamine neurons rapidly adapted to the information provided by reward-predicting stimuli. Responses shifted relative to the expected reward value, and the gain adjusted to the variance of reward value. In this way, dopamine neurons maintained their reward sensitivity over a large range of reward values.
Deep reinforcement learning (DRL) has achieved remarkable milestones in the field of artificial intelligence. However, the reward functions for most real-world tasks are characterized by delays and ...sparsity, posing significant challenges for DRL methods. To tackle the issues of delayed and sparse rewards, there have been many approaches based on the prior knowledge of expert trajectories proposed, such as GAIL and its variants. However, if only suboptimal demonstrations available, they usually struggle to overcome the performance disadvantage due to the complexity and fragility of adversarial training. To address these problems, this paper introduces a novel framework combining Self-Imitation learning with Reward Relabeling based Reinforcement learning, thus dubbed SIR3. It is capable of accelerating online learning using suboptimal demonstrations in environments even with extremely sparse rewards and meanwhile encouraging exploration of better policies. SIR3 devises a task-independent reward relabeling mechanism to generate reward signals for both the expert examples and online experience. This design provides the agent with more informative guidance, even when the number of suboptimal demonstrations is minimal. During the training process, the integration of imitation learning and RL losses enables the agent to dynamically mimic rewarding trajectories, possibly collected from experts or self-explored. Experimental findings on widely recognized MuJoCo benchmarks reveal that SIR3 can efficiently learn excellent policies surpassing suboptimal demonstrations, achieving superior training efficiency and performance relative to SOTA methods. Notably, in some environments, it secures a performance edge over an order of magnitude.
•An innovative method that leverages suboptimal demonstrations and online samples.•A simple, yet effective reward relabeling method to address the sparse-reward issue.•A novel mechanism for integrating self-imitation learning and reinforcement learning.
Dysfunctional reward processing is a leading candidate mechanism for the development of certain depressive symptoms, such as anhedonia. However, to our knowledge, there has not yet been a systematic ...assessment of whether and to what extent depression is associated with impairments on behavioral reward-processing tasks.
To determine whether depression is associated with impairments in reward-processing behavior.
The MEDLINE/PubMed, Embase, and PsycInfo databases were searched for studies that investigated reward processing using performance on behavioral tasks by individuals with depression and nondepressed control groups, published between January 1, 1946, and August 16, 2019.
Studies that contained data regarding performance by depressed and healthy control groups on reward-processing tasks were included in the systematic review and meta-analysis.
Summary statistics comparing performance between depressed and healthy groups on reward-processing tasks were converted to standardized mean difference (SMD) scores, from which summary effect sizes for overall impairment in reward processing and 4 subcomponent categories were calculated. Study quality, heterogeneity, replicability-index, and publication bias were also assessed.
Performance on reward-processing tasks.
The final data set comprised 48 case-control studies (1387 healthy control individuals and 1767 individuals with major depressive disorder). The mean age was 37.85 years and 58% of the participants were women. These studies used tasks assessing option valuation (n = 9), reward bias (n = 6), reward response vigor (n = 12), reinforcement learning (n = 20), and grip force (n = 1). Across all tasks, depression was associated with small to medium impairments in reward-processing behavior (SMD = 0.345; 95% CI, 0.209-0.480). When examining reward-processing subcomponent categories, impairment was associated with tasks assessing option valuation (SMD = 0.309; 95% CI, 0.147-0.471), reward bias (SMD = 0.644; 95% CI, 0.270-1.017), and reinforcement learning (SMD = 0.352; 95% CI, 0.115-0.588) but not reward response vigor (SMD = 0.083; 95% CI, -0.144 to 0.309). The medication status of the major depressive disorder sample did not explain any of the variance in the overall effect size. There was significant between-study heterogeneity overall and in all subcomponent categories other than option valuation. Significant publication bias was identified overall and in the reinforcement learning category.
Relative to healthy control individuals, individuals with depression exhibit reward-processing impairments, particularly for tests of reward bias, option valuation, and reinforcement learning. Understanding the neural mechanisms driving these associations may assist in designing novel interventions.
Whether cognitive, motivational and hedonic aspects of reward anticipation and consumption can be reliably assessed with explicit and implicit measures, and if different motivational (decision ...utility) and hedonic (experienced utility) processes get recruited by distinct reward types, remain partly unsolved questions that are relevant for theories of social and non-social decision-making. We investigated these topics using a novel experimental paradigm, including carefully matched social and nonsocial rewards, and by focusing on facial responses. Facial expressions are indeed an often-cited implicit measure of rewards’ hedonic impact. For example, food rewards elicit powerful facial responses – characterized by lip smacking, tongue protrusion, and relaxation of the middle face – in human newborns, juvenile monkeys, and adult rats. The same stimuli elicit more nuanced facial reactions in adult humans, which can be best captured with facial electromyography (fEMG). However, little is known about facial expressions preceding reward consumption, reflecting the motivation to obtain and possibly the expected pleasantness of a reward, and whether similar facial expressions are elicited by different types of rewards. To investigate these questions, a novel within-subject experimental paradigm was developed. During the anticipation and consumption of social (affective touch) and nonsocial (food) rewards, explicit (ratings of wanting and liking, physical effort) and implicit (fEMG) measures of wanting and liking were taken in 43 healthy adult participants. Reduced activation of the Corrugator Supercilii (CS) muscle (reflecting less frowning and indicating greater positive response) was found in trials with higher wanting and effort during the anticipation of food rewards, as well as in trials with higher liking and effort during the consumption of food rewards. The CS muscle is thus a sensitive measure of wanting and liking of food rewards both during their anticipation and consumption. Crucially, thanks to careful reward matching, these results cannot be explained by differences in subjective wanting, liking, or effort produced to obtain the two types of rewards. No significant modulation of the Zygomaticus Major (ZM) muscle was found for social or food rewards. Explorative analyses however indicated that the ZM may activate during the delivery of the most wanted touch, but not for the most wanted food. The absence of significant effects of social rewards on the activation of CS and ZM muscles are discussed in relation to the specifics of this innovative task comparing two types of matched rewards in the same participants. The present findings contribute to the understanding of the processes underlying motivational and hedonic aspects of rewards, and may therefore inform models of social and non-social decision-making.
While trait positive emotionality and state positive-valence affect have long been the subject of intense study, the importance of differentiating among several "discrete" positive emotions has only ...recently begun to receive serious attention. In this article, we synthesize existing literature on positive emotion differentiation, proposing that the positive emotions are best described as branches of a "family tree" emerging from a common ancestor mediating adaptive management of fitness-critical resources (e.g., food). Examples are presented of research indicating the importance of differentiating several positive emotion constructs. We then offer a new theoretical framework, built upon a foundation of phylogenetic, neuroscience, and behavioral evidence, that accounts for core features as well as mechanisms for differentiation. We propose several directions for future research suggested by this framework and develop implications for the application of positive emotion research to translational issues in clinical psychology and the science of behavior change.
Objective: Determine whether a novel psychosocial treatment for positive affect improves clinical status and reward sensitivity more than a form of cognitive behavioral therapy that targets negative ...affect and whether improvements in reward sensitivity correlate with improvements in clinical status. Method: In this assessor-blinded, parallel-group, multisite, two-arm randomized controlled clinical superiority trial, 85 treatment-seeking adults with severely low positive affect, moderate-to-severe depression or anxiety, and functional impairment received 15 weekly individual therapy sessions of positive affect treatment (PAT) or negative affect treatment (NAT). Clinical status measures were self-reported positive affect, interviewer-rated anhedonia, and self-reported depression and anxiety. Target measures were eleven physiological, behavioral, cognitive, and self-report measures of reward anticipation-motivation, response to reward attainment, and reward learning. All analyses were intent-to-treat. Results: Compared to NAT, individuals receiving PAT achieved superior improvements in the multivariate clinical status measures at posttreatment, b = .37, 95% CI .15, .59, t(109) = 3.34, p = .001, q = .004, d = .64. Compared to NAT, individuals receiving PAT also achieved higher multivariate reward anticipation-motivation, b = .21, 95% CI .05, .37, t(268) = 2.61, p = .010, q = .020, d = .32, and higher multivariate response to reward attainment, b = .24, 95% CI .02, .45, t(266) = 2.17, p = .031, q = .041, d = .25, at posttreatment. Measures of reward learning did not differ between the two groups. Improvements in reward anticipation-motivation and in response to reward attainment correlated with improvements in the clinical status measures. Conclusions: Targeting positive affect results in superior improvements in clinical status and reward sensitivity than targeting negative affect. This is the first demonstration of differential target engagement across two psychological interventions for anxious or depressed individuals with low positive affect.
What is the public health significance of this article?
This study demonstrates that positive affect treatment was significantly more effective than an intervention that targeted negative affect for adults with severely low positive affect, moderate-to-severe depression or anxiety, and functional impairment. The findings of this study suggest that positive affect treatment improves positive affect and aspects of reward hyposensitivity.
Choices and response times in two-alternative decision-making tasks can be modeled by assuming that individuals steadily accrue evidence in favor of each alternative until a response boundary for one ...of them is crossed, at which point that alternative is chosen. Prior studies have reported that evidence accumulation during decision-making tasks takes longer in adults with psychopathology than in healthy controls, indicating that slow evidence accumulation may be transdiagnostic. However, few studies have examined perceptual decision making in anxiety disorders, where hypervigilance might enhance performance. Therefore, this study used the Hierarchical Drift Diffusion model to investigate evidence accumulation in adults with social anxiety disorder (SAD) and healthy controls as they performed a probabilistic reward task (PRT), in which social rewards were delivered for correct perceptual judgments. Adults with SAD completed the PRT before and after gaze-contingent music reward therapy (GCMRT), which trains attention allocation and has shown efficacy for SAD. Healthy controls also completed the PRT twice. Results revealed excellent performance in adults with SAD, especially after GCMRT: relative to controls, they showed faster evidence accumulation, better discriminability, and earned more rewards. These data highlight a positive effect of attention training on performance in anxious adults and show how a behavioral trait that is typically problematic-hypervigilance in SAD-can nevertheless confer advantages in certain contexts. The data also indicate that, in contrast to other forms of psychopathology, SAD is not characterized by slow evidence accumulation, at least in the context of the social PRT.
Abstract Since Costello’s (1972) seminal Behavior Therapy article on loss of reinforcers or reinforcer effectiveness in depression, the role of reward sensitivity and processing in both depression ...and bipolar disorder has become a central area of investigation. In this article, we review the evidence for a model of reward sensitivity in mood disorders, with unipolar depression characterized by reward hyposensitivity and bipolar disorders by reward hypersensitivity. We address whether aberrant reward sensitivity and processing are correlates of, mood-independent traits of, vulnerabilities for, and/or predictors of the course of depression and bipolar spectrum disorders, covering evidence from self-report, behavioral, neurophysiological, and neural levels of analysis. We conclude that substantial evidence documents that blunted reward sensitivity and processing are involved in unipolar depression and heightened reward sensitivity and processing are characteristic of hypomania/mania. We further conclude that aberrant reward sensitivity has a trait component, but more research is needed to clearly demonstrate that reward hyposensitivity and hypersensitivity are vulnerabilities for depression and bipolar disorder, respectively. Moreover, additional research is needed to determine whether bipolar depression is similar to unipolar depression and characterized by reward hyposensitivity, or whether like bipolar hypomania/mania, it involves reward hypersensitivity.
PDF
