Humans have been shown to strategically explore. They can identify situations in which gathering information about distant and uncertain options is beneficial for the future. Because primates rely on ...scarce resources when they forage, they are also thought to strategically explore, but whether they use the same strategies as humans and the neural bases of strategic exploration in monkeys are largely unknown. We designed a sequential choice task to investigate whether monkeys mobilize strategic exploration based on whether information can improve subsequent choice, but also to ask the novel question about whether monkeys adjust their exploratory choices based on the contingency between choice and information, by sometimes providing the counterfactual feedback about the unchosen option. We show that monkeys decreased their reliance on expected value when exploration could be beneficial, but this was not mediated by changes in the effect of uncertainty on choices. We found strategic exploratory signals in anterior and mid-cingulate cortex (ACC/MCC) and dorsolateral prefrontal cortex (dlPFC). This network was most active when a low value option was chosen, which suggests a role in counteracting expected value signals, when exploration away from value should to be considered. Such strategic exploration was abolished when the counterfactual feedback was available. Learning from counterfactual outcome was associated with the recruitment of a different circuit centered on the medial orbitofrontal cortex (OFC), where we showed that monkeys represent chosen and unchosen reward prediction errors. Overall, our study shows how ACC/MCC-dlPFC and OFC circuits together could support exploitation of available information to the fullest and drive behavior towards finding more information through exploration when it is beneficial.
Abstract Staying engaged is necessary to maintain goal-directed behaviors. Despite this, engagement exhibits continuous, intrinsic fluctuations. Even in experimental settings, animals, unlike most ...humans, repeatedly and spontaneously move between periods of complete task engagement and disengagement. We, therefore, looked at behavior in male macaques ( macaca mulatta) in four tasks while recording fMRI signals. We identified consistent autocorrelation in task disengagement. This made it possible to build models capturing task-independent engagement. We identified task general patterns of neural activity linked to impending sudden task disengagement in mid-cingulate gyrus. By contrast, activity centered in perigenual anterior cingulate cortex (pgACC) was associated with maintenance of performance across tasks. Importantly, we carefully controlled for task-specific factors such as the reward history and other motivational effects, such as response vigor, in our analyses. Moreover, we showed pgACC activity had a causal link to task engagement: transcranial ultrasound stimulation of pgACC changed task engagement patterns.
Reward prediction errors (RPEs) and risk preferences have two things in common: both can shape decision making behavior, and both are commonly associated with dopamine. RPEs drive value learning and ...are thought to be represented in the phasic release of striatal dopamine. Risk preferences bias choices towards or away from uncertainty; they can be manipulated with drugs that target the dopaminergic system. Based on the common neural substrate, we hypothesize that RPEs and risk preferences are linked on the level of behavior as well. Here, we develop this hypothesis theoretically and test it empirically. First, we apply a recent theory of learning in the basal ganglia to predict how RPEs influence risk preferences. We find that positive RPEs should cause increased risk-seeking, while negative RPEs should cause risk-aversion. We then test our behavioral predictions using a novel bandit task in which value and risk vary independently across options. Critically, conditions are included where options vary in risk but are matched for value. We find that our prediction was correct: participants become more risk-seeking if choices are preceded by positive RPEs, and more risk-averse if choices are preceded by negative RPEs. These findings cannot be explained by other known effects, such as nonlinear utility curves or dynamic learning rates.
Animals learn from the past to make predictions. These predictions are adjusted after prediction errors, i.e., after surprising events. Generally, most reward prediction errors models learn the ...average expected amount of reward. However, here we demonstrate the existence of distinct mechanisms for detecting other types of surprising events. Six macaques learned to respond to visual stimuli to receive varying amounts of juice rewards. Most trials ended with the delivery of either 1 or 3 juice drops so that animals learned to expect 2 juice drops on average even though instances of precisely 2 drops were rare. To encourage learning, we also included sessions during which the ratio between 1 and 3 drops changed. Additionally, in all sessions, the stimulus sometimes appeared in an unexpected location. Thus, 3 types of surprising events could occur: reward amount surprise (i.e., a scalar reward prediction error), rare reward surprise, and visuospatial surprise. Importantly, we can dissociate scalar reward prediction errors-rewards that deviated from the average reward amount expected-and rare reward events-rewards that accorded with the average reward expectation but that rarely occurred. We linked each type of surprise to a distinct pattern of neural activity using functional magnetic resonance imaging. Activity in the vicinity of the dopaminergic midbrain only reflected surprise about the amount of reward. Lateral prefrontal cortex had a more general role in detecting surprising events. Posterior lateral orbitofrontal cortex specifically detected rare reward events regardless of whether they followed average reward amount expectations, but only in learnable reward environments.
The concept of "prediction error" - the difference between what occurred and was expected - is key to understanding the cognitive processes of human decision making. Expectations have to be learned ...so the concept of prediction error critically depends on context, specifically the temporal context of probabilistically related events and their changes across time (i.e. volatility). While past research suggests context differently affects some cognitive processes in East Asian and Western individuals, it is currently unknown whether this extends to computationally-grounded measures of learning and prediction error. Here we compared Chinese and British nationals in an associative learning task that quantifies behavioural effects of prediction error, and-through a hierarchical Bayesian learning model-also captures how individuals learn about probabilistic relationships and their volatility. For comparison, we also administered a psychophysical task, the tilt illusion, to assess cultural differences in susceptibility to spatial context. We found no cultural differences in the effect of spatial context on perception. In the domain of temporal context there was no effect of culture on sensitivity to prediction error, or learning about volatility, but some suggestion that Chinese individuals may learn more readily about probabilistic relationships.
A note on empathy in games Grohn, Jan; Huck, Steffen; Valasek, Justin Mattias
Journal of economic behavior & organization,
12/2014, Volume:
108
Journal Article
Peer reviewed
In this note we shall discuss a concept that – despite its prominence in both Hume (1739) and Smith (1759), its obvious relevance for social behavior, and its not so infrequent use in colloquial ...language – has never gained a foothold in economic theory: the concept of empathy. Specifically, we illustrate how some insights from the psychological literature on empathy can be incorporated into a standard utility framework, and demonstrate the potential interaction of beliefs and utility through the channel of empathy.
Provider: - Institution: - Data provided by Europeana Collections- In diesem Artikel diskutieren wir das Konzept der Empathie. Dieses konnte in der ökonomischen Theorie nie wirklich Fuß fassen, trotz ...seiner Bedeutung sowohl bei Hume (1739) als auch Smith (1759), seiner offensichtlichen Relevanz für soziales Verhalten und seines durchaus verbreiteten Gebrauchs in der Umgangssprache. Insbesondere zeigen wir, wie einige Erkenntnisse aus der psychologischen Literatur über Empathie in ein Standardkonzept von "Nutzen" integriert werden können und demonstrieren die potenzielle Interaktion von Erwartungen und Nutzen über den Weg der Empathie. (Autorenreferat)- In this note we shall discuss a concept that - despite its prominence in both Hume (1739) and Smith (1759), its obvious relevance for social behavior, and its not so infrequent use in colloquial language - has never gained a foothold in economic theory: the concept of empathy. Specifically, we illustrate how some insights from the psychological literature on empathy can be incorporated into a standard utility framework, and demonstrate the potential interaction of beliefs and utility through the channel of empathy. (author's abstract)- All metadata published by Europeana are available free of restriction under the Creative Commons CC0 1.0 Universal Public Domain Dedication. However, Europeana requests that you actively acknowledge and give attribution to all metadata sources including Europeana
PDF
