Dorsal anterior cingulate cortex (dACC) carries a wealth of value-related information necessary for regulating behavioral flexibility and persistence. It signals error and reward events informing ...decisions about switching or staying with current behavior. During decision-making, it encodes the average value of exploring alternative choices (search value), even after controlling for response selection difficulty, and during learning, it encodes the degree to which internal models of the environment and current task must be updated. dACC value signals are derived in part from the history of recent reward integrated simultaneously over multiple time scales, thereby enabling comparison of experience over the recent and extended past. Such ACC signals may instigate attentionally demanding and difficult processes such as behavioral change via interactions with prefrontal cortex. However, the signal in dACC that instigates behavioral change need not itself be a conflict or difficulty signal.
The neural mechanisms mediating sensory-guided decision-making have received considerable attention, but animals often pursue behaviors for which there is currently no sensory evidence. Such ...behaviors are guided by internal representations of choice values that have to be maintained even when these choices are unavailable. We investigated how four macaque monkeys maintained representations of the value of counterfactual choices-choices that could not be taken at the current moment but which could be taken in the future. Using functional magnetic resonance imaging, we found two different patterns of activity co-varying with values of counterfactual choices in a circuit spanning the hippocampus, the anterior lateral prefrontal cortex and the anterior cingulate cortex. Anterior cingulate cortex activity also reflected whether the internal value representations would be translated into actual behavioral change. To establish the causal importance of the anterior cingulate cortex for this translation process, we used a novel technique, transcranial focused ultrasound stimulation, to reversibly disrupt anterior cingulate cortex activity.
Neural Mechanisms of Foraging Kolling, Nils; Behrens, Timothy E. J.; Mars, Rogier B. ...
Science (American Association for the Advancement of Science),
04/2012, Volume:
336, Issue:
6077
Journal Article
Peer reviewed
Open access
Behavioral economic studies involving limited numbers of choices have provided key insights into neural decision-making mechanisms. By contrast, animals' foraging choices arise in the context of ...sequences of encounters with prey or food. On each encounter, the animal chooses whether to engage or, if the environment is sufficiently rich, to search elsewhere. The cost of foraging is also critical. We demonstrate that humans can alternate between two modes of choice, comparative decision-making and foraging, depending on distinct neural mechanisms in ventromedial prefrontal cortex (vmPFC) and anterior cingulate cortex (ACC) using distinct reference frames; in ACC, choice variables are represented in invariant reference to foraging or searching for alternatives. Whereas vmPFC encodes values of specific well-defined options, ACC encodes the average value of the foraging environment and cost of foraging.
Humans have been shown to strategically explore. They can identify situations in which gathering information about distant and uncertain options is beneficial for the future. Because primates rely on ...scarce resources when they forage, they are also thought to strategically explore, but whether they use the same strategies as humans and the neural bases of strategic exploration in monkeys are largely unknown. We designed a sequential choice task to investigate whether monkeys mobilize strategic exploration based on whether information can improve subsequent choice, but also to ask the novel question about whether monkeys adjust their exploratory choices based on the contingency between choice and information, by sometimes providing the counterfactual feedback about the unchosen option. We show that monkeys decreased their reliance on expected value when exploration could be beneficial, but this was not mediated by changes in the effect of uncertainty on choices. We found strategic exploratory signals in anterior and mid-cingulate cortex (ACC/MCC) and dorsolateral prefrontal cortex (dlPFC). This network was most active when a low value option was chosen, which suggests a role in counteracting expected value signals, when exploration away from value should to be considered. Such strategic exploration was abolished when the counterfactual feedback was available. Learning from counterfactual outcome was associated with the recruitment of a different circuit centered on the medial orbitofrontal cortex (OFC), where we showed that monkeys represent chosen and unchosen reward prediction errors. Overall, our study shows how ACC/MCC-dlPFC and OFC circuits together could support exploitation of available information to the fullest and drive behavior towards finding more information through exploration when it is beneficial.
When choosing between two options, correlates of their value are represented in neural activity throughout the brain. Whether these representations reflect activity that is fundamental to the ...computational process of value comparison, as opposed to other computations covarying with value, is unknown. We investigated activity in a biophysically plausible network model that transforms inputs relating to value into categorical choices. A set of characteristic time-varying signals emerged that reflect value comparison. We tested these model predictions using magnetoencephalography data recorded from human subjects performing value-guided decisions. Parietal and prefrontal signals matched closely with model predictions. These results provide a mechanistic explanation of neural signals recorded during value-guided choice and a means of distinguishing computational roles of different cortical regions whose activity covaries with value.
To survive, humans must estimate their own ability and the abilities of others. We found that, although people estimated their abilities on the basis of their own performance in a rational manner, ...their estimates of themselves were partly merged with the performance of others. Reciprocally, their ability estimates for others also reflected their own, as well as the others’, performance. Self-other mergence operated in a context-dependent manner: interacting with high or low performers, respectively, enhanced and diminished own ability estimates in cooperative contexts, but the opposite occurred in competitive contexts. Self-other mergence not only influenced subjective evaluations, it also affected how people subsequently objectively adjusted their performance. Perigenual anterior cingulate cortex tracked one’s own performance. Dorsomedial frontal area 9 tracked others’ performances, but also integrated contextual and self-related information. Self-other mergence increased with the strength of self and other representations in area 9, suggesting it carries interdependent representations of self and other.
•People confuse their own performance with the performance of others•Cooperation leads to positive and competition to negative self-other confusion•Confusion is reflected in area 9 indicating interdependent self-other processing•Learning from own performance history is represented in perigenual ACC
Wittmann et al. find that we confuse own performance with the performance of others, depending on whether we cooperate or compete with them. This confusion is reflected in area 9, indicating it interdependently processes information about ourselves and others.
Real-world decisions have benefits occurring only later and dependent on additional decisions taken in the interim. We investigated this in a novel decision-making task in humans (n = 76) while ...measuring brain activity with fMRI (n = 24). Modeling revealed that participants computed the prospective value of decisions: they planned their future behavior taking into account how their decisions might affect which states they would encounter and how they themselves might respond in these states. They considered their own likely future behavioral biases (e.g., failure to adapt to changes in prospective value) and avoided situations in which they might be prone to such biases. Three neural networks in adjacent medial frontal regions were linked to distinct components of prospective decision making: activity in dorsal anterior cingulate cortex, area 8 m/9, and perigenual anterior cingulate cortex reflected prospective value, anticipated changes in prospective value, and the degree to which prospective value influenced decisions.
•People use prospective information about future state when making sequential choices•People have insight into their future biases and pre-emptively avoid key situations•Distinct cingulate-prefrontal interactions encode prospective and myopic value•Perigenual cingulate motivates perseveration in sequential behavior
Kolling and Scholl et al. examined extended sequential decision-making problems. Participants used prospective information to inform even initial decision planning several steps ahead. Dorsal cingulate activity signaled prospective value while perigenual cingulate activity motivated decision sequence performance.
Real-life decision-making often comprises sequences of successive decisions about whether to take opportunities as they are encountered or keep searching for better ones instead. We investigated ...individual differences related to such sequential decision-making and link them especially to apathy and compulsivity in a large online sample (discovery sample: n = 449 and confirmation sample: n = 756). Our cognitive model revealed distinct changes in the way participants evaluated their environments and planned their own future behaviour. Apathy was linked to decision inertia, i.e., automatically persisting with a sequence of searches for longer than appropriate given the value of searching. Thus, despite being less motivated, they did not avoid the effort associated with longer searches. In contrast, compulsivity was linked to self-reported insensitivity to the cost of continuing with a sequence of searches. The objective measures of behavioural cost insensitivity were clearly linked to compulsivity only in the discovery sample. While the confirmation sample showed a similar effect, it did not reach significance. Nevertheless, in both samples, participants reported awareness of such bias (experienced as "overchasing"). In addition, this awareness made them report preemptively avoiding situations related to the bias. However, we found no evidence of them actually preempting more in the task, which might mean a misalignment of their metacognitive beliefs or that our behavioural measures were incomplete. In summary, individual variation in distinct, fundamental aspects of sequential decision-making can be linked to variation in 2 measures of behavioural traits associated with psychological illness in the normal population.
In many natural environments the value of a choice gradually gets better or worse as circumstances change. Discerning such trends makes predicting future choice values possible. We show that humans ...track such trends by comparing estimates of recent and past reward rates, which they are able to hold simultaneously in the dorsal anterior cingulate cortex (dACC). Comparison of recent and past reward rates with positive and negative decision weights is reflected by opposing dACC signals indexing these quantities. The relative strengths of time-linked reward representations in dACC predict whether subjects persist in their current behaviour or switch to an alternative. Computationally, trend-guided choice can be modelled by using a reinforcement-learning mechanism that computes a longer-term estimate (or expectation) of prediction errors. Using such a model, we find a relative predominance of expected prediction errors in dACC, instantaneous prediction errors in the ventral striatum and choice signals in the ventromedial prefrontal cortex.
In complex environments, many potential cues can guide a decision or be assigned responsibility for the outcome of the decision. We know little, however, about how humans and animals select relevant ...information sources that should guide behavior. We show that subjects solve this relevance selection and credit assignment problem by selecting one cue and its association with a particular outcome as the main focus of a hypothesis. To do this, we examined learning while using a task design that allowed us to estimate the focus of each subject's hypotheses on a trial-by-trial basis. When a prediction is confirmed by the outcome, then credit for the outcome is assigned to that cue rather than an alternative. Activity in medial frontal cortex is associated with the assignment of credit to the cue that is the main focus of the hypothesis. However, when the outcome disconfirms a prediction, the focus shifts between cues, and the credit for the outcome is assigned to an alternative cue. This process of reselection for credit assignment to an alternative cue is associated with lateral orbitofrontal cortex.
Learners should infer which features of environments are predictive of significant events, such as rewards. This "credit assignment" problem is particularly challenging when any of several cues might be predictive. We show that human subjects solve the credit assignment problem by implicitly "hypothesizing" which cue is relevant for predicting subsequent outcomes, and then credit is assigned according to this hypothesis. This process is associated with a distinctive pattern of activity in a part of medial frontal cortex. By contrast, when unexpected outcomes occur, hypotheses are redirected toward alternative cues, and this process is associated with activity in lateral orbitofrontal cortex.
PDF
