Abstract
Deciding whether to forgo a good choice in favour of exploring a potentially more rewarding alternative is one of the most challenging arbitrations both in human reasoning and in artificial ...intelligence. Humans show substantial variability in their exploration, and theoretical (but only limited empirical) work has suggested that excessive exploration is a critical mechanism underlying the psychiatric dimension of impulsivity. In this registered report, we put these theories to test using large online samples, dimensional analyses, and computational modelling. Capitalising on recent advances in disentangling distinct human exploration strategies, we not only demonstrate that impulsivity is associated with a specific form of exploration—value-free random exploration—but also explore links between exploration and other psychiatric dimensions.
Adolescence is associated with quickly changing environmental demands which require excellent adaptive skills and high cognitive flexibility. Feedback-guided adaptive learning and cognitive ...flexibility are driven by reward prediction error (RPE) signals, which indicate the accuracy of expectations and can be estimated using computational models. Despite the importance of cognitive flexibility during adolescence, only little is known about how RPE processing in cognitive flexibility deviates between adolescence and adulthood.
In this study, we investigated the developmental aspects of cognitive flexibility by means of computational models and functional magnetic resonance imaging (fMRI). We compared the neural and behavioral correlates of cognitive flexibility in healthy adolescents (12–16years) to adults performing a probabilistic reversal learning task. Using a modified risk-sensitive reinforcement learning model, we found that adolescents learned faster from negative RPEs than adults. The fMRI analysis revealed that within the RPE network, the adolescents had a significantly altered RPE-response in the anterior insula. This effect seemed to be mainly driven by increased responses to negative prediction errors.
In summary, our findings indicate that decision making in adolescence goes beyond merely increased reward-seeking behavior and provides a developmental perspective to the behavioral and neural mechanisms underlying cognitive flexibility in the context of reinforcement learning.
•Adolescents and adults show differences in processing RPEs.•Adolescents learn faster from negative prediction errors.•The anterior insula activation may cause altered sensitivity to RPEs.
Changes in response contingencies require adjusting ones assumptions about outcomes of behaviors. Such adaptation processes are driven by reward prediction error (RPE) signals which reflect the ...inadequacy of expectations. Signals resembling RPEs are known to be encoded by mesencephalic dopamine neurons projecting to the striatum and frontal regions. Although regions that process RPEs, such as the dorsal anterior cingulate cortex (dACC), have been identified, only indirect evidence links timing and network organization of RPE processing in humans. In electroencephalography (EEG), which is well known for its high temporal resolution, the feedback-related negativity (FRN) has been suggested to reflect RPE processing. Recent studies, however, suggested that the FRN might reflect surprise, which would correspond to the absolute, rather than the signed RPE signals. Furthermore, the localization of the FRN remains a matter of debate.
In this simultaneous EEG–functional magnetic resonance imaging (fMRI) study, we localized the FRN directly using the superior spatial resolution of fMRI without relying on any spatial constraint or other assumption. Using two different single-trial approaches, we consistently found a cluster within the dACC. One analysis revealed additional activations of the salience network. Furthermore, we evaluated the effect of signed RPEs and surprise signals on the FRN amplitude. We considered that both signals are usually correlated and found that only surprise signals modulate the FRN amplitude. Last, we explored the pathway of RPE signals using dynamic causal modeling (DCM). We found that the surprise signals are directly projected to the source region of the FRN. This finding contradicts earlier theories about the network organization of the FRN, but is in line with a recent theory stating that dopamine neurons also encode surprise-like saliency signals.
Our findings crucially advance the understanding of the FRN. We found compelling evidence that the FRN originates from the dACC. Furthermore, we clarified the functional role of the FRN, and determined the role of the dACC within the RPE network. These findings should enable us to study the processing of surprise and adjustment signals in the dACC in healthy and also in psychiatric patients.
•The feedback-related negativity (FRN) is associated with surprise signals.•The FRN is rather associated with absolute than signed reward prediction errors.•EEG-informed fMRI consistently locates the FRN in the dorsal anterior cingulum.•Surprise signals are directly projected to the dorsal anterior cingulum.
A prominent source of polarised and entrenched beliefs is confirmation bias, where evidence against one's position is selectively disregarded. This effect is most starkly evident when opposing ...parties are highly confident in their decisions. Here we combine human magnetoencephalography (MEG) with behavioural and neural modelling to identify alterations in post-decisional processing that contribute to the phenomenon of confirmation bias. We show that holding high confidence in a decision leads to a striking modulation of post-decision neural processing, such that integration of confirmatory evidence is amplified while disconfirmatory evidence processing is abolished. We conclude that confidence shapes a selective neural gating for choice-consistent information, reducing the likelihood of changes of mind on the basis of new information. A central role for confidence in shaping the fidelity of evidence accumulation indicates that metacognitive interventions may help ameliorate this pervasive cognitive bias.
Error processing and conflict monitoring are essential executive functions for goal directed actions and adaptation to conflicting information. Although medial frontal regions such as the anterior ...cingulate cortex (ACC) and the pre-supplementary motor area (pre-SMA) are known to be involved in these functions, there is still considerable heterogeneity regarding their spatio-temporal activations. The timing of these functions has been associated with two separable event-related potentials (ERPs) usually localized to the medial frontal wall, one during error processing (ERN — error related negativity) and one during conflict monitoring (N2).
In this study we aimed to spatially and temporally dissociate conflict and error processing using simultaneously recorded EEG and fMRI data from a modified Flanker task in healthy adults. We demonstrate a spatial dissociation of conflict monitoring and error processing along the medial frontal wall, with selective conflict level dependent activation of the SMA/pre-SMA. Activation to error processing was located in the ACC, rostral cingulate zone (RCZ) and pre-SMA. The EEG-informed fMRI analysis revealed that stronger ERN amplitudes are associated with increased activation in a large coherent cluster comprising the ACC, RCZ and pre-SMA, while N2 amplitudes increased with activation in the pre-SMA. Conjunction analysis of EEG-informed fMRI revealed common activation of ERN and N2 in the pre-SMA and divergent activation in the RCZ. No conjoint activation between error processing and conflict monitoring was found with standard fMRI analysis along the medial frontal wall.
Our fMRI findings clearly demonstrate that conflict monitoring and error processing are spatially dissociable along the medial frontal wall. Moreover, the overlap of ERN- and N2-informed fMRI activation in the pre-SMA provides new evidence that these ERP components share conflict related processing functions and are thus not completely separable.
•ACC activation to error processing, pre-SMA to conflict monitoring•More negative ERN amplitude was associated with increased RCZ activation•More negative N2 amplitude was associated with increased pre-SMA activation•Common activation of ERN and N2 in the pre-SMA•Spatial dissociation of conflict and error processing along the medial frontal wall
Optimal decision making mandates organisms learn the relevant features of choice options. Likewise, knowing how much effort we should expend can assume paramount importance. A mesolimbic network ...supports reward learning, but it is unclear whether other choice features, such as effort learning, rely on this same network. Using computational fMRI, we show parallel encoding of effort and reward prediction errors (PEs) within distinct brain regions, with effort PEs expressed in dorsomedial prefrontal cortex and reward PEs in ventral striatum. We show a common mesencephalic origin for these signals evident in overlapping, but spatially dissociable, dopaminergic midbrain regions expressing both types of PE. During action anticipation, reward and effort expectations were integrated in ventral striatum, consistent with a computation of an overall net benefit of a stimulus. Thus, we show that motivationally relevant stimulus features are learned in parallel dopaminergic pathways, with formation of an integrated utility signal at choice.
Successful behaviour depends on the right balance between maximising reward and soliciting information about the world. Here, we show how different types of information-gain emerge when casting ...behaviour as surprise minimisation. We present two distinct mechanisms for goal-directed exploration that express separable profiles of active sampling to reduce uncertainty. 'Hidden state' exploration motivates agents to sample unambiguous observations to accurately infer the (hidden) state of the world. Conversely, 'model parameter' exploration, compels agents to sample outcomes associated with high uncertainty, if they are informative for their representation of the task structure. We illustrate the emergence of these types of information-gain, termed active inference and active learning, and show how these forms of exploration induce distinct patterns of 'Bayes-optimal' behaviour. Our findings provide a computational framework for understanding how distinct levels of uncertainty systematically affect the exploration-exploitation trade-off in decision-making.
Display omitted
•A Toolbox to integrate preprocessing of physiological data and fMRI noise modeling.•Robust preprocessing via iterative peak detection, shown for noisy data and patients.•Flexible ...support of peripheral data formats and noise models (RETROICOR, RVHRCOR).•Fully automated noise correction and performance assessment for group studies.•Integration in fMRI pre-processing pipelines as SPM Toolbox (Batch Editor GUI).
Physiological noise is one of the major confounds for fMRI. A common class of correction methods model noise from peripheral measures, such as ECGs or pneumatic belts. However, physiological noise correction has not emerged as a standard preprocessing step for fMRI data yet due to: (1) the varying data quality of physiological recordings, (2) non-standardized peripheral data formats and (3) the lack of full automatization of processing and modeling physiology, required for large-cohort studies.
We introduce the PhysIO Toolbox for preprocessing of physiological recordings and model-based noise correction. It implements a variety of noise models, such as RETROICOR, respiratory volume per time and heart rate variability responses (RVT/HRV). The toolbox covers all intermediate steps − from flexible read-in of data formats to GLM regressor/contrast creation − without any manual intervention.
We demonstrate the workflow of the toolbox and its functionality for datasets from different vendors, recording devices, field strengths and subject populations. Automatization of physiological noise correction and performance evaluation are reported in a group study (N=35).
The PhysIO Toolbox reproduces physiological noise patterns and correction efficacy of previously implemented noise models. It increases modeling robustness by outperforming vendor-provided peak detection methods for physiological cycles. Finally, the toolbox offers an integrated framework with full automatization, including performance monitoring, and flexibility with respect to the input data.
Through its platform-independent Matlab implementation, open-source distribution, and modular structure, the PhysIO Toolbox renders physiological noise correction an accessible preprocessing step for fMRI data.
Patients with obsessive-compulsive disorder (OCD) can be described as cautious and hesitant, manifesting an excessive indecisiveness that hinders efficient decision making. However, excess caution in ...decision making may also lead to better performance in specific situations where the cost of extended deliberation is small. We compared 16 juvenile OCD patients with 16 matched healthy controls whilst they performed a sequential information gathering task under different external cost conditions. We found that patients with OCD outperformed healthy controls, winning significantly more points. The groups also differed in the number of draws required prior to committing to a decision, but not in decision accuracy. A novel Bayesian computational model revealed that subjective sampling costs arose as a non-linear function of sampling, closely resembling an escalating urgency signal. Group difference in performance was best explained by a later emergence of these subjective costs in the OCD group, also evident in an increased decision threshold. Our findings present a novel computational model and suggest that enhanced information gathering in OCD can be accounted for by a higher decision threshold arising out of an altered perception of costs that, in some specific contexts, may be advantageous.
Celotno besedilo
Dostopno za:
DOBA, IZUM, KILJ, NUK, PILJ, PNG, SAZU, SIK, UILJ, UKNU, UL, UM, UPUK
A well-established notion in cognitive neuroscience proposes that multiple brain systems contribute to choice behaviour. These include: (1) a model-free system that uses values cached from the ...outcome history of alternative actions, and (2) a model-based system that considers action outcomes and the transition structure of the environment. The widespread use of this distinction, across a range of applications, renders it important to index their distinct influences with high reliability. Here we consider the two-stage task, widely considered as a gold standard measure for the contribution of model-based and model-free systems to human choice. We tested the internal/temporal stability of measures from this task, including those estimated via an established computational model, as well as an extended model using drift-diffusion. Drift-diffusion modeling suggested that both choice in the first stage, and RTs in the second stage, are directly affected by a model-based/free trade-off parameter. Both parameter recovery and the stability of model-based estimates were poor but improved substantially when both choice and RT were used (compared to choice only), and when more trials (than conventionally used in research practice) were included in our analysis. The findings have implications for interpretation of past and future studies based on the use of the two-stage task, as well as for characterising the contribution of model-based processes to choice behaviour.
Celotno besedilo
Dostopno za:
DOBA, IZUM, KILJ, NUK, PILJ, PNG, SAZU, SIK, UILJ, UKNU, UL, UM, UPUK
PDF
