One potential application of forensic “brain reading” is to test whether a suspect has previously experienced a crime scene. Here, we investigated whether it is possible to decode real life ...autobiographic exposure to spatial locations using fMRI. In the first session, participants visited four out of eight possible rooms on a university campus. During a subsequent scanning session, subjects passively viewed pictures and videos from these eight possible rooms (four old, four novel) without giving any responses. A multivariate searchlight analysis was employed that trained a classifier to distinguish between “seen” versus “unseen” stimuli from a subset of six rooms. We found that bilateral precuneus encoded information that can be used to distinguish between previously seen and unseen rooms and that also generalized to the two stimuli left out from training. We conclude that activity in bilateral precuneus is associated with the memory of previously visited rooms, irrespective of the identity of the room, thus supporting a parietal contribution to episodic memory for spatial locations. Importantly, we could decode whether a room was visited in real life without the need of explicit judgments about the rooms. This suggests that recognition is an automatic response that can be decoded from fMRI data, thus potentially supporting forensic applications of concealed information tests for crime scene recognition.
Participants were exposed to different spatial locations (four seen and four unseen rooms) a day before the scanning session in which participants were passively viewing images and videos from the locations. Recognition memory of previously visited rooms could be decoded from fMRI data of the precuneus.
In multivariate pattern analysis of neuroimaging data, ‘second-level’ inference is often performed by entering classification accuracies into a t-test vs chance level across subjects. We argue that ...while the random-effects analysis implemented by the t-test does provide population inference if applied to activation differences, it fails to do so in the case of classification accuracy or other ‘information-like’ measures, because the true value of such measures can never be below chance level. This constraint changes the meaning of the population-level null hypothesis being tested, which becomes equivalent to the global null hypothesis that there is no effect in any subject in the population. Consequently, rejecting it only allows to infer that there are some subjects in which there is an information effect, but not that it generalizes, rendering it effectively equivalent to fixed-effects analysis. This statement is supported by theoretical arguments as well as simulations. We review possible alternative approaches to population inference for information-based imaging, converging on the idea that it should not target the mean, but the prevalence of the effect in the population. One method to do so, ‘permutation-based information prevalence inference using the minimum statistic’, is described in detail and applied to empirical data.
•A second level t-test applied to accuracies in MVPA does not provide population inference.•The same holds for other measures used in information-based imaging.•The reason is that the true value of ‘information-like’ measures cannot be below chance level.•This is in contrast to the use of the t-test in univariate analysis which does support generalization.•Population inference in MVPA can be achieved by targeting the effect prevalence instead of the mean.
Recent advances in human neuroimaging have shown that it is possible to accurately decode a person's conscious experience based only on non-invasive measurements of their brain activity. Such 'brain ...reading' has mostly been studied in the domain of visual perception, where it helps reveal the way in which individual experiences are encoded in the human brain. The same approach can also be extended to other types of mental state, such as covert attitudes and lie detection. Such applications raise important ethical issues concerning the privacy of personal thought.
Items held in working memory can be either attended or not, depending on their current behavioral relevance. It has been suggested that unattended contents might be solely retained in an ...activity-silent form. Instead, we demonstrate here that encoding unattended contents involves a division of labor. While visual cortex only maintains attended items, intraparietal areas and the frontal eye fields represent both attended and unattended items.
The increase in spatiotemporal resolution of neuroimaging devices is accompanied by a trend towards more powerful multivariate analysis methods. Often it is desired to interpret the outcome of these ...methods with respect to the cognitive processes under study. Here we discuss which methods allow for such interpretations, and provide guidelines for choosing an appropriate analysis for a given experimental goal: For a surgeon who needs to decide where to remove brain tissue it is most important to determine the origin of cognitive functions and associated neural processes. In contrast, when communicating with paralyzed or comatose patients via brain–computer interfaces, it is most important to accurately extract the neural processes specific to a certain mental state. These equally important but complementary objectives require different analysis methods. Determining the origin of neural processes in time or space from the parameters of a data-driven model requires what we call a forward model of the data; such a model explains how the measured data was generated from the neural sources. Examples are general linear models (GLMs). Methods for the extraction of neural information from data can be considered as backward models, as they attempt to reverse the data generating process. Examples are multivariate classifiers. Here we demonstrate that the parameters of forward models are neurophysiologically interpretable in the sense that significant nonzero weights are only observed at channels the activity of which is related to the brain process under study. In contrast, the interpretation of backward model parameters can lead to wrong conclusions regarding the spatial or temporal origin of the neural signals of interest, since significant nonzero weights may also be observed at channels the activity of which is statistically independent of the brain process under study. As a remedy for the linear case, we propose a procedure for transforming backward models into forward models. This procedure enables the neurophysiological interpretation of the parameters of linear backward models. We hope that this work raises awareness for an often encountered problem and provides a theoretical basis for conducting better interpretable multivariate neuroimaging analyses.
•Backward models cannot be interpreted in terms of the studied brain processes.•This affects common classification and regression techniques like SVM and LASSO.•The problem does not occur for forward models (e.g., GLMs).•We propose a way to transform linear backward models into linear forward models.•This makes backward models interpretable in terms of the studied brain processes.
Multi-voxel pattern analysis (MVPA) is a fruitful and increasingly popular complement to traditional univariate methods of analyzing neuroimaging data. We propose to replace the standard ‘decoding’ ...approach to searchlight-based MVPA, measuring the performance of a classifier by its accuracy, with a method based on the multivariate form of the general linear model. Following the well-established methodology of multivariate analysis of variance (MANOVA), we define a measure that directly characterizes the structure of multi-voxel data, the pattern distinctness D. Our measure is related to standard multivariate statistics, but we apply cross-validation to obtain an unbiased estimate of its population value, independent of the amount of data or its partitioning into ‘training’ and ‘test’ sets. The estimate D^ can therefore serve not only as a test statistic, but also as an interpretable measure of multivariate effect size. The pattern distinctness generalizes the Mahalanobis distance to an arbitrary number of classes, but also the case where there are no classes of trials because the design is described by parametric regressors. It is defined for arbitrary estimable contrasts, including main effects (pattern differences) and interactions (pattern changes). In this way, our approach makes the full analytical power of complex factorial designs known from univariate fMRI analyses available to MVPA studies. Moreover, we show how the results of a factorial analysis can be used to obtain a measure of pattern stability, the equivalent of ‘cross-decoding’.
•Cross-validated MANOVA is proposed as a replacement of classification.•The cvMANOVA overcomes several limitations of standard classification.•A measure of pattern distinctness D^ is defined.•D^ is a test statistic and provides an unbiased estimate of multivariate effect size.•cvMANOVA can quantify pattern differences, pattern changes, and pattern stability.
Listening to music often evokes intense emotions 1, 2. Recent research suggests that musical pleasure comes from positive reward prediction errors, which arise when what is heard proves to be better ...than expected 3. Central to this view is the engagement of the nucleus accumbens—a brain region that processes reward expectations—to pleasurable music and surprising musical events 4–8. However, expectancy violations along multiple musical dimensions (e.g., harmony and melody) have failed to implicate the nucleus accumbens 9–11, and it is unknown how music reward value is assigned 12. Whether changes in musical expectancy elicit pleasure has thus remained elusive 11. Here, we demonstrate that pleasure varies nonlinearly as a function of the listener’s uncertainty when anticipating a musical event, and the surprise it evokes when it deviates from expectations. Taking Western tonal harmony as a model of musical syntax, we used a machine-learning model 13 to mathematically quantify the uncertainty and surprise of 80,000 chords in US Billboard pop songs. Behaviorally, we found that chords elicited high pleasure ratings when they deviated substantially from what the listener had expected (low uncertainty, high surprise) or, conversely, when they conformed to expectations in an uninformative context (high uncertainty, low surprise). Neurally, we found using fMRI that activity in the amygdala, hippocampus, and auditory cortex reflected this interaction, while the nucleus accumbens only reflected uncertainty. These findings challenge current neurocognitive models of music-evoked pleasure and highlight the synergistic interplay between prospective and retrospective states of expectation in the musical experience.
Display omitted
•Musical pleasure depends on prospective and retrospective states of expectation•A machine-learning model quantified the uncertainty and surprise of pop song chords•Chords with low uncertainty and high surprise, and vice versa, evoked high pleasure•Joint effects of uncertainty and surprise found in the amygdala and auditory cortex
Cheung et al. use a machine-learning model to mathematically quantify the predictive uncertainty and surprise of 80,000 chords in 745 commercially successful pop songs. The authors further show that chord uncertainty and surprise jointly modulate musical pleasure, as well as activity in the amygdala, hippocampus, and auditory cortex using fMRI.
It has been suggested that visual images are memorized across brief periods of time by vividly imagining them as if they were still there. In line with this, the contents of both working memory and ...visual imagery are known to be encoded already in early visual cortex. If these signals in early visual areas were indeed to reflect a combined imagery and memory code, one would predict them to be weaker for individuals with reduced visual imagery vividness. Here, we systematically investigated this question in two groups of participants. Strong and weak imagers were asked to remember images across brief delay periods. We were able to reliably reconstruct the memorized stimuli from early visual cortex during the delay. Importantly, in contrast to the prediction, the quality of reconstruction was equally accurate for both strong and weak imagers. The decodable information also closely reflected behavioral precision in both groups, suggesting it could contribute to behavioral performance, even in the extreme case of completely aphantasic individuals. Our data thus suggest that working memory signals in early visual cortex can be present even in the (near) absence of phenomenal imagery.
Working memory signals in early visual cortex are thought to arise because people engage in vivid imagery to maintain visual information. We test this by measuring working memory signals in visual areas of people with strong and weak imagery abilities. We observed strong working memory signals irrespective of imagery ability, and signal strength was equally predictive of task performance in both groups. Thus, working memory in visual cortex is not necessarily linked to imagery.
There has been a long controversy as to whether subjectively 'free' decisions are determined by brain activity ahead of time. We found that the outcome of a decision can be encoded in brain activity ...of prefrontal and parietal cortex up to 10 s before it enters awareness. This delay presumably reflects the operation of a network of high-level control areas that begin to prepare an upcoming decision long before it enters awareness.
Predicting free choices for abstract intentions Soon, Chun Siong; He, Anna Hanxi; Bode, Stefan ...
Proceedings of the National Academy of Sciences - PNAS,
04/2013, Letnik:
110, Številka:
15
Journal Article
Recenzirano
Odprti dostop
Unconscious neural activity has been repeatedly shown to precede and potentially even influence subsequent free decisions. However, to date, such findings have been mostly restricted to simple motor ...choices, and despite considerable debate, there is no evidence that the outcome of more complex free decisions can be predicted from prior brain signals. Here, we show that the outcome of a free decision to either add or subtract numbers can already be decoded from neural activity in medial prefrontal and parietal cortex 4 s before the participant reports they are consciously making their choice. These choice-predictive signals co-occurred with the so-called default mode brain activity pattern that was still dominant at the time when the choice-predictive signals occurred. Our results suggest that unconscious preparation of free choices is not restricted to motor preparation. Instead, decisions at multiple scales of abstraction evolve from the dynamics of preceding brain activity.
PDF
