Early visual areas contain specific information about visual items maintained in working memory, suggesting a role for early visual cortex in more complex cognitive functions 1–4. It is an open ...question, however, whether these areas also underlie the ability to internally generate images de novo (i.e., mental imagery). Research on mental imagery has to this point focused mostly on whether mental images activate early sensory areas, with mixed results 5–7. Recent studies suggest that multivariate pattern analysis of neural activity patterns in visual regions can reveal content-specific representations during cognitive processes, even though overall activation levels are low 1–4. Here, we used this approach 8, 9 to study item-specific activity patterns in early visual areas (V1–V3) when these items are internally generated. We could reliably decode stimulus identity from neural activity patterns in early visual cortex during both working memory and mental imagery. Crucially, these activity patterns resembled those evoked by bottom-up visual stimulation, suggesting that mental images are indeed “perception-like” in nature. These findings suggest that the visual cortex serves as a dynamic “blackboard” 10, 11 that is used during both bottom-up stimulus processing and top-down internal generation of mental content.
•Mental images can be decoded from activity patterns in human early visual cortex•Activity patterns evoked by mental images resemble stimulus-driven activity patterns•Mental imagery and working memory have shared representations in early visual cortex
What makes a colored image, e.g. an abstract painting or a landscape, look pleasing? We hypothesized that a preference for complex color compositions, such as paintings and images of natural scenes, ...might be related to how natural the colors are perceived. We tested this possibility with two experiments in which the degree of naturalness of images was manipulated by rotating their color gamut rigidly in the color space CIELAB. This changed just the hue composition, but preserved saturation and lightness. In the first experiment we obtained individual scaling curves for perceived naturalness and for preference as a function of the angle of gamut rotation for a small set of images. The naturalness and preference scaling curves were found to be largely similar and their maxima were close to the original image. In the second experiment, we tested whether this effect generalized to a larger set of images. We used a simultaneous 5AFC procedure where in each trial participants had to select the most natural or the most preferred image from five different rotations of the color gamut. The results confirmed the first experiment and showed that, in general, the images perceived as the more natural tend to be the ones that are preferred. Together these results show that perceived naturalness and preference are indeed perceptually closely related and may be driven by related mechanisms.
In daily life, people show remarkable flexibility in adapting to novel circumstances. Although there is general agreement on which brain areas are involved in cognitive flexibility, little is known ...about the precise representational content of these cognitive control areas in different sub-processes involved in cognitive control. In the present study, we used an adaptation approach to differentiate the brain areas selectively representing the what and the how components of cognitive control in task preparation. When selectively repeating the task goal (the what component) without repeating the stimulus–response (S-R) mapping (the how component), task goal preferential adaptation was found in the left lateral prefrontal cortex, the medial prefrontal cortex and the left posterior parietal cortex. Within these areas, task goal specific adaptation was found in the left inferior frontal gyrus, the posterior part of the left inferior parietal lobule and the precuneus. Selectively repeating the S-R mapping, by contrast, resulted in S-R mapping preferential adaptation in the bilateral pre-central gyrus extending bilaterally to the intra-parietal lobule, indicating representation of the how component in these areas. Adaptation general to both task goal and S-R mapping was only found in Broca's area extending to the inferior frontal junction, suggesting that the what and the how components of cognitive control are similarly represented in this part of the brain.
► Brain areas representing the what and the how of cognitive control were examined. ► Adaptation to selectively repeating the task goal and the SR-mapping was studied. ► Adaptation offers an opportunity to study representations in cognitive control areas.
Multivariate pattern analysis (MVPA) of fMRI data has allowed the investigation of neural representations of stimuli on the basis of distributed patterns of activity within a brain region, ...independently from overall brain activity. For instance, several studies on early visual cortex have reported reliable MVPA decoding of the identity of a stimulus representation that was kept in working memory or internally generated, despite the fact that the overall BOLD response was low or even at baseline levels. Here we ask how it is possible that reliable stimulus information can be decoded from early visual cortex even when the overall BOLD signal remains low. We reanalyzed a data set in which human participants (N = 24) imagined or kept in working memory an oriented visual grating. We divided voxels from V1, V2, and V3 into groups based on orientation preference, and compared the time course of mean BOLD responses to preferred and non-preferred orientations with the time course of the multivariate decoding performance.
Decoding accuracy related to a numerically small, but reliable univariate difference in the mean BOLD response to preferred and non-preferred stimuli. The time course of the difference in BOLD responses to preferred and non-preferred orientations was highly similar to the time course of the multivariate pattern classification accuracy. The reliability of the classification strongly correlated with the magnitude of differences in BOLD signal between preferred and non-preferred stimuli. These activity differences were small compared to the large overall BOLD modulations. This suggests that a substantial part of the task-related BOLD response to visual stimulation might not be stimulus-specific. Rather, stimulus-evoked BOLD signals in early visual cortex during a task context may be an amalgam of small stimulus-specific responses and large task-related but non-stimulus-specific responses. The latter are not evident during the maintenance or internal generation of stimulus representations, but provide an explanation of how reliable stimulus information can be decoded from early visual cortex even though its overall BOLD signal remains low.
•Orientation-selective V1 voxels are more active when maintaining preferred stimuli.•Orientation-selective activity differences in V1 correlate with decoding accuracy.•BOLD is an amalgam of stimulus-specific and non-stimulus-specific processes.
•Color can be an important contributor to our appreciation of art.•Naïve observers prefer original paintings over colour-manipulated versions.•Observers also prefer original colours in spatially ...scrambled images of our paintings.•In abstract works color preference is not anchored to spatial configurations.
Walking around an art museum we can see how colours influence our aesthetic preferences: many great works of art would not be as impressive in grey scales. Is the beauty of colours in abstract paintings anchored to the spatial composition of the paintings, or can it be preserved even with random spatial arrangements? To test whether colour can have an independent contribution to aesthetic appreciation, we asked participants to select the preferred image among pairs of colour-manipulated versions of the same painting. We changed hue, but preserved lightness and saturation, by rotating the colour volume around the L* axis in CIELAB space. To test the influence of the spatial structure, the images of the paintings were presented: (1) in their original format, (2) spatially scrambled but preserving the colour composition, and (3) in a control condition with both colour and spatial scrambling. Relative preference as a function of hue angle was obtained for the four paintings in their original and modified forms.
For the original paintings, we found that participants generally preferred the colour angles that matched the original version of the paintings. Crucially, participants preferred the same colour distributions for spatially scrambled paintings as for the original paintings. For the control condition, there were no preferred colour configurations. This suggests that the aesthetic preference of colours in our abstract paintings is not anchored to particular spatial compositions, but is at least partly preserved even when the spatial composition is destroyed. Paintings thus can contain an aesthetic component that is exclusively related to colour.
A relatively new analysis technique, known as neural decoding or multivariate pattern analysis (MVPA), has become increasingly popular for cognitive neuroimaging studies over recent years. These ...techniques promise to uncover the representational contents of neural signals, as well as the underlying code and the dynamic profile thereof. A field in which these techniques have led to novel insights in particular is that of visual working memory (VWM). In the present study, we subjected human volunteers to a combined VWM/imagery task while recording their neural signals using magnetoencephalography (MEG). We applied multivariate decoding analyses to uncover the temporal profile underlying the neural representations of the memorized item. Analysis of gaze position however revealed that our results were contaminated by systematic eye movements, suggesting that the MEG decoding results from our originally planned analyses were confounded. In addition to the eye movement analyses, we also present the original analyses to highlight how these might have readily led to invalid conclusions. Finally, we demonstrate a potential remedy, whereby we train the decoders on a functional localizer that was specifically designed to target bottom-up sensory signals and as such avoids eye movements. We conclude by arguing for more awareness of the potentially pervasive and ubiquitous effects of eye movement-related confounds.
PDF
