Studies in anesthetized animals have suggested that activity in early visual cortex is mainly driven by visual input and is well described by a feedforward processing hierarchy. However, evidence ...from experiments on awake animals has shown that both eye movements and behavioral state can strongly modulate responses of neurons in visual cortex; the functional significance of this modulation, however, remains elusive. Using visual-flow feedback manipulations during locomotion in a virtual reality environment, we found that responses in layer 2/3 of mouse primary visual cortex are strongly driven by locomotion and by mismatch between actual and expected visual feedback. These data suggest that processing in visual cortex may be based on predictive coding strategies that use motor-related and visual input to detect mismatches between predicted and actual visual feedback.
► Both visual and motor-related responses are present in mouse primary visual cortex ► Motor-related activity occurs even in the complete absence of visual input ► Mismatch between predicted and actual visual feedback drives responses ► Feedback mismatch responses are contingent on previous experience
Keller et al. measure activity in primary visual cortex of behaving mice. They find that locomotion and mismatch between actual and expected visual feedback, and not visual input as such, are the main determinants of neural activity.
We determined how learning modifies neural representations in primary visual cortex (V1) during acquisition of a visually guided behavioral task. We imaged the activity of the same layer 2/3 neuronal ...populations as mice learned to discriminate two visual patterns while running through a virtual corridor, where one pattern was rewarded. Improvements in behavioral performance were closely associated with increasingly distinguishable population-level representations of task-relevant stimuli, as a result of stabilization of existing and recruitment of new neurons selective for these stimuli. These effects correlated with the appearance of multiple task-dependent signals during learning: those that increased neuronal selectivity across the population when expert animals engaged in the task, and those reflecting anticipation or behavioral choices specifically in neuronal subsets preferring the rewarded stimulus. Therefore, learning engages diverse mechanisms that modify sensory and non-sensory representations in V1 to adjust its processing to task requirements and the behavioral relevance of visual stimuli.
•V1 neurons increasingly discriminate task-relevant stimuli with learning•Chronic imaging reveals single cell changes underlying this population effect•Learning-related changes are reduced when animals ignore task-relevant stimuli•Anticipatory and behavioral choice-related signals emerge in reward-predicting cells
By tracking the same visual cortex neurons across days, Poort et al. demonstrate how learning a visual task leads to increasingly distinguishable representations of relevant stimuli. These changes parallel the emergence of diverse non-sensory signals in specific neuronal subsets.
Processing in cortical circuits is driven by combinations of cortical and subcortical inputs. These inputs are often conceptually categorized as bottom-up, conveying sensory information, and ...top-down, conveying contextual information. Using intracellular recordings in mouse primary visual cortex, we measured neuronal responses to visual input, locomotion, and visuomotor mismatches. We show that layer 2/3 (L2/3) neurons compute a difference between top-down motor-related input and bottom-up visual flow input. Most L2/3 neurons responded to visuomotor mismatch with either hyperpolarization or depolarization, and the size of this response was correlated with distinct physiological properties. Consistent with a subtraction of bottom-up and top-down input, visual and motor-related inputs had opposing influence on L2/3 neurons. In infragranular neurons, we found no evidence of a difference computation and responses were consistent with positive integration of visuomotor inputs. Our results provide evidence that L2/3 functions as a bidirectional comparator of top-down and bottom-up input.
•Layer 2/3 neurons show widespread subthreshold mismatch responses•Mismatch response sign is predicted by visual flow and locomotion-related responses•Layer 5/6 has a scarcity of depolarizing mismatch responses•Visual flow and locomotion speed have opposing signs of influence only in layer 2/3
Jordan and Keller use whole cell recordings in mice navigating in virtual reality to show that neurons only in superficial cortical layers have a special property: they integrate visual flow and locomotion speed with opposing signs, allowing them to compute bidirectional mismatches between actual and expected visual flow speeds.
Homeostatic plasticity is important to maintain a set level of activity in neuronal circuits and has been most extensively studied in cell cultures following activity blockade. It is still unclear, ...however, whether activity changes associated with mechanisms of homeostatic plasticity occur in vivo, for example after changes in sensory input. Here, we show that activity levels in the visual cortex are significantly decreased after sensory deprivation by retinal lesions, followed by a gradual increase in activity levels in the 48 hr after deprivation. These activity changes are associated with synaptic scaling, manifested in vitro by an increase in mEPSC amplitude and in vivo by an increase in spine size. Together, these data show that homeostatic activity changes occur in vivo in parallel with synaptic scaling.
•Deprivation results in compensatory homeostatic increases in spine size in vivo•Neuronal activity in the visual cortex drops after complete removal of visual input•Activity levels are restored in the 48 hr after visual deprivation•Ex vivo measurements show increased spine size is paralleled by synaptic scaling
Keck et al. investigate homeostatic plasticity in the mouse visual cortex in vivo and find that cortical activity levels decrease after visual deprivation and gradually recover in the subsequent 48 hr, accompanied by a corresponding increase in synaptic strength.
This perspective describes predictive processing as a computational framework for understanding cortical function in the context of emerging evidence, with a focus on sensory processing. We discuss ...how the predictive processing framework may be implemented at the level of cortical circuits and how its implementation could be falsified experimentally. Lastly, we summarize the general implications of predictive processing on cortical function in healthy and diseased states.
In this perspective, Keller and Mrsic-Flogel describe the advantages of predictive processing as a computational framework for understanding cortical function in the context of emerging evidence with a focus on sensory processing.
In primary visual cortex, a subset of neurons responds when a particular stimulus is encountered in a certain location in visual space. This activity can be modeled using a visual receptive field. In ...addition to visually driven activity, there are neurons in visual cortex that integrate visual and motor-related input to signal a mismatch between actual and predicted visual flow. Here we show that these mismatch neurons have receptive fields and signal a local mismatch between actual and predicted visual flow in restricted regions of visual space. These mismatch receptive fields are aligned to the retinotopic map of visual cortex and are similar in size to visual receptive fields. Thus, neurons with mismatch receptive fields signal local deviations of actual visual flow from visual flow predicted based on self-motion and could therefore underlie the detection of objects moving relative to the visual flow caused by self-motion.
Display omitted
•V1 layer 2/3 neurons signal visuomotor mismatch in local parts of the visual field•Resolution of mismatch receptive fields matches that of visual receptive fields•Mismatch receptive fields are aligned to the visual retinotopy
Zmarz and Keller show that in V1 neurons, sensorimotor mismatch responses, like visual responses, are confined to specific regions of the visual field. Thus the concept of receptive fields can be directly integrated with the theory of predictive coding.
The emergence of sensory-guided behavior depends on sensorimotor coupling during development. How sensorimotor experience shapes neural processing is unclear. Here, we show that the coupling between ...motor output and visual feedback is necessary for the functional development of visual processing in layer 2/3 (L2/3) of primary visual cortex (V1) of the mouse. Using a virtual reality system, we reared mice in conditions of normal or random visuomotor coupling. We recorded the activity of identified excitatory and inhibitory L2/3 neurons in response to transient visuomotor mismatches in both groups of mice. Mismatch responses in excitatory neurons were strongly experience dependent and driven by a transient release from inhibition mediated by somatostatin-positive interneurons. These data are consistent with a model in which L2/3 of V1 computes a difference between an inhibitory visual input and an excitatory locomotion-related input, where the balance between these two inputs is finely tuned by visuomotor experience.
Display omitted
•The development of mismatch responses depends on visuomotor experience•Mismatch neurons balance excitatory motor-related input against visual inhibition•Visual inhibition onto mismatch neurons is mediated by somatostatin interneurons•Normal visuomotor experience rapidly restores normal visual processing
The coupling of sensory and motor experience during development shapes visual perception by tuning a cortical circuit that compares inhibitory visual input and excitatory motor input and is able to detect mismatches between actual and expected sensory experience.
Learned associations between stimuli in different sensory modalities can shape the way we perceive these stimuli. However, it is not well understood how these interactions are mediated or at what ...level of the processing hierarchy they occur. Here we describe a neural mechanism by which an auditory input can shape visual representations of behaviorally relevant stimuli through direct interactions between auditory and visual cortices in mice. We show that the association of an auditory stimulus with a visual stimulus in a behaviorally relevant context leads to experience-dependent suppression of visual responses in primary visual cortex (V1). Auditory cortex axons carry a mixture of auditory and retinotopically matched visual input to V1, and optogenetic stimulation of these axons selectively suppresses V1 neurons that are responsive to the associated visual stimulus after, but not before, learning. Our results suggest that cross-modal associations can be communicated by long-range cortical connections and that, with learning, these cross-modal connections function to suppress responses to predictable input.
The cortex is organized as a hierarchical processing structure. Feedback from higher levels of the hierarchy, known as top-down signals, have been shown to be involved in attentional and contextual ...modulation of sensory responses. Here we argue that top-down input to the primary visual cortex (V1) from A24b and the adjacent secondary motor cortex (M2) signals a prediction of visual flow based on motor output. A24b/M2 sends a dense and topographically organized projection to V1 that targets most neurons in layer 2/3. By imaging the activity of A24b/M2 axons in V1 of mice learning to navigate a 2D virtual environment, we found that their activity was strongly correlated with locomotion and resulting visual flow feedback in an experience-dependent manner. When mice were trained to navigate a left-right inverted virtual environment, correlations of neural activity with behavior reversed to match visual flow. These findings are consistent with a predictive coding interpretation of visual processing.
•Mouse A24b/M2 sends a dense topographically organized input to V1•Motor-related signals from A24b/M2 drive motor and mismatch signals in V1•Training to navigate a left-right inverted world reverses A24b/M2 visuomotor coding•Stimulation of A24b/M2 axons in V1 in navigating mice elicits turning behavior
Top-down input to visual cortex from prefrontal areas is involved in attentional and contextual modulation of sensory responses. Leinweber et al. argue that, in the mouse, top-down input to V1 from A24b/M2 carries a prediction of visual flow given movement.
Motor cortex (M1) lesions result in motor impairments, yet how M1 contributes to the control of movement remains controversial. To investigate the role of M1 in sensory guided motor coordination, we ...trained mice to navigate a virtual corridor using a spherical treadmill. This task required directional adjustments through spontaneous turning, while unexpected visual offset perturbations prompted induced turning. We found that M1 is essential for execution and learning of this visually guided task. Turn-selective layer 2/3 and layer 5 pyramidal tract (PT) neuron activation was shaped differentially with learning but scaled linearly with turn acceleration during spontaneous turns. During induced turns, however, layer 2/3 neurons were activated independent of behavioral response, while PT neurons still encoded behavioral response magnitude. Our results are consistent with a role of M1 in the detection of sensory perturbations that result in deviations from intended motor state and the initiation of an appropriate corrective response.
•Motor cortex (M1) is necessary for the motor response to an unexpected perturbation•M1 is not necessary when the same movement is carried out spontaneously•Layer 2/3 neurons are differentially activated by unexpected visual perturbations•Activity in layer 5 PT neurons correlates with behavioral responses
The role of motor cortex in movement control is controversial. Heindorf et al. demonstrate that motor cortex mediates corrective behavioral responses to unexpected visual perturbations, paralleled by layer-specific cortical responses distinct from the ones during the same movement without perturbation.
PDF
