Decentralization is a central characteristic of biological motor control that allows for fast responses relying on local sensory information. In contrast, the current trend of Deep Reinforcement ...Learning (DRL) based approaches to motor control follows a centralized paradigm using a single, holistic controller that has to untangle the whole input information space. This motivates to ask whether decentralization as seen in biological control architectures might also be beneficial for embodied sensori-motor control systems when using DRL. To answer this question, we provide an analysis and comparison of eight control architectures for adaptive locomotion that were derived for a four-legged agent, but with their degree of decentralization varying systematically between the extremes of fully centralized and fully decentralized. Our comparison shows that learning speed is significantly enhanced in distributed architectures—while still reaching the same high performance level of centralized architectures—due to smaller search spaces and local costs providing more focused information for learning. Second, we find an increased robustness of the learning process in the decentralized cases—it is less demanding to hyperparameter selection and less prone to becoming trapped in poor local minima. Finally, when examining generalization to uneven terrains—not used during training—we find best performance for an intermediate architecture that is decentralized, but integrates only local information from both neighboring legs. Together, these findings demonstrate beneficial effects of distributing control into decentralized units and relying on local information. This appears as a promising approach towards more robust DRL and better generalization towards adaptive behavior.
Display omitted
•Deep Reinforcement Learning in motor control is heavily affected by overfitting.•In the proposed decentralized approach, control is distributed into local modules.•Training of these decentralized modules progressed much faster, driven by local decomposed rewards.•Decentralized controller produced more robust behavior when tested for generalization.
During brain maturation, the occurrence of the extracellular matrix (ECM) terminates juvenile plasticity by mediating structural stability. Interestingly, enzymatic removal of the ECM restores ...juvenile forms of plasticity, as for instance demonstrated by topographical reconnectivity in sensory pathways. However, to which degree the mature ECM is a compromise between stability and flexibility in the adult brain impacting synaptic plasticity as a fundamental basis for learning, lifelong memory formation, and higher cognitive functions is largely unknown. In this study, we removed the ECM in the auditory cortex of adult Mongolian gerbils during specific phases of cortex-dependent auditory relearning, which was induced by the contingency reversal of a frequency-modulated tone discrimination, a task requiring high behavioral flexibility. We found that ECM removal promoted a significant increase in relearning performance, without erasing already established—that is, learned—capacities when continuing discrimination training. The cognitive flexibility required for reversal learning of previously acquired behavioral habits, commonly understood to mainly rely on frontostriatal circuits, was enhanced by promoting synaptic plasticity via ECM removal within the sensory cortex. Our findings further suggest experimental modulation of the cortical ECM as a tool to open short-term windows of enhanced activity-dependent reorganization allowing for guided neuroplasticity.
Perception greatly benefits from integrating multiple sensory cues into a unified percept. To study the neural mechanisms of sensory integration, model systems are required that allow the ...simultaneous assessment of activity and the use of techniques to affect individual neural processes in behaving animals. While rodents qualify for these requirements, little is known about multisensory integration and areas involved for this purpose in the rodent. Using optical imaging combined with laminar electrophysiological recordings, the rat parietal cortex was identified as an area where visual and somatosensory inputs converge and interact. Our results reveal similar response patterns to visual and somatosensory stimuli at the level of current source density (CSD) responses and multi-unit responses within a strip in parietal cortex. Surprisingly, a selective asymmetry was observed in multisensory interactions: when the somatosensory response preceded the visual response, supra-linear summation of CSD was observed, but the reverse stimulus order resulted in sub-linear effects in the CSD. This asymmetry was not present in multi-unit activity however, which showed consistently sub-linear interactions. These interactions were restricted to a specific temporal window, and pharmacological tests revealed significant local intra-cortical contributions to this phenomenon. Our results highlight the rodent parietal cortex as a system to model the neural underpinnings of multisensory processing in behaving animals and at the cellular level.
Highlights • Separable conceptualizations of auditory cortex functions are reviewed. • Category formation allows dissociation of stimulus and meaning representations. • Specific patterns of ...coordinated cortical activity mediate category formation. • Auditory cortex dopamine boosts a cortico-efferent positive feedback circuit.
Recent data on learning-related changes in animal and human auditory cortex indicate functions beyond mere stimulus representation and simple recognition memory for stimuli. Rather, auditory cortex ...seems to process and represent stimuli in a task-dependent fashion. This implies plasticity in neural processing, which can be observed at the level of single neuron firing and the level of spatiotemporal activity patterns in cortical areas. Auditory cortex is a structure in which behaviorally relevant aspects of stimulus processing are highly developed because of the fugitive nature of auditory stimuli.
Brain function requires the flexible coordination of billions of neurons across multiple scales. This could be achieved by scale-free, critical dynamics balanced at the edge of order and disorder. ...Criticality has been demonstrated in several, often reduced neurophysiological model systems. In the intact human brain criticality has yet been only verified for the resting state. A more direct link between the concept of criticality and oscillatory brain physiology, which is strongly related to cognition, is yet missing. In the present study we therefore carried out a frequency-specific analysis of criticality in the MEG, recorded while subjects were in a defined cognitive state through mindfulness meditation. In a two-step approach we assessed whether the macroscopic neural avalanche dynamics is scale-free by evaluating the goodness of a power-law fits of cascade size and duration distributions of MEG deflections in different frequency bands. In a second step we determined the closeness of the power-law exponents to a critical value of -1.5. Power-law fitting was evaluated by permutation testing, fitting of alternative distributions, and cascade shape analysis. Criticality was verified by defined relationships of exponents of cascade size and duration distributions. Behavioral relevance of criticality was tested by correlation of indices of criticality with individual scores of the Mindful Attention Awareness Scale. We found that relevant scale-free near-critical dynamics originated only from broad-band high-frequency (> 100 Hz) MEG activity, which has been associated with action potential firing, and therefore links criticality on the macroscopic level of MEG to critical spike avalanches on a microscopic level. Whereas a scale-free dynamics was found under mindfulness meditation and rest, avalanche dynamics shifted towards a critical point during meditation by reduction of neural noise. Together with our finding that during mindfulness meditation avalanches show differences in topography relative to rest, our results show that self-regulated attention as required during meditation can serve as a control parameter of criticality in scale-free brain dynamics.
The auditory thalamus is the central nexus of bottom-up connections from the inferior colliculus and top-down connections from auditory cortical areas. While considerable efforts have been made to ...investigate feedforward processing of sounds in the auditory thalamus (medial geniculate body, MGB) of non-human primates, little is known about the role of corticofugal feedback in the MGB of awake non-human primates. Therefore, we developed a small, repositionable cooling probe to manipulate corticofugal feedback and studied neural responses in both auditory cortex and thalamus to sounds under conditions of normal and reduced cortical temperature. Cooling-induced increases in the width of extracellularly recorded spikes in auditory cortex were observed over the distance of several hundred micrometers away from the cooling probe. Cortical neurons displayed reduction in both spontaneous and stimulus driven firing rates with decreased cortical temperatures. In thalamus, cortical cooling led to increased spontaneous firing and either increased or decreased stimulus driven activity. Furthermore, response tuning to modulation frequencies of temporally modulated sounds and spatial tuning to sound source location could be altered (increased or decreased) by cortical cooling. Specifically, best modulation frequencies of individual MGB neurons could shift either toward higher or lower frequencies based on the vector strength or the firing rate. The tuning of MGB neurons for spatial location could both sharpen or widen. Elevation preference could shift toward higher or lower elevations and azimuth tuning could move toward ipsilateral or contralateral locations. Such bidirectional changes were observed in many parameters which suggests that the auditory thalamus acts as a filter that could be adjusted according to behaviorally driven signals from auditory cortex. Future work will have to delineate the circuit elements responsible for the observed effects.
Dopaminergic neurotransmission in primary auditory cortex (AI) has been shown to be involved in learning and memory functions. Moreover, dopaminergic projections and D1/D5 receptor distributions ...display a layer-dependent organization, suggesting specific functions in the cortical circuitry. However, the circuit effects of dopaminergic neurotransmission in sensory cortex and their possible roles in perception, learning, and memory are largely unknown. Here, we investigated layer-specific circuit effects of dopaminergic neuromodulation using current source density (CSD) analysis in AI of Mongolian gerbils. Pharmacological stimulation of D1/D5 receptors increased auditory-evoked synaptic currents in infragranular layers, prolonging local thalamocortical input via positive feedback between infragranular output and granular input. Subsequently, dopamine promoted sustained cortical activation by prolonged recruitment of long-range corticocortical networks. A detailed circuit analysis combining layer-specific intracortical microstimulation (ICMS), CSD analysis, and pharmacological cortical silencing revealed that cross-laminar feedback enhanced by dopamine relied on a positive, fast-acting recurrent corticoefferent loop, most likely relayed via local thalamic circuits. Behavioral signal detection analysis further showed that activation of corticoefferent output by infragranular ICMS, which mimicked auditory activation under dopaminergic influence, was most effective in eliciting a behaviorally detectable signal. Our results show that D1/D5-mediated dopaminergic modulation in sensory cortex regulates positive recurrent corticoefferent feedback, which enhances states of high, persistent activity in sensory cortex evoked by behaviorally relevant stimuli. In boosting horizontal network interactions, this potentially promotes the readout of task-related information from cortical synapses and improves behavioral stimulus detection.
Primary sensory cortex integrates sensory information from afferent feedforward thalamocortical projection systems and convergent intracortical microcircuits. Both input systems have been ...demonstrated to provide different aspects of sensory information. Here we have used high-density recordings of laminar current source density (CSD) distributions in primary auditory cortex of Mongolian gerbils in combination with pharmacological silencing of cortical activity and analysis of the residual CSD, to dissociate the feedforward thalamocortical contribution and the intracortical contribution to spectral integration. We found a temporally highly precise integration of both types of inputs when the stimulation frequency was in close spectral neighborhood of the best frequency of the measurement site, in which the overlap between both inputs is maximal. Local intracortical connections provide both directly feedforward excitatory and modulatory input from adjacent cortical sites, which determine how concurrent afferent inputs are integrated. Through separate excitatory horizontal projections, terminating in cortical layers II/III, information about stimulus energy in greater spectral distance is provided even over long cortical distances. These projections effectively broaden spectral tuning width. Based on these data, we suggest a mechanism of spectral integration in primary auditory cortex that is based on temporally precise interactions of afferent thalamocortical inputs and different short- and long-range intracortical networks. The proposed conceptual framework allows integration of different and partly controversial anatomical and physiological models of spectral integration in the literature.
Approaching or looming signals are often related to extremely relevant environmental events (e.g. threats or collisions) making these signals critical for survival. However, the neural network ...underlying multisensory looming processing is not yet fully understood. Using functional magnetic resonance imaging (fMRI) we identified the neural correlates of audiovisual looming processing in humans: audiovisual looming (vs. receding) signals enhance fMRI-responses in low-level visual and auditory areas plus multisensory cortex (superior temporal sulcus; plus parietal and frontal structures). When characterizing the fMRI-response profiles for multisensory looming stimuli, we found significant enhancements relative to the mean and maximum of unisensory responses in looming-sensitive visual and auditory cortex plus STS. Superadditive enhancements were observed in visual cortex. Subject-specific region-of-interest analyses further revealed superadditive response profiles within all sensory-specific looming-sensitive structures plus bilateral STS for audiovisual looming vs. summed unisensory looming conditions. Finally, we observed enhanced connectivity of bilateral STS with low-level visual areas in the context of looming processing. This enhanced coupling of STS with unisensory regions might potentially serve to enhance the salience of unisensory stimulus features and is accompanied by superadditive fMRI-responses. We suggest that this preference in neural signaling for looming stimuli effectively informs animals to avoid potential threats or collisions.
► The ability to discern approaching from receding events is critical for survival. ► Approaching (Looming) stimuli selectively enhance perceptual sensitivity. ► We examine the neural underpinnings of audiovisual looming signals in humans. ► Looming signals activate a network of sensory-specific and multisensory brain areas. ► Enhanced connectivity between regions might enhance the salience of stimuli features.
PDF
