Humans can naturally and effectively find salient regions in complex scenes. Motivated by this observation, attention mechanisms were introduced into computer vision with the aim of imitating this ...aspect of the human visual system. Such an attention mechanism can be regarded as a dynamic weight adjustment process based on features of the input image. Attention mechanisms have achieved great success in many visual tasks, including image classification, object detection, semantic segmentation, video understanding, image generation, 3D vision, multimodal tasks, and self-supervised learning. In this survey, we provide a comprehensive review of various attention mechanisms in computer vision and categorize them according to approach, such as channel attention, spatial attention, temporal attention, and branch attention; a related repository
https://github.com/MenghaoGuo/Awesome-Vision-Attentions
is dedicated to collecting related work. We also suggest future directions for attention mechanism research.
Video stabilization techniques are essential for most hand-held captured videos due to high-frequency shakes. Several 2D-, 2.5D-, and 3D-based stabilization techniques have been presented previously, ...but to the best of our knowledge, no solutions based on deep neural networks had been proposed to date. The main reason for this omission is shortage in training data as well as the challenge of modeling the problem using neural networks. In this paper, we present a video stabilization technique using a convolutional neural network. Previous works usually propose an off-line algorithm that smoothes a holistic camera path based on feature matching. Instead, we focus on low-latency, real-time camera path smoothing that does not explicitly represent the camera path and does not use future frames. Our neural network model, called StabNet, learns a set of mesh-grid transformations progressively for each input frame from the previous set of stabilized camera frames and creates stable corresponding latent camera paths implicitly. To train the network, we collect a dataset of synchronized steady and unsteady video pairs via a specially designed hand-held hardware. Experimental results show that our proposed online method performs comparatively to the traditional off-line video stabilization methods without using future frames while running about 10 times faster. More importantly, our proposed StabNet is able to handle low-quality videos, such as night-scene videos, watermarked videos, blurry videos, and noisy videos, where the existing methods fail in feature extraction or matching.
Cerebral cortex expansion is a hallmark of mammalian brain evolution; yet, how increased neurogenesis is coordinated with structural and functional development remains largely unclear. The T-box ...protein TBR2/EOMES is preferentially enriched in intermediate progenitors and supports cortical neurogenesis expansion. Here we show that TBR2 regulates fine-scale spatial and circuit organization of excitatory neurons in addition to enhancing neurogenesis in the mouse cortex. TBR2 removal leads to a significant reduction in neuronal, but not glial, output of individual radial glial progenitors as revealed by mosaic analysis with double markers. Moreover, in the absence of TBR2, clonally related excitatory neurons become more laterally dispersed and their preferential synapse development is impaired. Interestingly, TBR2 directly regulates the expression of Protocadherin 19 (PCDH19), and simultaneous PCDH19 expression rescues neurogenesis and neuronal organization defects caused by TBR2 removal. Together, these results suggest that TBR2 coordinates neurogenesis expansion and precise microcircuit assembly via PCDH19 in the mammalian cortex.
The frontal area of the cerebral cortex provides long-range inputs to sensory areas to modulate neuronal activity and information processing. These long-range circuits are crucial for accurate ...sensory perception and complex behavioral control; however, little is known about their precise circuit organization. Here we specifically identified the presynaptic input neurons to individual excitatory neuron clones as a unit that constitutes functional microcircuits in the mouse sensory cortex. Interestingly, the long-range input neurons in the frontal but not contralateral sensory area are spatially organized into discrete vertical clusters and preferentially form synapses with each other over nearby non-input neurons. Moreover, the assembly of distant presynaptic microcircuits in the frontal area depends on the selective synaptic communication of excitatory neuron clones in the sensory area that provide inputs to the frontal area. These findings suggest that highly precise long-range reciprocal microcircuit-to-microcircuit communication mediates frontal-sensory area interactions in the mammalian cortex.
Display omitted
•Sensory excitatory neuron clones receive defined long-range presynaptic inputs•Presynaptic neurons in frontal area are organized into discrete radial clusters•Presynaptic neurons in frontal area selectively form synapses with each other•Reciprocal microcircuit communication connects frontal and sensory cortices
Ren et al. identify presynaptic neurons innervating individual excitatory neuron clones that constitute microcircuits in the sensory cortex and reveal that input neurons in the frontal cortex are organized in discrete vertical clusters and preferentially form synapses with each other.
Radial glial progenitors (RGPs) are responsible for producing nearly all neocortical neurons. To gain insight into the patterns of RGP division and neuron production, we quantitatively analyzed ...excitatory neuron genesis in the mouse neocortex using Mosaic Analysis with Double Markers, which provides single-cell resolution of progenitor division patterns and potential in vivo. We found that RGPs progress through a coherent program in which their proliferative potential diminishes in a predictable manner. Upon entry into the neurogenic phase, individual RGPs produce ?8-9 neurons distributed in both deep and superficial layers, indicating a unitary output in neuronal production. Removal of OTX1, a transcription factor transiently expressed in RGPs, results in both deep- and superficial-layer neuron loss and a reduction in neuronal unit size. Moreover, ?1/6 of neurogenic RGPs proceed to produce glia. These results suggest that progenitor behavior and histogenesis in the mammalian neocortex conform to a remarkably orderly and deterministic program.
A fundamental feature of the mammalian neocortex is its columnar organization. In the visual cortex, functional columns consisting of neurons with similar orientation preferences have been ...characterized extensively, but how these columns are constructed during development remains unclear. The radial unit hypothesis posits that the ontogenetic columns formed by clonally related neurons migrating along the same radial glial fibre during corticogenesis provide the basis for functional columns in adult neocortex. However, a direct correspondence between the ontogenetic and functional columns has not been demonstrated. Here we show that, despite the lack of a discernible orientation map in mouse visual cortex, sister neurons in the same radial clone exhibit similar orientation preferences. Using a retroviral vector encoding green fluorescent protein to label radial clones of excitatory neurons, and in vivo two-photon calcium imaging to measure neuronal response properties, we found that sister neurons preferred similar orientations whereas nearby non-sister neurons showed no such relationship. Interestingly, disruption of gap junction coupling by viral expression of a dominant-negative mutant of Cx26 (also known as Gjb2) or by daily administration of a gap junction blocker, carbenoxolone, during the first postnatal week greatly diminished the functional similarity between sister neurons, suggesting that the maturation of ontogenetic into functional columns requires intercellular communication through gap junctions. Together with the recent finding of preferential excitatory connections among sister neurons, our results support the radial unit hypothesis and unify the ontogenetic and functional columns in the visual cortex.
Orderly division of radial glial progenitors (RGPs) in the developing mammalian cerebral cortex generates deep and superficial layer neurons progressively. However, the mechanisms that control RGP ...behavior and precise neuronal output remain elusive. Here, we show that the oxidative stress level progressively increases in the developing mouse cortex and regulates RGP behavior and neurogenesis. As development proceeds, numerous gene pathways linked to reactive oxygen species (ROS) and oxidative stress exhibit drastic changes in RGPs. Selective removal of PRDM16, a transcriptional regulator highly expressed in RGPs, elevates ROS level and induces expression of oxidative stress-responsive genes. Coinciding with an enhanced level of oxidative stress, RGP behavior was altered, leading to abnormal deep and superficial layer neuron generation. Simultaneous expression of mitochondrially targeted catalase to reduce cellular ROS levels significantly suppresses cortical defects caused by PRDM16 removal. Together, these findings suggest that oxidative stress actively regulates RGP behavior to ensure proper neurogenesis in the mammalian cortex.
Considerable progress has been made in converting human pluripotent stem cells (hPSCs) into functional neurons. However, the protracted timing of human neuron specification and functional maturation ...remains a key challenge that hampers the routine application of hPSC-derived lineages in disease modeling and regenerative medicine. Using a combinatorial small-molecule screen, we previously identified conditions to rapidly differentiate hPSCs into peripheral sensory neurons. Here we generalize the approach to central nervous system (CNS) fates by developing a small-molecule approach for accelerated induction of early-born cortical neurons. Combinatorial application of six pathway inhibitors induces post-mitotic cortical neurons with functional electrophysiological properties by day 16 of differentiation, in the absence of glial cell co-culture. The resulting neurons, transplanted at 8 d of differentiation into the postnatal mouse cortex, are functional and establish long-distance projections, as shown using iDISCO whole-brain imaging. Accelerated differentiation into cortical neuron fates should facilitate hPSC-based strategies for disease modeling and cell therapy in CNS disorders.
Neuronal production in the mammalian cortex depends on extensive mitoses of radial glial progenitors (RGPs) residing in the ventricular zone (VZ). We examined the function of centrioles in RGPs ...during cortical neurogenesis in mice by conditional removal of SAS-4, a protein that is required for centriole biogenesis. SAS-4 deletion led to a progressive loss of centrioles, accompanied by RGP detachment from the VZ. Delocalized RGPs did not become outer subventricular zone RGPs (oRGs). Although they remained proliferative, ectopic RGPs, as well as those in the VZ, with a centrosomal deficit exhibited prolonged mitosis, p53 upregulation and apoptosis, resulting in neuronal loss and microcephaly. Simultaneous removal of p53 fully rescued RGP death and microcephaly, but not RGP delocalization and randomized mitotic spindle orientation. Our findings define the functions of centrioles in anchoring RGPs in the VZ and ensuring their efficient mitoses, and reveal the robust adaptability of RGPs in the developing cortex.
PDF
