Highly discriminative 3D shape representations can be formed by encoding the spatial relationship among virtual words into the Bag of Words (BoW) method. To achieve this challenging task, several ...unresolved issues in the encoding procedure must be overcome for 3D shapes, including: 1) arbitrary mesh resolution; 2) irregular vertex topology; 3) orientation ambiguity on the 3D surface; and 4) invariance to rigid and non-rigid shape transformations. In this paper, a novel spatially enhanced 3D shape representation called bag of spatial context correlations (BoSCCs) is proposed to address all these issues. Adopting a novel local perspective, BoSCC is able to describe a 3D shape by an occurrence frequency histogram of spatial context correlation patterns, which makes BoSCC become more compact and discriminative than previous global perspective-based methods. Specifically, the spatial context correlation is proposed to simultaneously encode the geometric and spatial information of a 3D local region by the correlation among spatial contexts of vertices in that region, which effectively resolves the aforementioned issues. The spatial context of each vertex is modeled by Markov chains in a multi-scale manner, which thoroughly captures the spatial relationship by the transition probabilities of intra-virtual words and the ones of inter-virtual words. The high discriminability and compactness of BoSCC are effective for classification and retrieval, especially in the scenarios of limited samples and partial shape retrieval. Experimental results show that BoSCC outperforms the state-of-the-art spatially enhanced BoW methods in three common applications: global shape retrieval, shape classification, and partial shape retrieval.
Contextual information has been shown to be powerful for semantic segmentation. This work proposes a novel Context-based Tandem Network (CTNet) by interactively exploring the spatial contextual ...information and the channel contextual information, which can discover the semantic context for semantic segmentation. Specifically, the Spatial Contextual Module (SCM) is leveraged to uncover the spatial contextual dependency between pixels by exploring the correlation between pixels and categories. Meanwhile, the Channel Contextual Module (CCM) is introduced to learn the semantic features including the semantic feature maps and class-specific features by modeling the long-term semantic dependence between channels. The learned semantic features are utilized as the prior knowledge to guide the learning of SCM, which can make SCM obtain more accurate long-range spatial dependency. Finally, to further improve the performance of the learned representations for semantic segmentation, the results of the two context modules are adaptively integrated to achieve better results. Extensive experiments are conducted on four widely-used datasets, i.e., PASCAL-Context, Cityscapes, ADE20K and PASCAL VOC2012. The results demonstrate the superior performance of the proposed CTNet by comparison with several state-of-the-art methods. The source code and models are available at https://github.com/syp2ysy/CTNet .
Objects in unmanned aerial vehicle (UAV) images are generally small due to the high-photography altitude. Although many efforts have been made in object detection, how to accurately and quickly ...detect small objects is still one of the remaining open challenges. In this paper, we propose a feature fusion and scaling-based single shot detector (FS-SSD) for small object detection in the UAV images. The FS-SSD is an enhancement based on FSSD, a variety of the original single shot multibox detector (SSD). We add an extra scaling branch of the deconvolution module with an average pooling operation to form a feature pyramid. The original feature fusion branch is adjusted to be better suited to the small object detection task. The two feature pyramids generated by the deconvolution module and feature fusion module are utilized to make predictions together. In addition to the deep features learned by the FS-SSD, to further improve the detection accuracy, spatial context analysis is proposed to incorporate the object spatial relationships into object redetection. The interclass and intraclass distances between different object instances are computed as a spatial context, which proves effective for multiclass small object detection. Six experiments are conducted on the PASCAL VOC dataset and the two UAV image datasets. The experimental results demonstrate that the proposed method can achieve a comparable detection speed but an accuracy superior to those of the six state-of-the-art methods.
Shadow detection and shadow removal are fundamental and challenging tasks, requiring an understanding of the global image semantics. This paper presents a novel deep neural network design for shadow ...detection and removal by analyzing the spatial image context in a direction-aware manner. To achieve this, we first formulate the direction-aware attention mechanism in a spatial recurrent neural network (RNN) by introducing attention weights when aggregating spatial context features in the RNN. By learning these weights through training, we can recover direction-aware spatial context (DSC) for detecting and removing shadows. This design is developed into the DSC module and embedded in a convolutional neural network (CNN) to learn the DSC features at different levels. Moreover, we design a weighted cross entropy loss to make effective the training for shadow detection and further adopt the network for shadow removal by using a euclidean loss function and formulating a color transfer function to address the color and luminosity inconsistencies in the training pairs. We employed two shadow detection benchmark datasets and two shadow removal benchmark datasets, and performed various experiments to evaluate our method. Experimental results show that our method performs favorably against the state-of-the-art methods for both shadow detection and shadow removal.
The integration of spatial context in the classification of hyperspectral images is known to be an effective way in improving classification accuracy. In this paper, a novel spectral-spatial ...classification framework based on edge-preserving filtering is proposed. The proposed framework consists of the following three steps. First, the hyperspectral image is classified using a pixelwise classifier, e.g., the support vector machine classifier. Then, the resulting classification map is represented as multiple probability maps, and edge-preserving filtering is conducted on each probability map, with the first principal component or the first three principal components of the hyperspectral image serving as the gray or color guidance image. Finally, according to the filtered probability maps, the class of each pixel is selected based on the maximum probability. Experimental results demonstrate that the proposed edge-preserving filtering based classification method can improve the classification accuracy significantly in a very short time. Thus, it can be easily applied in real applications.
Availability of accurate land cover information over large areas is essential to the global environment sustainability; digital classification using medium-resolution remote sensing data would ...provide an effective method to generate the required land cover information. However, low accuracy of existing per-pixel based classification methods for medium-resolution data is a fundamental limiting factor. While convolutional neural networks (CNNs) with deep layers have achieved unprecedented improvements in object recognition applications that rely on fine image structures, they cannot be applied directly to medium-resolution data due to lack of such fine structures. In this paper, considering the spatial relation of a pixel to its neighborhood, we propose a new deep patch-based CNN system tailored for medium-resolution remote sensing data. The system is designed by incorporating distinctive characteristics of medium-resolution data; in particular, the system computes patch-based samples from multidimensional top of atmosphere reflectance data. With a test site from the Florida Everglades area (with a size of 771 square kilometers), the proposed new system has outperformed pixel-based neural network, pixel-based CNN and patch-based neural network by 24.36%, 24.23% and 11.52%, respectively, in overall classification accuracy. By combining the proposed deep CNN and the huge collection of medium-resolution remote sensing data, we believe that much more accurate land cover datasets can be produced over large areas.
Automatic road extraction from high-resolution remote sensing imagery has various applications like urban planning and automatic navigation. Existing methods for automatic road extraction however, ...focus on regional accuracy but not on the boundary quality; and most of these road extraction methods yield discontinuous results due to noise and occlusions. To address these two problems, a Boundary and Topological-aware Road extraction Network (BT-RoadNet) is proposed. BT-RoadNet is a coarse-to-fine architecture composed of two encoder-to-decoder networks, a Coarse Map Predicting Module (CMPM) and Fine Map Predicting Module (FMPM). The CMPM learns to predict coarse road segmentation maps, in which a Spatial Context Module (SCM) is employed as a bridge to solve discontinuous problems. The FMPM is used to refine the coarse road maps by learning the difference between the coarse road extraction result and the ground truth. Experiments were conducted on the open Massachusetts Road Dataset, a newly annotated Wuhan University (WHU) Road Dataset, and three large satellite images. Quantitative and qualitative analysis demonstrate that the proposed BT-RoadNet can enhance road network extraction to deal with interruptions caused by shadows and occlusions, extract roads with different scales and materials, and handle roads under construction that have incomplete spectral and geometric properties.
Decades of memory research demonstrate the importance of temporal organization in recall dynamics, using laboratory stimuli (i.e., word lists) at seconds- to minutes-long delays. Little is known, ...however, about such organization in recall of richer and more remote real-world experiences, in which the focus is usually on memory content without reference to event order. Here, 119 younger and older adults freely recalled extended real-world experiences, for which the encoding sequence was controlled, after 2 days or 1 week. We paired analytical tools from the list-learning and autobiographical memory literatures to measure spontaneous contextual dynamics and details in these recall narratives. Recall dynamics were organized by temporal context (contiguity and forward asymmetry), and organization was reduced in older age, despite similar serial position effects and recall initiation across age groups. Across participants, organization was positively associated with richness of episodic detail, providing evidence for a link between reexperiencing past events and reinstating their spatiotemporal context.
It is increasingly recognized that mental disorders are affected by both personal characteristics and environmental exposures. The built, natural, and social environments can either contribute to or ...buffer against metal disorders. Environmental exposure assessments related to mental health typically rely on neighborhoods within which people currently live. In this article, I call into question such neighborhood-based exposure assessments at one point in time, because human life unfolds over space and across time. To circumvent inappropriate exposure assessments and to better grasp the etiologies of mental disease, I argue that people are exposed to multiple health-supporting and harmful exposures not only during their daily lives, but also over the course of their lives. This article aims to lay a theoretical foundation elucidating the impact of dynamic environmental exposures on mental health outcomes. I examine, first, the possibilities and challenges for mental health research to integrate people's environmental exposures along their daily paths and, second, how exposures over people's residential history might affect mental health later in life. To push the borders of scientific inquiries, I stress that only such mobility-based approaches facilitate an exploration of exposure duration, exposure sequences, and exposure accumulation.
Display omitted
•The environmental context affects people's mental health.•Environmental exposure assessments are frequently restricted to the neighborhoods in which people currently live.•Dynamic exposures along people's daily paths may trigger mental disorders.•Exposures over people's residential history might affect mental health later in life.•Only dynamic exposure assessments enable the inclusion of the duration and sequence of exposures and risk accumulation over time.
Spatial context is a defining feature of episodic memories, which are often characterized as being events occurring in specific spatiotemporal contexts. In this review, I summarize research ...suggesting a common neural basis for episodic and spatial memory and relate this to the role of spatial context in episodic memory. I review evidence that spatial context serves as a scaffold for episodic memory and imagination, in terms of both behavioral and neural effects demonstrating a dependence of episodic memory on spatial representations. These effects are mediated by a posterior‐medial set of neocortical regions, including the parahippocampal cortex, retrosplenial cortex, posterior cingulate cortex, precuneus, and angular gyrus, which interact with the hippocampus to represent spatial context in remembered and imagined events. I highlight questions and areas that require further research, including differentiation of hippocampal function along its long axis and subfields, and how these areas interact with the posterior‐medial network.
This article is categorized under:
Psychology > Memory
Neuroscience > Cognition
Spatial contexts (e.g., a familiar building on campus) provide a foundation for remembered events (e.g., attending class in that building) and imagined future events (e.g., picturing one's graduation in that building). This paper reviews the behavioral and neural evidence supporting the role of spatial context in episodic memory and imagination.
PDF
