Models that represent meaning as high-dimensional numerical vectors—such as latent semantic analysis (LSA), hyperspace analogue to language (HAL), bound encoding of the aggregate language environment ...(BEAGLE), topic models, global vectors (GloVe), and word2vec—have been introduced as extremely powerful machine-learning proxies for human semantic representations and have seen an explosive rise in popularity over the past 2 decades. However, despite their considerable advancements and spread in the cognitive sciences, one can observe problems associated with the adequate presentation and understanding of some of their features. Indeed, when these models are examined from a cognitive perspective, a number of unfounded arguments tend to appear in the psychological literature. In this article, we review the most common of these arguments and discuss (a) what exactly these models represent at the implementational level and their plausibility as a cognitive theory, (b) how they deal with various aspects of meaning such as polysemy or compositionality, and (c) how they relate to the debate on embodied and grounded cognition. We identify common misconceptions that arise as a result of incomplete descriptions, outdated arguments, and unclear distinctions between theory and implementation of the models. We clarify and amend these points to provide a theoretical basis for future research and discussions on vector models of semantic representation.
•We compare classical distributional semantics models to a new, prediction-based class of models.•In evaluation we use psycholinguistically relevant tasks, including a semantic priming megastudy.•We ...find that the new class of model generally provides a better or comparable fit to behavioral data.•We release pre-trained semantic spaces for Dutch and English and an open-source interface.
Recent developments in distributional semantics (Mikolov, Chen, Corrado, & Dean, 2013; Mikolov, Sutskever, Chen, Corrado, & Dean, 2013) include a new class of prediction-based models that are trained on a text corpus and that measure semantic similarity between words. We discuss the relevance of these models for psycholinguistic theories and compare them to more traditional distributional semantic models. We compare the models’ performances on a large dataset of semantic priming (Hutchison et al., 2013) and on a number of other tasks involving semantic processing and conclude that the prediction-based models usually offer a better fit to behavioral data. Theoretically, we argue that these models bridge the gap between traditional approaches to distributional semantics and psychologically plausible learning principles. As an aid to researchers, we release semantic vectors for English and Dutch for a range of models together with a convenient interface that can be used to extract a great number of semantic similarity measures.
Semantic association computation is the process of quantifying the strength of a semantic connection between two textual units, based on different types of semantic relations. Semantic association ...computation is a key component of various applications belonging to a multitude of fields, such as computational linguistics, cognitive psychology, information retrieval and artificial intelligence. The field of semantic association computation has been studied for decades. The aim of this paper is to present a comprehensive survey of various approaches for computing semantic associations, categorized according to their underlying sources of background knowledge. Existing surveys on semantic computation have focused on a specific aspect of semantic associations, such as utilizing distributional semantics in association computation or types of spatial models of semantic associations. However, this paper has put a multitude of computational aspects and factors in one picture. This makes the article worth reading for those researchers who want to start off in the field of semantic associations computation. This paper introduces the fundamental elements of the association computation process, evaluation methodologies and pervasiveness of semantic measures in a variety of fields, relying on natural language semantics. Along the way, there is a detailed discussion on the main categories of background knowledge sources, classified as formal and informal knowledge sources, and the underlying design models, such as spatial, combinatorial and network models, that are used in the association computation process. The paper classifies existing approaches of semantic association computation into two broad categories, based on their utilization of background knowledge sources:
knowledge-rich
approaches; and
knowledge-lean
approaches. Each category is divided further into sub-categories, according to the type of underlying knowledge sources and design models of semantic association. A comparative analysis of strengths and limitations of various approaches belonging to each research stream is also presented. The paper concludes the survey by analyzing the pivotal factors that affect the performance of semantic association measures.
Handbook of Latent Semantic Analysis Landauer, Thomas K., Ed; McNamara, Danielle S., Ed; Dennis, Simon, Ed ...
Routledge, Taylor & Francis Group,
02/2007
Book
"The Handbook of Latent Semantic Analysis" is the authoritative reference for the theory behind Latent Semantic Analysis (LSA), a burgeoning mathematical method used to analyze how words make ...meaning, with the desired outcome to program machines to understand human commands via natural language rather than strict programming protocols. The first book of its kind to deliver such a comprehensive analysis, this volume explores every area of the method and combines theoretical implications as well as practical matters of LSA. Readers are introduced to a powerful new way of understanding language phenomena, as well as innovative ways to perform tasks that depend on language or other complex systems. The "Handbook" clarifies misunderstandings and pre-formed objections to LSA, and provides examples of exciting new educational technologies made possible by LSA and similar techniques. It raises issues in philosophy, artificial intelligence, and linguistics, while describing how LSA has underwritten a range of educational technologies and information systems. Alternate approaches to language understanding are addressed and compared to LSA. This work is essential reading for anyone--newcomers to this area and experts alike--interested in how human language works or interested in computational analysis and uses of text. Educational technologists, cognitive scientists, philosophers, and information technologists in particular will consider this volume especially useful.
Adult semantic memory has been traditionally conceptualized as a relatively static memory system that consists of knowledge about the world, concepts, and symbols. Considerable work in the past few ...decades has challenged this static view of semantic memory, and instead proposed a more fluid and flexible system that is sensitive to context, task demands, and perceptual and sensorimotor information from the environment. This paper (1) reviews traditional and modern computational models of semantic memory, within the umbrella of network (free association-based), feature (property generation norms-based), and distributional semantic (natural language corpora-based) models, (2) discusses the contribution of these models to important debates in the literature regarding knowledge representation (localist vs. distributed representations) and learning (error-free/Hebbian learning vs. error-driven/predictive learning), and (3) evaluates how modern computational models (neural network, retrieval-based, and topic models) are revisiting the traditional “static” conceptualization of semantic memory and tackling important challenges in semantic modeling such as addressing temporal, contextual, and attentional influences, as well as incorporating grounding and compositionality into semantic representations. The review also identifies new challenges regarding the abundance and availability of data, the generalization of semantic models to other languages, and the role of social interaction and collaboration in language learning and development. The concluding section advocates the need for integrating representational accounts of semantic memory with process-based accounts of cognitive behavior, as well as the need for explicit comparisons of computational models to human baselines in semantic tasks to adequately assess their psychological plausibility as models of human semantic memory.
As for the classification network that is constantly emerging with each passing day, different classification network as the backbone of the semantic segmentation network may show different ...performance. This paper selected the road extraction data set of CVPR DeepGlobe, and compared the performance differences of VGG-16 as the backbone of Unet, ResNet34, ResNet101 and Xception as the backbone of AD-LinkNet. When VGG-16 is used as the backbone of the semantic segmentation network, it performs better in the face of long and wide road extraction. As the backbone of the semantic segmentation network, ResNet has a higher ability to extract small roads. When Xception is used as the backbone of the semantic segmentation network, it not only retains the characteristics of ResNet34, but also can effectively deal with the complex situation of extracting target covered by occlusions.
The organization of semantic memory, including memory for word meanings, has long been a central question in cognitive science. Although there is general agreement that word meaning representations ...must make contact with sensory-motor and affective experiences in a non-arbitrary fashion, the nature of this relationship remains controversial. One prominent view proposes that word meanings are represented directly in terms of their experiential content (i.e., sensory-motor and affective representations). Opponents of this view argue that the representation of word meanings reflects primarily taxonomic structure, that is, their relationships to natural categories. In addition, the recent success of language models based on word co-occurrence (i.e., distributional) information in emulating human linguistic behavior has led to proposals that this kind of information may play an important role in the representation of lexical concepts. We used a semantic priming paradigm designed for representational similarity analysis (RSA) to quantitatively assess how well each of these theories explains the representational similarity pattern for a large set of words. Crucially, we used partial correlation RSA to account for intercorrelations between model predictions, which allowed us to assess, for the first time, the unique effect of each model. Semantic priming was driven primarily by experiential similarity between prime and target, with no evidence of an independent effect of distributional or taxonomic similarity. Furthermore, only the experiential models accounted for unique variance in priming after partialling out explicit similarity ratings. These results support experiential accounts of semantic representation and indicate that, despite their good performance at some linguistic tasks, the distributional models evaluated here do not encode the same kind of information used by the human semantic system.
Display omitted
•We used RSA to evaluate three major theories of word meaning representation.•Automatic semantic priming was measured item-wise with high reliability.•Results strongly support representation in terms of experiential information.•Distributional information did not independently contribute to semantic priming.•RSA and semantic priming can be used to determine the featural content of concepts.
Weakly Supervised Incremental Semantic Segmentation (WISS) aims to enable deep neural networks to incrementally learn new classes using only image-level labels without catastrophic forgetting. ...Despite WISS eliminating the usage of costly and time-consuming pixel-by-pixel annotations, the image-level labels can not provide details about the location of new classes, resulting in inferior performance. To address these issues, we take inspiration from zero-shot learning to model the inter-class semantic relation utilizing class names as text prompts, thereby facilitating knowledge transfer between classes. However, some class names of the segmentation datasets are polysemous. Thus, we design a new prompt template to better capture the semantic relation by appending synonyms and definitions of the corresponding classes. Guided by this semantic relation, we propose semantic relation weighted distillation to transfer the knowledge from old to new classes, significantly improving plasticity while reducing forgetting. Additionally, we introduce a novel superclass-level distillation aimed at preserving shared global knowledge within the superclass, further alleviating catastrophic forgetting. We extensively evaluate our method by integrating it into state-of-the-art WISS approaches on Pascal VOC and COCO datasets. We observe consistent gains in performance across diverse experimental scenarios. Code is available at https://github.com/Magic-Nova77/PGSD.
The semantic image segmentation task consists of classifying each pixel of an image into an instance, where each instance corresponds to a class. This task is a part of the concept of scene ...understanding or better explaining the global context of an image. In the medical image analysis domain, image segmentation can be used for image-guided interventions, radiotherapy, or improved radiological diagnostics. In this review, we categorize the leading deep learning-based medical and non-medical image segmentation solutions into six main groups of deep architectural, data synthesis-based, loss function-based, sequenced models, weakly supervised, and multi-task methods and provide a comprehensive review of the contributions in each of these groups. Further, for each group, we analyze each variant of these groups and discuss the limitations of the current approaches and present potential future research directions for semantic image segmentation.
PDF
