Complex reasoning aims to draw a correct inference based on complex rules. As a hallmark of human intelligence, it involves a degree of explicit reading comprehension, interpretation of logical ...knowledge and complex rule application. In this paper, we take a step forward in complex reasoning by systematically studying the three challenging and domain-general tasks of the Law School Admission Test (LSAT), including analytical reasoning, logical reasoning and reading comprehension. We propose a hybrid reasoning system to integrate these three tasks and achieve impressive overall performance on the LSAT tests. The experimental results demonstrate that our system endows itself a certain complex reasoning ability, especially the fundamental reading comprehension and challenging logical reasoning capacities. Further analysis also shows the effectiveness of combining the pre-trained models with the task-specific reasoning module, and integrating symbolic knowledge into discrete interpretable reasoning steps in complex reasoning. We further shed a light on the potential future directions, like unsupervised symbolic knowledge extraction, model interpretability, few-shot learning and comprehensive benchmark for complex reasoning.
Amharic is morphologically complex and under-resourced language, posing difficulties in the development of natural language processing applications. This paper presents the development of semantic ...role labeler for Amharic text using end-to-end deep neural network architecture. The system implicitly captures morphological, semantic and contextual features of a word at different levels of the architecture, and incorporates the syntactic structure of an input sentence. The proposed neural network architecture has four core layers from bottom to top, namely non-contextual word embedding, contextual word embedding, fully connected and sequence decoding layers. The non-contextual word embedding layer is formed from the concatenation of character-based, word-based and sentence-based word embeddings. This layer captures the morphological and semantic features of a given word by making use of BiLSTM recurrent neural network. At the contextual word embedding layer, a context sensitive embedding of a word is generated by applying a new LSTM layer on the top of the non-contextual concatenated word embedding layer. A fully connected network layer is added on top of contextual word embedding layer to supplement it by extracting dependencies among training samples in the corpus. At the sequence decoding layer, a sequence of semantic role labels is predicted using a linear-chain conditional random field algorithm by capturing the dependency among semantic role labels. In addition to the four core layers, the architecture has dropout layers to prevent overfitting problem. The proposed system achieves 94.96% accuracy and 81.2% F1 score when it is tested using test data.
Image captions are abstract expressions of content representations using text sentences, helping readers to better understand and analyse information between different media. With the advantage of ...encoder-decoder neural networks, captions can provide a rational structure for tasks such as image coding and caption prediction. This work introduces a Convolutional Neural Network (CNN) to Bidirectional Content-Adaptive Recurrent Unit (Bi-CARU) (CNN-to-Bi-CARU) model that performs bidirectional structure to consider contextual features and captures major feature from image. The encoded feature coded form image is respectively passed into the forward and backward layer of CARU to refine the word prediction, providing contextual text output for captioning. An attention layer is also introduced to collect the feature produced by the context-adaptive gate in CARU, aiming to compute the weighting information for relationship extraction and determination. In experiments, the proposed CNN-to-Bi-CARU model outperforms other advanced models in the field, achieving better extraction of contextual information and detailed representation of image captions. The model obtains a score of 41.28 on BLEU@4, 31.23 on METEOR, 61.07 on ROUGE-L, and 133.20 on CIDEr-D, making it competitive in the image captioning of MSCOCO dataset.
An ontology-based knowledge representation scheme thrives by establishing relations between different terms belonging to the underlying domain. However, automating the identification of all possible ...relations that may exist between the different terms pertaining to a vast domain, such as agriculture, is quite challenging. One of the major challenges comes from extracting the inter-subdomain relations which are quite common in the agriculture domain. This present work proposes a knowledge-based scheme for extraction of inter-subdomain relations. In particular, it deals with three such cases, viz. crop and soil, crop and disease, soil and region. A baseline system has been designed by analysing official documents retrieved from different repositories. In an improved version of the proposed scheme co-reference resolution techniques have been used to achieve much higher success with average precision and average recall more than 75% and 85%, respectively.
Abstract
Background
Text Matching (TM) is a fundamental task of natural language processing widely used in many application systems such as information retrieval, automatic question answering, ...machine translation, dialogue system, reading comprehension, etc. In recent years, a large number of deep learning neural networks have been applied to TM, and have refreshed benchmarks of TM repeatedly. Among the deep learning neural networks, convolutional neural network (CNN) is one of the most popular networks, which suffers from difficulties in dealing with small samples and keeping relative structures of features. In this paper, we propose a novel deep learning architecture based on capsule network for TM, called CapsTM, where capsule network is a new type of neural network architecture proposed to address some of the short comings of CNN and shows great potential in many tasks.
Methods
CapsTM is a five-layer neural network, including an input layer, a representation layer, an aggregation layer, a capsule layer and a prediction layer. In CapsTM, two pieces of text are first individually converted into sequences of embeddings and are further transformed by a highway network in the input layer. Then, Bidirectional Long Short-Term Memory (BiLSTM) is used to represent each piece of text and attention-based interaction matrix is used to represent interactive information of the two pieces of text in the representation layer. Subsequently, the two kinds of representations are fused together by BiLSTM in the aggregation layer, and are further represented with capsules (vectors) in the capsule layer. Finally, the prediction layer is a connected network used for classification. CapsTM is an extension of ESIM by adding a capsule layer before the prediction layer.
Results
We construct a corpus of Chinese medical question matching, which contains 36,360 question pairs. This corpus is randomly split into three parts: a training set of 32,360 question pairs, a development set of 2000 question pairs and a test set of 2000 question pairs. On this corpus, we conduct a series of experiments to evaluate the proposed CapsTM and compare it with other state-of-the-art methods. CapsTM achieves the highest F-score of 0.8666.
Conclusion
The experimental results demonstrate that CapsTM is effective for Chinese medical question matching and outperforms other state-of-the-art methods for comparison.
The article examines the Universal Dependencies (UD) annotation scheme. The UD project is an international initiative to produce treebanks of the world’s languages, whereby the treebanks have been ...annotated in a cross-linguistically consistent manner. A central aspect of the UD annotation scheme is its analysis of function words. The scheme advocates subordinating function words to content words. This article discusses linguistic and practical motivations behind the UD decision to subordinate function words to content words. It demonstrates that UD choices in this area are not supported linguistically. At the same time, the near convertibility of the UD treebanks to a more linguistically motivated annotation format means that the UD initiative remains of great value to linguistics in general.
PDF
