Recent advances in multilingual language modeling have brought the idea of a truly universal parser closer to reality. However, such models are still not immune to the “curse of multilinguality”: ...Cross-language interference and restrained model capacity remain major obstacles. To address this, we propose a novel language adaptation approach by introducing
to a multilingual parser. Contextual language adapters make it possible to learn adapters via language embeddings while sharing model parameters across languages based on contextual parameter generation. Moreover, our method allows for an easy but effective integration of existing
into the parsing model. Because not all typological features are available for every language, we further combine typological feature prediction with parsing in a multi-task model that achieves very competitive parsing performance without the need for an external prediction system for missing features.
The resulting parser, UDapter, can be used for dependency parsing as well as sequence labeling tasks such as POS tagging, morphological tagging, and NER. In dependency parsing, it outperforms strong monolingual and multilingual baselines on the majority of
high-resource and low-resource (zero-shot) languages, showing the success of the proposed adaptation approach. In sequence labeling tasks, our parser surpasses the baseline on high resource languages, and performs very competitively in a zero-shot setting. Our in-depth analyses show that adapter generation via typological features of languages is key to this success.
"This book is the result of a group of researchers from different disciplines asking themselves one question: what does it take to develop a computer interface that listens, talks, and can answer ...questions in a domain? First, obviously, it takes specialized modules for speech recognition and synthesis, human interaction management (dialogue, input fusion, and multimodal output fusion), basic question understanding, and answer finding. While all modules are researched as independent subfields, this book describes the development of state-of-the-art modules and their integration into a single, working application capable of answering medical (encyclopedic) questions such as ""How long is a person with measles contagious?"" or ""How can I prevent RSI?"". The contributions in this book, which grew out of the IMIX project funded by the Netherlands Organisation for Scientific Research, document the development of this system, but also address more general issues in natural language processing, such as the development of multidimensional dialogue systems, the acquisition of taxonomic knowledge from text, answer fusion, sequence processing for domain-specific entity recognition, and syntactic parsing for question answering. Together, they offer an overview of the most important findings and lessons learned in the scope of the IMIX project, making the book of interest to both academic and commercial developers of human-machine interaction systems in Dutch or any other language. Highlights include: integrating multi-modal input fusion in dialogue management (Van Schooten and Op den Akker), state-of-the-art approaches to the extraction of term variants (Van der Plas, Tiedemann, and Fahmi, Tjong Kim Sang, Hofmann, and De Rijke), and multi-modal answer fusion (two chapters by Van Hooijdonk, Bosma, Krahmer, Maes, Theune, and Marsi). Watch the IMIX movie at www.youtube.com/watch?v=swA8_Y56aak Like IBM's Watson, the IMIX system described in the book gives naturally phrased responses to naturally posed questions. Where Watson can only generate synthetic speech, the IMIX system also recognizes speech. On the other hand, Watson is able to win a television quiz, while the IMIX system is domain-specific, answering only to medical questions. ""The Netherlands has always been one of the leaders in the general field of Human Language Technology, and IMIX is no exception. It was a very ambitious program, with a remarkably successful performance leading to interesting results. The teams covered a remarkable amount of territory in the general sphere of multimodal question answering and information delivery, question answering, information extraction and component technologies."" Eduard Hovy, USC, USA, Jon Oberlander, University of Edinburgh, Scotland, and Norbert Reithinger, DFKI, Germany"
In this paper, we present a unified feature-based theory of complement, adjunct, and subject extraction, in which there is no need either for valence reducing lexical rules or for phonologically null ...traces. Our analysis rests on the assumption that the mapping between argument structure and valence is defined by realization constraints which are satisfied by all lexical heads. Arguments can be realized as local dependents, in which case they are selected via the head's valence features. Alternatively, arguments may be realized in a long-distance dependency construction, in which case they are selected via the head's SLASH features. Furthermore, we argue that English post-verbal adjuncts, as well as complements, are syntactic dependents selected by the verb, thus providing a uniform analysis of complement and adjunct extraction. Finally, we show that our analysis provides an alternative treatment of subject extraction and we offer a new account of the that-trace effect.
We present a minimally-supervised approach for learning part–whole relations from texts. Unlike previous techniques, we focused on sparse, domain-specific texts. The novelty in our approach lies in ...the use of Wikipedia as a knowledge-base, from which we first acquire a set of reliable patterns that express part–whole relations. This is achieved by a minimally-supervised algorithm. We then use the patterns acquired to extract part–whole relation triples from a collection of sparse, domain-specific texts. Our strategy, of learning in one domain and applying the knowledge in another domain is based upon the notion of domain-adaption. It allows us to overcome the challenges of learning the relations directly from the sparse, domain-specific corpus. Our experimental evaluations reveal that, despite its general-purpose nature, Wikipedia can be exploited as a source of knowledge for improving the performance of domain-specific part–whole relation extraction. As our other contributions, we propose a mechanism that mitigates the negative impact of semantic-drift on minimally-supervised algorithms. Also, we represent the patterns in the extracted relations using sophisticated syntactic structures that avoid the limitations of traditional surface string representations. In addition, we show that domain-specific part–whole relations cannot be conclusively classified in existing taxonomies.
We propose a novel framework for overcoming the challenges in extracting causal relations from domain-specific texts. Our technique is minimally-supervised, alleviating the need for ...manually-annotated, expensive training data. As our main contribution, we show that open-domain corpora can be exploited as knowledge bases to overcome data sparsity issues posed by domain-specific relation extraction, and that they enable substantial performance gains. We also address longstanding challenges of extant minimally-supervised approaches. To suppress the negative impact of semantic drift, we propose a technique based on the Latent Relational Hypothesis. In addition, our approach discovers both explicit (e.g. “to cause”) and implicit (e.g. “to destroy”) causal patterns/relations. Unlike existing minimally-supervised techniques, we adopt a principled seed selection strategy, which enables us to discover a more diverse set of causal patterns/relations. Our experiments reveal that our approach outperforms a state-of-the-art baseline in discovering causal relations from a real-life, domain-specific corpus.
•Novel technique that accurately extracts implicit and explicit causal patterns from sparse, ungrammatical domain-specific texts•Approach is minimally-supervised, eschewing the need for expensive annotated data.•Exploits open-domain knowledge base to support domain-specific term extraction•Addresses the issue of semantic drift using Latent Relational Hypothesis•The proposed technique outperforms a state-of-the-art baseline over real-life texts.
► Novel technique for term extraction from sparse, ungrammatical texts. ► Accurately detects terms, even those with extremely low frequency (sparse). ► Extracts terms that contain any number of ...words, even those with more than 2 words. ► Exploits open-domain knowledge base to support domain-specific term extraction. ► The proposed technique outperforms a state-of-the-art baseline over real-life texts.
Existing term extraction systems have predominantly targeted large and well-written document collections, which provide reliable statistical and linguistic evidence to support term extraction. In this article, we address the term extraction challenges posed by sparse, ungrammatical texts with domain-specific contents, such as customer complaint emails and engineers’ repair notes. To this aim, we present ExtTerm, a novel term extraction system. Specifically, as our core innovations, we accurately detect rare (low frequency) terms, overcoming the issue of data sparsity. These rare terms may denote critical events, but they are often missed by extant TE systems. ExtTerm also precisely detects multi-word terms of arbitrarily lengths, e.g. with more than 2 words. This is achieved by exploiting fundamental theoretical notions underlying term formation, and by developing a technique to compute the collocation strength between any number of words. Thus, we address the limitation of existing TE systems, which are primarily designed to identify terms with 2 words. Furthermore, we show that open-domain (general) resources, such as Wikipedia, can be exploited to support domain-specific term extraction. Thus, they can be used to compensate for the unavailability of domain-specific knowledge resources. Our experimental evaluations reveal that ExtTerm outperforms a state-of-the-art baseline in extracting terms from a domain-specific, sparse and ungrammatical real-life text collection.
Various supervised algorithms for mining causal relations from large corpora exist. These algorithms have focused on relations explicitly expressed with causal verbs, e.g. “to cause”. However, the ...challenges of extracting causal relations from domain-specific texts have been overlooked. Domain-specific texts are rife with causal relations that are implicitly expressed using verbal and non-verbal patterns, e.g. “reduce”, “drop in”, “due to”. Also, readily-available resources to support supervised algorithms are inexistent in most domains. To address these challenges, we present a novel approach for causal relation extraction. Our approach is minimally-supervised, alleviating the need for annotated data. Also, it identifies both explicit and implicit causal relations. Evaluation results revealed that our technique achieves state-of-the-art performance in extracting causal relations from domain-specific, sparse texts. The results also indicate that many of the domain-specific relations were unclassifiable in existing taxonomies of causality.
PDF
