This paper investigates the ability of multilingual BERT (mBERT) language model to transfer syntactic knowledge cross-lingually, verifying if and to which extent syntactic dependency relationships ...learnt in a language are maintained in other languages. In detail, the main contributions of this paper are: (i) an analysis of the cross-lingual syntactic transfer capability of mBERT model; (ii) a detailed comparison of cross-language syntactic transfer among languages belonging to different branches of the Indo-European languages, namely English, Italian and French, which present very different syntactic constructions; (iii) a study on the transferability of a syntactic phenomenon peculiar of Italian language, namely the pronoun dropping (pro-drop), also known as omissibility of the subject. To this end, a structural probe devoted to reconstruct the dependency parse tree of a sentence has been exploited, representing the input sentences with the contextual embeddings from mBERT layers. The results of the experimental assessment have shown a transfer of syntactic knowledge of the mBERT model among these languages. Moreover, the behaviour of the probe in the transition from pro-drop to non-pro-drop languages and vice versa has proven to be more effective in case of languages sharing a common linguistic matrix. The possibility of transferring syntactical knowledge, especially in the case of specific phenomena, meets both a theoretical need and can have important practical implications in syntactic tasks, such as dependency parsing.
•An analysis of cross-lingual syntax transfer capability of multilingual BERT model.•Use of a structural probe to reconstruct the dependency parse tree of a sentence.•A cross-lingual test on an aligned and parallel annotated treebank.•Three different languages considered: English, Italian and French.•Study of cross-language transfer of omissibility of the subject syntactic phenomenon.
The work presented in this paper investigates the ability of BERT neural language model pretrained in Italian to embed syntactic dependency relationships into its layers, by approximating a ...Dependency Parse Tree. To this end, a structural probe, namely a supervised model able to extract linguistic structures from a language model, has been trained leveraging the contextual embeddings from the layers of BERT. An experimental assessment has been performed using an Italian version of BERT-base model and a set of datasets for Italian labelled with Universal Dependencies formalism. The results, achieved using standard metrics of dependency parsers, have shown that a knowledge of the Italian syntax is embedded in central-upper layers of the BERT model, according to what observed in literature for the English case. In addition, the probe has been also used to experimentally evaluate the BERT model behaviour in case of two specific syntactic phenomena in Italian, namely null-subject and subject-verb-agreement, showing better performance than an Italian state-of-the-art parser. These findings can open a path for the development of new hybrid approaches, exploiting the probe to integrate or improve limits or weaknesses in analysing articulated constructions of Italian syntax, traditionally complex to be parsed.
The meeting between Natural Language Processing (NLP) and Quantum Computing has been very successful in recent years, leading to the development of several approaches of the so-called Quantum Natural ...Language Processing (QNLP). This is a hybrid field in which the potential of quantum mechanics is exploited and applied to critical aspects of language processing, involving different NLP tasks. Approaches developed so far span from those that demonstrate the quantum advantage only at the theoretical level to the ones implementing algorithms on quantum hardware. This paper aims to list the approaches developed so far, categorizing them by type, i.e., theoretical work and those implemented on classical or quantum hardware; by task, i.e., general purpose such as syntax-semantic representation or specific NLP tasks, like sentiment analysis or question answering; and by the resource used in the evaluation phase, i.e., whether a benchmark dataset or a custom one has been used. The advantages offered by QNLP are discussed, both in terms of performance and methodology, and some considerations about the possible usage QNLP approaches in the place of state-of-the-art deep learning-based ones are given.
Gestures are an inseparable part of the language system (McNeill 2005; Kendon 2004), they are semantically co-expressive with speech serving different semantic functions to accompany oral modality ...(Lin 2017; McNeill 2016). To study these phenomena, we analyse the co-gesture behavior of several Italian politicians during face-to-face interviews. We add a new annotation layer to the PoliModal corpus (Trotta et al. 2020) focused on semantic function of hand movements (Lin 2017; Colletta et al. 2015; Kendon 2004). Then, we explore the patterns of co-occurrence of speech and gestures for the single politicians and from a party perspective. In particular, we address following research questions: i) Are there categories of verbs that systematically accompany hand movements in political interviews? ii) Since the corpus used presents an annotation of "speech constants" (Voghera 2001), is the Lexical Retrieval hypothesis confirmed or are gestures used in correlation with other and different constants of speech? The Lexical Retrieval hypothesis assumes that (a) gesturing occurs during hesitation pauses or in pauses before words indicating problems with lexical retrieval (Dittmann and Llewellyn 1969; Butterworth and Beattie 1978), and (b) that the inability to gesture can cause verbal disfluencies. Finally, we analyse semantic patterns of gesture-speech relationship.
In the last decade, the demand for readily accessible corpora has touched all areas of natural language processing, including coreference resolution. However, it is one of the least considered ...sub-fields in recent developments. Moreover, almost all existing resources are only available for the English language. To overcome this lack, this work proposes a methodology to create a corpus for coreference resolution in Italian exploiting knowledge of annotated resources in other languages. Starting from OntonNotes, the methodology translates and refines English utterances to obtain utterances respecting Italian grammar, dealing with language-specific phenomena and preserving coreference and mentions. A quantitative and qualitative evaluation is performed to assess the well-formedness of generated utterances, considering readability, grammaticality, and acceptability indexes. The results have confirmed the effectiveness of the methodology in generating a good dataset for coreference resolution starting from an existing one. The goodness of the dataset is also assessed by training a coreference resolution model based on BERT language model, achieving the promising results. Even if the methodology has been tailored for English and Italian languages, it has a general basis easily extendable to other languages, adapting a small number of language-dependent rules to generalize most of the linguistic phenomena of the language under examination.
In recent years, the impact of Neural Language Models has changed every field of Natural Language Processing. In this scenario, coreference resolution has been among the least considered task, ...especially in language other than English. This work proposes a coreference resolution system for Italian, based on a neural end-to-end architecture integrating ELECTRA language model and trained on OntoCorefIT, a novel Italian dataset built starting from OntoNotes. Even if some approaches for Italian have been proposed in the last decade, to the best of our knowledge, this is the first neural coreference resolver aimed specifically to Italian. The performance of the system is evaluated with respect to three different metrics and also assessed by replacing ELECTRA with the widely-used BERT language model, since its usage has proven to be effective in the coreference resolution task in English. A qualitative analysis has also been conducted, showing how different grammatical categories affect performance in an inflectional and morphological-rich language like Italian. The overall results have shown the effectiveness of the proposed solution, providing a baseline for future developments of this line of research in Italian.
In last years, coreference resolution has received a sensibly performance boost exploiting different pre-trained Neural Language Models, from BERT to SpanBERT until Longformer. This work is aimed at ...assessing, for the first time, the impact of ELECTRA model on this task, moved by the experimental evidence of an improved contextual representation and better performance on different downstream tasks. In particular, ELECTRA has been employed as representation layer in an assessed neural coreference architecture able to determine entity mentions among spans of text and to best cluster them. The architecture itself has been optimized: i) by simplifying the modality of representation of spans of text but still considering both the context they appear and their entire content, ii) by maximizing both the number and length of input textual segments to exploit better the improved contextual representation power of ELECTRA, iii) by maximizing the number of spans of text to be processed, since potentially representing mentions, preserving computational efficiency. Experimental results on the OntoNotes dataset have shown the effectiveness of this solution from both a quantitative and qualitative perspective, and also with respect to other state-of-the-art models, thanks to a more proficient token and span representation. The results also hint at the possible use of this solution also for low-resource languages, simply requiring a pre-trained version of ELECTRA instead of language-specific models trained to handle either spans of text or long documents.
Il lavoro mira a fornire una rappresentazione delle preferenze di selezione verbali per la lingua italiana. L'esperimento si ricollega alle metodologie basate su corpora e si articola in due fasi: ...l'estrazione degli argomenti dai corpora scelti e la generalizzazione delle preferenze di selezione utilizzando un'ontologia lessicale. Le risorse utilizzate sono: LexIt, un lessico di valenza per i verbi italiani, come risorsa lessicale, e MultiWordNet, come ontologia. L'obiettivo e fornire un livello di rappresentazione dettagliato del comportamento verbale navigando l'intera rete semantica e facendo emergere comportamenti piu specifici nelle preferenze di selezione degli argomenti verbali. KEYWORDS: Linguistica computazionale; LexIt; MultiWordNet; Selezione verbale; Verbi
Il lavoro mira a fornire una rappresentazione delle preferenze di selezione verbali per la lingua italiana. L'esperimento si ricollega alle metodologie basate su corpora e si articola in due fasi: ...l'estrazione degli argomenti dai corpora scelti e la generalizzazione delle preferenze di selezione utilizzando un'ontologia lessicale. Le risorse utilizzate sono: LexIt, un lessico di valenza per i verbi italiani, come risorsa lessicale, e MultiWordNet, come ontologia. L'obiettivo è fornire un livello di rappresentazione dettagliato del comportamento verbale navigando l'intera rete semantica e facendo emergere comportamenti più specifici nelle preferenze di selezione degli argomenti verbali. Testo dell'editore.The present article aims at providing a representation of verb selection preferences for the Italian language. The experiment connects methods based on corpora, and it has been carried on in two steps: first, topic mining from the selected corpora; then, the generalization of verb selection preferences using a lexical ontology. The resources used for the experiment are the following: LexIt, a lexicon for Italian verbs as a lexical resource, and MultiWordNet as an ontology. The aim of the study is provinding a detailed representation level of the verbal behaviour and selection through the semantic network, highlighting specific behaviours in preferences of verb selection. Publisher's Text.
Nowadays, the spread of deceptive reviews is a problem that has reached critical dimensions, having a significant economic impact on business activities. This paper aims to estimate – at the ...quantitative and qualitative levels – the possibility of using particular words to disambiguate between truthful and deceptive text, focusing on reviews produced in the cultural heritage domain. For this purpose, a lexicon-based methodology has used two different lexicons: sentiment information, intensifiers, downtoners, and negation operators. As known in the literature, these elements are crucial in a classification process related to deceptiveness. The evaluation phase has considered quantitative metrics such as accuracy and F1 score and ad hoc developed metrics that consider specific linguistic parameters such as polarity and tone of voice intensifiers. A qualitative analysis of a subset of the corpus has also been carried out to understand better factors that impact the classification of deceptive review. Several linguistic features have been considered, ranging from the number of intensifiers to their type and position in phrases and sentences. A comparison between the performances of two different lexicons used has been added to the analysis.
•Lexicon-based approach for classifying Italian fake Cultural Heritage reviews.•A methodology based on intensifiers and downtoners from sentiment analysis.•New Italian Cultural Heritage Corpus for Deceptive Reviews Classification.•Comparison of manual and automatically created lexicons.•New metric considering polarity and linguistic features.
PDF
