Cross-Lingual Information Retrieval (CLIR) provides flexibility to users to query in their regional (source) languages regardless the target documents languages. CLIR uses trending translation ...techniques Statistical Machine Translation (SMT) and Neural Machine Translation (NMT). SMT and NMT achieve good results for foreign languages but not for Indian languages due to non-absoluteness of the parallel corpus. Source language user queries may contain the Out Of Vocabulary (OOV) words which are not present in the parallel corpus such words may be skipped without performing translation by SMT. In this paper, a context-based translation algorithm is proposed to translate the OOV words by utilizing two unlabeled & unrelated large raw corpora (in source and target language) and a small bi-lingual parallel corpus. Since SMT performs better than NMT for Hindi to English translation as per the literature, therefore, experimental results are evaluated for FIRE datasets against baseline SMT. The proposed algorithm improves evaluation measures, Recall up to 6.04% (0.8785) for FIRE 2010 and up to 3.96% (0.7365) for FIRE 2011, & Mean Average Precision (MAP) up to 14.37% (0.3239) for FIRE 2010 and up to 5.46% (0.1988) for FIRE 2011, in comparison to the baseline SMT which achieves 0.8284 and 0.7084 Recall for FIRE 2010 and 2011, & 0.2832 and 0.1885 MAP for FIRE 2010 and 2011. An analysis for the number of OOV words shows that the proposed algorithm reduces the number of OOV more effectively, up to 0.81% for FIRE 2010 and 1.73% for FIRE 2011.
In this paper we argue that the time is ripe for translator educators to engage with Statistical Machine Translation (SMT) in more profound ways than they have done to date. We explain the basic ...principles of SMT and reflect on the role of humans in SMT workflows. Against a background of diverging opinions on the latter, we argue for a holistic approach to the integration of SMT into translator training programmes, one that empowers rather than marginalises translators. We discuss potential barriers to the use of SMT by translators generally and in translator training in particular, and propose some solutions to problems thus identified. More specifically, cloud-based services are proposed as a means of overcoming some of the technical and ethical challenges posed by more advanced uses of SMT in the classroom. Ultimately the paper aims to pave the way for the design and implementation of a new translator-oriented SMT syllabus at our own University and elsewhere.
In the past couple of decades, machine translation (MT), whether phrase-based statistical or neural MT, has made considerable progress and has become increasingly common in specialised translation ...workflows. More studies on MT in legal contexts have been conducted in the past several years, and MT has been and continues to be used in international and supranational institutions that rely on legal translation and whose voluminous collections of translated text have been used to train contemporary systems. The time is ripe for a synthesis of developments in the area that may reinforce our understanding of what we might expect from MT from the perspective of legal translators. This systematic review of legal translation and MT studies teases out potential strengths/weaknesses of MT and opportunities/threats of using it from a legal translator perspective, focusing on lexical and morphosyntactic challenges that may set legal translation apart from other specialised translation areas. A SWOT analysis from this perspective may offer value as a pedagogical framework for fostering domain-specific MT literacy in the legal translation classroom. Students will develop an understanding of what might reasonably be expected from MT in legal translation contexts, while reinforcing their understanding of legal translation challenges.
•A syntax-based reordering model (RM) for SMT system is proposed.•Our RM predicts the orientation between syntactic dependants of the source sentence.•We enrich the proposed RM with semantic ...features, so it can perform semantic generalization.•Our RM outperforms the baseline and two competing RMs in terms of BLEU and TER.
We present a syntax-based reordering model (RM) for hierarchical phrase-based statistical machine translation (HPB-SMT) enriched with semantic features. Our model brings a number of novel contributions: (i) while the previous dependency-based RM is limited to the reordering of head and dependant constituent pairs, we also model the reordering of pairs of dependants; (ii) Our model is enriched with semantic features (Wordnet synsets) in order to allow the reordering model to generalize to pairs not seen in training but with equivalent meaning. (iii) We evaluate our model on two language directions: English-to-Farsi and English-to-Turkish. These language pairs are particularly challenging due to the free word order, rich morphology and lack of resources of the target languages.
We evaluate our RM both intrinsically (accuracy of the RM classifier) and extrinsically (MT). Our best configuration outperforms the baseline classifier by 5–29% on pairs of dependants and by 12–30% on head and dependant pairs while the improvement on MT ranges between 1.6% and 5.5% relative in terms of BLEU depending on language pair and domain. We also analyze the value of the feature weights to obtain further insights on the impact of the reordering-related features in the HPB-SMT model. We observe that the features of our RM are assigned significant weights and that our features are complementary to the reordering feature included by default in the HPB-SMT model.
Internet of Things (IoT) networks have been widely deployed to achieve communication among machines and humans. Machine translation can enable human-machine interactions for IoT equipment. In this ...paper, we propose to combine the neural machine translation (NMT) and statistical machine translation (SMT) to improve translation precision. In our design, we propose a hybrid deep learning (DL) network that uses the statistical feature extracted from the words as the data set. Namely, we use the SMT model to score the generated words in each decoding step of the NMT model, instead of directly processing their outputs. These scores will be converted to the generation probability corresponding to words by classifiers and used for generating the output of the hybrid MT system. For the NMT, the DL network consists of the input layer, embedding layer, recurrent layer, hidden layer, and output layer. At the offline training stage, the NMT network is jointly trained with SMT models. Then at the online deployment stage, we load the fine-trained models and parameters to generate the outputs. Experimental results on French-to-English translation tasks show that the proposed scheme can take advantage of both NMT and SMT methods, thus higher translation precision could be achieved.
In statistical machine translation, translation prediction considers not only the aligned source word itself but also its source contextual information. Learning context representation is a promising ...method for improving translation results, particularly through neural networks. Most of the existing methods process context words sequentially and neglect source long-distance dependencies. In this paper, we propose a novel neural approach to source dependence-based context representation for translation prediction. The proposed model is capable of not only encoding source long-distance dependencies but also capturing functional similarities to better predict translations (i.e., word form translations and ambiguous word translations). To verify our method, the proposed mode is incorporated into phrase-based and hierarchical phrase-based translation models, respectively. Experiments on large-scale Chinese-to-English and English-to-German translation tasks show that the proposed approach achieves significant improvement over the baseline systems and outperforms several existing context-enhanced methods.
Adequately selecting variable names is a difficult activity for practitioners. In 2018, Jaffe et al. proposed the use of statistical machine translation (SMT) to suggest descriptive variable names ...for decompiled code. A large corpus of decompiled C code was used to train the SMT model. Our paper presents the results of a partial replication of Jaffe’s experiment. We apply the same technique and methodology to a dataset made of code written in the Pharo programming language. We selected Pharo since its syntax is simple – it fits on half of a postcard – and because the optimizations performed by the compiler are limited to method scope. Our results indicate that SMT may recover between 8.9% and 69.88% of the variable names depending on the training set. Our replication concludes that: (i) the accuracy depends on the code similarity between the training and testing sets; (ii) the simplicity of the Pharo syntax and the satisfactory decompiled code alignment have a positive impact on predicting variable names; and (iii) a relatively small code corpus is sufficient to train the SMT model, which shows the applicability of the approach to less popular programming languages. Additionally, to assess SMT’s potential in improving original variable names, ten Pharo developers reviewed 400 SMT name suggestions, with four reviews per variable. Only 15 suggestions (3.75%) were unanimously viewed as improvements, while 45 (11.25%) were perceived as improvements by at least two reviewers, highlighting SMT’s limitations in providing suitable alternatives.
The Model of Automated Knowledge Communication outlined in this postdoctoral thesis aims to broaden the scope of current conceptualizations of specialized communication to account for self-adapting, ...AI-/LLM-based machines since these are presently changing the practice of multilingual specialized communication in fundamental ways – hence affording research to adapt. The model facilitates the development of research on specialized communication specifically by integrating, firstly, new approaches toward concepts of expert knowledge and, secondly, the involvement of sophisticated machines able to act increasingly autonomously. The monograph sets out with an overview on the objects of research the model can be applied to as it incorporates approaches of cybernetics, technical sociology, and media sociology. The description of the model is followed by its validation on basis of seven case studies from which some suggestions for further research are derived.
•Hybrid Arabic-to-French SMT using rule-based pre-processing and language analysis.•Morphologically reduced rules that reduce the morphology of Arabic.•Swapping rules for a structural matching on ...pronouns and verbs.•A gain in terms of BLEU score after applying some of these rules.•A learning curve showing the findings under scarce- or large-resources conditions.
Arabic is a highly inflected language and a morpho-syntactically complex language with many differences compared to several languages that are heavily studied. It may thus require good pre-processing as it presents significant challenges for Natural Language Processing (NLP), specifically for Machine Translation (MT). This paper aims to examine how Statistical Machine Translation (SMT) can be improved using rule-based pre-processing and language analysis. We describe a hybrid translation approach coupling an Arabic–French statistical machine translation system using the Moses decoder with additional morphological rules that reduce the morphology of the source language (Arabic) to a level that makes it closer to that of the target language (French). Moreover, we introduce additional swapping rules for a structural matching between the source language and the target language. Two structural changes involving the positions of the pronouns and verbs in both the source and target languages have been attempted. The results show an improvement in the quality of translation and a gain in terms of BLEU score after introducing a pre-processing scheme for Arabic and applying these rules based on morphological variations and verb re-ordering (VS into SV constructions) in the source language (Arabic) according to their positions in the target language (French). Furthermore, a learning curve shows the improvement in terms on BLEU score under scarce- and large-resources conditions. The proposed approach is completed without increasing the amount of training data or radically changing the algorithms that can affect the translation or training engines.
PDF
