The multimodal nature of texts to music involves the complex interaction of verbal, auditory, in some cases visual and other components, which determines the functioning of the textual unity. In this ...regard, the translation of multimodal texts, and in particular the translation of opera librettos, puts forward special requirements for the translator, due to which even the knowledge of the subject area, terminology and style of the source text is not always sufficient to achieve high quality translation. Based on the analysis of the translation principles of one of the most prominent translators of opera librettos Viktor Kolomiitsov (1868-1936) and his translated texts, the article aims to determine the basic requirements to the translation of libretto as a multimodal text, combining the verbal and auditory components in the text unity. It also aims to answer the research question: what exactly is the special feature of multimodal text translation focused on auditory mode, and in case of opera libretto translations - what resources can be used to perform the translation? The material for the study included the texts to L. Beethoven and R. Wagner’s works and their translations. The comparative analysis of V. Kolomiytsov’s translations and several other translators’ approaches allowed us to confirm the assumption about the complexity of auditory multimodal texts translation, identify the main problems of libretto translation, and demonstrate different translation solutions in the transfer of the rhythmic logic of the source text, its syntactic features, alliteration and vocabulary. The procedure of text analysis involved elements of philological and comparative analyses of translations, as well as statistical calculations. It was revealed that the connection of the verbal component with the melodic line, its accents and rhythm predetermines the structure of the text to vocal works. For the translator, this means additional limitations, which are quite difficult to overcome. Equirhythmic translation correlated with music is one of the most difficult types of translation, in which preservation of “form” (rhyme, meter, number of syllables, coincidence of accents, etc.) plays an important role and predetermines the quality of auditory multimodal text translation.
Studying the city as text is influenced by a phenomenon called the linguistic turn. This is because the text of the city is composed of different semiotic systems. This study employs multimodal ...discourse analysis as its theoretical framework. Through a close examination of the organizational, interactional and representational metafunction of the city text, the authors aim to highlight its characteristics as a communicational resource to provide a multimodal reading of the city. They demonstrate how applying a social semiotic approach to the physical dimensions of a street contributes to a particular understanding of how this public space is used or navigated and how the formal management of the flow of cars and people is materialized. Here the city is classified into different levels of resolution based on urban morphology’s description of the urban tissue. The text of the city is constructed through dialogue among meaning metafunctions on one level and among different levels. Examining different aspects of meaning in the text of Imam Khomeini Street and the city of Rasht indicates how, in such a text, the producers and participants communicate through the material aspect of the city. Uses of semiotic materials are ideological and their affordances have evolved in specific socio-political moments. In this street, the city as an institution and as a place of authority having the biggest share in the production of this text persuades other participants to follow and reproduce a certain narration of the reality. Controlling and training the body, on one hand, and using the urban space for advertising the dominant discourse, on the other hand, are the boldest motives of this street’s design, and the formation of the different dimensions of meaning is somehow at the service of the consolidation of a particular kind of everyday life.
Homogeneous texts are being replaced by heterogeneous texts that are called audio-visual ones; texts and teletexts belong to the most formed of them. On the ground of cinema and television the texts ...with the unique structure that combines verbal and non-verbal sense carriers apoeared. The aim of this work is to describe peculiarities of meaning articulation in the dynamic scope of such screen texts. To reach the aim the task to show how the basic elements of the screen “language” materialize in verbal and non-verbal units of the screen “speech” of polycode-multimodal text and how it constructs the general meaning of audio-visual narration is set. Also contamination of levels of meaning is being studied in this work and how it influences the trope. The research is being held on the case of the film text “Faust” by A. Sokurov. The analysis is being done on the space-time continuum, i.e. with the consideration of the movement in the film text. Units of the screen “speech” are being extracted from the chosen segments and analysed as the main sources of the meaning of audio-visual message as a whole; also the peculiarities of the meaning articulation on the denotational and connotational levels in space-time scope of polycode-multimodal text are being described. Apart from that, the process of semiotic interpolation of verbal and non-verbal units that leads to the generating of the transitional meaning and of the trope on the grounds of displacement and adjacency of the meanings is being studied. The author comes to the conclusion that integrity of verbal and non-verbal units in the space-time scope of the polycode-multimodal text, that results in trope generation, also leads to the semiotic situation of simulacrativity, whose main goal is to make the spectator believe that what is happening on the screen is real. This article is aimed at students, post-graduates, professors, linguists and others interested in general and linguistic semiotics.
In this paper we present our work process, methods and concerns in developing and implementing a model for analysis of multimodal texts. Using a design-based research approach and a multimodal social ...semiotic framework on communication, we investigated three different groups of students using video recordings and participatory observations. The results revealed the development of the model through three different design cycles into the current version and how students contributed to this process. In addition, we examined how the design process can be understood from the concept of student agency and how it contributes to the understanding of theory. The findings show that by applying a design process, students are offered student agency while creating an opportunity for a joint exploration between students and teachers, which is a rewarding experience that contributes to the development of all participants.
More recently, as images, memes and graphics interchange formats have dominated social feeds, typographic/infographic visual content has emerged as an important social media component. This ...multimodal text combines text and image, defining a novel visual language that must be analysed because it has the potential to modify, confirm or grade the sentiment's polarity. The problem is how to effectively use information from the visual and textual content in image‐text posts. This article presents a new deep learning‐based multimodal sentiment analysis (MSA) model using multimodal data such as images, text and multimodal text (image with embedded text). The text analytic unit, the discretization control unit, the picture analytic component and the decision‐making component are all included in this system. The discretization unit separates the text from the picture using the variant and channel augmented maximally stable extremal regions (VCA‐MSERs) technique, which are then analysed as discrete elements and fed into the appropriate image and text analytics units. The text analytics system utilizes a stacked recurrent neural network with multilevel attention and feedback module (SRNN‐MAFM) to detect the sentiment of the text. A deep convolutional neural network (CNN) structure with parallel‐dilated convolution and self‐attention module (PDC‐SAM) is developed to forecast the emotional response to visual content. Finally, the decision component employs a Boolean framework including an OR function to evaluate and classify the output into three fine‐grained sentiment classes: positive, neutral and negative. The proposed work is simulated in the python platform using the STS‐Gold, Flickr 8k and B‐T4SA datasets for sentiment analysis of text and visual and multimodal text. Simulation outcomes proved that the suggested method achieved better accuracy of 97.8%, 97.7% and 90% for text, visual and MSA individually compared to other methods.
Experimental Study of Demotivators and Memes Perception Complexity Gorbacheva, Aleksandra; Berlin Khenis, Alexandra; Puchkova, Alexandra ...
Vestnik Volgogradskogo gosudarstvennogo universiteta. Serii͡a︡ 2, I͡A︡zykoznanie.,
06/2021, Letnik:
20, Številka:
2
Journal Article
Recenzirano
Odprti dostop
The article presents the results of a pilot study of the perception of the demotivator and meme genres. It was a part of an experimental study of psychophysiological and psycholinguistic features of ...perception and understanding of multimodal extremist texts. The aim of the study is to develop and test the hypothesis about the influence of genre on perception of multimodal texts. To test the hypothesis, we analyze the respondents' eye movement data from the main experimental study (n = 60; 31 forensic linguists with anti-extremism practice, 29 non-experts). Research methods were eye-tracking and quantitative data processing. The following statistically reliable data were obtained: compared to memes, respondents looked at demotivators (1) for a longer time, made (2) shorter fixations, (3) with more of them, and also made (4) faster and (5) shorter saccades. These parameters may indicate a denser scanning pattern of viewing demotivators compared to memes and greater cognitive expenditure in assessing the semantic content of demotivator texts. The results of the study suggest a connection between genre and the degree of multimodal texts perception complexity. This provides an opportunity for further research in this direction and, in the future, will enable the development of norms of cognitive load of judicial linguists who analyse multimodal extremist texts.
This article is devoted to the analysis of the sign as the means of constructing reality and mind control of the text consumer. The texts under study are semiotically expanded texts of the cinema and ...television. The research is mainly dedicated to the study of the chosen material at the level of the screenshot without regard to their syntagmatic organization and rhythmic stratification. Summarizing the results of the research the author comes to the conclusion that there is a semiotic situation in heterogeneous texts that enhances the construction of reality of different levels of simulacrativity. The author manages to renovate the topic of simulacra (“empty signs”), having singled out their lingual-semiotic peculiarities at the level of the screenshot of the semiotically expanded texts.
PDF
