In this paper we explore how simple auditory/verbal features of the spoken language, such as voice characteristics (pitch) and language cues (empathy/humor expression) influence the quality of ...interaction with a social robot receptionist. For our experiment two robot characters were created: Olivia, the more extrovert, exuberant, and humorous robot with a higher voice pitch and Cynthia, the more introvert, calmer and more serious robot with a lower voice pitch. Our results showed that the voice pitch seemed to have a strong influence on the way users rated the overall interaction quality, as well as the robot’s appeal and overall enjoyment. Further, the humor appeared to improve the users’ perception of task enjoyment, robot personality and speaking style while the empathy showed effects on the way users evaluated the robot’s receptive behavior and the interaction ease. With our study, we would like to stress in particular the importance of voice pitch in human robot interaction and to encourage further research on this topic.
Full text
Available for:
EMUNI, FIS, FZAB, GEOZS, GIS, IJS, IMTLJ, KILJ, KISLJ, MFDPS, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, SBMB, SBNM, UKNU, UL, UM, UPUK, VKSCE, ZAGLJ
•We present an audiovisual dataset for investigation of mimicry behaviour.•We report baseline performances from per-session mimicry classification experiments.•Performance is session-dependent, due ...to variability in subject expressiveness.•Current mimicry classification methods need more development for spontaneous data.
People mimic verbal and nonverbal expressions and behaviour of their counterparts in various social interactions. Research in psychology and social sciences has shown that mimicry has the power to influence social judgment and various social behaviours, including negotiation and debating, courtship, empathy and helping behaviour. Hence, automatic recognition of mimicry behaviour would be a valuable tool in various domains, and especially in negotiation skills enhancement and medical help provision training. In this work, we present the MAHNOB mimicry database, a set of fully synchronised, multi-sensory, audiovisual recordings of naturalistic dyadic interactions, suitable for investigation of mimicry and negotiation behaviour. The database contains 11 h of recordings, split over 54 sessions of dyadic interactions between 12 confederates and their 48 counterparts, being engaged either in a socio-political discussion or negotiating a tenancy agreement. To provide a benchmark for efforts in machine understanding of mimicry behaviour, we report a number of baseline experiments based on visual data only. Specifically, we consider face and head movements, and report on binary classification of video sequences into mimicry and non-mimicry categories based on the following widely-used methodologies: two similarity-based methods (cross correlation and time warping), and a state-of-the-art temporal classifier (Long Short Term Memory Recurrent Neural Network). The best reported results are session-dependent, and affected by the sparsity of positive examples in the data. This suggests that there is much room for improvement upon the reported baseline experiments.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UL, UM, UPCLJ, UPUK
Despite the robust evidence that congruent background music in the physical store environment positively affects consumer reactions, less is known about its effects in an online context. The present ...study aims (1) to examine whether congruency via multiple elicited crossmodal correspondences between background music and the online store environment (e.g., perceived lightness, loudness, and coldness of the cue/environment) leads to more positive affective, evaluative, and behavioral consumer reactions and (2) to investigate the moderating role of shopping goals on this crossmodal congruency effect. Previous research showed that low task-relevant atmospheric cues like music can have a negative effect on consumers when they visit a website with a purchase goal in mind. An online experiment was conducted with 239 respondents randomly assigned to a shopping goal (experiential browsing vs. goal-directed searching) and a music condition (no music, crossmodally congruent music, or crossmodally incongruent music). Our results show that crossmodally incongruent background music (vs. no music) leads to more positive consumer reactions for experiential browsers and more negative consumer reactions for goal-directed searchers. Conversely, crossmodally congruent background music (vs. no music) has a positive effect on experiential browsers and no adverse effect on goal-directed searchers. Additionally, the presence of crossmodally congruent background music leads to more positive consumer reactions than the presence of crossmodally incongruent background music, independent of the shopping goal. We extend previous research on multisensory congruency effects by showing the added value of establishing congruency between music and the store environment via multiple elicited crossmodal correspondences in the online environment, countering previously found negative effects of low-task relevant atmospheric cues for goal-directed searchers.
In the previous decade, breakthroughs in the central nervous system bioinformatics and computational innovation have prompted significant developments in brain-computer interface (BCI), elevating it ...to the forefront of applied science and research. BCI revitalization enables neurorehabilitation strategies for physically disabled patients (e.g., disabled patients and hemiplegia) and patients with brain injury (e.g., patients with stroke). Different methods have been developed for electroencephalogram (EEG)-based BCI applications. Due to the lack of a large set of EEG data, methods using matrix factorization and machine learning were the most popular. However, things have changed recently because a number of large, high-quality EEG datasets are now being made public and used in deep learning-based BCI applications. On the other hand, deep learning is demonstrating great prospects for solving complex relevant tasks such as motor imagery classification, epileptic seizure detection, and driver attention recognition using EEG data. Researchers are doing a lot of work on deep learning-based approaches in the BCI field right now. Moreover, there is a great demand for a study that emphasizes only deep learning models for EEG-based BCI applications. Therefore, we introduce this study to the recent proposed deep learning-based approaches in BCI using EEG data (from 2017 to 2022). The main differences, such as merits, drawbacks, and applications are introduced. Furthermore, we point out current challenges and the directions for future studies. We argue that this review study will help the EEG research community in their future research.
The brain-computer interface (BCI) field has grown dramatically over the past few years, but there are still no coordinated efforts to ensure efficient communication and collaboration among key ...stakeholders. The European Commission (EC) has recently renewed their efforts to establish such a coordination effort by funding a coordination and support action for the BCI community called 'BNCI Horizon 2020' after the 'Future BNCI' project. Major goals of this new project include developing a roadmap for the next decade and beyond, encouraging discussion and collaboration within the BCI community, fostering communication with the general public, and the foundation of an international BCI Society. We present a short overview of current and past EU-funded BCI projects and provide evidence of a growing research and industrial community. Efficient communication also entails the establishment of clear terminology, which is a major goal of BNCI Horizon 2020. To this end, we give a brief overview of current BCI-related terms and definitions. A major networking activity in the project was the BNCI Horizon 2020 Retreat in Hallstatt, Austria. Over 60 experts participated in this event to discuss the future of the BCI field in a series of plenary talks, targeted discussions, and parallel focus sessions. A follow-up event was the EU BCI Day at the 6th International Brain-Computer Interface Conference in Graz, Austria. This networking event included plenary talks by eight companies and representatives from all seven ongoing EU research projects, poster presentations, demos, and discussions. Another goal of BNCI Horizon 2020 is the foundation of an official BCI Society. In this article, we summarize the current status of this process. Finally, we present visions for future BCI applications developed within BNCI Horizon 2020 using input from external BCI experts as well. We identify common themes and conclude with six exemplary use cases.
Full text
Available for:
BFBNIB, GIS, IJS, KISLJ, NUK, PNG, UL, UM
We present a perspective article on the state of multisensory human–food interaction (MHFI) research and lay out some reflections for research and development in this area of inquiry, based on a ...revision of the different spaces that we have co-created with researchers in this space. We begin by conceptualizing and defining MHFI, before moving onto presenting some of its major themes, as well as possible ways in which such themes can guide future research in the area. This article provides key definitions and foundations for the area of MHFI, as well as a first point of contact for those interested in it.
PDF
