Parents recognize the potential benefits of technology for their young children but are wary of too much screen time and its potential deficits in terms of social engagement and physical activity. To ...address these concerns, related literature suggests technology usages with a blend of digital and physical learning experiences. Towards this end, we developed Kid Space, incorporating immersive computing experiences designed to engage children more actively in physical movement and social collaboration during play-based learning. The technology features an animated peer learner, Oscar, who aims to understand and respond to children’s actions and utterances using extensive multimodal sensing and sensemaking technologies. To investigate student engagement during Kid Space learning experiences, an exploratory case study was designed using a formative research method with eight first-grade students. Multimodal data (audio and video) along with observational, interview, and questionnaire data were collected and analyzed. The results show that the students demonstrated high levels of engagement, less attention focused on the screen (projected wall), and more physical activity. In addition to these promising results, the study also enabled us to understand actionable insights to improve Kid Space for future deployments (e.g., the need for real-time personalization). We plan to incorporate the lessons learned from this preliminary study and deploy Kid Space with real-time personalization features for longer periods with more students.
Full text
Available for:
DOBA, EMUNI, FIS, FZAB, GEOZS, GIS, IJS, IMTLJ, IZUM, KILJ, KISLJ, MFDPS, NLZOH, NUK, OILJ, PILJ, PNG, SAZU, SBCE, SBJE, SBMB, SBNM, SIK, UILJ, UKNU, UL, UM, UPUK, VKSCE, ZAGLJ
Educational technology research has found that parents of young children widely share concerns about extended screen time, lack of physical activity, and lack of social interaction. Kid Space was ...developed to address these concerns by enabling multi-modal and immersive collaborative play-based learning. Kid Space utilizes multiple sensing technologies with an immersive physical space through a human-scale wall projection and incorporates a conversational AI agent to interact with children, understand individual progress, and personalize learning experiences in a blended physical and digital environment. To evaluate Kid Space in the wild, we conducted a multi-method user study involving a quasi-experimental design and exploratory case study with 14 students and three educators in an elementary school. Mixed methods for data collection and analysis were used to understand the students' and educators' perceptions of Kid Space and its impact on the students’ educational outcomes (learning engagement, experience, and performance). The findings showed (1) positive perceptions toward Kid Space, (2) high levels of engagement - with decreased screen time (41% of the time), increased physical activity (99.3% of the time), and increased social interactions with conversational AI agent and the other collaborating student (52% of the time), and (3) significant learning gains after experiencing Kid Space (24% gain, paired t-test: p < 0.01). These positive results are accompanied by critical user insights for improving future iterations of Kid Space to validate long-term educational outcomes.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
The goal of this research is to investigate the effect of emotion-aware interventions on students' behavioral and emotional states. To this end, we collected data from 12 students in the 9th grade in ...a high school in Turkey. The data collection took place in two sessions of an English Course. While the students were reading articles and solving relevant questions, our data collection application running in the background recorded the videos of the individual students through a camera and captured students' screens in a nonintrusive manner. In total, we had 12.5 h of student data. We employed the human expert labeling process (HELP) (Asian et al. in Workshop proceedings at international conference on intelligent tutoring systems (ITS), pp 156-165, 2016) to have the data labeled (150 h of data labeling in total). The data collection application was designed in a way that it also collected emotional self-labels (i.e., emotional states as self-reported by students at any time of learning). We leveraged emotional self-label information to suggest various realtime interventions for the students. The results obtained using the final expert labels showed that the percentage of the students' Satisfied state was significantly higher after interventions. The results also demonstrated that although the interventions were triggered by the emotional states as self-labeled by the students and tailored to improve such states, there was a major positive impact of these interventions on students' behavioral states. This preliminary study showed that even with a limited set of emotion-aware interventions based on self-labels, students' states could be impacted positively. Conducting large-scale pilots leveraging more advanced interventions is a future direction for our research.
Full text
Available for:
BFBNIB, DOBA, EMUNI, FIS, FZAB, GEOZS, GIS, IJS, IMTLJ, IZUM, KILJ, KISLJ, MFDPS, NLZOH, NMLJ, NUK, OBVAL, OILJ, PILJ, PNG, SAZU, SBCE, SBJE, SBMB, SBNM, SIK, UILJ, UKNU, UL, UM, UPUK, VKSCE, ZAGLJ, ZRSKP
Biometric identification from three-dimensional (3-D) facial surface characteristics has become popular, especially in high security applications. In this paper, we propose a fully automatic ...expression insensitive 3-D face recognition system. Surface deformations due to facial expressions are a major problem in 3-D face recognition. The proposed approach deals with such challenging conditions in several aspects. First, we employ a fast and accurate region-based registration scheme that uses common region models. These common models make it possible to establish correspondence to all the gallery samples in a single registration pass. Second, we utilize curvature-based 3-D shape descriptors. Last, we apply statistical feature extraction methods. Since all the 3-D facial features are regionally registered to the same generic facial component, subspace construction techniques may be employed. We show that linear discriminant analysis significantly boosts the identification accuracy. We demonstrate the recognition ability of our system using the multiexpression Bosphorus and the most commonly used 3-D face database, Face Recognition Grand Challenge (FRGCv2). Our experimental results show that in both databases we obtain comparable performance to the best rank-1 correct classification rates reported in the literature so far: 98.19% for the Bosphorus and 97.51% for the FRGCv2 database. We have also carried out the standard receiver operating characteristics (ROC III) experiment for the FRGCv2 database. At an FAR of 0.1%, the verification performance was 86.09%. This shows that model-based registration is beneficial in identification scenarios where speed-up is important, whereas for verification one-to-one registration can be more beneficial.
With advances in sensor technology, the three-dimensional (3-D) face has become an emerging biometric modality, preferred especially in high security applications. However, dealing with occlusions ...covering the facial surface is a great challenge, which should be handled to enable applicability to fully automatic security systems. In this paper, we propose a fully automatic 3-D face recognition system which is robust to occlusions. We basically consider two problems: 1) occlusion handling for surface registration, and 2) missing data handling for classification based on subspace analysis techniques. For the alignment problem, we employ an adaptively-selected-model-based registration scheme, where a face model is selected for an occluded face such that only the valid nonoccluded patches are utilized. After registering to the model, occlusions are detected and removed. In the classification stage, a masking strategy, which we call masked projection, is proposed to enable the use of subspace analysis techniques with incomplete data. Furthermore, a regional scheme suitable for occlusion handling is incorporated in classification to improve the overall results. Experimental results on two databases with realistic facial occlusions, namely, the Bosphorus and the UMB-DB, are reported. Experimental results confirm that registration based on the adaptively selected model together with the masked subspace analysis classification offer an occlusion robust face recognition system.
Previous research showed that the parents acknowledged the technology's benefits for their young children's learning, however, they are still worried about the extended screen time, lack of physical ...activity and lack of social interactions. To address these concerns, we developed Kid Space to enable pedagogically appropriate technology use for children in early childhood education by combining various sensing technologies with a multi‐modal conversational artificial intelligence system that can interact with children, understand individual progress and provide personalised learning experiences. To understand the impact of Kid Space on the parents' initial concerns about technology use by their young children, we conducted a multi‐method user study: (1) a quasi‐experimental design and (2) formative research method using an exploratory case study with a set of children and their parents experiencing Kid Space in their homes. The results show that after experiencing Kid Space with their children, the parents felt significantly less concerned about screen time, social interactions and physical activity and reported positive perceptions towards pedagogical value of Kid Space. Detailed analysis on the multi‐modal data quantitatively and qualitatively validated why Kid Space alleviated these concerns. Future research is needed to validate long‐term educational value of Kid Space and generate insights for improvement for next iterations.
Practitioner notes
What is already known about this topic
Play‐based learning is critical for young children's education, but digital games create major concerns around extended screen time, lack of physical activity and lack of social interactions.
Blending digital and physical spaces could support pedagogically appropriate technology use for young children. Towards this end, there are some exemplary studies in the state–of‐the‐art reporting positive educational outcomes as an effect of utilising such spaces. However, none of these studies supported children's most natural mode of communication in their interactions with the systems—speaking.
Pedagogical conversational agents (PCAs) are promising, but they are tricky when it comes to young children's speech because of unique technical challenges resulted from how children use language and communicate with digital systems.
What this paper adds
To our best knowledge, Kid Space is one of the earliest implementations of a PCA with a multi‐modal artificial intelligence (AI) system utilising physical and digital learning manipulatives for maths learning with a focus on early childhood education. The key contributions of this paper are (1) the design and development of an end‐to‐end multi‐modal system enabling Wizard‐of‐Oz experimentation for initial evaluations with users, (2) the creation of a multi‐modal, in‐the‐wild labelled dataset with children–agent, children–parent and children–physical/digital space interactions enabling advancements for AI components for later evaluations with users and (3) the generation of rich insights from an initial research study on user perceptions and engagement as well as actionable findings to improve Kid Space experiences for next iterations and inform key design features for similar systems.
Implications for practice and/or policy
The results of the study implied a set of areas for improvement—or design features—for Kid Space and other similar pedagogical conversational systems developed for children's home usages: (1) easier setup and usage with optimised setup size addressing diverse space limitations at homes, (2) minimised latency between Oscar (the conversational pedagogical agent) and child interactions (eg, adding multimodal dialogue system to reduce the need for a human wizard), (3) more advanced personalisation, social (including more verbal interactions) and pedagogical skills for Oscar with increased contextual awareness (eg, sending children's engagement), (4) scalability and higher visual quality of content with diverse games and learning outcomes, (5) parental control features over Kid Space platform and Oscar (eg, time limit, content, etc.) and (6) accessibility features (eg, captions turned on for multilingual children) and support for neurodiversity.
Full text
Available for:
BFBNIB, FZAB, GIS, IJS, KILJ, NLZOH, NUK, OILJ, SBCE, SBMB, UL, UM, UPUK
Face is a highly utilized biometric, and 3D modality is preferred due to better handling of variations such as pose and illumination. However, occlusions covering the face alter the 3D surface and ...degrade the recognition performance. To improve recognition rates, the occluded parts should be detected prior to any surface comparison. In this paper, we consider two different occlusion detection approaches: The first one is based on statistical facial surface modeling, where pixel-wise Gaussian Mixture Models are trained. The second algorithm considers occlusion detection as a binary image segmentation problem: The regional cues of depth values are incorporated with neighborhood cues, and the acquired surface is modeled as a graph. The surface pixels are labeled as either face or occlusion via the graph cut technique. Experiments on the Bosphorus and the UMB-DB databases, including realistic occlusion variations, show that both methods improve occlusion detection and face recognition rates as compared to the baseline technique.
"Serious games" are becoming extremely relevant to individuals who have specific needs, such as children with an autism spectrum condition (ASC). Often, individuals with an ASC have difficulties in ...interpreting verbal and nonverbal communication cues during social interactions. The ASC-Inclusion EU-FP7 funded project aims to provide children who have an ASC with a platform to learn emotion expression and recognition, through play in the virtual world. In particular, the ASC-Inclusion platform focuses on the expression of emotion via facial, vocal, and bodily gestures. The platform combines multiple analysis tools, using onboard microphone and webcam capabilities. The platform utilizes these capabilities via training games, text-based communication, animations, video, and audio clips. This paper introduces current findings and evaluations of the ASC-Inclusion platform and provides detailed description for the different modalities.
This paper presents an evaluation of several 3D face recognizers on the Bosphorus database which was gathered for studies on expression and pose invariant face analysis. We provide identification ...results of three 3D face recognition algorithms, namely generic face template based ICP approach, one-to-all ICP approach, and depth image-based Principal Component Analysis (PCA) method. All of these techniques treat faces globally and are usually accepted as baseline approaches. In addition, 2D texture classifiers are also incorporated in a fusion setting. Experimental results reveal that even though global shape classifiers achieve almost perfect identification in neutral-to-neutral comparisons, they are sub-optimal under extreme expression variations. We show that it is possible to boost the identification accuracy by focusing on the rigid facial regions and by fusing complementary information coming from shape and texture modalities.
With the advances in computing technologies, we have been undergoing a shift towards a digital world. As an inevitable result of this shift, the technology penetrates into education in myriad forms. ...Intelligent tutoring systems (ITS) are essential outcomes of this penetration, emerging to satisfy the needs of learners and instructors. Their working principle is based on collecting and processing data of all students through various modalities to understand the strengths and needs of learners. Yet, more important is that ITSs untangle the overlooked problem of traditional education: One size does not fit all, and there is a need for personalized tutoring for each individual. It is well known that that learning is emotional as well as intellectual. To truly meet the needs of education, we need empathic companions, ones that are affectively aware and thus can accompany the learner for an enhanced learning experience.
PDF
