Today, the cutting edge of computer vision research greatly depends on the availability of large datasets, which are critical for effectively training and testing new methods. Manually annotating ...visual data, however, is not only a labor‐intensive process but also prone to errors. In this study, we present NOVA, a versatile framework to create realistic‐looking 3D rendered worlds containing procedurally generated humans with rich pixel‐level ground truth annotations. NOVA can simulate various environmental factors such as weather conditions or different times of day, and bring an exceptionally diverse set of humans to life, each having a distinct body shape, gender and age. To demonstrate NOVA's capabilities, we generate two synthetic datasets for person tracking. The first one includes 108 sequences, each with different levels of difficulty like tracking in crowded scenes or at nighttime and aims for testing the limits of current state‐of‐the‐art trackers. A second dataset of 97 sequences with normal weather conditions is used to show how our synthetic sequences can be utilized to train and boost the performance of deep‐learning based trackers. Our results indicate that the synthetic data generated by NOVA represents a good proxy of the real‐world and can be exploited for computer vision tasks.
Graphical :
Providing a depth-rich Virtual Reality (VR) experience to users without causing discomfort remains to be a challenge with today’s commercially available head-mounted displays (HMDs), which enforce ...strict measures on stereoscopic rendering parameters for the sake of keeping visual discomfort to a minimum. However, these measures often lead to an unimpressive VR experience with shallow depth feeling. Using radial basis function interpolation and projection matrix manipulations, we propose a novel method for dynamic tuning of stereoscopic rendering parameters toward enhanced VR experience and demonstrate that it is ready to be used with existing consumer HMDs for automated stereoscopic camera control. We present our method with an approach that is primarily intended for guided VR experiences such as VR walkthroughs or VR rides, assuming a depth layout defined by a path of depth-authored key points in the virtual environment. For this purpose, we also introduce a novel immersive interface for authoring unique 3D stereoscopic cinematographies for VR HMD experiences. The results of the user study indicate that our approach, with proper depth authoring, is able to enhance user experience in terms of overall perceived depth and picture quality while maintaining visual comfort on a par with the HMD’s default arrangement. We also investigate the effects of using depth-of-field blurring in combination with our approach and observe that the quality of the outcome varies based on the overall scene depth.
Display omitted
•A novel method of real-time automated stereoscopic camera control for enhanced VR experience.•Via an efficient depth tuning scheme based on RBF interpolation and projection manipulations using depth-authored keypoints.•Also featuring an immersive authoring interface to create unique depth narratives.•User evaluation validates improved VR experience in terms of perceived depth and image quality.
Previous studies showed that virtual reality (VR) environments can affect emotional state and cause significant changes in physiological responses. Aside from these effects, inadvertently induced ...cybersickness is a notorious problem faced in VR. In this study, to further investigate the effects of virtual environments (VEs) with different context, three dynamic VEs were created. Each VE had a particular purpose: evoking no emotion in Campfire (CF), unpleasant emotions in Hospital (HH), and cybersickness symptoms in Roller Coaster (RC). We made use of objective measurements of physiological responses such as pupil dilation, blinks, fixations, saccades, and heart rate, as well as subjective self‐assessments via pre‐ and post‐VE session questionnaires. While previous studies investigate different subsets of these measures, our study makes a comprehensive analysis of them jointly in dynamic VEs. The results of the study indicate that cybersickness produced higher saccade mean speed, whereas unpleasant context caused higher fixation count, saccade rate, and pupil dilation. Moreover, CF decreased anxiety, whereas HH and RC increased it and they also decreased comfort. Participants felt cybersickness in all VEs even in CF which is designed to minimize the effects.
In this study, to investigate the emotional and physiological effects of virtual environments (VEs) with different contexts, three dynamic VEs were created. Each VE had a particular purpose: evoking no emotion in Campfire (CF), unpleasant emotions in Hospital (HH), and cybersickness symptoms in Roller Coaster (RC). Results showed that cybersickness produced higher saccade mean speed, while unpleasant context caused higher fixation count, saccade rate, and pupil dilation; CF decreased anxiety, whereas HH and RC increased it and decreased comfort.
Learning robust representations is critical for the success of person re-identification and attribute recognition systems. However, to achieve this, we must use a large dataset of diverse person ...images as well as annotations of identity labels and/or a set of different attributes. Apart from the obvious concerns about privacy issues, the manual annotation process is both time consuming and too costly. In this paper, we instead propose to use synthetic person images for addressing these difficulties. Specifically, we first introduce Synthetic18K, a large-scale dataset of over 1 million computer generated person images of 18K unique identities with relevant attributes. Moreover, we demonstrate that pretraining of simple deep architectures on Synthetic18K for person re-identification and attribute recognition and then fine-tuning on real data leads to significant improvements in prediction performances, giving results better than or comparable to state-of-the-art models.
•Our work learns robust representations for person re-id and attribute recognition.•We introduce Synthetic18K dataset for person re-id and attribute recognition.•Synthetic18K provides a strong alternative to the widely used ImageNet pre-training.•Our pre-trained models give highly competitive results against the state-of-the-art.
•A broad analysis on estimating visual saliency in immersive, real-time rendered VR.•Contribution of depth-cue to visual saliency decreases with reduced navigation pace.•The methods based on boundary ...connectivity and surroundedness work best.
Display omitted
As virtual reality (VR) headsets with head-mounted-displays (HMDs) are becoming more and more prevalent, new research questions are arising. One of the emergent questions is how best to employ visual saliency prediction in VR applications using current line of advanced HMDs. Due to the complex nature of human visual attention mechanism, the problem needs to be investigated from different points of view using different approaches. Having such an outlook, this work extends the previous effort on exploring a set of well-studied visual saliency cues and saliency prediction methods making use of these cues with the aim of assessing how applicable they are for estimating visual saliency in immersive VR environments that are rendered in real-time and experienced with consumer HMDs. To that end, a new user study was conducted with a larger sample and reveals the effects of experiencing dynamic computer-generated scenes with reduced navigation speeds on visual saliency. Using these scenes that have varying visual experiences in terms of contents and range of depth-of-field, the study also compares VR viewing to 2D desktop viewing with an expanded set of results. The presented evaluation offers the most in-depth view of visual saliency in immersive, real-time rendered VR to date. The analysis encompassing the results of both studies indicate that decreasing navigation speed reduces the contribution of depth-cue to visual saliency and has a boosting effect for cues based on 2D image features only. While there are content-dependent variances among their scores, it is seen that the saliency prediction methods based on boundary connectivity and surroundedness work best in general for the given settings.
Deep Reinforcement Learning (DRL) has been effectively performed in various complex environments, such as playing video games. In many game environments, DeepMind’s baseline Deep Q-Network (DQN) game ...agents performed at a level comparable to that of humans. However, these DRL models require many experience samples to learn and lack the adaptability to changes in the environment and handling complexity. In this study, we propose Attention-Augmented Deep Q-Network (AADQN) by incorporating a combined top-down and bottom-up attention mechanism into the DQN game agent to highlight task-relevant features of input. Our AADQN model uses a particle-filter -based top-down attention that dynamically teaches an agent how to play a game by focusing on the most task-related information. In the evaluation of our agent’s performance across eight games in the Atari 2600 domain, which vary in complexity, we demonstrate that our model surpasses the baseline DQN agent. Notably, our model can achieve greater flexibility and higher scores at a reduced number of time steps.Across eight game environments, AADQN achieved an average relative improvement of 134.93%. Pong and Breakout games both experienced improvements of 9.32% and 56.06%, respectively. Meanwhile, SpaceInvaders and Seaquest, which are more intricate games, demonstrated even higher percentage improvements, with 130.84% and 149.95%, respectively. This study reveals that AADQN is productive for complex environments and produces slightly better results in elementary contexts.
This study aimed to evaluate the usability of the virtual cafeteria (VC) and determine its suitability for further studies in portion size education and rehabilitation of nutrition.
The study was ...conducted with 73 participants (aged 18–40 years). The VC, where the participants performed the task of assembling a meal, was created as a virtual reality simulation of a buffet-style cafeteria (94 food and 10 beverage items). The participants were asked to complete the System Usability Scale, which regards ≥70 points as acceptable, and to give comments about the VC.
The mean System Usability Scale score was 79.4 ± 12.71 (range, 22.2–97.2). Approximately 68% of the participants described positive qualities for the VC. The participants with technical background reported the VC as more usable (96%) than the others (74%) (χ2 = 5,378; df = 1, P = 0.025).
Offered as a novel tool for education and rehabilitation of nutrition, the VC was confirmed to feature good usability.
Virtual reality (VR) has become a new tool in psychological research and application. Although several studies have investigated its triggering and reducing role in anxiety via VR-based exposure and ...response prevention (VR-ERP) across different psychopathologies, its efficiency in contamination fear and reduction of disgust are yet to be examined. The study contributes to the field by investigating the effectiveness of the VR-ERP on the reduction of anxiety, disgust and urge to wash levels using an experimental design.
A total of 21 non-clinical participants with high contamination fear were randomly assigned to experimental (n = 9) and control (n = 12) groups. The experimental group was exposed to repeated VR-ERP sessions.
After a minimum of three VR-ERP sessions, the experimental group scored significantly lower than the control group on the anxiety, disgust, and urge to wash hands scores.
A small sample size, an analogue sample, and a generic virtual scenario content were potential limitations of the study.
The results indicated that the VR-ERP can be an efficient and alternative exposure tool in the reduction of anxiety, disgust, and urge to wash hands. The theoretical and clinical applications of the VR-ERP in the treatment of contamination-based OCD symptoms were discussed in light of our findings.
•Anxiety, disgust and urge to wash scores were used as contamination-based ratings.•The effectiveness of VR exposure was examined for contamination-based ratings.•Contamination related ratings increased as the degree of dirtiness increased.•VR-ERP sessions were effective to decrease contamination related ratings.•The post-test scores in the experimental group significantly declined.
We present a novel method for retargeting human motion to arbitrary 3D mesh models with as little user interaction as possible. Traditional motion‐retargeting systems try to preserve the original ...motion, while satisfying several motion constraints. Our method uses a few pose‐to‐pose examples provided by the user to extract the desired semantics behind the retargeting process while not limiting the transfer to being only literal. Thus, mesh models with different structures and/or motion semantics from humanoid skeletons become possible targets. Also considering the fact that most publicly available mesh models lack additional structure (e.g. skeleton), our method dispenses with the need for such a structure by means of a built‐in surface‐based deformation system. As deformation for animation purposes may require non‐rigid behaviour, we augment existing rigid deformation approaches to provide volume‐preserving and squash‐and‐stretch deformations. We demonstrate our approach on well‐known mesh models along with several publicly available motion‐capture sequences.
We present a novel method for retargeting human motion to arbitrary 3D mesh models with as little user interaction as possible. Traditional motion‐retargeting systems try to preserve the original motion, while satisfying several motion constraints. Our method uses a few pose‐to‐pose examples provided by the user to extract the desired semantics behind the retargeting process while not limiting the transfer to being only literal. Thus, mesh models with different structures and/or motion semantics from humanoid skeletons become possible targets.
Allowing us to visit places we have never set foot before and to experience them as if we were there, virtual reality is a new medium with unique characteristics compared to its predecessors with its ...main features such as its ability to manipulate perceptions of time and space, the interaction options it provides, enabling to host many participants at the same time, and empowering the user to direct the narrative flow of the experience. It is clear that interest in virtual reality is high and will continue to increase. Innovative uses of technology can provide significant benefits in today's populations, especially for the Generation Z, who closely follow technological developments and can adopt them quickly. Education and medicine are among the major fields where virtual reality has led to significant transformations in recent years. In this review article, the applications of virtual reality in the fields of education and medicine are examined within the framework of scientific research. In the light of this review, the experiences gained in practice, the challenges encountered, and recommendations to overcome these challenges were given and predictions regarding the future place of virtual reality were discussed together with the effects of the new normal that took shape with the COVID-19 pandemic.
Sanal gerçeklik, daha önce ayak basmadığımız yerleri sanal olarak ziyaret edip sanki oradaymış gibi tecrübe etmemize imkan verirken zaman ve mekan algılarını manipüle etme becerisi, sağladığı etkileşim seçenekleri, aynı anda birçok katılımcıyı barındırabilmesi ve katılımcılara deneyimin anlatı akışını yönlendirme olanağı vermesi gibi başlıca özellikleriyle öncüllerine kıyasla benzersiz özellikleri olan yeni bir medyadır. Sanal gerçekliğe ilginin yüksek olduğu ve artmaya devam edeceği açıktır. Teknolojinin yenilikçi kullanımları, günümüz popülasyonlarında, özellikle teknolojik gelişmeleri yakından takip eden ve bunları hızlı bir şekilde benimseyebilen Z kuşağı için önemli faydalar sağlayabilmektedir. Eğitim ve tıp, son yıllarda sanal gerçekliğin önemli dönüşümlere yol açtığı alanların başında gelmektedir. Sanal gerçekliğin bu alanlardaki uygulamaları giderek çeşitlenmekte, olgunlaşmakta ve bunlara dair araştırmalarda olumlu sonuçların alındığı görülmektedir. Bu derleme makalede, sanal gerçekliğin eğitim ve tıp alanlarındaki uygulamaları, yapılan bilimsel araştırmalar çerçevesinde incelenmiştir. Bu incelemenin ışığında uygulamada edinilen tecrübeler, karşılaşılan zorluklar ve bu zorlukları aşmaya dair öneriler verilmiş ve sanal gerçekliğin gelecekteki yerine dair öngörüler COVID-19 pandemisi ile şekillenen yeni normalin etkileri ile birlikte ele alınmıştır.
PDF
