AR/MR Remote Collaboration on Physical Tasks: A Review Wang, Peng; Bai, Xiaoliang; Billinghurst, Mark ...
Robotics and computer-integrated manufacturing,
December 2021, 2021-12-00, 20211201, Letnik:
72
Journal Article
Recenzirano
This paper provides a review of research into using Augmented Reality (AR) and Mixed Reality(MR) for remote collaboration on physical tasks. AR/MR-based remote collaboration on physical tasks has ...recently become more prominent in academic research and engineering applications. It has great potential in many fields, such as real-time remote medical consultation, education, training, maintenance, remote assistance in engineering, and other remote collaborative tasks. However, to the best of our knowledge there has not been any comprehensive review of research in AR/MR remote collaboration on physical tasks. Therefore, this paper presents a comprehensive survey of research between 2000 and 2018 in this domain. We collected 215 papers, more than 80% of which were published between 2010 and 2018, and all relevant works are discussed at length. Then we elaborate on the review from typical architectures, applications (e.g., industry, telemedicine, architecture, teleducation and others), and empathic computing. Next, we made an in-depth review of the papers from seven aspects: (1) collection and classification research, (2) using 3D scene reconstruction environments and live panorama, (3) periodicals and conducting research, (4) local and remote user interfaces, (5) features of user interfaces commonly used, (6) architecture and sharing non-verbal cues, (7) applications and toolkits. We find that most papers (160 articles, 74.4%) are published in conferences, using co-located collaboration to emulate remote collaboration is adopted by more than half (126, 58.6%) of the reviewed papers, the shared non-verbal cues can be mainly classified into five types (Virtual Replicas or Physical Proxy(VRP), AR Annotations or a Cursor Pointer(ARACP), avatar, gesture, and gaze), the local/remote interface is mainly divided into four categories (Head-Mounted Displays(HMD), Spatial Augmented Reality(SAR), Windows-Icon-Menu-Pointer(WIMP) and Hand-Held Displays(HHD)). From this, we can draw ten conclusions. Following this we report on issues for future works. The paper also provides an overall academic roadmap and useful insight into the state-of-the-art of AR/MR remote collaboration on physical tasks. This work will be useful for current and future researchers who are interested in collaborative AR/MR systems.
•Development of a novel adaptive Mixed Reality remote collaboration system.•Exploring adaptive generation of AR instructions via context-awareness.•Conducting user study to explore the effect of ...adaptive generation of instructions.•Discussion of the benefits and implications of adaptive generation of instructions.
Mixed Reality (MR) remote collaboration supports a shared workspace and various non-verbal communication cues that enable remote experts to express ideas in an immersive Virtual Reality (VR) space to guide local workers in industrial physical tasks by creating Augmented Reality (AR) instructions, such as remote assembly/disassembly guidance, emergency maintenance, and training. However, due to the information non-symmetrical caused by geographical distribution, it can be challenging and high-workload for remote experts to create clear and detailed AR instructions for local workers to match spatial relationships with the local workspace, while comprehensively considering the situation information (e.g., task-related objects and partner’s environment) in complex industrial physical tasks. Advances in ambient intelligence bring ideas for building a new remote collaboration framework that supports the adaptive generation of instructions. In the study, we developed a novel MR remote collaboration prototype system, which can automatically and adaptively generate clear, detailed, and standardized AR instructions based on remote experts’ simple and intuitive interactions and local contextual information to assist remote experts in guiding local workers to complete industrial physical tasks, especially industrial assembly tasks. The effect of two interfaces on industrial assembly tasks involving relatively complex process information and environments was explored in a user study: one was a baseline solution that uses a typical MR remote collaboration interface to support remote experts to create instructions (RECI) using popular non-verbal communication cues in the shared local 3D stereoscopic scene, and the other was our novel interface that additionally supported the adaptive generation of instructions via context awareness (AGICA) on top of the typical interface. The result of the user study showed that AGICA significantly improves collaboration performance, reduces errors and workload, and enhances usability and user experience compared to RECI. This demonstrates that supporting the adaptive generation of instructions is a feasible way to enhance MR remote collaboration in industrial assembly tasks with relatively complex process information and environments. Our research findings have a certain guiding significance for the design of MR remote collaboration systems in such tasks.
As Virtual Reality(VR), Augmented Reality(AR), Mixed Reality(MR) technology becomes more accessible, it is important to explore VR/AR/MR technologies that can be used for remote collaboration on ...physical tasks. Previous research has shown that gesture-based interaction is intuitive and expressive for remote collaboration, and using 3D CAD models can provide clear instructions for assembly tasks. In this paper, therefore, we describe a new MR remote collaboration system which combines the use of gesture and CAD models in a complementary manner. The prototype system enables a remote expert in VR to provide instructions based on 3D gesture and CAD models (3DGAM) for a local worker who uses AR to see these instructions. Using this interface, we conducted a formal user study to explore the effect of sharing 3D gesture and CAD models in an assembly training task. We found that the combination of 3D gesture and CAD models can improve remote collaboration on an assembly task with respect to the performance time and user experience. Finally, we provide some conclusions and directions for future research.
Introduction
Physical fitness and health are closely linked. However, regular workplace assessments of paramedic health and fitness are not conducted, despite increasing evidence that the role ...requires a high level of fitness due to increasing physical and psychological demands. Describing the physical attributes required to complete the day-to-day tasks of a paramedic is problematic as data has not been collated. Recent registration, and significant illness and injury rates of paramedics in Australia, make systematic identification of the role more important in order to set a standard criteria.
Objective
This systematic literature review set out to define: 1) paramedic health status and the physical tasks performed; and 2) level of requisite fitness required to complete paramedic tasks.
Results
Using systematic search strategies, 24 articles were appraised and generally conclude that paramedics have a poorer health status and fitness levels compared to the general population, despite often performing physical heavy handling tasks.
Conclusion
General themes were highlighted throughout the literature, identifying that further research needs to be conducted to identify the day-to-day tasks of a paramedic and what level of requisite fitness is required. Current health and fitness is lacking for physical heavy handling activities performed in the paramedic role.
Although sharing gestures and gaze can improve AR remote collaboration, most current systems only enable collaborators to share 2D or 3D gestures, and the unimodal HCI interface remains dominant. To ...address this problem, we describe a novel remote collaborative platform based on 2.5D gestures and gaze (2.5DGG), which supports an expert who collaborates with a worker (e.g., during assembly or training tasks). We investigate the impact of sharing the remote site’s 2.5DGG using spatial AR (SAR) remote collaboration in manufacturing. Compared to other systems, there is a key advantage that it can provide more natural and intuitive multimodal interaction based on 2.5DGG. We track the remote experts’ gestures and eye gaze using Leap Motion and aGlass, respectively, in a VR space displaying the live video stream of the local physical workspace and visualize them onto the local work scenario by a projector. The results of an exploratory user study demonstrate that 2.5DGG has a clear difference in performance time and collaborative experience, and it is better than the traditional one.
Introduction
The Bachelor Degree in Emergency Medical Care (BEMC) is a unique program in that students operate in both emergency care and rescue contexts, unlike international paramedic degree ...programs which focus only on emergency care. The learning activities associated with the rescue content are physically strenuous and therefore BEMC students need to be physically and mentally prepared to engage in diverse austere environments. Although South African BEMC programs have a common medical rescue curriculum, approaches to the training and assessment of physical preparedness vary between the institutions. The objective of this research was to explore the knowledge gap through the review of literature that describes the unique physical preparedness requirements in the field of emergency care education.
Methods
We conducted a scoping review in the form of a narrative literature review.
Results
Seventy-five (n) articles were initially identified, however, only four were relevant to the objective of this study. This highlighted the paucity of literature describing the unique physical fitness requirements of the BEMC program and the current challenges experienced by educators in the field.
Conclusion
While physical preparedness training exists in higher education institutes and there are assessments conducted at these higher education institutes, none of these assessment tools have yet been scientifically validated which creates a challenge for educators. The current knowledge gap within EMC education is therefore the absence of a scientifically validated task-oriented physical preparedness assessment tool which addresses the desired physical attributes and abilities of EMC students linked to the BEMC curriculum and associated learning experiences.
Climbing With a Head-Mounted Display Woodham, Alexander; Billinghurst, Mark; Helton, William S.
Human factors,
05/2016, Letnik:
58, Številka:
3
Journal Article
Recenzirano
Objective:
We explored the dual-task costs of climbers performing a visual communication task using a head-mounted display (HMD) while simultaneously climbing along a vertical surface.
Background:
...Climbing is affected by secondary auditory cognitive tasks, and climbing impairs later recall of secondary task information; the effects of visually presented tasks are less clear. Given that HMDs are projected to be adopted into emergency response work, questions are raised about the effects of HMD use during climbing or other physical tasks.
Method:
Climbers performed five conditions—a climbing-only condition, two dual-task climbing conditions (words presented on the HMD with and without auditory warnings while climbing), and two seated control conditions (words presented on the HMD with and without auditory warnings)—in a repeated-measures design. Motion data were also collected to examine participant motion around word presentation.
Results:
We found a decrease in both climbing performance and word recall under dual-task conditions, paralleling results found in previous research using auditory tasks. Participants slowed around word presentations on the HMD. Additional comparisons to previous research indicate that physical tasks may be more detrimental to word recall than are seated tasks and that visual stimuli might hinder climbing performance more than do audible stimuli.
Conclusion:
Complex physical activity, like climbing, is disruptive to memory rehearsal and later recall, and cognitive tasks disrupt physical performance.
Application:
Avoiding cognitive HMD tasks requiring later recall during complex physical activity is advisable. However, these systems may be developed to provide intelligent assistance, or memory augmentation, in these settings.
Plastic Approach to Acting Rimovna, Yusupova Marina
Annals of the Romanian society for cell biology,
01/2021, Letnik:
25, Številka:
2
Journal Article
...it implies the development of all aspects of the creative personality of the performer himself. Methods In this regard, it is necessary to emphasize and recall the principle of "infinitely ...expanding consciousness" (according to M.K. Mamardashvili), which paid great attention to the consciousness of creators, since: "If it is sluggish, not loaded with culture, if the achievements of previous creators are poorly assimilated, their ideas and discoveries, if their own high meaning of work in the theatre has not been born, there will be no professional. "The law of exercise": the same reactions to the same stimuli, being fixed, are automated "7, p.87. Behaviourism in the 20s of the XX-century came to its classical form and brought its understanding of human behaviour to anthropology, sociology, and pedagogy, and naturally, could not help but affect Meyerhold's search for a circular scheme of interaction between the centre of any system and its executing elements. ...these techniques are revealed in music.
The article shows the actuality of the use of physical tasks in the course of physics as a means of training future technologists of light industry. The work of a light industry technologist is ...considered by the author as a solution to engineering problems related to the transfer of materials from one state to another. Scientific novelty of the results of the study: the necessity and possibility of preparing students of technological universities to solve professional problems is justified; Structure and composition of professional tasks are identified. The practical significance of the results of the study is that: a system of physical tasks used in the process of physics training has been developed, ensuring that future technologists are able to solve professional problems on the basis of physical knowledge and skills.
The author investigates the evolution of job skill distribution using task data derived from the U.K. Skills Surveys of 1997, 2001, and 2006, and the 1992 Employment Survey in Britain. He determines ...the extent to which employee involvement in the workplace and computer technologies promote the use of higher order cognitive and interactive skills. He finds that literacy, other communication tasks, and self-planning skills have grown especially fast. Numerical and problem-solving skills have also become more important, but repetitive physical skills have largely remained unchanged. He finds that employee involvement and computer technologies privilege the use of greater generic skills but substitute for repetitive physical tasks. However, the classification of all tasks as either routine or non-routine is found to be problematic. Finally, the author finds a strong connection between the rising use of more academic skills and the education level required for entry into the labor market.
PDF
