The increasing availability of head-mounted displays (HMDs) for home use motivates the study of the possible effects that adopting this new hardware might have on users. Moreover, while the impact of ...display type has been studied for different kinds of tasks, it has been scarcely explored in procedural training. Our study considered three different types of displays used by participants for training in aviation safety procedures with a serious game. The three displays were respectively representative of: (i) desktop VR (a standard desktop monitor), (ii) many setups for immersive VR used in the literature (an HMD with narrow field of view and a 3-DOF tracker), and (iii) new setups for immersive home VR (an HMD with wide field of view and 6-DOF tracker). We assessed effects on knowledge gain, and different self-reported measures (self-efficacy, engagement, presence). Unlike previous studies of display type that measured effects only immediately after the VR experience, we considered also a longer time span (2 weeks). Results indicated that the display type played a significant role in engagement and presence. The training benefits (increased knowledge and self-efficacy) were instead obtained, and maintained at two weeks, regardless of the display used. The paper discusses the implications of these results.
The numerous works on media retargeting call for a methodological approach for evaluating retargeting results. We present the first comprehensive perceptual study and analysis of image retargeting. ...First, we create a benchmark of images and conduct a large scale user study to compare a representative number of state-of-the-art retargeting methods. Second, we present analysis of the users' responses, where we find that humans in general agree on the evaluation of the results and show that some retargeting methods are consistently more favorable than others. Third, we examine whether computational image distance metrics can predict human retargeting perception. We show that current measures used in this context are not necessarily consistent with human rankings, and demonstrate that better results can be achieved using image features that were not previously considered for this task. We also reveal specific qualities in retargeted media that are more important for viewers. The importance of our work lies in promoting better measures to assess and guide retargeting algorithms in the future. The full benchmark we collected, including all images, retargeted results, and the collected user data, are available to the research community for further investigation at http://people.csail.mit.edu/mrub/retargetme.
Accurate and fast understanding of the patient's anatomy is crucial in surgical decision making and particularly important in visceral surgery. Sophisticated visualization techniques such as 3D ...Volume Rendering can aid the surgeon and potentially lead to a benefit for the patient. Recently, we proposed a novel volume rendering technique called Adaptive Volumetric Illumination Sampling (AVIS) that can generate realistic lighting in real-time, even for high resolution images and volumes but without introducing additional image noise. In order to evaluate this new technique, we conducted a randomized, three-period crossover study comparing AVIS to conventional Direct Volume Rendering (DVR) and Path Tracing (PT). CT datasets from 12 patients were evaluated by 10 visceral surgeons who were either senior physicians or experienced specialists. The time needed for answering clinically relevant questions as well as the correctness of the answers were analyzed for each visualization technique. In addition to that, the perceived workload during these tasks was assessed for each technique, respectively. The results of the study indicate that AVIS has an advantage in terms of both time efficiency and most aspects of the perceived workload, while the average correctness of the given answers was very similar for all three methods. In contrast to that, Path Tracing seems to show particularly high values for mental demand and frustration. We plan to repeat a similar study with a larger participant group to consolidate the results.
•Research literature suggests that fewer than ten personas should be created to avoid overloading persona users with too many choices.•We conducted an in-person user study with 37 participants to ...investigate how persona users’ recall of persona details, perceptions of personas, and user behavior differed when using 5 and 15 personas with the help of an interactive persona system.•There were no differences in terms of participants recalling less persona information, recalling the information wrong, or perceived the personas as less empathetic or more confusion when using fewer personas.•In turn, using more personas, the participants exhibited higher overall engagement, were provided with a more demographically diverse end-user representation, and subsequently chose personas from more varied demographic groups.•The participants also adjusted their persona usage behavior by making more use of navigational system features and information snapshot within the interactive system, which suggests interactive persona systems help users cope with more personas even when exceeding the conventional “less than ten” rule of thumb.
Studies in human-computer interaction recommend creating fewer than ten personas, based on stakeholders’ limitations to cognitively process and use personas. However, no existing studies offer empirical support for having fewer rather than more personas. Investigating this matter, thirty-seven participants interacted with five and fifteen personas using an interactive persona system, choosing one persona to design for. Our study results from eye-tracking and survey data suggest that when using interactive persona systems, the number of personas can be increased from the conventionally suggested ‘less than ten’, without significant negative effects on user perceptions or task performance, and with the positive effects of increasing engagement with the personas, having a more diverse representation of the end-user population, as well as users accessing personas from more varied demographic groups for a design task. Using the interactive persona system, users adjusted their information processing style by spending less time on each persona when presented with fifteen personas, while still absorbing a similar amount of information than with five personas, implying that more efficient information processing strategies are applied with more personas. The results highlight the importance of designing interactive persona systems to support users’ browsing of more personas.
This paper presents results of a user study of the effects of virtual reality technology on VR Sickness and User Experience. In our study the participants watched two different panoramic (360) ...videos, one with relaxing content (beach clip) and second one with action content (roller coaster video clip). Videos were watched on four different head mounted displays (HMDs) and on the 2D television as a reference display. To assess VR Sickness discomfort levels, we have used the Simulator Sickness Questionnaire (SSQ), and for user experience the User Experience Questionnaire (UEQ) was used. For quick assessments of VR Sickness discomfort levels, we have also used Subjective Units of Distress Scale (SUDS). We have found a strong correlation between SUDS and total SSQ score and between total SSQ score and SSQ-D score. Shown negative correlation between VR Sickness discomfort levels (assessed by SSQ and UEQ Questionnaire), and user experience (assessed by UEQ Questionnaire), indicates that presence of VR Sickness symptoms affects the user experience.
•Assessing VR Sickness discomfort levels using the SSQ and Questionnaire.•User Experience measuring by the UEQ Questionnaire.•Shown negative correlation between VR Sickness discomfort levels and User Experience.•Ne-grain feature, we used recurrent operation over once calculated convolution of sliding window.
•Human-centered XAI design is required to create human-understandable explanations.•We present a human-centered design approach to develop explanations for AI systems.•User requirements for ...explanations from clinical support systems are derived.•A set of re-usable explanation design patterns for decision-support systems is created.
Display omitted
Much of the research on eXplainable Artificial Intelligence (XAI) has centered on providing transparency of machine learning models. More recently, the focus on human-centered approaches to XAI has increased. Yet, there is a lack of practical methods and examples on the integration of human factors into the development processes of AI-generated explanations that humans prove to uptake for better performance. This paper presents a case study of an application of a human-centered design approach for AI-generated explanations. The approach consists of three components: Domain analysis to define the concept & context of explanations, Requirements elicitation & assessment to derive the use cases & explanation requirements, and the consequential Multi-modal interaction design & evaluation to create a library of design patterns for explanations. In a case study, we adopt the DoReMi-approach to design explanations for a Clinical Decision Support System (CDSS) for child health. In the requirements elicitation & assessment, a user study with experienced paediatricians uncovered what explanations the CDSS should provide. In the interaction design & evaluation, a second user study tested the consequential interaction design patterns. This case study provided a first set of user requirements and design patterns for an explainable decision support system in medical diagnosis, showing how to involve expert end users in the development process and how to develop, more or less, generic solutions for general design problems in XAI.
We report on a controlled user study comparing three visualization environments for common 3D exploration. Our environments differ in how they exploit natural human perception and interaction ...capabilities. We compare an augmented-reality head-mounted display (Microsoft HoloLens), a handheld tablet, and a desktop setup. The novel head-mounted HoloLens display projects stereoscopic images of virtual content into a user's real world and allows for interaction in-situ at the spatial position of the 3D hologram. The tablet is able to interact with 3D content through touch, spatial positioning, and tangible markers, however, 3D content is still presented on a 2D surface. Our hypothesis is that visualization environments that match human perceptual and interaction capabilities better to the task at hand improve understanding of 3D visualizations. To better understand the space of display and interaction modalities in visualization environments, we first propose a classification based on three dimensions: perception, interaction, and the spatial and cognitive proximity of the two. Each technique in our study is located at a different position along these three dimensions. We asked 15 participants to perform four tasks, each task having different levels of difficulty for both spatial perception and degrees of freedom for interaction. Our results show that each of the tested environments is more effective for certain tasks, but that generally the desktop environment is still fastest and most precise in almost all cases.
What’s (Not) Working in Programmer User Studies? Davis, Matthew C.; Aghayi, Emad; Latoza, Thomas D. ...
ACM transactions on software engineering and methodology,
07/2023, Volume:
32, Issue:
5
Journal Article
Peer reviewed
Open access
A key goal of software engineering research is to improve the environments, tools, languages, and techniques programmers use to efficiently create quality software. Successfully designing these tools ...and demonstrating their effectiveness involves engaging with tool users—software engineers. Researchers often want to conduct user studies of software engineers to collect direct evidence. However, running user studies can be difficult, and researchers may lack solution strategies to overcome the barriers, so they may avoid user studies. To understand the challenges researchers face when conducting programmer user studies, we interviewed 26 researchers. Based on the analysis of interview data, we contribute (i) a taxonomy of 18 barriers researchers encounter; (ii) 23 solution strategies some researchers use to address 8 of the 18 barriers in their own studies; and (iii) 4 design ideas, which we adapted from the behavioral science community, that may lower 8 additional barriers. To validate the design ideas, we held an in-person all-day focus group with 16 researchers.
The use of clarifying questions (CQs) is a fairly new and useful technique to aid systems in recognizing the intent, context, and preferences behind user queries. Yet, understanding the extent of the ...effect of CQs on user behavior and the ability to identify relevant information remains relatively unexplored. In this work, we conduct a large user study to understand the interaction of users with CQs in various quality categories, and the effect of CQ quality on user search performance in terms of finding relevant information, search behavior, and user satisfaction. Analysis of implicit interaction data and explicit user feedback demonstrates that high-quality CQs improve user performance and satisfaction. By contrast, low- and mid-quality CQs are harmful, and thus allowing the users to complete their tasks without CQ support may be preferred in this case. We also observe that user engagement, and therefore the need for CQ support, is affected by several factors, such as search result quality or perceived task difficulty. The findings of this study can help researchers and system designers realize why, when, and how users interact with CQs, leading to a better understanding and design of search clarification systems.
A comparative study of image retargeting Rubinstein, Michael; Gutierrez, Diego; Sorkine, Olga ...
ACM transactions on graphics,
12/2010, Volume:
29, Issue:
6
Journal Article
Peer reviewed
The numerous works on media retargeting call for a methodological approach for evaluating retargeting results. We present the first comprehensive perceptual study and analysis of image retargeting. ...First, we create a benchmark of images and conduct a large scale user study to compare a representative number of state-of-the-art retargeting methods. Second, we present analysis of the users' responses, where we find that humans in general agree on the evaluation of the results and show that some retargeting methods are consistently more favorable than others. Third, we examine whether computational image distance metrics can predict human retargeting perception. We show that current measures used in this context are not necessarily consistent with human rankings, and demonstrate that better results can be achieved using image features that were not previously considered for this task. We also reveal specific qualities in retargeted media that are more important for viewers. The importance of our work lies in promoting better measures to assess and guide retargeting algorithms in the future. The full benchmark we collected, including all images, retargeted results, and the collected user data, are available to the research community for further investigation at
http://people.csail.mit.edu/mrub/retargetme.
PDF
