Markerless tracking of hands and fingers is a promising enabler for human-computer interaction. However, adoption has been limited because of tracking inaccuracies, incomplete coverage of motions, ...low framerate, complex camera setups, and high computational requirements. In this paper, we present a fast method for accurately tracking rapid and complex articulations of the hand using a single depth camera. Our algorithm uses a novel detectionguided optimization strategy that increases the robustness and speed of pose estimation. In the detection step, a randomized decision forest classifies pixels into parts of the hand. In the optimization step, a novel objective function combines the detected part labels and a Gaussian mixture representation of the depth to estimate a pose that best fits the depth. Our approach needs comparably less computational resources which makes it extremely fast (50 fps without GPU support). The approach also supports varying static, or moving, camera-to-scene arrangements. We show the benefits of our method by evaluating on public datasets and comparing against previous work.
Control Theoretic Models of Pointing Müller, Jörg; Oulasvirta, Antti; Murray-Smith, Roderick
ACM transactions on computer-human interaction,
09/2017, Letnik:
24, Številka:
4
Journal Article
Recenzirano
Odprti dostop
This article presents an empirical comparison of four models from manual control theory on their ability to model targeting behaviour by human users using a mouse: McRuer’s Crossover, Costello’s ...Surge, second-order lag (2OL), and the Bang-bang model. Such dynamic models are generative, estimating not only movement time, but also pointer position, velocity, and acceleration on a moment-to-moment basis. We describe an experimental framework for acquiring pointing actions and automatically fitting the parameters of mathematical models to the empirical data. We present the use of time-series, phase space, and Hooke plot visualisations of the experimental data, to gain insight into human pointing dynamics. We find that the identified control models can generate a range of dynamic behaviours that captures aspects of human pointing behaviour to varying degrees. Conditions with a low index of difficulty (ID) showed poorer fit because their unconstrained nature leads naturally to more behavioural variability. We report on characteristics of human surge behaviour (the initial, ballistic sub-movement) in pointing, as well as differences in a number of controller performance measures, including
overshoot, settling time, peak time,
and
rise time
. We describe trade-offs among the models. We conclude that control theory offers a promising complement to Fitts’ law based approaches in HCI, with models providing representations and predictions of human pointing dynamics, which can improve our understanding of pointing and inform design.
Objective
The objective was to better understand how people adapt multitasking behavior when circumstances in driving change and how safe versus unsafe behaviors emerge.
Background
Multitasking ...strategies in driving adapt to changes in the task environment, but the cognitive mechanisms of this adaptation are not well known. Missing is a unifying account to explain the joint contribution of task constraints, goals, cognitive capabilities, and beliefs about the driving environment.
Method
We model the driver’s decision to deploy visual attention as a stochastic sequential decision-making problem and propose hierarchical reinforcement learning as a computationally tractable solution to it. The supervisory level deploys attention based on per-task value estimates, which incorporate beliefs about risk. Model simulations are compared against human data collected in a driving simulator.
Results
Human data show adaptation to the attentional demands of ongoing tasks, as measured in lane deviation and in-car gaze deployment. The predictions of our model fit the human data on these metrics.
Conclusion
Multitasking strategies can be understood as optimal adaptation under uncertainty, wherein the driver adapts to cognitive constraints and the task environment’s uncertainties, aiming to maximize the expected long-term utility. Safe and unsafe behaviors emerge as the driver has to arbitrate between conflicting goals and manage uncertainty about them.
Application
Simulations can inform studies of conditions that are likely to give rise to unsafe driving behavior.
Optimization methods have revolutionized almost every field of engineering design, so why not user interface design? The author reviews progress and challenges in model-driven UI optimization, in ...which an optimizer utilizes predictive models of human perception, behavior, and experience to anticipate users' responses to computer-generated designs.
Tracking the articulated 3D motion of the hand has important applications, for example, in human-computer interaction and teleoperation. We present a novel method that can capture a broad range of ...articulated hand motions at interactive rates. Our hybrid approach combines, in a voting scheme, a discriminative, part-based pose retrieval method with a generative pose estimation method based on local optimization. Color information from a multi-view RGB camera setup along with a person-specific hand model are used by the generative method to find the pose that best explains the observed images. In parallel, our discriminative pose estimation method uses fingertips detected on depth data to estimate a complete or partial pose of the hand by adopting a part-based pose retrieval strategy. This part-based strategy helps reduce the search space drastically in comparison to a global pose retrieval strategy. Quantitative results show that our method achieves state-of-the-art accuracy on challenging sequences and a near-real time performance of 10 fps on a desktop computer.
On-Skin Interfaces Oulasvirta, Antti
Computer (Long Beach, Calif.),
2017, Letnik:
50, Številka:
10
Journal Article
Recenzirano
Odprti dostop
The convergence of advances in electrical engineering and material science has opened up new opportunities for using the skin as an interactive device. This theme issue presents two articles focusing ...on emerging input capabilities and design.
We discuss the state-of-the-art and future directions of the development, evaluation, and application of computational cognitive models for human-automated vehicle interaction. The capabilities of ...automated vehicles are rapidly increasing and changing human interaction with and around the vehicle. Yet, at the same time, fully automated vehicles that do not require human interaction are not available. Therefore, systems are needed in which the human and the vehicle interact together. We discuss how computational cognitive models that can describe, predict, and/or anticipate human behavior and thought can play a crucial role in this regard. Such research comes from many different disciplines including cognitive science, human-computer interaction, human factors, transportation research, and artificial intelligence. This special issue brings together state-of-the-art research from these fields. We identify four broader directions for future research: (1) to continue Allen Newell's research agenda for cognitive modeling, but now apply it to the field of human-automated vehicle interaction; (2) to move from isolated theory-slicing to integrated theories, (3) to consider cognitive models both for analysis of interaction and for use in embedded systems; (4) to move from models that mostly describe to models that can predict.
Designing a good scatterplot can be difficult for non-experts in visualization, because they need to decide on many parameters, such as marker size and opacity, aspect ratio, color, and rendering ...order. This paper contributes to research exploring the use of perceptual models and quality metrics to set such parameters automatically for enhanced visual quality of a scatterplot. A key consideration in this paper is the construction of a cost function to capture several relevant aspects of the human visual system, examining a scatterplot design for some data analysis task. We show how the cost function can be used in an optimizer to search for the optimal visual design for a user's dataset and task objectives (e.g., "reliable linear correlation estimation is more important than class separation"). The approach is extensible to different analysis tasks. To test its performance in a realistic setting, we pre-calibrated it for correlation estimation, class separation, and outlier detection. The optimizer was able to produce designs that achieved a level of speed and success comparable to that of those using human-designed presets (e.g., in R or MATLAB). Case studies demonstrate that the approach can adapt a design to the data, to reveal patterns without user intervention.
PDF
