Human movement researchers are often restricted to laboratory environments and data capture techniques that are time and/or resource intensive. Markerless pose estimation algorithms show great ...potential to facilitate large scale movement studies 'in the wild', i.e., outside of the constraints imposed by marker-based motion capture. However, the accuracy of such algorithms has not yet been fully evaluated. We computed 3D joint centre locations using several pre-trained deep-learning based pose estimation methods (OpenPose, AlphaPose, DeepLabCut) and compared to marker-based motion capture. Participants performed walking, running and jumping activities while marker-based motion capture data and multi-camera high speed images (200 Hz) were captured. The pose estimation algorithms were applied to 2D image data and 3D joint centre locations were reconstructed. Pose estimation derived joint centres demonstrated systematic differences at the hip and knee (~ 30-50 mm), most likely due to mislabeling of ground truth data in the training datasets. Where systematic differences were lower, e.g., the ankle, differences of 1-15 mm were observed depending on the activity. Markerless motion capture represents a highly promising emerging technology that could free movement scientists from laboratory environments but 3D joint centre locations are not yet consistently comparable to marker-based motion capture.
This study describes the development, evaluation and application of a computer vision and deep learning system capable of capturing sprinting and skeleton push start step characteristics and mass ...centre velocities (sled and athlete). Movement data were captured concurrently by a marker-based motion capture system and a custom markerless system. High levels of agreement were found between systems, particularly for spatial based variables (step length error 0.001 ± 0.012 m) while errors for temporal variables (ground contact time and flight time) were on average within ± 1.5 frames of the criterion measures. Comparisons of sprinting and pushing revealed decreased mass centre velocities as a result of pushing the sled but step characteristics were comparable to sprinting when aligned as a function of step velocity. There were large asymmetries between the inside and outside leg during pushing (e.g. 0.22 m mean step length asymmetry) which were not present during sprinting (0.01 m step length asymmetry). The observed asymmetries suggested that force production capabilities during ground contact were compromised for the outside leg. The computer vision based methods tested in this research provide a viable alternative to marker-based motion capture systems. Furthermore, they can be deployed into challenging, real world environments to non-invasively capture data where traditional approaches are infeasible.
The ability to accurately and non-invasively measure 3D mass centre positions and their derivatives can provide rich insight into the physical demands of sports training and competition. This study ...examines a method for non-invasively measuring mass centre velocities using markerless human pose estimation and Kalman smoothing. Marker (Qualysis) and markerless (OpenPose) motion capture data were captured synchronously for sprinting and skeleton push starts. Mass centre positions and velocities derived from raw markerless pose estimation data contained large errors for both sprinting and skeleton pushing (mean ± SD = 0.127 ± 0.943 and -0.197 ± 1.549 m·s
, respectively). Signal processing methods such as Kalman smoothing substantially reduced the mean error (±SD) in horizontal mass centre velocities (0.041 ± 0.257 m·s
) during sprinting but the precision remained poor. Applying pose estimation to activities which exhibit unusual body poses (e.g., skeleton pushing) appears to elicit more erroneous results due to poor performance of the pose estimation algorithm. Researchers and practitioners should apply these methods with caution to activities beyond sprinting as pose estimation algorithms may not generalise well to the activity of interest. Retraining the model using activity specific data to produce more specialised networks is therefore recommended.
This study presented a fully automated deep learning based markerless motion capture workflow and evaluated its performance against marker-based motion capture during overground running, walking and ...counter movement jumping. Multi-view high speed (200 Hz) image data were collected concurrently with marker-based motion capture (criterion data), permitting a direct comparison between methods. Lower limb kinematic data for 15 participants were computed using 2D pose estimation, our 3D fusion process and OpenSim based inverse kinematics modelling. Results demonstrated high levels of agreement for lower limb joint angles, with mean differences ranging “0.1° − 10.5° for hip (3 DoF) joint rotations, and 0.7° − 3.9° for knee (1 DoF) and ankle (2 DoF) rotations. These differences generally fall within the documented uncertainties of marker-based motion capture, suggesting that our markerless approach could be used for appropriate biomechanics applications. We used an open-source, modular and customisable workflow, allowing for integration with other popular biomechanics tools such as OpenSim. By developing open-source tools, we hope to facilitate the democratisation of markerless motion capture technology and encourage the transparent development of markerless methods. This presents exciting opportunities for biomechanics researchers and practitioners to capture large amounts of high quality, ecologically valid data both in the laboratory and in the wild.
Emotional facial expressions critically impact social interactions and cognition. However, emotion research to date has generally relied on the assumption that people represent categorical emotions ...in the same way, using standardized stimulus sets and overlooking important individual differences. To resolve this problem, we developed and tested a task using
to derive assumption-free, participant-generated emotional expressions. One hundred and five participants generated a subjective representation of happy, angry, fearful and sad faces. Population-level consistency was observed for happy faces, but fearful and sad faces showed a high degree of variability. High test-retest reliability was observed across all emotions. A separate group of 108 individuals accurately identified happy and angry faces from the first study, while fearful and sad faces were commonly misidentified. These findings are an important first step towards understanding individual differences in emotion representation, with the potential to reconceptualize the way we study atypical emotion processing in future research.
This study describes the development, evaluation and application of a computer vision and deep learning system capable of capturing sprinting and skeleton push start step characteristics and mass ...centre velocities (sled and athlete). Movement data were captured concurrently by a marker-based motion capture system and a custom markerless system. High levels of agreement were found between systems, particularly for spatial based variables (step length error 0.001 ± 0.012 m) while errors for temporal variables (ground contact time and flight time) were on average within ± 1.5 frames of the criterion measures. Comparisons of sprinting and pushing revealed decreased mass centre velocities as a result of pushing the sled but step characteristics were comparable to sprinting when aligned as a function of step velocity. There were large asymmetries between the inside and outside leg during pushing (e.g. 0.22 m mean step length asymmetry) which were not present during sprinting (0.01 m step length asymmetry). The observed asymmetries suggested that force production capabilities during ground contact were compromised for the outside leg. The computer vision based methods tested in this research provide a viable alternative to marker-based motion capture systems. Furthermore, they can be deployed into challenging, real world environments to non-invasively capture data where traditional approaches are infeasible.
Skeleton is a winter sport where performance is greatly affected by the velocity an athlete can achieve during their start up to the point where they load themselves onto their sled. As such, it is ...of interest to athletes and coaching staff to be able to monitor the performance of their athletes and how they respond to different training schedules and techniques. This paper proposes a non-invasive vision based method for measuring the velocity of a skeleton athlete and their sled during the push start. Mean differences in estimated velocity between ground truth data and our proposed system were -0.005 (± 0.186) m.s −1 for the athlete mass centre and -0.017 (± 0.133) m.s −1 for the sled. The results compare favourably to techniques previously presented in the biomechanics and sport science literature.
A user-centric method for fast, interactive, robust and high-quality shadow removal is presented. Our algorithm can perform detection and removal in a range of difficult cases: such as highly ...textured and colored shadows. To perform detection an on-the-fly learning approach is adopted guided by two rough user inputs for the pixels of the shadow and the lit area. After detection, shadow removal is performed by registering the penumbra to a normalized frame which allows us efficient estimation of non-uniform shadow illumination changes, resulting in accurate and robust removal. Another major contribution of this work is the first validated and multi-scene category ground truth for shadow removal algorithms. This data set containing 186 images eliminates inconsistencies between shadow and shadow-free images and provides a range of different shadow types such as soft, textured, colored and broken shadow. Using this data, the most thorough comparison of state-of-the-art shadow removal methods to date is performed, showing our proposed new algorithm to outperform the state-of-the-art across several measures and shadow category. To complement our dataset, an online shadow removal benchmark website is also presented to encourage future open comparisons in this challenging field of research.
We introduce a video-based approach for producing water surface models. Recent advances in this field output high-quality results but require dedicated capturing devices and only work in limited ...conditions. In contrast, our method achieves a good tradeoff between the visual quality and the production cost: It automatically produces a visually plausible animation using a single viewpoint video as the input. Our approach is based on two discoveries: first, shape from shading (SFS) is adequate to capture the appearance and dynamic behavior of the example water; second, shallow water model can be used to estimate a velocity field that produces complex surface dynamics. We will provide qualitative evaluation of our method and demonstrate its good performance across a wide range of scenes.
PDF
