•This work compares Kinect Structured-Light with Kinect Time-of-Flight cameras.•The results offer descriptions under which condition one is superior to the other.•Solid insight of the devices is ...given to make decisions on their application.•We propose a set of nine tests for comparing both Kinects, five of which are novel.
Recently, the new Kinect One has been issued by Microsoft, providing the next generation of real-time range sensing devices based on the Time-of-Flight (ToF) principle. As the first Kinect version was using a structured light approach, one would expect various differences in the characteristics of the range data delivered by both devices.
This paper presents a detailed and in-depth comparison between both devices. In order to conduct the comparison, we propose a framework of seven different experimental setups, which is a generic basis for evaluating range cameras such as Kinect. The experiments have been designed with the goal to capture individual effects of the Kinect devices as isolatedly as possible and in a way, that they can also be adopted, in order to apply them to any other range sensing device. The overall goal of this paper is to provide a solid insight into the pros and cons of either device. Thus, scientists who are interested in using Kinect range sensing cameras in their specific application scenario can directly assess the expected, specific benefits and potential problem of either device.
•Camera viewing angles could affect depth sensors ability to track gait kinematics.•Kinect v2 and Orbbec Astra track kinematics well but only for a frontal view angle.•Azure Kinect is recommended to ...track gait kinematics at non-frontal viewing angles.
Depth sensors could be a portable, affordable, marker-less alternative to three-dimension motion capture systems for gait analysis, but the effects of camera viewing angles on their joint angle tracking performance have not been fully investigated.
This study evaluated the accuracies of three depth sensors Azure Kinect (AK); Kinect v2 (K2); Orbbec Astra (OA) for tracking kinematic gait patterns during treadmill walking at five camera viewing angles (0°/22.5°/45°/67.5°/90°).
Ten healthy subjects performed fifteen treadmill walking trials (3 speeds × 5 viewing angles) using the three depth sensors to measure joint angles in sagittal hip, frontal hip, sagittal knee, and sagittal ankle. Ten walking steps were recorded and averaged for each walking trial. Range of motion in terms of maximum and minimum joint angles measured by the depth sensors were compared with the Vicon motion capture system as the gold standard. Depth sensors tracking accuracies were compared against the Vicon reference using root-mean-square error (RMSE) on the joint angle time series. Effects of different walking speeds, viewing angles, and depth sensors on the tracking accuracy were observed using three-way repeated-measure analysis of variance (ANOVA).
ANOVA results on RMSE showed significant interaction effects between viewing angles and depth sensors for sagittal hip F(8,72) = 4.404, p = 0.005 and for sagittal knee F(8,72)=13.211, p < 0.001 joint angles. AK had better tracking performance when subjects walked at non-frontal camera viewing angles (22.5°/45°/67.5°/90°); while K2 performed better at frontal viewing angle (0°). The superior tracking performance of AK compared with K2/OA might be attributed to the improved depth sensor resolution and body tracking algorithm.
Researchers should be cautious about camera viewing angle when using depth sensors for kinematic gait measurements. Our results demonstrated Azure Kinect had good tracking performance of sagittal hip and sagittal knee joint angles during treadmill walking tests at non-frontal camera viewing angles.
► Introduction to the recent advances in depth sensing technologies. ► Survey of human motion analysis using depth images, including action recognition. ► Covers significant literature that uses ...Kinect or time-of-flight images. ► Details public software libraries/human action/activity datasets based on Kinect. ► Discusses current research and prospective future research directions.
Analysis of human behaviour through visual information has been a highly active research topic in the computer vision community. This was previously achieved via images from a conventional camera, however recently depth sensors have made a new type of data available. This survey starts by explaining the advantages of depth imagery, then describes the new sensors that are available to obtain it. In particular, the Microsoft Kinect has made high-resolution real-time depth cheaply available. The main published research on the use of depth imagery for analysing human activity is reviewed. Much of the existing work focuses on body part detection and pose estimation. A growing research area addresses the recognition of human actions. The publicly available datasets that include depth imagery are listed, as are the software libraries that can acquire it from a sensor. This survey concludes by summarising the current state of work on this topic, and pointing out promising future research directions. For both researchers and practitioners who are familiar with this topic and those who are new to this field, the review will aid in the selection, and development, of algorithms using depth data.
•Occupancy detection and profiling have been demonstrated by using pure depth data.•Depth data are unsuitable to reveal person's identity, allowing to preserve privacy.•The presented framework scales ...well with the number of people.•Validated with two different depth sensors with initially promising results.•Relevant applications for building energy management have been also demonstrated.
In this paper, a computational framework for occupancy detection and profiling based exclusively on depth data is presented. 3D depth sensors offer many advantages against traditional video cameras. Occupants’ privacy can be assured more effectively because depth information is unsuitable to reveal the person's identity. Notable low-level computer vision tasks can be simplified, thus lightening the computational load. The presented framework is suitable for wall-mounting setups as well as for ceiling-mounting setups, and scales well with the number of people. To take full advantage of depth data and to accommodate specificities of crowded environments, several improvements to the standard computer vision pipeline are suggested. Firstly, the running Gaussian average background model is adapted to work with depth distances in crowded scenes. Secondly, the classical complete linkage agglomerative clustering is boosted by adding edge-based constraints specifically designed for people segmentation in depth data. Thirdly, to reliable discriminate people, specific depth-based features are defined to be used with a Real AdaBoost classifier. The preliminary results achieved by using two different depth sensors and synthetic data are very promising, outperforming existing approaches. Relevant applications for building energy management, such as occupancy profiling and construction of trajectories and density maps, have been also demonstrated.
Jumping spiders (Salticidae) rely on accurate depth perception for predation and navigation. They accomplish depth perception, despite their tiny brains, by using specialized optics. Each principal ...eye includes a multitiered retina that simultaneously receives multiple images with different amounts of defocus, and from these images, distance is decoded with relatively little computation. We introduce a compact depth sensor that is inspired by the jumping spider. It combines metalens optics, which modifies the phase of incident light at a subwavelength scale, with efficient computations to measure depth from image defocus. Instead of using a multitiered retina to transduce multiple simultaneous images, the sensor uses a metalens to split the light that passes through an aperture and concurrently form 2 differently defocused images at distinct regions of a single planar photosensor. We demonstrate a system that deploys a 3-mm-diameter metalens to measure depth over a 10-cm distance range, using fewer than 700 floating point operations per output pixel. Compared with previous passive depth sensors, our metalens depth sensor is compact, single-shot, and requires a small amount of computation. This integration of nanophotonics and efficient computation brings artificial depth sensing closer to being feasible on millimeter-scale, microwatts platforms such as microrobots and microsensor networks.
A depth camera or 3-dimensional scanner was used as a sensor for traditional methods to quantify the identified concrete spalling damage in terms of volume. However, to quantify the concrete spalling ...damage automatically, the first step is to detect (i.e., identify) the concrete spalling. The multiple spots of spalling can be possible within a single structural element or in multiple structural elements. However, there is, as of yet, no method to detect concrete spalling automatically using deep learning methods. Therefore, in this paper, a faster region-based convolutional neural network (Faster R-CNN)-based concrete spalling damage detection method is proposed with an inexpensive depth sensor to quantify multiple instances of spalling simultaneously in the same surface separately and consider multiple surfaces in structural elements. A database composed of 1091 images (with 853 × 1440 pixels) labeled for volumetric damage is developed, and the deep learning network is then modified, trained, and validated using the proposed database. The damage quantification is automatically performed by processing the depth data, identifying surfaces, and isolating the damage after merging the output from the Faster R-CNN with the depth stream of the sensor. The trained Faster R-CNN presented an average precision (AP) of 90.79%. Volume quantifications show a mean precision error (MPE) of 9.45% when considering distances from 100 cm to 250 cm between the element and the sensor. Also, an MPE of 3.24% was obtained for maximum damage depth measurements across the same distance range.
•The first method of an automatic concrete spalling volumetric quantification without limitations of setup of a depth camera•An advanced deep learning approach integrated with a structural surface fitting algorithm for automation•The proposed method can quantify multiple volumetric sites of damage in the same structural surface and on multiple surfaces•Fully automated damage detection, localization, and quantification realized by automatically identifying the damaged surface•The proposed method can be used as a prototype for the automatic quantification of damage using other type of depth camera
Survey on 3D Hand Gesture Recognition Cheng, Hong; Yang, Lu; Liu, Zicheng
IEEE transactions on circuits and systems for video technology,
2016-Sept., 2016-9-00, 20160901, Letnik:
26, Številka:
9
Journal Article
Recenzirano
Three-dimensional hand gesture recognition has attracted increasing research interests in computer vision, pattern recognition, and human-computer interaction. The emerging depth sensors greatly ...inspired various hand gesture recognition approaches and applications, which were severely limited in the 2D domain with conventional cameras. This paper presents a survey of some recent works on hand gesture recognition using 3D depth sensors. We first review the commercial depth sensors and public data sets that are widely used in this field. Then, we review the state-of-the-art research for 3D hand gesture recognition in four aspects: 1) 3D hand modeling; 2) static hand gesture recognition; 3) hand trajectory gesture recognition; and 4) continuous hand gesture recognition. While the emphasis is on 3D hand gesture recognition approaches, the related applications and typical systems are also briefly summarized for practitioners.
To overcome the travelling difficulty for the visually impaired group, this paper presents a novel ETA (Electronic Travel Aids)-smart guiding device in the shape of a pair of eyeglasses for giving ...these people guidance efficiently and safely. Different from existing works, a novel multi-sensor fusion based obstacle avoiding algorithm is proposed, which utilizes both the depth sensor and ultrasonic sensor to solve the problems of detecting small obstacles, and transparent obstacles, e.g. the French door. For totally blind people, three kinds of auditory cues were developed to inform the direction where they can go ahead. Whereas for weak sighted people, visual enhancement which leverages the AR (Augment Reality) technique and integrates the traversable direction is adopted. The prototype consisting of a pair of display glasses and several low-cost sensors is developed, and its efficiency and accuracy were tested by a number of users. The experimental results show that the smart guiding glasses can effectively improve the user's travelling experience in complicated indoor environment. Thus it serves as a consumer device for helping the visually impaired people to travel safely.
PDF
