•A new evaluation procedure for action localization is proposed.•We introduce performance graphs showing quantity as a function of quality.•A single performance measure integrates out quality ...constraints.•Soft upper bounds estimated from experimental data.•The entry algorithms in the ICPR 2012 HARL competition are evaluated.
Evaluating the performance of computer vision algorithms is classically done by reporting classification error or accuracy, if the problem at hand is the classification of an object in an image, the recognition of an activity in a video or the categorization and labeling of the image or video. If in addition the detection of an item in an image or a video, and/or its localization are required, frequently used metrics are Recall and Precision, as well as ROC curves. These metrics give quantitative performance values which are easy to understand and to interpret even by non-experts. However, an inherent problem is the dependency of quantitative performance measures on the quality constraints that we need impose on the detection algorithm. In particular, an important quality parameter of these measures is the spatial or spatio-temporal overlap between a ground-truth item and a detected item, and this needs to be taken into account when interpreting the results.
We propose a new performance metric addressing and unifying the qualitative and quantitative aspects of the performance measures. The performance of a detection and recognition algorithm is illustrated intuitively by performance graphs which present quantitative performance values, like Recall, Precision and F-Score, depending on quality constraints of the detection. In order to compare the performance of different computer vision algorithms, a representative single performance measure is computed from the graphs, by integrating out all quality parameters. The evaluation method can be applied to different types of activity detection and recognition algorithms. The performance metric has been tested on several activity recognition algorithms participating in the ICPR 2012 HARL competition.
To reduce the computational and memory cost of ConvNets, structured pruning consists in removing features in convolutional and/or fully-connected layers of a pretrained network. As a novel ...contribution, we study the selection of features as a convex quadratic program with linear constraints. In addition, the updating of weights is treated as the resolution of a linear least-squares system. The quadratic objective function arises from the least-squares reconstruction error on the outputs of pruned layers. Experiments show that our method gives accuracies on par with other reconstruction error-based methods, while achieving a clear gain on computation time.
We describe a narrow band region approach for deformable curves and surfaces in the perspective of 2D and 3D image segmentation. Basically, we develop a region energy involving a fixed-width band ...around the curve or surface. Classical region-based methods, like the Chan–Vese model, often make strong assumptions on the intensity distributions of the searched object and background. In order to be less restrictive, our energy achieves a trade-off between local features of gradient-like terms and global region features. Relying on the theory of parallel curves and surfaces, we perform a mathematical derivation to express the region energy in a curvature-based form allowing efficient computation on explicit models. We introduce two different region terms, each one being dedicated to a particular configuration of the target object. Evolution of deformable models is performed by means of energy minimization using gradient descent. We provide both explicit and implicit implementations. The explicit models are a parametric snake in 2D and a triangular mesh in 3D, whereas the implicit models are based on the level set framework, regardless of the dimension. Experiments are carried out on MRI and CT medical images, in 2D and 3D, as well as 2D color photographs.
We propose a method for human activity recognition from RGB data that does not rely on any pose information during test time, and does not explicitly calculate pose information internally. Instead, a ...visual attention module learns to predict glimpse sequences in each frame. These glimpses correspond to interest points in the scene that are relevant to the classified activities. No spatial coherence is forced on the glimpse locations, which gives the attention module liberty to explore different points at each frame and better optimize the process of scrutinizing visual information. Tracking and sequentially integrating this kind of unstructured data is a challenge, which we address by separating the set of glimpses from a set of recurrent tracking/recognition workers. These workers receive glimpses, jointly performing subsequent motion tracking and activity prediction. The glimpses are soft-assigned to the workers, optimizing coherence of the assignments in space, time and feature space using an external memory module. No hard decisions are taken, i.e. each glimpse point is assigned to all existing workers, albeit with different importance. Our methods outperform the state-of-the-art on the largest human activity recognition dataset available to-date, NTU RGB+D, and on the Northwestern-UCLA Multiview Action 3D Dataset.
•We use a global shape model to segment leaves in complex natural images.•Prior knowledge and shape constraints improve segmentation performance.•We designed explicit botany-inspired descriptors for ...local shapes.•Information is combined to classify leaf images into a list of species.•Dedicated high-level features prove to perform better than generic statistical ones.
With the aim of elaborating a mobile application, accessible to anyone and with educational purposes, we present a method for tree species identification that relies on dedicated algorithms and explicit botany-inspired descriptors. Focusing on the analysis of leaves, we developed a working process to help recognize species, starting from a picture of a leaf in a complex natural background. A two-step active contour segmentation algorithm based on a polygonal leaf model processes the image to retrieve the contour of the leaf. Features we use afterwards are high-level geometrical descriptors that make a semantic interpretation possible, and prove to achieve better performance than more generic and statistical shape descriptors alone. We present the results, both in terms of segmentation and classification, considering a database of 50 European broad-leaved tree species, and an implementation of the system is available in the iPhone application Folia.1https://itunes.apple.com/app/folia/id547650203.1
Among the various existing and mathematically equivalent definitions of the skeleton, we consider the set of critical points of the Euclidean distance transform of the shape. The problem of detecting ...these points and using them to generate a skeleton that is stable, thin and homotopic to the shape has been the focus of numerous papers. Skeleton branches correspond to ridges of the distance map, i.e., continuous lines of points that are local maxima of the distance in at least one direction. Extracting these ridges is a non-trivial task on a discrete grid. In this context, the average outward flux, used in the Hamilton–Jacobi skeleton (Siddiqi et al. in Int J Comput Vis 48(3):215–231,
2002
), and the ridgeness measure (Leborgne et al. in J Vis Commun Image Represent 31:165–176,
2015
) have been proposed as ridge detectors. We establish the mathematical relation between these detectors and, extending the work in Dimitrov et al. (Computer vision and pattern recognition, pp 835–841,
2003
), we study various local shape configurations, on which closed-form expressions or approximations of the average outward flux and ridgeness can be derived. In addition, we conduct experiments to assess the accuracy of skeletons generated using these measures and study the influence of their respective parameters.
► A novel frequency based background modeling method is proposed. ► Local Fourier transform is used for feature extraction. ► Spectral background analysis is performed. ► Foreground objects are ...detected using the approach. ► The results are compared with the results of mixture of Gaussian method.
Background models are used for object detection in many computer vision algorithms. In this article, we propose a novel background modeling method based on frequency for spatially varying and time repetitive textured background. The local Fourier transform is applied to construct a pixel-wise representation of local frequency components. We apply our method for object detection in moving background conditions. Experimental results of our frequency-based background model are evaluated both qualitatively and quantitatively.
Minimum cost paths have been extensively studied theoretical tools for interactive image segmentation. The existing geodesically linked active contour (GLAC) model, which basically consists of a set ...of vertices connected by paths of minimal cost, blends the benefits of minimal paths and region-based active contours. This results in a closed piecewise-smooth curve, over which an edge or region energy functional can be formulated. As an important shortcoming, the GLAC in its initial formulation does not guarantee the curve to be simple, consistent with respect to the purpose of segmentation. In this paper, we draw our inspiration from the GLAC and other boundary-based interactive segmentation algorithms, in the sense that we aim to extract a contour given a set of user-provided points, by connecting these points using paths. The key idea is to select a combination among a set of possible paths, such that the resulting structure represents a relevant closed curve. Instead of considering minimal paths only, we switch to a more general formulation, which we refer to as
admissible
paths. These basically correspond to the roads travelling along the bottom of distinct valleys between given endpoints. We introduce a novel term to favor the simplicity of the generated contour, as well as a local search method to choose the best combination among possible paths.
•We propose a propagation algorithm to obtain a skeleton with “good properties”.•Our algorithm combines RDMA and Laplacian of Gaussian filtering.•The complexity is linearly proportional to the number ...of pixels of image.•Our method is compared to three existing methods.
The skeleton is an essential shape descriptor providing a compact representation of a shape that can be used in the context of real object recognition. However, due to the discretization, the required properties to use it for graph matching (homotopy to the shape, consequently connectivity, thinness, robustness to noise) may be difficult to obtain simultaneously. In this paper, we propose a new skeletonization algorithm having all these properties, based on the Euclidean distance map. More precisely, the algorithm cleverly combines the centers of maximal balls included in the shape and the ridges of the distance map. Post-processing is then applied to thin and prune the resulting skeleton. We compare the proposed method to three fairly recent methods and demonstrate its good properties.
PDF
