Activity detection and classification are very important for autonomous monitoring of humans for applications, including assistive living, rehabilitation, and surveillance. Wearable sensors have ...found wide-spread use in recent years due to their ever-decreasing cost, ease of deployment and use, and ability to provide continuous monitoring as opposed to sensors installed at fixed locations. Since many smart phones are now equipped with a variety of sensors, such as accelerometer, gyroscope, and camera, it has become more feasible to develop activity monitoring algorithms employing one or more of these sensors with increased accessibility. We provide a complete and comprehensive survey on activity classification with wearable sensors, covering a variety of sensing modalities, including accelerometer, gyroscope, pressure sensors, and camera- and depth-based systems. We discuss differences in activity types tackled by this breadth of sensing modalities. For example, accelerometer, gyroscope, and magnetometer systems have a history of addressing whole body motion or global type activities, whereas camera systems provide the context necessary to classify local interactions, or interactions of individuals with objects. We also found that these single sensing modalities laid the foundation for hybrid works that tackle a mix of global and local interaction-type activities. In addition to the type of sensors and type of activities classified, we provide details on each wearable system that include on-body sensor location, employed learning approach, and extent of experimental setup. We further discuss where the processing is performed, i.e., local versus remote processing, for different systems. This is one of the first surveys to provide such breadth of coverage across different wearable sensor systems for activity classification.
Robust and reliable detection of falls is crucial especially for elderly activity monitoring systems. In this letter, we present a fall detection system using wearable devices, e.g., smartphones, and ...tablets, equipped with cameras and accelerometers. Since the portable device is worn by the subject, monitoring is not limited to confined areas, and extends to wherever the subject may travel, as opposed to static sensors installed in certain rooms. Moreover, a camera provides an abundance of information, and the results presented here show that fusing camera and accelerometer data not only increases the detection rate, but also decreases the number of false alarms compared to only accelerometer-based or only camera-based systems. We employ histograms of edge orientations together with the gradient local binary patterns for the camera-based part of fall detection. We compared the performance of the proposed method with that of using original histograms of oriented gradients (HOG) as well as a modified version of HOG. Experimental results show that the proposed method outperforms using original HOG and modified HOG, and provides lower false positive rates for the camera-based detection. Moreover, we have employed an accelerometer-based fall detection method, and fused these two sensor modalities to have a robust fall detection system. Experimental results and trials with actual Samsung Galaxy phones show that the proposed method, combining two different sensor modalities, provides much higher sensitivity, and a significant decrease in the number of false positives during daily activities, compared to accelerometer-only and camera-only methods.
To make efficient use of limited spectral resources, we in this work propose a deep actor-critic reinforcement learning based framework for dynamic multichannel access. We consider both a single-user ...case and a scenario in which multiple users attempt to access channels simultaneously. We employ the proposed framework as a single agent in the single-user case, and extend it to a decentralized multi-agent framework in the multi-user scenario. In both cases, we develop algorithms for the actor-critic deep reinforcement learning and evaluate the proposed learning policies via experiments and numerical results. In the single-user model, in order to evaluate the performance of the proposed channel access policy and the framework's tolerance against uncertainty, we explore different channel switching patterns and different switching probabilities. In the case of multiple users, we analyze the probabilities of each user accessing channels with favorable channel conditions and the probability of collision. We also address a time-varying environment to identify the adaptive ability of the proposed framework. Additionally, we provide comparisons (in terms of both the average reward and time efficiency) between the proposed actor-critic deep reinforcement learning framework, Deep-Q network (DQN) based approach, random access, and the optimal policy when the channel dynamics are known.
Hybrid automatic repeat request (HARQ) is a high-performance communication protocol, leading to the effective use of the wireless channel and the resources with only limited feedback about the ...channel state information (CSI) to the transmitter. In this paper, the throughput of HARQ with incremental redundancy (IR) and fixed transmission rate is studied in the presence of statistical queuing constraints imposed as limitations on buffer overflow probabilities. In particular, tools from the theory of renewal processes and stochastic network calculus are employed to characterize the maximum arrival rates that can be supported by the wireless channel when HARQ-IR is adopted. Effective capacity is employed as the throughput metric, and a closed-form expression for the effective capacity of HARQ-IR is determined for small values of the quality-of-service (QoS) exponent. The impact of the fixed transmission rate, queuing constraints, and hard-deadline limitations on the throughput is investigated, and comparisons with type-I HARQ and HARQ with Chase Combining are provided.
Driver behavior analysis plays an important role in driver assistance systems. A driver's face and head pose hold the key towards understanding whether the driver's attention and concentration are on ...the road while driving. Naturalistic driving studies (NDS) allow observing drivers in real-time under naturalistic traffic conditions. Yet, data collected in NDS often comprise low-resolution videos usually with more challenging camera positions compared to controlled studies. For instance, when the camera is not directly facing the driver, classifying head pose becomes more challenging, since the variation between different classes becomes much smaller. In this paper, we propose three different approaches to classify a driver's head pose from naturalistic videos, which were captured by a camera providing a side view, instead of directly facing the driver. These approaches employ a sequence of five key points on the driver's face. We compare these three proposed approaches with each other as well as with three different baselines by using leave-one-driver-out cross-validation on nine different drivers. Results show that our proposed method employing a Bidirectional Gated Recurrent Unit (BiGRU) outperforms the best performing baseline by 11% in terms of overall accuracy.
The proliferation of 3D sensors, due to the increased demand for 3D data, induced the 3D computer vision research in the last decade, and 3D data processing has gained a lot of interest. As in many ...other applications in computer vision, deep learning-based methods were quickly applied to 3D data classification and have become the state-of-the-art in this area. More recently, capsule networks, which are novel neural structures, have been introduced to enhance the ability of neural networks to better capture the parts-relationship, which yields more accurate classification with less training data. Moreover, deploying deep machine learning models on mobile platforms requires the models to be optimized due to limited memory and computational constraints. In this work, we propose methods to boost the accuracies of a standard 3D CNN-based and a Capsule Network-based classifier, help the training to better generalize the data distribution with limited data, and optimize the models for resource-constrained environments, such as mobile platforms. We also show that the introduction of capsules to 3D object classification pipeline improves the classification performance with limited training data, while a specifically optimized weight pruning method keeps the model compact enough for mobile deployment. Our broad spectrum of experiments show that proposed methods improve the performance of the base model while significantly reducing the memory and computation requirements.
Person re-identification across a network of cameras, with disjoint views, has been studied extensively due to its importance in wide-area video surveillance. This is a challenging task due to ...several reasons including changes in illumination and target appearance, and variations in camera viewpoint and camera intrinsic parameters. The approaches developed to re-identify a person across different camera views need to address these challenges. More recently, neural network-based methods have been proposed to solve the person re-identification problem across different camera views, achieving state-of-the-art performance. In this paper, we present an effective and generalizable attack model that generates adversarial images of people, and results in very significant drop in the performance of the existing state-of-the-art person re-identification models. The results demonstrate the extreme vulnerability of the existing models to adversarial examples, and draw attention to the potential security risks that might arise due to this in video surveillance. Our proposed attack is developed by decreasing the dispersion of the internal feature map of a neural network to degrade the performance of several different state-of-the-art person re-identification models. We also compare our proposed attack with other state-of-the-art attack models on different person re-identification approaches, and by using four different commonly used benchmark datasets. The experimental results show that our proposed attack outperforms the state-of-art attack models on the best performing person re-identification approaches by a large margin, and results in the most drop in the mean average precision values.
AbstractThe current paper implements a methodology for automatically detecting vehicle maneuvers from vehicle telemetry data under naturalistic driving settings. Previous approaches have treated ...vehicle maneuver detection as a classification problem, although both time series segmentation and classification are required since input telemetry data are continuous. Our objective is to develop an end-to-end pipeline for the frame-by-frame annotation of naturalistic driving studies videos into various driving events including stop and lane-keeping events, lane changes, left-right turning movements, and horizontal curve maneuvers. To address the time series segmentation problem, the study developed an energy-maximization algorithm (EMA) capable of extracting driving events of varying durations and frequencies from continuous signal data. To reduce overfitting and false alarm rates, heuristic algorithms were used to classify events with highly variable patterns such as stops and lane-keeping. To classify segmented driving events, four machine-learning models were implemented, and their accuracy and transferability were assessed over multiple data sources. The duration of events extracted by EMA was comparable to actual events, with accuracies ranging from 59.30% (left lane change) to 85.60% (lane-keeping). Additionally, the overall accuracy of the 1D-convolutional neural network model was 98.99%, followed by the long-short-term-memory model at 97.75%, then the random forest model at 97.71%, and the support vector machine model at 97.65%. These model accuracies were consistent across different data sources. The study concludes that implementing a segmentation-classification pipeline significantly improves both the accuracy of driver maneuver detection and the transferability of shallow and deep ML models across diverse datasets.
Robust detection of events and activities, such as falling, sitting, and lying down, is a key to a reliable elderly activity monitoring system. While fast and precise detection of falls is critical ...in providing immediate medical attention, other activities like sitting and lying down can provide valuable information for early diagnosis of potential health problems. In this paper, we present a fall detection and activity classification system using wearable cameras. Since the camera is worn by the subject, monitoring is not limited to confined areas, and extends to wherever the subject may go including indoors and outdoors. Furthermore, since the captured images are not of the subject, privacy concerns are alleviated. We present a fall detection algorithm employing histograms of edge orientations and strengths, and propose an optical flow-based method for activity classification. The first set of experiments has been performed with prerecorded video sequences from eight different subjects wearing a camera on their waist. Each subject performed around 40 trials, which included falling, sitting, and lying down. Moreover, an embedded smart camera implementation of the algorithm was also tested on a CITRIC platform with subjects wearing the CITRIC camera, and each performing 50 falls and 30 non-fall activities. Experimental results show the success of the proposed method.
PDF
