The detection of driving events could be useful for reducing accidents, fleet management and insurance premiums etc. Currently, top of the range vehicles and large fleets employ expensive driver ...monitoring systems. However, most drivers do not have access to such systems. The required monitoring platform would have to deliver the required performance while also being affordable and accessible. A candidate with considerable promise is the smartphone with sensors built-in that could be exploited for the detection of driving events. However, to date it has not been possible to achieve the required correct, missed and false detection rates in addition to the computational efficiency for real-time operations. This paper proposes a novel bagging tree and dynamic time warping (DTW) integrated algorithm for the detection of driving events employing acceleration and orientation data from a smartphone's low cost three-axis accelerometers and gyroscopes. The bagging tree-based machine learning algorithm provides the initial maneuver detection results, as well as the location of the event start and end points. Event detection is then achieved by calculating the similarity of the results predicted through the bagging tree algorithm with the corresponding templates extracted from the experience datasets, while also applying a number of constraints to verify the calculated results. Field test results show that the proposed integrated algorithm is superior to the state-of-the-art, achieving a high correct detection accuracy of 97.5%, a low missed detection of 2.5% and a false detection rate of 2.9%. The corresponding results for the best alternative candidate method are 90.2%, 9.8% and 11.7%. Furthermore, the improvement in computational efficiency offered by our proposed approach is three to more than ten times greater than that of the other state-of-the-art algorithms.
During action recognition in videos, irrelevant motions in the background can greatly degrade the performance of recognizing specific actions with which we actually concern ourself here. In this ...paper, a novel deep neural network, called factorized action-scene network (FASNet), is proposed to encode and fuse the most relevant and informative semantic cues for action recognition. Specifically, we decompose the FASNet into two components. One is a newly designed encoding network, named content attention network (CANet), which encodes local spatial-temporal features to learn the action representations with good robustness to the noise of irrelevant motions. The other is a fusion network, which integrates the pretrained CANet to fuse the encoded spatial-temporal features with contextual scene feature extracted from the same video, for learning more descriptive and discriminative action representations. Moreover, different from the existing deep learning based tasks for generic action recognition, which applies softmax loss function as the training guidance, we formulate two loss functions for guiding the proposed model to accomplish more specific action recognition tasks, i.e., the multilabel correlation loss for multilabel action recognition and the triplet loss for complex event detection. Extensive experiments on the Hollywood2 dataset and the TRECVID MEDTest 14 dataset show that our method achieves superior performance compared with the state-of-the-art methods.
This paper is concerned about the challenge of timely and accurate detection and identification of events involving line outages and faults in power grids. Given the increasing number of installed ...phasor measurement units (PMU), their measurements are prime sources to be exploited to address this challenge since their high refresh rates allows to capture post- and pre- event phasor values. The challenge is mainly due to the fact that the number of installed PMUs is typically not sufficient to receive measurements from terminal bus(es) of every line. This challenge can be addressed by applying model-based sparse estimation methods or data-driven approaches such as Convolutional Neural Networks (CNN) and Graphical Neural Networks (GCN). These methods have been shown to be effective with high rates of success in detecting and identifying faults and line outages despite the availability of relatively sparse set of installed PMUs. It should however be noted that success rates of these methods are very strongly related to strategic placement of PMUs, as their locations significantly influence the accuracy of line outage detection and fault identification. This dependency constitutes the main motivation of the paper whose focus is development of a strategic PMU placement algorithm to maximize the performance of the event detection and identification methods that use PMU measurements as their inputs. This algorithm needs to be designed to yield the best possible locations to install a limited number of PMUs, optimizing their utility for model-based sparse estimation methods as well as advanced data-driven approaches. This strategic placement is aimed at enhancing the precision of event detection within the power grid by optimizing the locations of PMUs to help reduce collinearity in the system parameters. It is demonstrated that the proposed PMU placement achieves an accuracy exceeding 90% with only 83 PMUs for the 1354 Bus system, specifically for line outage detection purposes. Conversely, other methods achieve less than 90% accuracy for the same 1354 Bus system. For fault detection, the proposed PMU placement algorithm for the 196 Bus system with 50 PMUs achieves 85% accuracy, while the maximum accuracy attained by alternative methods does not exceed 60%.
•Emotion-related EEG signal segmentation strategy based on facial landmarks.•SNN-based framework for subject-independent short-term emotion recognition from EEG.•SNN hyperparameter optimisation ...related to spike encoding and output representation.•The emerged SNN connectivity helps in understanding the neural mechanisms of emotion.
Emotion recognition still poses a challenge lying at the core of the rapidly growing area of affective computing and is crucial for establishing a successful human–computer interaction. Identification and understanding of emotions are achieved through various measures, such as subjective self-reports, face-tracking, voice analysis, gaze-tracking, as well as the analysis of autonomic and central neurophysiological measurements. Current approaches to emotion recognition based on electroencephalography (EEG) mostly rely on various handcrafted features extracted over relatively long time windows of EEG during participants exposure to appropriate affective stimuli. In this paper, we present a short-term emotion recognition framework based on spiking neural network (SNN) modelling of spatio-temporal EEG patterns. Our method relies on EEG signal segmentation based on detection of short-term changes in facial landmarks, and as such includes no computation of handcrafted EEG features. Differences between participants’ EEG properties are taken into account via subject-dependent spike encoding in the formulated subject-independent emotion recognition task. We test our methods on the publicly available DEAP and MAHNOB-HCI databases due to the availability of both EEG and frontal face video data. Through an exhaustive hyperparameter optimisation strategy, we show that the proposed SNN-based representation of EEG spiking patterns provides valuable information for short- term emotion recognition. The obtained accuracies are 78.97% and 79.39% in arousal classification, and 67.76% and 72.12% in valence classification, on the DEAP and MAHNOB-HCI datasets, respectively. Furthermore, through the application of a brain-inspired SNN model, this study provides novel insight and helps in the understanding of the neural mechanisms involved in emotional processing in the context of audiovisual stimuli, such as affective videos. The presented results encourage the use of the proposed EEG processing methodology as a complement to existing features and methods commonly used for EEG-based emotion recognition, especially for short-term arousal recognition.
Outdoor acoustic event detection is an exciting research field but challenged by the need for complex algorithms and deep learning techniques, typically requiring many computational, memory, and ...energy resources. These challenges discourage IoT implementations, where an efficient use of resources is required. However, current embedded technologies and microcontrollers have increased their capabilities without penalizing energy efficiency. This paper addresses the application of sound event detection at the very edge, by optimizing deep learning techniques on resource-constrained embedded platforms for the IoT. The contribution is two-fold: firstly, a two-stage student-teacher approach is presented to make state-of-the-art neural networks for sound event detection fit on current microcontrollers; secondly, we test our approach on an ARM Cortex M4, particularly focusing on issues related to 8-bits quantization. Our embedded implementation can achieve 68% accuracy in recognition on Urbansound8k, not far from state-of-the-art performance, with an inference time of 125 ms for each second of the audio stream, and power consumption of 5.5 mW in just 34.3 kB of RAM.
Self-paced learning (SPL) is a recently proposed methodology designed by mimicking through the learning principle of humans/animals. A variety of SPL realization schemes have been designed for ...different computer vision and pattern recognition tasks, and empirically demonstrated to be effective in these applications. However, the literature is in lack of the theoretical understanding of SPL. Regarding this research gap, this study attempts to provide some new theoretical understanding of the SPL scheme. Specifically, we prove that the solution strategy on SPL accords with a majorization minimization algorithm implemented on an implicit objective function. Furthermore, we found that the loss function contained in this implicit objective has a similar configuration with the non-convex regularized penalty (NCRP) known in statistics and machine learning. Such connection inspires us to discover more intrinsic relationships between the SPL regimes and the NCRP forms, like smoothly clipped absolute deviation (SCAD), logarithmic penalty (LOG) and non-convex exponential penalty (EXP). The insight of the robustness under SPL can then be finely explained. We also analyze the capability of SPL regarding its easy loss-prior-embedding property, and provide an insightful interpretation of the effectiveness mechanism under current SPL variations. Moreover, we design a group-partial-order loss prior, which is especially useful for weakly labeled large-scale data processing tasks. By applying SPL with this loss prior to the FCVID dataset, which is currently one of the largest manually annotated video dataset, our method achieves state-of-the-art performance above existing methods, which further supports the proposed theoretical arguments.
Sound event detection (SED) is the task of tagging the absence or presence of audio events and their corresponding interval within a given audio clip. While SED can be done using supervised machine ...learning, where training data is fully labeled with access to per event timestamps and duration, our work focuses on weakly-supervised sound event detection (WSSED), where prior knowledge about an event's duration is unavailable. Recent research within the field focuses on improving segmentand eventlevel localization performance for specific datasets regarding specific evaluation metrics. Specifically, well-performing event-level localization requires fully labeled development subsets to obtain event duration estimates, which significantly benefits localization performance. Moreover, well-performing segment-level localization models output predictions at a coarse-scale (e.g.,1 second), hindering their deployment on datasets containing very short events (<; 1second). This work proposes a duration robust CRNN (CDur) framework, which aims to achieve competitive performance in terms of segmentand event-level localization. This paper proposes a new post-processing strategy named "Triple Threshold" and investigates two data augmentation methods along with a label smoothing method within the scope of WSSED. Evaluation of our model is done on the DCASE2017 and 2018 Task 4 datasets, and URBAN-SED. Our model outperforms other approaches on the DCASE2018 and URBAN-SED datasets without requiring prior duration knowledge. In particular, our model is capable of similar performance to strongly-labeled supervised models on the URBANSED dataset. Lastly, ablation experiments to reveal that without post-processing, our model's localization performance drop is significantly lower compared with other approaches.
For intelligent systems to make best use of the audio modality, it is important that they can recognize not just speech and music, which have been researched as specific tasks, but also general ...sounds in everyday environments. To stimulate research in this field we conducted a public research challenge: the IEEE Audio and Acoustic Signal Processing Technical Committee challenge on Detection and Classification of Acoustic Scenes and Events (DCASE). In this paper, we report on the state of the art in automatically classifying audio scenes, and automatically detecting and classifying audio events. We survey prior work as well as the state of the art represented by the submissions to the challenge from various research groups. We also provide detail on the organization of the challenge, so that our experience as challenge hosts may be useful to those organizing challenges in similar domains. We created new audio datasets and baseline systems for the challenge; these, as well as some submitted systems, are publicly available under open licenses, to serve as benchmarks for further research in general-purpose machine listening.
Abnormal event detection aims to identify the events that deviate from expected normal patterns. Existing methods usually extract normal spatio-temporal patterns of appearance and motion in a ...separate manner, which ignores low-level correlations between appearance and motion patterns and may fall short of capturing fine-grained spatio-temporal patterns. In this paper, we propose to simultaneously learn appearance and motion to obtain fine-grained spatio-temporal patterns. To this end, we present an adversarial 3D convolutional auto-encoder to learn the normal spatio-temporal patterns and then identify abnormal events by diverging them from the learned normal patterns in videos. The encoder captures the low-level correlations between spatial and temporal dimensions of videos, and generates distinctive features representing visual spatio-temporal information. The decoder reconstrucccts the original video from the encoded features representing by 3D de-convolutions and learns the normal spatio-temporal patterns in an unsupervised manner. We introduce the denoising reconstruction error and adversarial learning strategy to train the 3D convolutional auto-encoder to implicitly learn accurate data distributions that are considered normal patterns, which benefits enhancing the reconstruction ability of the auto-encoder to discriminate abnormal events. Both the theoretical analysis and the extensive experiments on four publicly available datasets demonstrate the effectiveness of our method.
PDF
