Statistical signal processing applications usually require the estimation of some parameters of interest given a set of observed data. These estimates are typically obtained either by solving a ...multi-variate optimization problem, as in the maximum likelihood (ML) or maximum a posteriori (MAP) estimators, or by performing a multi-dimensional integration, as in the minimum mean squared error (MMSE) estimators. Unfortunately, analytical expressions for these estimators cannot be found in most real-world applications, and the Monte Carlo (MC) methodology is one feasible approach. MC methods proceed by drawing random samples, either from the desired distribution or from a simpler one, and using them to compute consistent estimators. The most important families of MC algorithms are the Markov chain MC (MCMC) and importance sampling (IS). On the one hand, MCMC methods draw samples from a proposal density, building then an ergodic Markov chain whose stationary distribution is the desired distribution by accepting or rejecting those candidate samples as the new state of the chain. On the other hand, IS techniques draw samples from a simple proposal density and then assign them suitable weights that measure their quality in some appropriate way. In this paper, we perform a thorough review of MC methods for the estimation of static parameters in signal processing applications. A historical note on the development of MC schemes is also provided, followed by the basic MC method and a brief description of the rejection sampling (RS) algorithm, as well as three sections describing many of the most relevant MCMC and IS algorithms, and their combined use. Finally, five numerical examples (including the estimation of the parameters of a chaotic system, a localization problem in wireless sensor networks and a spectral analysis application) are provided in order to demonstrate the performance of the described approaches.
Multi-output inference tasks, such as multi-label classification, have become increasingly important in recent years. A popular method for multi-label classification is classifier chains, in which ...the predictions of individual classifiers are cascaded along a chain, thus taking into account inter-label dependencies and improving the overall performance. Several varieties of classifier chain methods have been introduced, and many of them perform very competitively across a wide range of benchmark datasets. However, scalability limitations become apparent on larger datasets when modelling a fully cascaded chain. In particular, the methods׳ strategies for discovering and modelling a good chain structure constitute a major computational bottleneck. In this paper, we present the classifier trellis (CT) method for scalable multi-label classification. We compare CT with several recently proposed classifier chain methods to show that it occupies an important niche: it is highly competitive on standard multi-label problems, yet it can also scale up to thousands or even tens of thousands of labels.
•A study of multi-output classification as graphical models.•An empirical comparison of existing strategies for modelling dependency among outputs.•A novel scalable approach based on a hill climbing heuristic: the classifier trellis.•An empirical cross-fold comparison with other methods.•A connection to structured output prediction and a comparison in a segmentation task.
Adaptive independent sticky MCMC algorithms Martino, Luca; Casarin, Roberto; Leisen, Fabrizio ...
EURASIP journal on advances in signal processing,
01/2018, Letnik:
2018, Številka:
1
Journal Article
Recenzirano
Odprti dostop
Monte Carlo methods have become essential tools to solve complex Bayesian inference problems in different fields, such as computational statistics, machine learning, and statistical signal ...processing. In this work, we introduce a novel class of adaptive Monte Carlo methods, called adaptive independent sticky Markov Chain Monte Carlo (MCMC) algorithms, to sample efficiently from any bounded target probability density function (pdf). The new class of algorithms employs adaptive non-parametric proposal densities, which become closer and closer to the target as the number of iterations increases. The proposal pdf is built using interpolation procedures based on a set of support points which is constructed iteratively from previously drawn samples. The algorithm’s efficiency is ensured by a test that supervises the evolution of the set of support points. This extra stage controls the computational cost and the convergence of the proposal density to the target. Each part of the novel family of algorithms is discussed and several examples of specific methods are provided. Although the novel algorithms are presented for univariate target densities, we show how they can be easily extended to the multivariate context by embedding them within a Gibbs-type sampler or the hit and run algorithm. The ergodicity is ensured and discussed. An overview of the related works in the literature is also provided, emphasizing that several well-known existing methods (like the adaptive rejection Metropolis sampling (ARMS) scheme) are encompassed by the new class of algorithms proposed here. Eight numerical examples (including the inference of the hyper-parameters of Gaussian processes, widely used in machine learning for signal processing applications) illustrate the efficiency of sticky schemes, both as stand-alone methods to sample from complicated one-dimensional pdfs and within Gibbs samplers in order to draw from multi-dimensional target distributions.
The correct identification of individuals through different biometric traits is becoming increasingly important. Apart from traditional biomarkers (like fingerprints), many alternative measures have ...been proposed during the last two decades: electrocardiogram (ECG) and electroencephalogram (EEG) signals, iris or facial recognition, conductual traits, etc. Several works have shown that ECG-based recognition is a feasible alternative, either for stand-alone or multi-biometric recognition systems. In this paper, we propose a novel framework for ECG-based biometric identification, consisting of a simple and robust feature extraction approach and a clustering-based feature reduction method, that enables for an efficient and scalable biometric identification. The proposed feature reduction approach is a two phase method: it uses a clustering algorithm to group features according to their similarities first, and then clusters are represented in terms of a prototype vector and associated to the available subjects. On its side, the proposed time-domain feature extraction method is a semi-fiducial procedure, where the well-known Pan–Tompkins algorithm is first used to detect the R wave peaks of the QRS complexes, and then fixed-width time segments are selected for further dimensionality reduction and feature extraction. The resulting combined methods are efficient, robust, scalable and attain excellent results (with up-to 98.6% sensitivity) on all the subjects of the Physikalisch-Technische Bundesanstalt (PTB) database, regardless of their pathological or healthy status. Additionally, we also show how the existing Auto Correlation/Discrete Cosine Transform (AC/DCT)-based non-fiducial feature extraction method can be integrated within our framework, allowing us to attain up to 90.6% sensitivity on the Massachusetts Institute of Technology-Beth Israel Hospital (MIT-BIH) arrhythmia database. Since this database is much noisier and has a much lower sampling rate (360 Hz instead of 1000 Hz), we claim that this is a very good result.
•We propose a novel efficient framework for ECG-based biometric identification.•A clustering-based classifier is used to reduce the computational/storage cost.•Hierarchical agglomerative clustering (HAC) is used to build the clusters.•A novel semi-fiducial time-domain feature extraction method is proposed.•Statistical analysis of P-QRS-T complexes is performed on MIT-BIH arrhythmia and PTB.
The electrocardiogram (ECG) was the first biomedical signal for which digital signal processing techniques were extensively applied. By its own nature, the ECG is typically a sparse signal, composed ...of regular activations (QRS complexes and other waveforms, such as the P and T waves) and periods of inactivity (corresponding to isoelectric intervals, such as the PQ or ST segments), plus noise and interferences. In this work, we describe an efficient method to construct an overcomplete and multi-scale dictionary for sparse ECG representation using waveforms recorded from real-world patients. Unlike most existing methods (which require multiple alternative iterations of the dictionary learning and sparse representation stages), the proposed approach learns the dictionary first, and then applies a fast sparse inference algorithm to model the signal using the constructed dictionary. As a result, our method is much more efficient from a computational point of view than other existing algorithms, thus becoming amenable to dealing with long recordings from multiple patients. Regarding the dictionary construction, we located first all the QRS complexes in the training database, then we computed a single average waveform per patient, and finally we selected the most representative waveforms (using a correlation-based approach) as the basic atoms that were resampled to construct the multi-scale dictionary. Simulations on real-world records from Physionet’s PTB database show the good performance of the proposed approach.
There are numerous articles that study the ground reaction forces during the golf swing, among which only a few analyze the pressure pattern distributed on the entire surface of the foot. The current ...study compares the pressure patterns on the foot insoles of fifty-five golfers, from three different performance levels, playing swings with driver and 5-iron clubs in the driving range. Five swings were selected for each club. During each swing, ultra-thin insole sensors (4 sensors/cm2) measure foot pressure at the frequency of 100 Hz. To perform statistical analysis, insole sensors are clustered to form seven areas, with the normalized pressure of each area being our dependent variable. A video camera was used to label the five key instants of the swing. Statistical analysis demonstrates a significant difference between the pressure distribution pattern of the left and right feet for both driver and 5-iron. However, the pressure distribution pattern remains almost the same when switching the club type from 5-iron to driver. We have also observed that there are significant differences between the pattern of professionals and players with medium and high handicap. The obtained pattern agrees with the principle of weight transfer with a different behavior between the medial and lateral areas of the foot.
The processing and analysis of multichannel signals which exhibit some synchronicity is of critical importance in biomedical signal processing. Though sparse reconstruction of such signals is well ...known, penalties enforcing both sparsity and partial synchronicity between channels have been less investigated. In this paper, we present an algorithm based on overlapping grouped LASSO to guarantee both sparsity and synchronicity for multichannel signals. We show that the proposed method can be implemented efficiently, using consensus learning approaches and recent work of the authors for accelerating the related optimization procedure. Results on synthetic data show the efficiency of the proposed approach. When applied on real intracardiac recordings, the proposed method succeeds in automatically detecting electrical pulses both for sinus rhythm and atrial fibrillation episodes.
•A new method for pulse detection in intracardiac multichannel ECG recordings is proposed.•Pulse localization is performed using Overlapping Group LASSO (OGL) with a consensus post-processing.•A fast implementation of the proposed method for a dictionary of translated waveforms is developed.•A novel criterion to assess the validity of the estimated arrival times is introduced.•Results on real data illustrates the effectiveness of the proposed method, both in cases of sinus rhythm and atrial fibrillation.
Multi-dimensional classification (MDC) is the supervised learning problem where an instance is associated with multiple classes, rather than with a single class, as in traditional classification ...problems. Since these classes are often strongly correlated, modeling the dependencies between them allows MDC methods to improve their performance – at the expense of an increased computational cost. In this paper we focus on the classifier chains (CC) approach for modeling dependencies, one of the most popular and highest-performing methods for multi-label classification (MLC), a particular case of MDC which involves only binary classes (i.e., labels). The original CC algorithm makes a greedy approximation, and is fast but tends to propagate errors along the chain. Here we present novel Monte Carlo schemes, both for finding a good chain sequence and performing efficient inference. Our algorithms remain tractable for high-dimensional data sets and obtain the best predictive performance across several real data sets.
•A Monte Carlo approach for efficient classifier chains.•Applied to learning from multi-label and multi-dimensional data.•A theoretical and empirical study of payoff functions in the search space.•An empirical cross-fold comparison with PCC and other related methods.
Bayesian methods have become very popular in signal processing lately, even though performing exact Bayesian inference is often unfeasible due to the lack of analytical expressions for optimal ...Bayesian estimators. In order to overcome this problem, Monte Carlo (MC) techniques are frequently used. Several classes of MC schemes have been developed, including Markov Chain Monte Carlo (MCMC) methods, particle filters and population Monte Carlo approaches. In this paper, we concentrate on the Gibbs-type approach, where automatic and fast samplers are needed to draw from univariate (full-conditional) densities. The Adaptive Rejection Metropolis Sampling (ARMS) technique is widely used within Gibbs sampling, but suffers from an important drawback: an incomplete adaptation of the proposal in some cases. In this work, we propose an alternative adaptive MCMC algorithm (IA 2 RMS) that overcomes this limitation, speeding up the convergence of the chain to the target, allowing us to simplify the construction of the sequence of proposals, and thus reducing the computational cost of the entire algorithm. Note that, although IA 2 RMS has been developed as an extremely efficient MCMC-within-Gibbs sampler, it also provides an excellent performance as a stand-alone algorithm when sampling from univariate distributions. In this case, the convergence of the proposal to the target is proved and a bound on the complexity of the proposal is provided. Numerical results, both for univariate (stand-alone IA 2 RMS) and multivariate (IA 2 RMS-within-Gibbs) distributions, show that IA 2 RMS outperforms ARMS and other classical techniques, providing a correlation among samples close to zero.
PDF
