This paper proposes a novel scheme for reduced-rank Gaussian process regression. The method is based on an approximate series expansion of the covariance function in terms of an eigenfunction ...expansion of the Laplace operator in a compact subset of
R
d
. On this approximate eigenbasis, the eigenvalues of the covariance function can be expressed as simple functions of the spectral density of the Gaussian process, which allows the GP inference to be solved under a computational cost scaling as
O
(
n
m
2
)
(initial) and
O
(
m
3
)
(hyperparameter learning) with
m
basis functions and
n
data points. Furthermore, the basis functions are independent of the parameters of the covariance function, which allows for very fast hyperparameter learning. The approach also allows for rigorous error analysis with Hilbert space theory, and we show that the approximation becomes exact when the size of the compact subset and the number of eigenfunctions go to infinity. We also show that the convergence rate of the truncation error is independent of the input dimensionality provided that the differentiability order of the covariance function increases appropriately, and for the squared exponential covariance function it is always bounded by
∼
1
/
m
regardless of the input dimensionality. The expansion generalizes to Hilbert spaces with an inner product which is defined as an integral over a specified input density. The method is compared to previously proposed methods theoretically and through empirical tests with simulated and real data.
This paper considers the application of the unscented Kalman filter (UKF) to continuous-time filtering problems, where both the state and measurement processes are modeled as stochastic differential ...equations. The mean and covariance differential equations which result in the continuous-time limit of the UKF are derived. The continuous-discrete UKF is derived as a special case of the continuous-time filter, when the continuous-time prediction equations are combined with the update step of the discrete-time UKF. The filter equations are also transformed into sigma-point differential equations, which can be interpreted as matrix square root versions of the filter equations.
Statistical signal processing applications usually require the estimation of some parameters of interest given a set of observed data. These estimates are typically obtained either by solving a ...multi-variate optimization problem, as in the maximum likelihood (ML) or maximum a posteriori (MAP) estimators, or by performing a multi-dimensional integration, as in the minimum mean squared error (MMSE) estimators. Unfortunately, analytical expressions for these estimators cannot be found in most real-world applications, and the Monte Carlo (MC) methodology is one feasible approach. MC methods proceed by drawing random samples, either from the desired distribution or from a simpler one, and using them to compute consistent estimators. The most important families of MC algorithms are the Markov chain MC (MCMC) and importance sampling (IS). On the one hand, MCMC methods draw samples from a proposal density, building then an ergodic Markov chain whose stationary distribution is the desired distribution by accepting or rejecting those candidate samples as the new state of the chain. On the other hand, IS techniques draw samples from a simple proposal density and then assign them suitable weights that measure their quality in some appropriate way. In this paper, we perform a thorough review of MC methods for the estimation of static parameters in signal processing applications. A historical note on the development of MC schemes is also provided, followed by the basic MC method and a brief description of the rejection sampling (RS) algorithm, as well as three sections describing many of the most relevant MCMC and IS algorithms, and their combined use. Finally, five numerical examples (including the estimation of the parameters of a chaotic system, a localization problem in wireless sensor networks and a spectral analysis application) are provided in order to demonstrate the performance of the described approaches.
This note considers the application of the unscented transform to optimal smoothing of nonlinear state-space models. In this note, a new Rauch-Tung-Striebel type form of the fixed-interval unscented ...Kalman smoother is derived. The new smoother differs from the previously proposed two-filter-formulation-based unscented Kalman smoother in the sense that it is not based on running two independent filters forward and backward in time. Instead, a separate backward smoothing pass is used, which recursively computes corrections to the forward filtering result. The smoother equations are derived as approximations to the formal Bayesian optimal smoothing equations. The performance of the new smoother is demonstrated with a simulation.
This article considers the application of variational Bayesian methods to joint recursive estimation of the dynamic state and the time-varying measurement noise parameters in linear state space ...models. The proposed adaptive Kalman filtering method is based on forming a separable variational approximation to the joint posterior distribution of states and noise parameters on each time step separately. The result is a recursive algorithm, where on each step the state is estimated with Kalman filter and the sufficient statistics of the noise variances are estimated with a fixed-point iteration. The performance of the algorithm is demonstrated with simulated data.
This article is concerned with Gaussian filtering in nonlinear continuous-discrete state-space models. We propose a novel Taylor moment expansion (TME) Gaussian filter, which approximates the moments ...of the stochastic differential equation with a temporal Taylor expansion. Differently from classical linearization or Itô-Taylor approaches, the Taylor expansion is formed for the moment functions directly and in time variable, not by using a Taylor expansion on the nonlinear functions in the model. We analyze the theoretical properties, including the positive definiteness of the covariance estimate and stability of the TME filter. By numerical experiments, we demonstrate that the proposed TME Gaussian filter significantly outperforms the state-of-the-art methods in terms of estimation accuracy and numerical stability.
This article proposes a Rao-Blackwellized particle filter (RBPF) for fully mixing state-space models that replace the Kalman filter within the RBPF method with a noise-adaptive Kalman filter. This ...extension aims to deal with unknown time-varying measurement variances. Consequently, a variational Bayesian (VB) adaptive Kalman filter estimates the conditionally linear states and the measurement noise variances whereas the nonlinear (or latent) states are handled by sequential Monte Carlo (MC) sampling. Thus, by modifying the underlying mathematical framework of RBPF, we construct the Monte Carlo Variational Bayesian (MCVB) filter. A stopping criterion for VB approximations is proposed by employing Tikhonov regularization. Additionally, an analysis of the numerical stability of the proposed filtering mechanism is presented. The performance of the MCVB filter is illustrated in simulations and mobile robot tracking experiments in the presence of measurement model uncertainties.
In this paper, we propose a new framework for solving state estimation problems with an additional sparsity-promoting L 1 -regularizer term. We first formulate such problems as minimization of the ...sum of linear or nonlinear quadratic error terms and an extra regularizer, and then present novel algorithms which solve the linear and nonlinear cases. The methods are based on a combination of the iterated extended Kalman smoother and variable splitting techniques such as alternating direction method of multipliers (ADMM). We present a general algorithmic framework for variable splitting methods, where the iterative steps involving minimization of the nonlinear quadratic terms can be computed efficiently by iterated smoothing. Due to the use of state estimation algorithms, the proposed framework has a low per-iteration time complexity, which makes it suitable for solving a large-scale or high-dimensional state estimation problem. We also provide convergence results for the proposed algorithms. The experiments show the promising performance and speed-ups provided by the methods.
Robust and accurate localization is a basic requirement for mobile autonomous systems. Pole-like objects, such as traffic signs, poles, and lamps are frequently used landmarks for localization in ...urban environments due to their local distinctiveness and long-term stability. In this paper, we present a novel, accurate, and fast pole extraction approach based on geometric features that runs online and has little computational demands. Our method performs all computations directly on range images generated from 3D LiDAR scans, which avoids processing 3D point clouds explicitly and enables fast pole extraction for each scan. We further use the extracted poles as pseudo labels to train a deep neural network for online range image-based pole segmentation. We test both our geometric and learning-based pole extraction methods for localization on different datasets with different LiDAR scanners, routes, and seasonal changes. The experimental results show that our methods outperform other state-of-the-art approaches. Moreover, boosted with pseudo pole labels extracted from multiple datasets, our learning-based method can run across different datasets and achieve even better localization results compared to our geometry-based method. We released our pole datasets to the public for evaluating the performance of pole extractors, as well as the implementation of our approach.
PDF
