Functional Bayesian Filter Li, Kan; Principe, Jose C.
IEEE transactions on signal processing,
2022, Letnik:
70
Journal Article
Recenzirano
Odprti dostop
We present a general nonlinear Bayesian filter for high-dimensional state estimation using the theory of reproducing kernel Hilbert space (RKHS). By applying the kernel method and the representer ...theorem to perform linear quadratic estimation in a functional space, we derive a Bayesian recursive state estimator for a general nonlinear dynamical system in the original input space. Unlike existing nonlinear extensions of the Kalman filter where the system dynamics are assumed known, the state-space representation for the Functional Bayesian Filter (FBF) is completely learned online from measurement data in the form of an infinite impulse response (IIR) filter or recurrent network in the RKHS, with universal approximation property. Using a positive definite kernel function satisfying Mercer's conditions to compute and evolve information quantities, the FBF exploits both the statistical and time-domain information about the signal, extracts higher-order moments, and preserves the properties of covariances without the ill effects due to conventional arithmetic operations. We apply this novel kernel adaptive filtering (KAF) to recurrent network training, chaotic time-series estimation and cooperative filtering using Gaussian and non-Gaussian noises, and inverse kinematics modeling. Simulation results show FBF outperforms existing Kalman-based algorithms.
To cluster data that are not linearly separable in the original feature space, <inline-formula> <tex-math notation="LaTeX">k </tex-math></inline-formula>-means clustering was extended to the kernel ...version. However, the performance of kernel <inline-formula> <tex-math notation="LaTeX">k </tex-math></inline-formula>-means clustering largely depends on the choice of the kernel function. To mitigate this problem, multiple kernel learning has been introduced into the <inline-formula> <tex-math notation="LaTeX">k </tex-math></inline-formula>-means clustering to obtain an optimal kernel combination for clustering. Despite the success of multiple kernel <inline-formula> <tex-math notation="LaTeX">k </tex-math></inline-formula>-means clustering in various scenarios, few of the existing work update the combination coefficients based on the diversity of kernels, which leads to the result that the selected kernels contain high redundancy and would degrade the clustering performance and efficiency. We resolve this problem from the perspective of subset selection in this article. In particular, we first propose an effective strategy to select a diverse subset from the prespecified kernels as the representative kernels, and then incorporate the subset selection process into the framework of multiple <inline-formula> <tex-math notation="LaTeX">k </tex-math></inline-formula>-means clustering. The representative kernels can be indicated as a significant combination weights. Due to the nonconvexity of the obtained objective function, we develop an alternating minimization method to optimize the combination coefficients of the selected kernels and the cluster membership alternatively. In particular, an efficient optimization method is developed to reduce the time complexity of optimizing the kernel combination weights. Finally, extensive experiments on benchmark and real-world data sets demonstrate the effectiveness and superiority of our approach in comparison with existing methods.
Kernel methods are a class of learning machines for the fast recognition of nonlinear patterns in any data set. In this paper, the applications of kernel methods for feature extraction in industrial ...process monitoring are systematically reviewed. First, we describe the reasons for using kernel methods and contextualize them among other machine learning tools. Second, by reviewing a total of 230 papers, this work has identified 12 major issues surrounding the use of kernel methods for nonlinear feature extraction. Each issue was discussed as to why they are important and how they were addressed through the years by many researchers. We also present a breakdown of the commonly used kernel functions, parameter selection routes, and case studies. Lastly, this review provides an outlook into the future of kernel-based process monitoring, which can hopefully instigate more advanced yet practical solutions in the process industries.
This paper introduces kernel versions of maximum autocorrelation factor (MAF) analysis and minimum noise fraction (MNF) analysis. The kernel versions are based upon a dual formulation also termed ...Q-mode analysis in which the data enter into the analysis via inner products in the Gram matrix only. In the kernel version, the inner products of the original data are replaced by inner products between nonlinear mappings into higher dimensional feature space. Via kernel substitution also known as the kernel trick these inner products between the mappings are in turn replaced by a kernel function and all quantities needed in the analysis are expressed in terms of this kernel function. This means that we need not know the nonlinear mappings explicitly. Kernel principal component analysis (PCA), kernel MAF, and kernel MNF analyses handle nonlinearities by implicitly transforming data into high (even infinite) dimensional feature space via the kernel function and then performing a linear analysis in that space. Three examples show the very successful application of kernel MAF/MNF analysis to: 1) change detection in DLR 3K camera data recorded 0.7 s apart over a busy motorway, 2) change detection in hyperspectral HyMap scanner data covering a small agricultural area, and 3) maize kernel inspection. In the cases shown, the kernel MAF/MNF transformation performs better than its linear counterpart as well as linear and kernel PCA. The leading kernel MAF/MNF variates seem to possess the ability to adapt to even abruptly varying multi and hypervariate backgrounds and focus on extreme observations.
Out-of-focus blur is a common image degradation phenomenon that occurs in case of lens defocusing. The out-of-focus blur kernel is usually modeled as a Gaussian function or a uniform disk in previous ...work. In this paper, we propose that it can be more accurately depicted using the generalized Gaussian (GG) function. This is motivated by the theoretical analysis of the out-of-focus blur and the practical observation of real blur kernels. We show that as the out-of-focus blur kernels are of specific shapes, the GG function can be further simplified to a single-parameter model. We estimate the parameter of the GG blur kernel from image patches containing step edges, and obtain the clear image by non-blind image deblurring. Experimental results validate that the proposed GG blur kernel estimation algorithm outperforms the state-of-the-art ones deploying either parametric (disk and Gaussian) or nonparametric kernels, and consequently benefits the image deblurring process.
Kernel adaptive filters (KAF) are a class of powerful nonlinear filters developed in Reproducing Kernel Hilbert Space (RKHS). The Gaussian kernel is usually the default kernel in KAF algorithms, but ...selecting the proper kernel size (bandwidth) is still an open important issue especially for learning with small sample sizes. In previous research, the kernel size was set manually or estimated in advance by Silverman׳s rule based on the sample distribution. This study aims to develop an online technique for optimizing the kernel size of the kernel least mean square (KLMS) algorithm. A sequential optimization strategy is proposed, and a new algorithm is developed, in which the filter weights and the kernel size are both sequentially updated by stochastic gradient algorithms that minimize the mean square error (MSE). Theoretical results on convergence are also presented. The excellent performance of the new algorithm is confirmed by simulations on static function estimation, short term chaotic time series prediction and real world Internet traffic prediction.
Recognition of facial expressions across various actors, contexts, and recording conditions in real-world videos involves identifying local facial movements. Hence, it is important to discover the ...formation of expressions from local representations captured from different parts of the face. So in this paper, we propose a dynamic kernel-based representation for facial expressions that assimilates facial movements captured using local spatio-temporal representations in a large universal Gaussian mixture model (uGMM). These dynamic kernels are used to preserve local similarities while handling global context changes for the same expression by utilizing the statistics of uGMM. We demonstrate the efficacy of dynamic kernel representation using three different dynamic kernels, namely, explicit mapping based, probability-based, and matching-based, on three standard facial expression datasets, namely, MMI, AFEW, and BP4D. Our evaluations show that probability-based kernels are the most discriminative among the dynamic kernels. However, in terms of computational complexity, intermediate matching kernels are more efficient as compared to the other two representations.
It is well known that the partial least squares (PLS) and its nonlinear extension kernel PLS (KPLS) are efficient tools widely utilized in fault detection. Nonetheless, these two techniques project ...obliquely into the process variable space, limiting their ability to discriminate between the quality-related and quality-unrelated faults. Although various enhanced ways based on the basic PLS and KPLS, such as the total PLS and total KPLS methods, have been published to cope with this problem, these methods fail to reduce the false alarm rates of the quality-unrelated fault when the fault amplitude increases, and they need to monitor four statistics to determine the fault categories. To address this issue, this paper proposes a three-stage data-driven method for complicated nonlinear industrial quality-related process monitoring, consisting of three parts: data preprocessing, modeling, and decomposing. The kernel direct orthogonalization (KDO) approach is initially investigated in this strategy, which is based on the traditional direct orthogonalization methodology and the kernel theory. The unnecessary fluctuations in the process variable space are eliminated after the data is preprocessed using the KDO approach. A standard KPLS model is built for the filtered component. Then, a quality-related KPLS (QRKPLS) method is proposed as the decomposing part to further divide the filtered process mapping matrix into two orthogonal parts, namely, the quality-related part and the quality-unrelated part. Additionally, the corresponding test statistics are calculated for these two parts, and the diagnosis logic is also given in this paper. The effectiveness and superiority of the novel established KDO-QRKPLS method is investigated using a widely numerical simulation as well as an industrial simulation.
PDF
