Although maximum mean discrepancy (MMD) has achieved great success in unsupervised domain adaptation (UDA), most of existing UDA methods ignore the issue of class weight bias across domains, which is ...ubiquitous and evidently gives rise to the degradation of UDA performance. In this work, we propose two improved MMD metrics, i.e., weighted MMD (WMMD) and class-specific MMD (CMMD), to alleviate the adverse effect caused by the changes of class prior distributions between source and target domains. In WMMD, class-specific auxiliary weights are deployed to reweigh the source samples. In CMMD, we calculate the MMD for each class of source and target samples. Since the class labels of target samples are unknown for UDA problem, we present a classification expectation-maximization algorithm to estimate the pseudo-labels of target samples on the fly and update the model parameters using estimated labels. The proposed methods can be flexibly incorporated into deep convolutional neural networks to form WMMD and CMMD based domain adaptation networks, which we called WDAN and CDAN, respectively. By combining WMMD with CMMD, we present a CWMMD based domain adaptation network (CWDAN) to further improve classification performance. Experiments show that, both WMMD and CMMD benefit the classification accuracy, and our CWDAN can achieve compelling UDA performance in comparison with MMD and the state-of-the-art UDA methods.
In this paper, we present an algorithm for the sparse signal recovery problem that incorporates damped Gaussian generalized approximate message passing (GGAMP) into expectation-maximization-based ...sparse Bayesian learning (SBL). In particular, GGAMP is used to implement the E-step in SBL in place of matrix inversion, leveraging the fact that GGAMP is guaranteed to converge with appropriate damping. The resulting GGAMP-SBL algorithm is much more robust to arbitrary measurement matrix A than the standard damped GAMP algorithm while being much lower complexity than the standard SBL algorithm. We then extend the approach from the single measurement vector case to the temporally correlated multiple measurement vector case, leading to the GGAMP-TSBL algorithm. We verify the robustness and computational advantages of the proposed algorithms through numerical experiments.
When recovering a sparse signal from noisy compressive linear measurements, the distribution of the signal's non-zero coefficients can have a profound effect on recovery mean-squared error (MSE). If ...this distribution was a priori known, then one could use computationally efficient approximate message passing (AMP) techniques for nearly minimum MSE (MMSE) recovery. In practice, however, the distribution is unknown, motivating the use of robust algorithms like LASSO-which is nearly minimax optimal-at the cost of significantly larger MSE for non-least-favorable distributions. As an alternative, we propose an empirical-Bayesian technique that simultaneously learns the signal distribution while MMSE-recovering the signal-according to the learned distribution-using AMP. In particular, we model the non-zero distribution as a Gaussian mixture and learn its parameters through expectation maximization, using AMP to implement the expectation step. Numerical experiments on a wide range of signal classes confirm the state-of-the-art performance of our approach, in both reconstruction error and runtime, in the high-dimensional regime, for most (but not all) sensing operators.
Specific optimization algorithms have been developed for the purpose of automated software reliability assessment tools. In this article, we propose the Monte Carlo expectation-maximization algorithm ...as another optimization algorithm, and carry out the performance comparison of the software reliability estimation algorithms through comprehensive numerical experiments.
Estimating and predicting traffic conditions in arterial networks using probe data has proven to be a substantial challenge. Sparse probe data represent the vast majority of the data available on ...arterial roads. This paper proposes a probabilistic modeling framework for estimating and predicting arterial travel-time distributions using sparsely observed probe vehicles. We introduce a model based on hydrodynamic traffic theory to learn the density of vehicles on arterial road segments, illustrating the distribution of delay within a road segment. The characterization of this distribution is essentially to use probe vehicles for traffic estimation: Probe vehicles report their location at random locations, and the travel times between location reports must be properly scaled to match the map discretization. A dynamic Bayesian network represents the spatiotemporal dependence on the network and provides a flexible framework to learn traffic dynamics from historical data and to perform real-time estimation with streaming data. The model is evaluated using data from a fleet of 500 probe vehicles in San Francisco, CA, which send Global Positioning System (GPS) data to our server every minute. The numerical experiments analyze the learning and estimation capabilities on a subnetwork with more than 800 links. The sampling rate of the probe vehicles does not provide detailed information about the location where vehicles encountered delay or the reason for any delay (i.e., signal delay, congestion delay, etc.). The model provides an increase in estimation accuracy of 35% when compared with a baseline approach to process probe-vehicle data.
Statistical shape models have been extensively used in a wide range of applications due to their effectiveness in providing prior shape information for object segmentation problems. The most popular ...method is the active shape model (ASM). However, accurately fitting the shape model to an object boundary under a cluttered environment is a challenging task. Under such assumptions, the model is often attracted toward invalid observations (outliers), leading to meaningless estimates of the object boundary. In this paper, we propose a novel algorithm that improves the robustness of ASM in the presence of outliers. The proposed framework assumes that both type of observations (valid observations and outliers) are detected in the image. A new strategy is devised for treating the data in different ways, depending on the observations being considered as valid or invalid. The proposed algorithm assigns a different weight to each observation. The shape parameters are recursively updated using the expectation-maximization method, allowing a correct and robust fit of the shape model to the object boundary in the image. Two estimation criteria are considered: 1) the maximum likelihood criterion and 2) the maximum a posteriori criterion that use priors for the unknown parameters. The methods are tested with synthetic and real images, comprising medical images of the heart and image sequences of the lips. The results are promising and show that this approach is robust in the presence of outliers, leading to a significant improvement over the standard ASM and other state-of-the-art methods.
Measurement error is a crucial factor that determines the accuracy of state estimation (SE). Conventional estimators have fixed models, and can yield optimal performance only when the measurement ...error statistics exactly meet the assumptions. In reality, however, the error distribution is usually unknown and time-varying, resulting in suboptimal state estimates in most cases. This paper develops the concept of adaptive SE for power systems measured by phasor measurement units (PMUs). First, the Gaussian-Laplacian Mixture (GLM) model is developed to fit the body and tail of unknown measurement error distributions. Then, an adaptive estimation framework is proposed based on the Expectation-Maximization (EM) algorithm. It is capable of tracking the actual error statistics online, and adjusting the parameters of SE to maintain near-optimality of state estimates under complex measurement error conditions. Simulation results demonstrate that the proposed adaptive estimator yields more accurate state estimates than the well-known Weighted Least Squares (WLS) and Weighted Least Absolute Value (WLAV) estimators by adapting itself to the variations of measurement error statistics.
In this work the dynamic compressive sensing (CS) problem of recovering sparse, correlated, time-varying signals from sub-Nyquist, non-adaptive, linear measurements is explored from a Bayesian ...perspective. While there has been a handful of previously proposed Bayesian dynamic CS algorithms in the literature, the ability to perform inference on high-dimensional problems in a computationally efficient manner remains elusive. In response, we propose a probabilistic dynamic CS signal model that captures both amplitude and support correlation structure, and describe an approximate message passing algorithm that performs soft signal estimation and support detection with a computational complexity that is linear in all problem dimensions. The algorithm, DCS-AMP, can perform either causal filtering or non-causal smoothing, and is capable of learning model parameters adaptively from the data through an expectation-maximization learning procedure. We provide numerical evidence that DCS-AMP performs within 3 dB of oracle bounds on synthetic data under a variety of operating conditions. We further describe the result of applying DCS-AMP to two real dynamic CS datasets, as well as a frequency estimation task, to bolster our claim that DCS-AMP is capable of offering state-of-the-art performance and speed on real-world high-dimensional problems.
There has been a recent surge of interest in the study of asymptotic reconstruction performance in various cases of generalized linear estimation problems in the teacher-student setting, especially ...for the case of i.i.d standard normal matrices. Here, we go beyond these matrices, and prove an analytical formula for the reconstruction performance of convex generalized linear models with rotationally-invariant data matrices with arbitrary bounded spectrum, rigorously confirming, under suitable assumptions, a conjecture originally derived using the replica method from statistical physics. The proof is achieved by leveraging on message passing algorithms and the statistical properties of their iterates, allowing to characterize the asymptotic empirical distribution of the estimator. For sufficiently strongly convex problems, we show that the two-layer vector approximate message passing algorithm (2-MLVAMP) converges, where the convergence analysis is done by checking the stability of an equivalent dynamical system, which gives the result for such problems. We then show that, under a concentration assumption, an analytical continuation may be carried out to extend the result to convex (non-strongly) problems. We illustrate our claim with numerical examples on mainstream learning methods such as sparse logistic regression and linear support vector classifiers, showing excellent agreement between moderate size simulation and the asymptotic prediction.
Dynamic statistical process monitoring methods have been widely studied and applied in modern industrial processes. These methods aim to extract the most predictable temporal information and develop ...the corresponding dynamic monitoring schemes. However, measurement noise is widespread in real-world industrial processes, and ignoring its effect will lead to suboptimal modeling and monitoring performance. In this article, a probabilistic predictable feature analysis (PPFA) is proposed for multivariate time series modeling, and a multistep dynamic predictive monitoring scheme is developed. The model parameters are estimated with an efficient expectation-maximization algorithm, where the genetic algorithm and the Kalman filter are designed and incorporated. Furthermore, a novel dynamic statistical monitoring index, the dynamic index, is proposed as an important supplement of T2 and SPE to detect dynamic anomalies. The effectiveness of the proposed algorithm is demonstrated via its application on the three-phase flow facility and a medium-speed coal mill.
PDF
