Combining traditional diversity and re-balancing techniques serves to design effective ensembles for solving imbalanced classification problems. Therefore, to explore the performance of new ...diversification procedures and new re-balancing methods is an attractive research subject which can provide even better performances. In this contribution, we propose to create ensembles of the recently introduced binary Bayesian classifiers, that show intrinsic re-balancing capacities, by means of a diversification mechanism which is based on applying different cost policies to each ensemble learner as well as appropriate aggregation schemes. Experiments with an extensive number of representative imbalanced datasets and their comparison with those of several selected high-performance classifiers show that the proposed approach provides the best overal results.
Missing values in a dataset is one of the most common difficulties in real applications. Many different techniques based on machine learning have been proposed in the literature to face this problem. ...In this work, the great representation capability of the stacked denoising auto-encoders is used to obtain a new method of imputating missing values based on two ideas: deletion and compensation. This method improves imputation performance by artificially deleting values in the input features and using them as targets in the training process. Nevertheless, although the deletion of samples is demonstrated to be really efficient, it may cause an imbalance between the distributions of the training and the test sets. In order to solve this issue, a compensation mechanism is proposed based on a slight modification of the error function to be optimized. Experiments over several datasets show that the deletion and compensation not only involve improvements in imputation but also in classification in comparison with other classical techniques.
This paper presents a novel scheme for nonlinear acoustic echo cancellation based on adaptive Volterra Filters with linear and quadratic kernels, which automatically prefers those diagonals ...contributing most to the output of the quadratic kernel with the goal of minimizing the overall mean-square error. In typical echo cancellation scenarios, not all coefficients will be equally relevant for the modeling of the nonlinear echo, but coefficients close to the main diagonal of the second-order kernel will describe most of the nonlinear echo distortions, such that not all diagonals need to be implemented. However, it is difficult to decide the most appropriate number of diagonals a priori, since there are many factors that influence this decision, such as the energy of the nonlinear echo, the shape of the room impulse response, or the step size used for the adaptation of kernel coefficients. Our proposed scheme incorporates adaptive scaling factors that control the influence of each group of adjacent diagonals contributing to the quadratic kernel output. An appropriate selection of these factors serves to emphasize or neglect diagonals of the model as required by the present situation. We provide adaptation rules for these factors based on previous works on combination of adaptive filters, and comprehensive simulations showing the reduced gradient noise reached by the new echo canceller.
Divisive Gaussian Processes for Nonstationary Regression Munoz-Gonzalez, Luis; Lazaro-Gredilla, Miguel; Figueiras-Vidal, Anibal R.
IEEE transaction on neural networks and learning systems,
11/2014, Letnik:
25, Številka:
11
Journal Article
Standard Gaussian process regression (GPR) assumes constant noise power throughout the input space and stationarity when combined with the squared exponential covariance function. This can be ...unrealistic and too restrictive for many real-world problems. Nonstationarity can be achieved by specific covariance functions, though prior knowledge about this nonstationarity can be difficult to obtain. On the other hand, the homoscedastic assumption is needed to allow GPR inference to be tractable. In this paper, we present a divisive GPR model which performs nonstationary regression under heteroscedastic noise using the pointwise division of two nonparametric latent functions. As the inference on the model is not analytically tractable, we propose a variational posterior approximation using expectation propagation (EP) which allows for accurate inference at reduced cost. We have also made a Markov chain Monte Carlo implementation with elliptical slice sampling to assess the quality of the EP approximation. Experiments support the usefulness of the proposed approach.
The broadcast scheduling problem (BSP) arises in frame design for packet radio networks (PRNs). The frame structure determines the main communication parameters: communication delay and throughput. ...The BSP is a combinatorial optimization problem which is known to be NP-hard. To solve it, we propose an algorithm with two main steps which naturally arise from the problem structure: the first one tackles the hardest contraints and the second one carries out the throughput optimization. This algorithm combines a Hopfield neural network for the constraints satisfaction and a genetic algorithm for achieving a maximal throughput. The algorithm performance is compared with that of existing algorithms in several benchmark cases; in all of them, our algorithm finds the optimum frame length and outperforms previous algorithms in the resulting throughput.
Nonlinear system identification based on support vector machines (SVM) has been usually addressed by means of the standard SVM regression (SVR), which can be seen as an implicit nonlinear ...autoregressive and moving average (ARMA) model in some reproducing kernel Hilbert space (RKHS). The proposal of this letter is twofold. First, the explicit consideration of an ARMA model in an RKHS (SVM-ARMA 2K ) is proposed. We show that stating the ARMA equations in an RKHS leads to solving the regularized normal equations in that RKHS, in terms of the autocorrelation and cross correlation of the (nonlinearly) transformed input and output discrete time processes. Second, a general class of SVM-based system identification nonlinear models is presented, based on the use of composite Mercer's kernels. This general class can improve model flexibility by emphasizing the input-output cross information (SVM-ARMA 4K ), which leads to straightforward and natural combinations of implicit and explicit ARMA models (SVR-ARMA 2K and SVR-ARMA 4K ). Capabilities of these different SVM-based system identification schemes are illustrated with two benchmark problems
Adaptive filtering schemes are subject to different tradeoffs regarding their steady-state misadjustment, speed of convergence, and tracking performance. To alleviate these compromises, a new ...approach has recently been proposed, in which two filters with complementary capabilities adaptively mix their outputs to get an overall filter of improved performance. Following this approach, we propose a new normalized rule for adapting the mixing parameter that controls the combination. The new update rule preserves the good features of the existing scheme and is more robust to changes in the filtering scenario, for instance when the signal-to-noise ratio (SNR) is time varying. The benefits of the normalized scheme are illustrated analytically and with a number of experiments in both stationary and tracking situations.
Real Adaboost ensembles with weighted emphasis (RA-we) on erroneous and critical (near the classification boundary) samples have recently been proposed, leading to improved performance when an ...adequate combination of these terms is selected. However, finding the optimal emphasis adjustment is not an easy task.
In this paper, we propose to make a fusion of the outputs of RA-we ensembles trained with different emphasis adjustments by means of a generalized voting scheme. The resulting committee of RA-we ensembles can retain the performance of the best RA-we component and even, occasionally, can improve it. Additionally, we present an ensemble selection strategy that removes from the committee RA-we ensembles with very poor performance.
Experimental results show that these committees frequently outperform RA and RA-we with cross validated emphasis.
Among all adaptive filtering algorithms, Widrow and Hoff's least mean square (LMS) has probably become the most popular because of its robustness, good tracking properties and simplicity. A drawback ...of LMS is that the step size implies a compromise between speed of convergence and final misadjustment. To combine different speed LMS filters serves to alleviate this compromise, as it was demonstrated by our studies on a two filter combination that we call combination of LMS filters (CLMS). Here, we extend this scheme in two directions. First, we propose a generalization to combine multiple LMS filters with different steps that provides the combination with better tracking capabilities. Second, we use a different mixing parameter for each weight of the filter in order to make independent their adaption speeds. Some simulation examples in plant identification and noise cancellation applications show the validity of the new schemes when compared to the CLMS filter and to other previous variable step approaches.
PDF
