•Stacking/blending models were first employed for daily ETo estimation.•Stacking/blending models were compared with basic and empirical models.•Stacking/blending models had better accuracy and ...portability across stations.•Stacking/blending models had higher accuracy when data or inputs were limited.•Blending models had similar accuracy to stacking models with less time costs.
Precise reference evapotranspiration (ETo) estimation and prediction are the first steps to realize efficient agricultural water resources management. As machine learning methods are widely applied in ETo estimation, we assess whether a high accuracy can be attained by stacking or integrating more models. Can the accuracy be increased indefinitely and at what cost? To this end, this study reports the first evaluation of stacking and blending ensemble models for daily ETo estimation. The stacking and blending models adopted a 2-layer structure: level-0 basic models included random forest (RF), support vector regression (SVR), multilayer perceptron neural network (MLP) and K-Nearest Neighbor regression (KNN); level-1 outputted the final result via linear regression (LR). The accuracy and computational costs of stacking and blending models were compared with those of the 4 basic models and 3 empirical models under 5 complete and limited input conditions. A station-cross validation on models with solar radiation input was further performed to study the portability of the tested models. The results indicated that both stacking and blending models performed better than the basic and empirical models regardless of input combination, and the former (R2 ranged from 0.6602 to 0.9977, with an average AIC of −7785.68) achieved a slightly higher accuracy than the latter models (R2: 0.6562–0.9974; average AIC: −7689.68). Meanwhile, the stacking and blending models were more portable (RMSE ranged from 0.5445 to 0.8799 and 0.5511–0.8767 mm day−1, respectively) than basic models across stations in different climate zones. In terms of computational cost, both stacking and blending models were able to achieve significantly better accuracy than basic models in reasonable time with smaller training data size, while the blending models could obtain similar high accuracy to stacking models in less time after increasing the size of the training data. Therefore, the stacking and blending ensemble models can be highly recommend for ETo estimation, especially when the available training data set or meteorological variables are limited.
The problem of generalizing deep neural networks from multiple source domains to a target one is studied under two settings: When unlabeled target data is available, it is a multi-source unsupervised ...domain adaptation (UDA) problem, otherwise a domain generalization (DG) problem. We propose a unified framework termed domain adaptive ensemble learning (DAEL) to address both problems. A DAEL model is composed of a CNN feature extractor shared across domains and multiple classifier heads each trained to specialize in a particular source domain. Each such classifier is an expert to its own domain but a non-expert to others. DAEL aims to learn these experts collaboratively so that when forming an ensemble, they can leverage complementary information from each other to be more effective for an unseen target domain. To this end, each source domain is used in turn as a pseudo-target-domain with its own expert providing supervisory signal to the ensemble of non-experts learned from the other sources. To deal with unlabeled target data under the UDA setting where real expert does not exist, DAEL uses pseudo labels to supervise the ensemble learning. Extensive experiments on three multi-source UDA datasets and two DG datasets show that DAEL improves the state of the art on both problems, often by significant margins.
Visible thermal person re-identification (VT-ReID) is a challenging cross-modality pedestrian retrieval problem due to the large intra-class variations and modality discrepancy across different ...cameras. Existing VT-ReID methods mainly focus on learning cross-modality sharable feature representations by handling the modality-discrepancy in feature level. However, the modality difference in classifier level has received much less attention, resulting in limited discriminability. In this paper, we propose a novel modality-aware collaborative ensemble (MACE) learning method with middle-level sharable two-stream network (MSTN) for VT-ReID, which handles the modality-discrepancy in both feature level and classifier level. In feature level, MSTN achieves much better performance than existing methods by capturing sharable discriminative middle-level features in convolutional layers. In classifier level, we introduce both modality-specific and modality-sharable identity classifiers for two modalities to handle the modality discrepancy. To utilize the complementary information among different classifiers, we propose an ensemble learning scheme to incorporate the modality sharable classifier and the modality specific classifiers. In addition, we introduce a collaborative learning strategy, which regularizes modality-specific identity predictions and the ensemble outputs. Extensive experiments on two cross-modality datasets demonstrate that the proposed method outperforms current state-of-the-art by a large margin, achieving rank-1/mAP accuracy 51.64%/50.11% on the SYSU-MM01 dataset, and 72.37%/69.09% on the RegDB dataset.
•A GBDT model can be converted into a single decision tree.•The generated tree approximates the accuracy of its source forest.•The developed tree provides interpretable classifications as opposed to ...GBDT.•The generated tree outperforms CARET induced trees in terms of predictive performance.•The complexity of the tree can be configured by the method user.
The increasing usage of machine-learning models in critical domains has recently stressed the necessity of interpretable machine-learning models. In areas like healthcare, finary – the model consumer must understand the rationale behind the model output in order to use it when making a decision. For this reason, it is impossible to use black-box models in these scenarios, regardless of their high predictive performance. Decision forests, and in particular Gradient Boosting Decision Trees (GBDT), are examples of this kind of model. GBDT models are considered the state-of-the-art in many classification challenges, reflected by the fact that the majority of Kaggle’s recent winners used GBDT methods as a part of their solution (such as XGBoost). But despite their superior predictive performance, they cannot be used in tasks that require transparency. This paper presents a novel method for transforming a decision forest of any kind into an interpretable decision tree. The method extends the tool-set available for machine learning practitioners, who want to exploit the interpretability of decision trees without significantly impairing the predictive performance gained by GBDT models like XGBoost. We show in an empirical evaluation that in some cases the generated tree is able to approximate the predictive performance of a XGBoost model while enabling better transparency of the outputs.
Wind energy is one of the sources which is still in development in Brazil. However, it already represents 17% of the National Interconnected System. Due to the high level of uncertainty and ...fluctuations in wind speed, predicting wind energy with high accuracy is challenging. In this context, this paper proposes a novel decomposition-ensemble learning approach that combines Complete Ensemble Empirical Mode Decomposition (CEEMD) and Stacking-ensemble learning (STACK) based on Machine Learning algorithms to forecast the wind energy of a turbine in a wind farm at Parazinho city, Brazil, using multi-step-ahead forecasting strategy. The approached forecasting models were k-Nearest Neighbors, Partial Least Squares Regression, Ridge Regression, Support Vector Regression, and Cubist Regression. Additionally, Box-Cox transformation, correlation matrix, and principal component analysis were used to pre-process the data. The performance of the proposed forecasting models was evaluated by using three performance metrics: mean absolute error, mean absolute percentage error, and root mean square error, and the Diebold-Mariano statistical test to evaluate the forecasting error signals. The proposed models outperform the CEEMD, STACK, and single models in all forecasting horizons, with a performance improvement that ranges 0.06%–97.53%. Indeed, the decomposition-ensemble learning model is an efficient and accurate model for wind energy forecasting.
Display omitted
•A novel decomposition-ensemble learning model is proposed for wind energy forecasting.•CEEMD method deal with the non-linearity and non-stationarity of the time series.•Different preprocessing approaches are employed to deal with the high-correlation of the system’s inputs.•STACK approach by divide-and-conquer scheme takes advantages of the models.•Proposed model improves the accuracy of wind energy forecasting multi-step ahead.
•We propose a deep learning-based capacity estimation method.•Our method incorporates the concepts of transfer learning and ensemble learning.•Transfer learning reduces the data-collection efforts by ...improving the learning performance.•Ensemble learning improves the accuracy and robustness in capacity estimation.•Cycling data from implantable and 18,650 cells are used to demonstrate the improvements.
It is often difficult for a machine learning model trained based on a small size of charge/discharge cycling data to produce satisfactory accuracy in the capacity estimation of lithium-ion (Li-ion) rechargeable batteries. However, in real-world applications, collecting long-term cycling data is a costly and time-consuming process. To overcome this difficulty, we propose a deep learning-based capacity estimation method that incorporates the concepts of transfer learning and ensemble learning. We target the applications where only a relatively small set of training data is available. Transfer learning is a knowledge learning method that leverages the knowledge learned from a source task to improve learning in a related but different target task. Ensemble learning can reduce the risk of choosing a learning algorithm with poor performance by combining prediction results from multiple learning algorithms. In this study, 10-year daily cycling data from eight implantable Li-ion cells is first used as the source dataset to pre-train eight deep convolutional neural network (DCNN) models. The learned parameters of the pre-trained DCNN models are then transferred from the source task to the target task, resulting in eight DCNN with transfer learning (DCNN-TL) models, respectively. These DCNN-TL models are then integrated to build an ensemble model called the DCNN with ensemble learning and transfer learning (DCNN-ETL). The effectiveness of the DCNN-ETL model is verified using a target dataset consisting of 20 commercial 18650 Li-ion cells, and the performance of the model on the target dataset is compared with that of five other data-driven methods including random forest regression, Gaussian process regression, DCNN, DCNN-TL, and DCNN-EL. The verification and comparison results demonstrate that the proposed DCNN-ETL method can produce a higher accuracy and robustness than these other data-driven methods in estimating the capacities of the Li-ion cells in the target task.
Uncertainty quantification (UQ) methods play a pivotal role in reducing the impact of uncertainties during both optimization and decision making processes. They have been applied to solve a variety ...of real-world problems in science and engineering. Bayesian approximation and ensemble learning techniques are two widely-used types of uncertainty quantification (UQ) methods. In this regard, researchers have proposed different UQ methods and examined their performance in a variety of applications such as computer vision (e.g., self-driving cars and object detection), image processing (e.g., image restoration), medical image analysis (e.g., medical image classification and segmentation), natural language processing (e.g., text classification, social media texts and recidivism risk-scoring), bioinformatics, etc. This study reviews recent advances in UQ methods used in deep learning, investigates the application of these methods in reinforcement learning, and highlights fundamental research challenges and directions associated with UQ.
•We provided an extensive review of uncertainty quantification methods in deep learning.•We covered popular and efficient Bayesian approaches for uncertainty quantification.•We listed notable ensemble techniques for quantifying uncertainty.•We discussed various applications of uncertainty quantification methods.•We summarized major open challenges and research gaps in uncertainty quantification.
•A review of ensembles for feature selection is described.•Current state-of-the-art is provided.•Types of ensembles, combination methods and evaluation measures are described.•Some challenges and ...future trends are provided.
Ensemble learning is a prolific field in Machine Learning since it is based on the assumption that combining the output of multiple models is better than using a single model, and it usually provides good results. Normally, it has been commonly employed for classification, but it can be used to improve other disciplines such as feature selection. Feature selection consists of selecting the relevant features for a problem and discard those irrelevant or redundant, with the main goal of improving classification accuracy. In this work, we provide the reader with the basic concepts necessary to build an ensemble for feature selection, as well as reviewing the up-to-date advances and commenting on the future trends that are still to be faced.
PDF
