•Potential of tree-based ensemble models for daily ET0 estimation with limited climatic data is explored.•Proposed ensemble models are compared with their corresponding SVM and ELM models.•ELM and ...SVM models offered the best combination of prediction accuracy and stability.•XGBoost and GBDT models have comparable accuracy and stability to those of ELM and SVM models.•XGBoost and GBDT models have less computational costs than the other models.
Accurate estimation of reference evapotranspiration (ET0) is of great importance for the regional water resources planning and irrigation scheduling design. The FAO-56 Penman-Monteith model is recommended as the reference model to predict ET0, but its application is commonly restricted by lack of complete meteorological data at many worldwide locations. This study evaluated the potential of machine learning models, particularly four relatively simple tree-based assemble algorithms (i.e. random forest (RF), M5 model tree (M5Tree), gradient boosting decision tree (GBDT) and extreme gradient boosting (XGBoost)), for estimating daily ET0 with limited meteorological data using a K-fold cross-validation method. For assessment of the tree-based models in terms of prediction accuracy, stability and computational costs, these models were further compared with their corresponding support vector machine (SVM) and extreme learning machine (ELM) models. Four input combinations of daily maximum and maximum temperature (Tmax and Tmin), relative humidity (Hr), wind speed (U2), global and extra-terrestrial solar radiation (Rs and Ra) with Tmax, Tmin and Ra as the base dataset were considered using meteorological data during 1961–2010 from eight representative weather stations in different climates of China. The results showed that, when lack of complete meteorological data, the machine learning models using Tmax, Tmin, Hr, U2 and Ra obtained satisfactory ET0 estimates in the temperate continental, mountain plateau and temperate monsoon zones of China (RMSE < 0.5 mm d−1). However, models with three input parameters of Tmax, Tmin and Rs were superior for daily ET0 prediction in the tropical and subtropical zones. The ELM and SVM models offered the best combination of prediction accuracy and stability. The simple tree-based XGBoost and GBDT models showed comparable accuracy and stability to the SVM and ELM models, but exhibited much less computational costs. Considering the complexity level, prediction accuracy, stability and computational costs of the studied models, the XGBoost and GBDT models have been recommended for daily ET0 estimation in different climatic zones of China and maybe elsewhere with similar climates around the world.
Predicting pillar stability is a vital task in hard rock mines as pillar instability can cause large-scale collapse hazards. However, it is challenging because the pillar stability is affected by ...many factors. With the accumulation of pillar stability cases, machine learning (ML) has shown great potential to predict pillar stability. This study aims to predict hard rock pillar stability using gradient boosting decision tree (GBDT), extreme gradient boosting (XGBoost), and light gradient boosting machine (LightGBM) algorithms. First, 236 cases with five indicators were collected from seven hard rock mines. Afterwards, the hyperparameters of each model were tuned using a five-fold cross validation (CV) approach. Based on the optimal hyperparameters configuration, prediction models were constructed using training set (70% of the data). Finally, the test set (30% of the data) was adopted to evaluate the performance of each model. The precision, recall, and F1 indexes were utilized to analyze prediction results of each level, and the accuracy and their macro average values were used to assess the overall prediction performance. Based on the sensitivity analysis of indicators, the relative importance of each indicator was obtained. In addition, the safety factor approach and other ML algorithms were adopted as comparisons. The results showed that GBDT, XGBoost, and LightGBM algorithms achieved a better comprehensive performance, and their prediction accuracies were 0.8310, 0.8310, and 0.8169, respectively. The average pillar stress and ratio of pillar width to pillar height had the most important influences on prediction results. The proposed methodology can provide a reliable reference for pillar design and stability risk management.
This paper proposes an effective computing framework for Short-Term Load Forecasting (STLF). The proposed technique copes with the stochastic variations of the load demand using a stacked ...generalization approach. This approach combines three models, namely, Light Gradient Boosting Machine (LGBM), eXtreme Gradient Boosting machine (XGB), and Multi-Layer Perceptron (MLP). The inner mechanism of Stacked XGB-LGBM-MLP model consists of generating a meta-data from XGB and LGBM models to compute the final predictions using MLP network. The performance of the proposed Stacked XGB-LGBM-MLP model is validated using two datasets from different locations: Malaysia and New England. The main contributions of this paper are: 1) A novel stacking ensemble-based algorithm is proposed; 2) An effective STLF technique is introduced; 3) A critical multi-study analysis for hyperparameter optimization with five techniques is comprehensively performed; 4) A performance comparative study using two datasets and reference models is conducted. Several case studies have been carried out to prove the performance superiority of the proposed model compared to both existing benchmark techniques and hybrid models.
•A novel Stacked XGB-LGBM-MLP model is proposed to improve the overall regression performance.•A novel STLF technique is explored and developed in this study.•A comparative analysis of five hyperparameter optimization algorithms was comprehensively presented for STLF.•An assessment of the proposed technique is conducted using two real datasets.•A comparative study with the recent benchmark techniques is performed.
•Machine learning models coupled with wavelet transforms for GWL forecasting.•eXtreme Gradient Boosting, Random Forests, and Support Vector Regression explored.•Bayesian optimization automatically ...selects XGB, RF, and SVR hyper-parameters.•XGB and RF inherently perform high dimensional input variable selection.•New hybrid approaches (WT-XGB and WT-RF) are more accurate than standalone models.
Groundwater level (GWL) forecasting is crucial for irrigation scheduling, water supply and land development. Machine learning (ML) (e.g., artificial neural networks) has been increasingly adopted to forecast GWL due to its ability to model nonlinearities between GWL and its drivers (e.g., rainfall). Although ML approaches have been successful at forecasting GWL, they are often inaccurate when GWL exhibits multiscale changes (e.g., due to urbanization). To address this shortcoming, wavelet transforms (WT) are routinely coupled with ML methods. Unfortunately, researchers frequently neglect key issues associated with WT that render such forecasts useless for real-world scenarios. This study demonstrates how new ML methods, such as eXtreme Gradient Boosting and Random Forests, can be properly coupled with WT to generate accurate GWL forecasts (1-–3 months ahead) for 7 wells in Kumamoto City in Southern Japan that can be used to help address current pressing issues such as groundwater quality and land subsidence.
•The CS of FRSCC was predicted using three ML models, namely XGBoost, DT, and Light GBM.•A database of 387 samples, 17 inputs, Monte Carlo and K-fold CV techniques were used for models ...development.•The XGBoost model has highest prediction and stability, followed by DT and Light GBM models.•Effect of 17 inputs on the output was evaluated by feature importance and Shap values.•Based on architectural analysis, XGB was proposed as the most accurate and reliable model.
Fiber-reinforced self-compacting concrete (FRSCC), a great combination of self-compacting concrete (SCC) and fiber, plays a vital role as a potential construction material. Improving the accuracy of FRSCC’ performance prediction methods is critical and challenging to reduce costly experiments and time. Therefore, this study developed and assessed the performance of three machine learning models, including Decision tree, Light Gradient Boosting Machine, and Extreme Gradient Boosting (XGBoost), for predicting the compressive strength (CS) of FRSCC. The models were developed based on 387 data samples with 17 input parameters. Monte Carlo and K-fold cross-validation techniques were used to assess the models' generalizability and predictive performance. The results showed that the XGBoost model has the highest predictive performance and stability, with typical results R2 = 0.992, RMSE = 1.892 MPa, MAE = 1.438 MPa. The sensitivity analysis of the models indicated that cement, coarse aggregate, fine aggregate, water, and sample age significantly influence the CS of FRSCC with inconsistent order. Finally, XGBoost was the most accurate and reliable model based on the final architecture analysis.
This paper extends the technique of gradient boosting with a focus on using domain-specific models instead of trees. The domain of mortality forecasting is considered as an application. The two novel ...contributions are to use well-known stochastic mortality models as weak learners in gradient boosting rather than trees, and to include a penalty that shrinks mortality forecasts in adjacent age groups and nearby geographical regions closer together. The proposed method demonstrates superior forecasting performance based on US male mortality data from 1969 to 2019. The proposed approach also enables us to interpret and visualize the results. The boosted model with age-based shrinkage yields the most accurate national-level mortality forecast. For state-level forecasts, spatial shrinkage provides further improvement in accuracy in addition to the benefits of age-based shrinkage. This improvement can be attributed to data sharing across states with large and small populations in adjacent regions and states with common risk factors.
Cardiovascular diseases (CVD) are among the most common serious illnesses affecting human health. CVDs may be prevented or mitigated by early diagnosis, and this may reduce mortality rates. ...Identifying risk factors using machine learning models is a promising approach. We would like to propose a model that incorporates different methods to achieve effective prediction of heart disease. For our proposed model to be successful, we have used efficient Data Collection, Data Pre-processing and Data Transformation methods to create accurate information for the training model. We have used a combined dataset (Cleveland, Long Beach VA, Switzerland, Hungarian and Stat log). Suitable features are selected by using the Relief, and Least Absolute Shrinkage and Selection Operator (LASSO) techniques. New hybrid classifiers like Decision Tree Bagging Method (DTBM), Random Forest Bagging Method (RFBM), K-Nearest Neighbors Bagging Method (KNNBM), AdaBoost Boosting Method (ABBM), and Gradient Boosting Boosting Method (GBBM) are developed by integrating the traditional classifiers with bagging and boosting methods, which are used in the training process. We have also instrumented some machine learning algorithms to calculate the Accuracy (ACC), Sensitivity (SEN), Error Rate, Precision (PRE) and F1 Score (F1) of our model, along with the Negative Predictive Value (NPR), False Positive Rate (FPR), and False Negative Rate (FNR). The results are shown separately to provide comparisons. Based on the result analysis, we can conclude that our proposed model produced the highest accuracy while using RFBM and Relief feature selection methods (99.05%).
Accurate assessment of undrained shear strength (USS) for soft sensitive clays is a great concern in geotechnical engineering practice. This study applies novel data-driven extreme gradient boosting ...(XGBoost) and random forest (RF) ensemble learning methods for capturing the relationships between the USS and various basic soil parameters. Based on the soil data sets from TC304 database, a general approach is developed to predict the USS of soft clays using the two machine learning methods above, where five feature variables including the preconsolidation stress (PS), vertical effective stress (VES), liquid limit (LL), plastic limit (PL) and natural water content (W) are adopted. To reduce the dependence on the rule of thumb and inefficient brute-force search, the Bayesian optimization method is applied to determine the appropriate model hyper-parameters of both XGBoost and RF. The developed models are comprehensively compared with three comparison machine learning methods and two transformation models with respect to predictive accuracy and robustness under 5-fold cross-validation (CV). It is shown that XGBoost-based and RF-based methods outperform these approaches. Besides, the XGBoost-based model provides feature importance ranks, which makes it a promising tool in the prediction of geotechnical parameters and enhances the interpretability of model.
Display omitted
•XGBoost and RF-based ELMs for predicting the undrained shear strength of soft clays.•Bayesian hyper-parameter optimization for determination of the hyper-parameters.•ELMs performance and transformation model comparison.•Feature relative importance ranked by the proposed XGBoost-based model.
Introduction: The present study discusses the importance of having a predictive method to determine the prognosis of patients with diseases like Covid-19. This method can assist physicians in making ...treatment decisions that improve survival rates and avoid unnecessary treatments. This research also highlights the importance of calibration, which is often overlooked in model evaluation. Without proper calibration, incorrect decisions can be made in disease treatment and preventive care. Therefore, the current study compares two highly accurate machine learning algorithms, Gradient boosting and Extreme gradient boosting, not only in terms of prediction accuracy but also in terms of model calibration and speed. Methods: This study involved analyzing data from Covid-19 patients who were admitted to two hospitals in Mashhad city, Razavi Khorasan province, over a span of 18 months. The k-fold cross-validation method was employed on the training dataset (K=5) to conduct the study. The accuracy and calibration of two methods (Gradient boosting and Extreme gradient boosting) in predicting survival were compared using the Concordance Index and calibration. Results: The Concordance Index values obtained for gradient boosting and Extreme gradient boosting models were 0.734 and 0.736, in the imbalanced and In the balanced data, the Concordance Index values were 0.893 for gradient boosting and 0.894 for Extreme gradient boosting. The surv.calib_beta index, the gradient boosting model had an estimated value of 0.59 in the imbalanced data and 0.66 in the balanced data. The Extreme gradient boosting model had an estimated value of 0.86 in the balanced data and 0.853 in the imbalanced data. The Extreme gradient boosting model was faster in the learning process compared to the gradient boosting model. Conclusion: The Gradient boosting and Extreme gradient boosting models exhibited similar prediction accuracy and discrimination power, but the Extreme gradient boosting model demonstrated relatively good calibration compare to Gradient boosting model.
This study introduces machine learning (ML) algorithms to predict hydrogen (H2) thermodynamic properties for geological storage, focusing on its mixtures with natural gas, nitrogen (N2), and carbon ...dioxide (CO2). Employing 1167 data samples, this research utilizes multiple linear regression (MLR), random forest (RF), gradient boosting (GB), and decision tree (DT) models, enhanced by sensitivity analysis for feature engineering. GB model's superior accuracy and efficiency over conventional statistical linear regression methods. The finding reveals that the GB model achieves superior performance, with an R2 value of 0.9998 for thermal capacity and the lowest mean absolute error (MAE)/root mean square error (RMSE) across all H2 properties—density (1.51%, 2.06 kg/m3), viscosity (0.000078%, 0.000117 cp), thermal conductivity (0.000427%, 0.000849 W/m.K), and thermal capacity (8.79%, 26.06 J/g.K). Moreover, this ML-based approach not only demonstrates remarkable accuracy and efficiency but also suggests the potential of smart paradigms to further enhance the safety and transportability of underground hydrogen storage projects. Ultimately, this work will help the reservoir simulation at field scale to be more robust and reliable.
Display omitted
•Machine learning employed in the prediction of hydrogen's thermodynamic properties.•Four advanced ML-based models constructed by variables such as pressure, temperature, and hydrogen fraction.•Gradient boosting algorithm comparatively showed the highest accuracies.•ML achieved the lowest MAE/RMSE values for viscosity 0.000078/0.000117.
PDF
