Deep neural nets (DNNs) mostly tend to outperform other machine learning (ML) approaches when the training data is abundant, high-dimensional, sparse, or consisting of raw data (e.g., pixels). For ...datasets with other characteristics – for example, dense tabular numerical data – algorithms such as Gradient Boosting Machines and Random Forest often achieve comparable or better performance at a fraction of the time and resources. These differences suggest that combining these approaches has potential to yield superior performance. Existing attempts to combine DNNs with other ML approaches, which usually consist of feeding the output of the latter into the former, often do not produce positive results. We argue that this lack of improvement stems from the fact that the final classifications fail to provide the DNN with an understanding of the other algorithms’ decision-making process (i.e., its “logic”). In this study we present F-PENN, a novel approach for combining decision forests and DNNs. Instead of providing the final output of the forest (or its trees) to the DNN, we provide the paths traveled by each sample. This information, when fed to the neural net, yields significant improvement in performance. We demonstrate the effectiveness of our approach by conducting extensive evaluation on 56 datasets and comparing F-PENN to four leading baselines: DNNs, Gradient Boosted Decision Trees (GBDT), Random Forest and DeepFM. We show that F-PENN outperforms the baselines in 69%–89% of dataset and achieves an overall average error reduction of 16%–26%.
•We present a novel method to combine ensemble models and neural networks.•We encode the sample trajectory in the ensemble to enrich samples representation.•The encoding method used is Word2Vec — what creates de-facto data augmentation.•We evaluate our model on 56 open-sourced datasets.•We present superior performances against several baselines.
Forecasting the direction of the daily changes of stock indices is an important yet difficult task for market participants. Advances on data mining and machine learning make it possible to develop ...more accurate predictions to assist investment decision making. This paper attempts to develop a learning architecture LR2GBDT for forecasting and trading stock indices, mainly by cascading the logistic regression (LR) model onto the gradient boosted decision trees (GBDT) model. Without any assumption on the underlying data generating process, raw price data and twelve technical indicators are employed for extracting the information contained in the stock indices. The proposed architecture is evaluated by comparing the experimental results with the LR, GBDT, SVM (support vector machine), NN (neural network) and TPOT (tree-based pipeline optimization tool) models on three stock indices data of two different stock markets, which are an emerging market (Shanghai Stock Exchange Composite Index) and a mature stock market (Nasdaq Composite Index and S&P 500 Composite Stock Price Index). Given the same test conditions, the cascaded model not only outperforms the other models, but also shows statistically and economically significant improvements for exploiting simple trading strategies, even when transaction cost is taken into account.
•A cascaded learning architecture LR2GBDT is proposed to predict the direction of the daily changes of stock indices.•Logistic regression and gradient boosted decision trees are combined in our approach.•Technical indicators and the output derived from LR are fed as input features.•The prediction accuracy and trading performance are improved by LR2GBDT.•The profitability with a simple long–short trading strategy in a daily investment horizon is also discussed.
•Four machine learning (KNN, RF, GBDT and XGBoost) models for predicting the glass forming ability (GFA) of amorphous alloys are developed.•The 10-fold cross-validation method divides the data set ...into validation set and training set, and makes full use of each data to prevent overfitting.•Grid-search was used to determine the hyperparameters, and cross-validation was used to determine this model's rationality.•The XGBoost model provides the highest accuracy in the glass forming ability prediction.•The machine learning based models show good predictive and generalization ability.
In this work, we adopted four machine learning (ML) models, i.e., random forest (RF), K nearest neighbor (KNN), gradient boosted decision trees (GBDTs) and eXtreme gradient boosting (XGBoost) to predict the glass forming ability (GFA) of amorphous alloys using the dataset of Deng. The critical casting diameter (Dmax) of these alloys represents their GFA. The correlation coefficient (R) and root mean square error (RMSE) of the RF, KNN, GBDTs as well as XGBoost models are 0.75 and 3.29, 0.734 and 3.431, 0.724 and 3.474, and 0.755 and 3.277, respectively. Based on 10-fold cross-validation, it is found that the XGBoost model exhibits the highest predictive performance than the other above-mentioned three ML models and twelve previously reported criteria. Our results imply that machine learning method is very powerful and efficient, and has great potential for designing new amorphous alloys with desired GFA.
The glass forming ability (GFA) is a problem of great concern in the research of amorphous materials. It is of great significance to understand the physical mechanism of GFA and to seek conditions ...and methods to improve it. In this study, we collected 820 experimental data from existing literature, and used gradient boosted decision trees (GBDT) model to predict the GFA. The GBDT model optimized by 10-fold cross-validation and grid search technology shows excellent predictive results. The determination coefficient (R2) and root mean square error (RMSE) are 0.652 and 2.85, respectively. Compared with the previously reported 27 criteria and ML models, GBDT model has the highest prediction ability. The result exhibit that the predictive performance of GBDT can be significantly improved by considering the atomic size difference, total electronegativity, mixing entropy and average atomic volume.
•The GBDT model for predicting the GFA of amorphous alloys was developed.•Grid-search and cross-validation were used to determine the hyperparameters.•The GBDT model provides the highest accuracy in GFA prediction.•The predictive performance of GFA can be significantly improved after adding TEN, δ, VA and Sm.
Rampant pasture burning has lead to various forest fires taking their toll over the health of many forests. Nanda Devi Biosphere Reserve, located in the northern part of India, witnessed a majority ...of these incidents in the recent past, though, it remains comprehensively untouched from research studies. The scale of these wildfires has led to an immense requirement of preventive measures to be taken for recuperating from such events. This requires for an in-depth analysis of the study area, its history of wildfires and their causes. These efforts would assist in laying a blueprint for a contingency plan in the event of a wildfire. This work proposes an evolutionary optimized gradient boosted decision trees for preparing wildfire susceptibility maps for the study area that would aid in the government’s forest preservation and disaster management activities. The study took 18 ignition factors of elevation, slope, aspect, plan curvature, topographic position index, topographic water index, normalized difference vegetation index, soil texture, temperature, rainfall, aridity index, potential evapotranspiration, relative humidity, wind speed, land cover and distance from roads, rivers and habitations into consideration. The study revealed that approximately 1432.025 km
2
of area was very highly susceptible to forest fires while 1202.356 km
2
was highly susceptible to forest fires. The proposed model was compared against various machine learning models such as random forest, neural networks and support vector machines, and it outperformed them by achieving an overall accuracy of 95.5%. The proposed model demonstrated good prospects for application in the field of hazard susceptibility mappings.
In the drought prone district of Dholpur in Rajasthan, India, groundwater is a lifeline for its inhabitants. With population explosion and rapid urbanization, the groundwater is being critically ...over-exploited. Hence the current groundwater potential mapping study was undertaken to ascertain the areas that are more likely to yield a larger volume of groundwater against those areas that have poor groundwater potential and accordingly perpetuate the much needed damage control. Thematic layers for 14 groundwater influencing factors were considered for the study region, including elevation, slope, aspect, plan curvature, profile curvature, topographic wetness index (TWI), geology, soil, land use, normalized difference vegetation index (NDVI), surface temperature, precipitation, distance from roads, and distance from rivers. These were then subjected to an overlay operation, with the groundwater inventory which comprised of the locations of observational groundwater wells. The resulting geospatial database was then used to train two decision tree based ensemble models: gradient boosted decision trees (GBDT) and random forest (RF). The predictive performance of these models was then compared using various performance metrics such as area under curve (AUC) of receiver operating characteristics (ROC), sensitivity, accuracy, etc. It was found that GBDT (AUC: 0.79) outperformed RF (AUC: 0.71). The validated GBDT model was then used to construct the groundwater potential zonation map. The generated map showed that about 20.2% of the region has very high potential, while 22.6% has high potential to yield groundwater, and approximately 19.9–17.5% of the study region has very low to low groundwater potential.
Conservation decision makers must negotiate social and technical complexities to achieve desired biodiversity outcomes. Quantitative models can inform decision making, by evaluating and predicting ...management outcomes, so that comparisons can be made between alternative courses of action. However, whether a proposed action is appropriate for implementation, regardless of its contribution to management outcomes, also requires consideration. Existing quantitative models have yet to fully incorporate the suitability of proposed management actions, which hinders their ability to inform decision making.
We used gradient boosted decision trees – a machine‐learning technique – to determine the suitability of alternative management actions available to a biodiversity conservation programme. We demonstrate our approach using the Predator Free 2050 programme – a large and complex conservation initiative that seeks to eradicate selected invasive vertebrates from the entirety of New Zealand by 2050. We created a nationally contiguous network of management tools to suppress populations of invasive species across the entire country. We then used our suitability predictions to explore three scenarios for selecting invasive species management tools, based on maximising (a) implementation probability, (b) humaneness and (c) cost‐savings.
Our models highlighted that an interplay of factors influence where management tools can potentially be implemented. Our management scenarios revealed what different contiguous management networks could look like for New Zealand over the next 10–15 years as an interim step to achieving Predator Free 2050. Each scenario differed in the tools selected for implementation in different places and in the overall economic costs associated with creating a contiguous management network. Some locations were identified as unsuitable for any existing management tools, indicating that future transformative technologies may be required to create a contiguous network.
Synthesis and applications. Conservation decision making must not only consider biodiversity outcomes but also whether selected management actions are appropriate in the first place. Here, we used machine‐learning techniques to determine the suitability of competing managements actions that are proposed to meet biodiversity objectives. Our approach provides an objective, transparent and reproducible framework to determine the suitability of actions at sites across large spatial extents, under complex social and technical constraints.
Whakarāpopototanga
Kia whakatutuki i ngā putanga kanorau koiora, arā anō ngā piki me ngā heke ā‐pāpori, ā‐hangarau hoki hei urungi mā ngā mana whakatau i te atawhainga o te ao tūroa nei. Auau tonu nei te whakamahinga o ngā tauira ine rahi ki te tautoko i ngā whiriwhiri whakataunga, mā te matapae i ngā putanga whakahaere, kia taea ai ngā whakatauritenga i waenga i ngā momo kōkiritanga mahi. Heoi anō, ahakoa kua tika rānei te mahi kua whakatakoto mō te whakatinanatanga, ahakoa hoki te pitomata o taua mahi ki te tautoko i ngā putanga whakahaere, me aromatawai tonu. Kāore anō kia kōkuhua katoatia e ngā tauira ine rahi onāianei te hāngaitanga o ngā mahi whakahaere kua whakatakoto, he mea whakaroiroi i tā rātou āhei ki te tautoko i te mahi whakatau.
I whakamahia e mātou ngā rākau whakapiki whakatau ā‐ronaki ‐ he āhuatanga ako‐ā‐pūrere ‐ i te whakamahinga āhua rerekē nei ki te whiriwhiri i te hāngaitanga o ngā mahi whakahaere kē kua whakatakoto mō tētahi kaupapa atawhai nuku kanorau koiora. Ka whakaaturia e mātou te ngā painga o tō mātou momo kōkiri e whakamahi ana i te kaupapa Predator Free 2050 ‐ he kaupapa kōkiri atawhai nuku nui, matatini anō hoki e whai ana ki te whakakore atu i ētahi momo kararehe urutomo i Aotearoa nei i mua i te tau 2050. Tā mātou i konei he aro ki te hanga whatunga haere tonu ā‐motu o ngā utauta whakahaere ki te kaupēhi, tāmi hoki i ngā tini kararehe urutomo puta katoa i te whenua. Kātahi ka whakamahia e mātou ā mātou matapae hāngaitanga ki te tūhura i ngā wheako whakaari i whakapikihia takitahitia nei i (i) te tūponotanga whakatinanatanga, (ii) te ngākau atawhaitanga, ka mutu, (iii) te pai o te utu, he take matua ēnei kua whakaarohia e ngā mana whakatau i te tīpakotanga o ngā utauta whakahaere kararehe urutomo.
He mea whakapuaki ō mātou tauira i te whitiwhitinga matatini o ngā take e whiriwhiri ki hea pea whakatinanahia ai ngā utauta whakahaere. I whakakitea e ō mātou wheako whakaari whakahaere he pēhea te āhua o ngā whatunga whakahaere haere tonu i Aotearoa i roto i te 10 ki te 15 tau e tū mai nei. He rerekē ia wheako whakaari i ngā utauta i tīpakona mō te whakatinanatanga ki tēnā wāhi, ki tēnā wāhi, i ngā utu whai pānga hoki ki te hanganga o tētahi whatunga whakahaere haere tonu. I tautuhia ētahi wāhi kāore e pai mō ngā utauta whakahaere onāianei, e tohu ana ka hiahiatia pea ngā momo hangarau panonitanga o nga rā ki tua ki te hanga i te whatunga haere tonu.
Te kōtuitui me ngā whakamahinga. I whakatauria e mātou te hāngaitanga o ngā momo mahi whakahaere ki te whakatutuki i ngā whāinga atawhai nuku kua whakatakoto mā te whakamahi i ngā āhuatanga ako‐ā‐pūrere. Ko te āhua o tā mātou momo kōkiri he whakatakoto i te poutarāwaho hei whiriwhiri i te hāngaitanga o ngā momo mahi ki tētahi wāhi ahakoa ko tēhea puta noa i ngā whānuitanga mokowā nui, i raro hoki i ngā herenga a‐pāpori, ā‐hangarau matatini rawa anō.
Conservation decision making must not only consider biodiversity outcomes but also whether selected management actions are appropriate in the first place. Here, we used machine‐learning techniques to determine the suitability of competing managements actions that are proposed to meet biodiversity objectives. Our approach provides an objective, transparent and reproducible framework to determine the suitability of actions at sites across large spatial extents, under complex social and technical constraints.
The ever-increasing frequency and intensity of intrusion attacks on computer networks worldwide has necessitated intense research efforts towards the design of attack detection and prediction ...mechanisms. While there are a variety of intrusion detection solutions available, the prediction of network intrusion events is still under active investigation. Over the past, statistical methods have dominated the design of attack prediction methods. However more recently, both shallow and deep learning techniques have shown promise for such data intensive regression tasks. This paper first explores the use of shallow learning techniques for predicting intrusions in computer networks by estimating the probability that a malicious source will repeat an attack in a given future time interval. The approach also highlights the limits to which shallow learning may be applied for such predictive tasks. The work then goes on to show that deep learning approaches are much more suited for network alert prediction tasks. A recurrent neural network based approach is shown to be more suited for alert prediction tasks. Both approaches are evaluated on the same dataset, comprising of millions of alerts taken from the alert sharing system Warden operated by CESNET.
•Method for optimising pin-in-paste technology was developed.•Multiple machine learning techniques were analysed and compared.•Sensitivity analysis of the applied model was performed.•The first pass ...yield of the production can be enhanced based on our results.•Managerial implications were also summarized based on the results.
Pin-in-paste technology is an advanced and constantly developing assembly method, where both the surface mounted and through-hole components are joined commonly with the same reflow soldering step. In the paper, detailed evaluation of machine learning based prediction methods was performed, including artificial neural networks (ANN), adaptive neuro fuzzy inference systems (AFNIS) and gradient boosted decision trees to optimize the process parameters of pin-in-paste. The methods were presented for predicting the hole-filling of solder paste during stencil printing and to enable an evaluation of the aforementioned methods for the specific problem. Experiments were performed to obtain the required input data (hole-filling against the process parameters) for the training of the prediction tools. A testboard with plated through-holes of different diameters (0.8, 1, 1.1, 1.4 mm) was applied. Type 4 lead-free solder paste (particle size 20–38 µm) was deposited with stencil printing into the through-holes with different printing speeds (between 20 mm/s and 70 mm/s). The extent of hole-filling was investigated with X-ray analysis. The optimal structure for the prediction method was determined for every approach by varying the size and configuration iteration-wise. Optimal number of hidden neurons was 34 for the full data set and 18 for the smaller – incomplete- test cases. The optimal structure for the ANFIS consists of 8 and 6 membership functions respectively. Triangular and Gaussian membership function types and 7 different training methods were assessed. The ANN, which was trained with Bayesien Regularization (prediction error <15%) was found to be the recommended one for the PIP technology. A sensitivity analysis was carried out serving as a basis for managerial implications, which were also described in the paper. The obtained results aid in improving the quality and reliability of products assembled with pin-in-paste technology, also enable the more precise control in the wake of Industry 4.0 recommendations.
Among natural hazards occurring offshore, submarine landslides pose a significant risk to offshore infrastructure installations attached to the seafloor. With the offshore being important for current ...and future energy production, there is a need to anticipate where future landslide events are likely to occur to support planning and development projects. Using the northern Gulf of Mexico (GoM) as a case study, this paper performs Landslide Susceptibility Mapping (LSM) using a gradient-boosted decision tree (GBDT) model to characterize the spatial patterns of submarine landslide probability over the United States Exclusive Economic Zone (EEZ) where water depths are greater than 120 m. With known spatial extents of historic submarine landslides and a Geographic Information System (GIS) database of known topographical, geomorphological, geological, and geochemical factors, the resulting model was capable of accurately forecasting potential locations of sediment instability. Results of a permutation modelling approach indicated that LSM accuracy is sensitive to the number of unique training locations with model accuracy becoming more stable as the number of training regions was increased. The influence that each input feature had on predicting landslide susceptibility was evaluated using the SHapely Additive exPlanations (SHAP) feature attribution method. Areas of high and very high susceptibility were associated with steep terrain including salt basins and escarpments. This case study serves as an initial assessment of the machine learning (ML) capabilities for producing accurate submarine landslide susceptibility maps given the current state of available natural hazard-related datasets and conveys both successes and limitations.
PDF
