•We design a tabular data GAN for oversampling that can handle categorical variables.•We assess our GAN in a credit scoring setting using multiple real-world datasets.•We find GAN-based oversampling ...to outperform advanced SMOTE-type benchmarks.•Ablations confirm the specific choices in the proposed GAN architecture.
Class imbalance impedes the predictive performance of classification models. Popular countermeasures include oversampling minority class cases by creating synthetic examples. The paper examines the potential of Generative Adversarial Networks (GANs) for oversampling. A few prior studies have used GANs for this purpose but do not reflect recent methodological advancements for generating tabular data using GANs. The paper proposes an approach based on a conditional Wasserstein GAN that can effectively model tabular datasets with numerical and categorical variables and pays special attention to the down-stream classification task through an auxiliary classifier loss. We focus on a credit scoring context in which binary classifiers predict the default risk of loan applications. Empirical comparisons in this context evidence the competitiveness of GAN-based oversampling compared to several standard oversampling regimes. We also clarify the conditions under which oversampling in general and the proposed GAN-based approach in particular raise predictive performance. In sum, our findings suggest that GAN architectures for tabular data and our extensions deserve a place in data scientists’ modelling toolbox.
•Extreme learning machines (ELM) are a new vehicle for credit risk management.•Systematic analysis of ELM usability, efficiency and forecast accuracy is performed.•Benchmark results confirm ELM ...effectiveness for individual and ensemble scorecards.
Classification algorithms are used in many domains to extract information from data, predict the entry probability of events of interest, and, eventually, support decision making. This paper explores the potential of extreme learning machines (ELM), a recently proposed type of artificial neural network, for consumer credit risk management. ELM possess some interesting properties, which might enable them to improve the quality of model-based decision support. To test this, we empirically compare ELM to established scoring techniques according to three performance criteria: ease of use, resource consumption, and predictive accuracy. The mathematical roots of ELM suggest that they are especially suitable as a base model within ensemble classifiers. Therefore, to obtain a holistic picture of their potential, we assess ELM in isolation and in conjunction with different ensemble frameworks. The empirical results confirm the conceptual advantages of ELM and indicate that they are a valuable alternative to other credit risk modelling methods.
•Large-scale benchmark of 41 classifiers across eight real-word credit scoring data sets.•Introduction of ensemble selection routines to the credit scoring community.•Analysis of six established and ...novel indicators to measure scorecard accuracy.•Assessment of the financial impact of different scorecards.
Many years have passed since Baesens et al. published their benchmarking study of classification algorithms in credit scoring Baesens, B., Van Gestel, T., Viaene, S., Stepanova, M., Suykens, J., & Vanthienen, J. (2003). Benchmarking state-of-the-art classification algorithms for credit scoring. Journal of the Operational Research Society, 54(6), 627–635.. The interest in prediction methods for scorecard development is unbroken. However, there have been several advancements including novel learning methods, performance measures and techniques to reliably compare different classifiers, which the credit scoring literature does not reflect. To close these research gaps, we update the study of Baesens et al. and compare several novel classification algorithms to the state-of-the-art in credit scoring. In addition, we examine the extent to which the assessment of alternative scorecards differs across established and novel indicators of predictive accuracy. Finally, we explore whether more accurate classifiers are managerial meaningful. Our study provides valuable insight for professionals and academics in credit scoring. It helps practitioners to stay abreast of technical advancements in predictive modeling. From an academic point of view, the study provides an independent assessment of recent scoring methods and offers a new baseline to which future approaches can be compared.
Display omitted
In credit scoring, feature selection aims at removing irrelevant data to improve the performance of the scorecard and its interpretability. Standard techniques treat feature selection as a ...single-objective task and rely on statistical criteria such as correlation. Recent studies suggest that using profit-based indicators may improve the quality of scoring models for businesses. We extend the use of profit measures to feature selection and develop a multi-objective wrapper framework based on the NSGA-II genetic algorithm with two fitness functions: the Expected Maximum Profit (EMP) and the number of features. Experiments on multiple credit scoring data sets demonstrate that the proposed approach develops scorecards that can yield a higher expected profit using fewer features than conventional feature selection strategies.
•A profit-driven feature selection framework for credit scoring is introduced.•The suggested framework is based on a multi-objective algorithm NSGA-II.•The algorithm optimizes profitability and comprehensibility of a scoring model.•Experiments on ten data sets demonstrate good performance of the proposed approach.
Crime prediction is crucial to criminal justice decision makers and efforts to prevent crime. The paper evaluates the explanatory and predictive value of human activity patterns derived from taxi ...trip, Twitter and Foursquare data. Analysis of a six-month period of crime data for New York City shows that these data sources improve predictive accuracy for property crime by 19% compared to using only demographic data. This effect is strongest when the novel features are used together, yielding new insights into crime prediction. Notably and in line with social disorganisation theory, the novel features cannot improve predictions for violent crimes.
•Using Twitter, taxi flow and Foursquare data improves property crime predictions.•Interactions are important and emphasise relevance of local crime opportunities.•Violent crime does not emerge from short-run human dynamics.•Crime prediction and prevention must account for spatial and structural difference.
•An extensive benchmark in financial time series forecasting is performed.•Best machine learning(ML) methods out-perform best econometric methods.•The ML methodology employed significantly affects ...forecasting accuracy.•Market maturity, forecast horizon & model-assessment method affect forecast accuracy.•Evidence against the informational value of technical indicators.
Financial time series forecasting is a popular application of machine learning methods. Previous studies report that advanced forecasting methods predict price changes in financial markets with high accuracy and that profit can be made trading on these predictions. However, financial economists point to the informational efficiency of financial markets, which questions price predictability and opportunities for profitable trading. The objective of the paper is to resolve this contradiction. To this end, we undertake an extensive forecasting simulation, based on data from thirty-four financial indices over six years. These simulations confirm that the best machine learning methods produce more accurate forecasts than the best econometric methods. We also examine the methodological factors that impact the predictive accuracy of machine learning forecasting experiments. The results suggest that the predictability of a financial market and the feasibility of profitable model-based trading are significantly influenced by the maturity of the market, the forecasting method employed, the horizon for which it generates predictions and the methodology used to assess the model and simulate model-based trading. We also find evidence against the informational value of indicators from the field of technical analysis. Overall, we confirm that advanced forecasting methods can be used to predict price changes in some financial markets and we discuss whether these results question the prevailing view in the financial economics literature that financial markets are efficient.
•Evaluation of LSTM deep neural networks for irradiance forecasting.•New experimental framework including virtual PV site construction.•Approximation of GHI readings using satellite ...images.•Large-scale benchmark of forecasting methods across 21 geo-locations.
Accurate forecasts of solar energy are important for photovoltaic (PV) based energy plants to facilitate an early participation in energy auction markets and efficient resource planning. The study concentrates on Long Short Term Memory (LSTM), a novel forecasting method from the family of deep neural networks, and compares its forecasting accuracy to alternative methods with a proven track record in solar energy forecasting. To provide a comprehensive and reliable assessment of LSTM, the study employs remote-sensing data for testing predictive accuracy at 21 locations, 16 of which are in mainland Europe and 5 in the US. To that end, a novel framework to conduct empirical forecasting comparisons is introduced, which includes the generation of virtual PV plants. The framework enables richer comparisons with higher coverage of geographical regions. Empirical results suggest that LSTM outperforms a large number of alternative methods with substantial margin and an average forecast skill of 52.2% over the persistence model. An implication for energy management practice is that LSTM is a promising technique, which deserves a place in forecasters’ toolbox. From an academic point of view, LSTM and the proposed framework for experimental design provide a valuable environment for future studies that assess new forecasting technology.
Open-ended responses are widely used in market research studies. Processing of such responses requires labour-intensive human coding. This paper focuses on unsupervised topic models and tests their ...ability to automate the analysis of open-ended responses. Since state-of-the-art topic models struggle with the shortness of open-ended responses, the paper considers three novel short text topic models: Latent Feature Latent Dirichlet Allocation, Biterm Topic Model and Word Network Topic Model. The models are fitted and evaluated on a set of real-world open-ended responses provided by a market research company. Multiple components such as topic coherence and document classification are quantitatively and qualitatively evaluated to appraise whether topic models can replace human coding. The results suggest that topic models are a viable alternative for open-ended response coding. However, their usefulness is limited when a correct one-to-one mapping of responses and topics or the exact topic distribution is needed.
Corporate data mining faces the challenge of systematic knowledge discovery in large data streams to support managerial decision making. While research in operations research, direct marketing and ...machine learning focuses on the analysis and design of data mining algorithms, the interaction of data mining with the preceding phase of data preprocessing has not been investigated in detail. This paper investigates the influence of different preprocessing techniques of attribute scaling, sampling, coding of categorical as well as coding of continuous attributes on the classifier performance of decision trees, neural networks and support vector machines. The impact of different preprocessing choices is assessed on a real world dataset from direct marketing using a multifactorial analysis of variance on various performance metrics and method parameterisations. Our case-based analysis provides empirical evidence that data preprocessing has a significant impact on predictive accuracy, with certain schemes proving inferior to competitive approaches. In addition, it is found that (1) selected methods prove almost as sensitive to different data representations as to method parameterisations, indicating the potential for increased performance through effective preprocessing; (2) the impact of preprocessing schemes varies by method, indicating different ‘best practice’ setups to facilitate superior results of a particular method; (3) algorithmic sensitivity towards preprocessing is consequently an important criterion in method evaluation and selection which needs to be considered together with traditional metrics of predictive power and computational efficiency in predictive data mining.
•A formal decision analysis clarifies the economics of the customer targeting problem.•Optimal targeting considers the marketing effect and customer response probability.•Causal hurdle models jointly ...estimate treatment effect and response probability.•Results from e-coupon targeting confirm the superiority of the analytical policy.
This study provides a formal analysis of the customer targeting problem when the cost for a marketing action depends on the customer response and proposes a framework to estimate the decision variables for campaign profit optimization. Targeting a customer is profitable if the impact and associated profit of the marketing treatment are higher than its cost. Despite the growing literature on uplift models to identify the strongest treatment-responders, no research has investigated optimal targeting when the costs of the treatment are unknown at the time of the targeting decision. Stochastic costs are ubiquitous in direct marketing and customer retention campaigns because marketing incentives are conditioned on a positive customer response. This study makes two contributions to the literature, which are evaluated on an e-commerce coupon targeting campaign. First, we formally analyze the targeting decision problem under response-dependent costs. Profit-optimal targeting requires an estimate of the treatment effect on the customer and an estimate of the customer response probability under treatment. The empirical results demonstrate that the consideration of treatment cost substantially increases campaign profit when used for customer targeting in combination with an estimate of the average or customer-level treatment effect. Second, we propose a framework to jointly estimate the treatment effect and the response probability by combining methods for causal inference with a hurdle mixture model. The proposed causal hurdle model achieves competitive campaign profit while streamlining model building. All codes are available at https://github.com/Humboldt-WI/response-dependent-costs.
PDF
