We introduce the problem of load nowcasting to the energy forecasting literature. The recent load of the objective area is predicted based on limited available metering data within this area. Thus, ...slightly different from load forecasting, we are predicting the recent past using limited available metering data from the supply side of the system. Next, to an industry benchmark model, we introduce multiple high-dimensional models for providing more accurate predictions. They evaluate metered interconnector and generation unit data of different types like wind and solar power, storages, and nuclear and fossil power plants. Additionally, we augment the model by seasonal and autoregressive components to improve the nowcasting performance. We consider multiple estimation techniques based on the lassoand ridge and study the impact of the choice of the training/calibration period. The methodology is applied to a European TSO dataset from 2014 to 2019. The overall results show that in comparison to the industry benchmark, an accuracy improvement in terms of MAE and RMSE of about 60% is achieved. The best model is based on the ridge estimator and uses a specific non-standard shrinkage target. Due to the linear model structure, we can easily interpret the model output.
We address the issue of public or bank holidays in electricity load modeling and forecasting. Special characteristics of public holidays such as their classification into fixed-date and weekday ...holidays are discussed in detail. We present state-of-the-art techniques to deal with public holidays such as removing them from the data set, treating them as Sunday dummy or introducing separate holiday dummies. We analyze pros and cons of these approaches and provide a large load forecasting study for Germany that compares the techniques using standard performance and significance measures. Finally, we give general recommendations for the treatment of public holidays in energy forecasting to suggest how the drawbacks particular to most of the state-of-the-art methods can be mitigated. This is especially useful, as the incorporation of holiday effects can improve the forecasting accuracy during public holidays periods by more than 80%, but even for non-holidays periods, the forecasting error can be reduced by approximately 10%.
We present a winning method of the IEEE DataPort Competition on Day-Ahead Electricity Demand Forecasting: Post-COVID Paradigm. The day-ahead load forecasting approach is based on a novel online ...forecast combination of multiple point prediction models. It contains four steps: i) data cleaning and preprocessing, ii) a new holiday adjustment procedure, iii) training of individual forecasting models, iv) forecast combination by smoothed Bernstein Online Aggregation (BOA). The approach is flexible and can quickly adjust to new energy system situations as they occurred during and after COVID-19 shutdowns. The ensemble of individual prediction models ranges from simple time series models to sophisticated models like generalized additive models (GAMs) and high-dimensional linear models estimated by lasso. They incorporate autoregressive, calendar, and weather effects efficiently. All steps contain novel concepts that contribute to the excellent forecasting performance of the proposed method. It is especially true for the holiday adjustment procedure and the fully adaptive smoothed BOA approach.
We present a methodology for probabilistic load forecasting that is based on lasso (least absolute shrinkage and selection operator) estimation. The model considered can be regarded as a bivariate ...time-varying threshold autoregressive(AR) process for the hourly electric load and temperature. The joint modeling approach incorporates the temperature effects directly, and reflects daily, weekly, and annual seasonal patterns and public holiday effects. We provide two empirical studies, one based on the probabilistic load forecasting track of the Global Energy Forecasting Competition 2014 (GEFCom2014-L), and the other based on another recent probabilistic load forecasting competition that follows a setup similar to that of GEFCom2014-L. In both empirical case studies, the proposed methodology outperforms two multiple linear regression based benchmarks from among the top eight entries to GEFCom2014-L.
In this paper we present a regression based model for day-ahead electricity spot prices. We estimate the considered linear regression model by the lasso estimation method. The lasso approach allows ...for many possible parameters in the model, but also shrinks and sparsifies the parameters automatically to avoid overfitting. Thus, it is able to capture the autoregressive intraday dependency structure of the electricity price well. We discuss in detail the estimation results which provide insights to the intraday behavior of electricity prices. We perform an out-of-sample forecasting study for several European electricity markets. The results illustrate well that the efficient lasso based estimation technique can exhibit advantages from two popular model approaches.
The liberalization of electricity markets and the development of renewable energy sources has led to new challenges for decision makers. These challenges are accompanied by an increasing uncertainty ...about future electricity price movements. The increasing amount of papers, which aim to model and predict electricity prices for a short period of time provided new opportunities for market participants. However, the electricity price literature seem to be very scarce on the issue of medium- to long-term price forecasting, which is mandatory for investment and political decisions. Our paper closes this gap by introducing a new approach to simulate electricity prices with hourly resolution for several months up to three years. Considering the uncertainty of future events we are able to provide probabilistic forecasts which are able to detect probabilities for price spikes even in the long-run. As market we decided to use the EPEX day-ahead electricity market for Germany and Austria. Our model extends the X-Model which mainly utilizes the sale and purchase curve for electricity day-ahead auctions. By applying our procedure we are able to give probabilities for the due to the EEG practical relevant event of six consecutive hours of negative prices. We find that using the supply and demand curve based model in the long-run yields realistic patterns for the time series of electricity prices and leads to promising results considering common error measures.
•There are no long-term probabilistic electricity price forecasting models yet.•Literature on probabilistic forecasting is sparse and heterogeneous.•Extending the X-model provides remarkable long-term forecasts.•Statististical methods and fundamental methods combined provide forecast stability.•Bootstrapping forecasted prices shows high accuracy for density forecasts.
The present paper considers the problem of choosing among a collection of competing electricity price forecasting models to address a stochastic decision-making problem. We propose an event-based ...evaluation framework applicable to any optimization problem, where uncertainty is captured through ensembles. The task of forecast evaluation is simplified from assessing a multivariate distribution over prices to assessing a univariate distribution over a binary outcome directly linked to the underlying decision-making problem. The applicability of our framework is demonstrated for two exemplary profit-maximization problems of a risk-neutral energy trader, (i) the optimal operation of a pumped-hydro storage plant and (ii) the optimal trading of subsidized renewable energy in Germany. We compare and contrast the approach with the full probabilistic and profit–loss-based evaluation frameworks. It is concluded that the event-based evaluation framework more reliably identifies economically equivalent forecasting models, and in addition, the results suggest that an event-based evaluation specifically tailored to the rare event is crucial for decision-making problems linked to rare events.
The M5 competition uncertainty track aims for probabilistic forecasting of sales of thousands of Walmart retail goods. We show that the M5 competition data face strong overdispersion and sporadic ...demand, especially zero demand. We discuss modeling issues concerning adequate probabilistic forecasting of such count data processes. Unfortunately, the majority of popular prediction methods used in the M5 competition (e.g. lightgbm and xgboost GBMs) fail to address the data characteristics, due to the considered objective functions. Distributional forecasting provides a suitable modeling approach to overcome those problems. The GAMLSS framework allows for flexible probabilistic forecasting using low-dimensional distributions. We illustrate how the GAMLSS approach can be applied to M5 competition data by modeling the location and scale parameters of various distributions, e.g. the negative binomial distribution. Finally, we discuss software packages for distributional modeling and their drawbacks, like the R package gamlss with its package extensions, and (deep) distributional forecasting libraries such as TensorFlow Probability.
This paper covers predicting high-resolution electricity peak demand features given lower-resolution data. This is a relevant setup as it answers whether limited higher-resolution monitoring helps to ...estimate future high-resolution peak loads when the high-resolution data is no longer available. That question is particularly interesting for network operators considering replacing high-resolution monitoring by predictive models due to economic considerations. We propose models to predict half-hourly minima and maxima of high-resolution (every minute) electricity load data while model inputs are of a lower resolution (30 min). We combine predictions of generalized additive models (GAM) and deep artificial neural networks (DNN), which are popular in load forecasting. We extensively analyze the prediction models, including the input parameters’ importance, focusing on load, weather, and seasonal effects. The proposed method won a data competition organized by Western Power Distribution, a British distribution network operator. In addition, we provide a rigorous evaluation study that goes beyond the competition frame to analyze the models’ robustness. The results show that the proposed methods are superior to the competition benchmark concerning the out-of-sample root mean squared error (RMSE). This holds regarding the competition month and the supplementary evaluation study, which covers an additional eleven months. Overall, our proposed model combination reduces the out-of-sample RMSE by 57.4% compared to the benchmark.
Display omitted
•Propose a winning method for predicting high-resolution power load with low-resolution data.•Analyze the descriptive data characteristics of relevant input data (esp. load and weather inputs) in detail for suitable usage for predictive analytics.•Design machine learning models, GAMs and DNNs, which combined yield the best performance among competitors.•Provide a rigorous predicting study that ensures the robustness of the method.•Study and interpret the explanatory power of relevant inputs and discuss hyperparameter tuning results.
PDF
