Machine Learning (ML) methods have been proposed in the academic literature as alternatives to statistical ones for time series forecasting. Yet, scant evidence is available about their relative ...performance in terms of accuracy and computational requirements. The purpose of this paper is to evaluate such performance across multiple forecasting horizons using a large subset of 1045 monthly time series used in the M3 Competition. After comparing the post-sample accuracy of popular ML methods with that of eight traditional statistical ones, we found that the former are dominated across both accuracy measures used and for all forecasting horizons examined. Moreover, we observed that their computational requirements are considerably greater than those of statistical methods. The paper discusses the results, explains why the accuracy of ML models is below that of statistical ones and proposes some possible ways forward. The empirical results found in our research stress the need for objective and unbiased ways to test the performance of forecasting methods that can be achieved through sizable and open competitions allowing meaningful comparisons and definite conclusions.
The M4 competition is the continuation of three previous competitions started more than 45 years ago whose purpose was to learn how to improve forecasting accuracy, and how such learning can be ...applied to advance the theory and practice of forecasting. The purpose of M4 was to replicate the results of the previous ones and extend them into three directions: First significantly increase the number of series, second include Machine Learning (ML) forecasting methods, and third evaluate both point forecasts and prediction intervals. The five major findings of the M4 Competitions are: 1. Out Of the 17 most accurate methods, 12 were “combinations” of mostly statistical approaches. 2. The biggest surprise was a “hybrid” approach that utilized both statistical and ML features. This method’s average sMAPE was close to 10% more accurate than the combination benchmark used to compare the submitted methods. 3. The second most accurate method was a combination of seven statistical methods and one ML one, with the weights for the averaging being calculated by a ML algorithm that was trained to minimize the forecasting. 4. The two most accurate methods also achieved an amazing success in specifying the 95% prediction intervals correctly. 5. The six pure ML methods performed poorly, with none of them being more accurate than the combination benchmark and only one being more accurate than Naïve2. This paper presents some initial results of M4, its major findings and a logical conclusion. Finally, it outlines what the authors consider to be the way forward for the field of forecasting.
•Generalize Theta for automatic time series prediction.•Consider both linear and non-linear trends.•Adjust the slope of such trends.•Introduce a multiplicative expression of the Theta model.•Provide ...both accurate point forecasts and precise prediction intervals.
The Theta method became popular due to its superior performance in the M3 forecasting competition. Since then, although it has been shown that Theta provides accurate forecasts for various types of data, being a solid benchmark to beat, limited research has been conducted to exploit its full potential and generalize its reach. This paper examines three extensions on Theta’s framework to boost its performance. This includes (i) considering both linear and non-linear trends, (ii) allowing to adjust the slope of such trends, and (iii) introducing a multiplicative expression of the underlying forecasting model along with the existing, additive one. The proposed modifications transform Theta into a generalized forecasting algorithm, suitable for automatic time series predictions. The proposed algorithm is evaluated using the series of the M, M3, and M4 competitions. Such an evaluation shows that the proposed approach produces more accurate forecasts than the original, classic Theta, both in terms of point forecasts and prediction intervals, and is also more accurate than other well-known methods for yearly series.
Temporal hierarchies have been widely used during the past few years as they are capable to provide more accurate coherent forecasts at different planning horizons. However, they still display some ...limitations, being mainly subject to the forecasting methods used for generating the base forecasts and the particularities of the examined series. This paper deals with such limitations by considering three different strategies: (i) combining forecasts of multiple methods, (ii) applying bias adjustments and (iii) selectively implementing temporal hierarchies to avoid seasonal shrinkage. The proposed strategies can be applied either separately or simultaneously, being complements to the method considered for reconciling the base forecasts and completely independent from each other. Their effect is evaluated using the monthly series of the M and M3 competitions. The results are very promising, displaying lots of potential for improving the performance of temporal hierarchies, both in terms of accuracy and bias.
In alignment with the European Union’s legislation, Greece submitted its final 10-year National Energy and Climate Plan (NECP) in December 2019, setting more ambitious energy and climate targets than ...those originally proposed in the draft version of the document. Apart from higher penetration of renewable energy sources (RES), the final NECP projects also zero carbon use in power generation till 2030. Although decarbonization has long been regarded beneficial for economies that base their energy production on coal, as it is the case with Greece, the macroeconomic and societal ramifications of faster transitions to carbon-free economies remain highly unexplored. Under this context, in this paper, we soft-link energy models, namely Times-Greece and Primes, with a macroeconomic model, namely Global Trade Analysis Project (GTAP), to measure the effects of the final and draft NECPs on the Greek economy and evaluate the impact of higher decarbonization speeds. We find that the faster transition scenario displays both economic and societal merits, increasing Gross Domestic Product (GDP) and household income by about 1% and 7%, respectively.
Forecasting is a challenging task that typically requires making assumptions about the observed data but also the future conditions. Inevitably, any forecasting process will result in some degree of ...inaccuracy. The forecasting performance will further deteriorate as the uncertainty increases. In this article, we focus on univariate time series forecasting and we review five approaches that one can use to enhance the performance of standard extrapolation methods. Much has been written about the “wisdom of the crowds” and how collective opinions will outperform individual ones. We present the concept of the “wisdom of the data” and how data manipulation can result in information extraction which, in turn, translates to improved forecast accuracy by aggregating (combining) forecasts computed on different perspectives of the same data. We describe and discuss approaches that are based on the manipulation of local curvatures (theta method), temporal aggregation, bootstrapping, sub-seasonal and incomplete time series. We compare these approaches with regards to how they extract information from the data, their computational cost, and their performance.
Machine learning has shown great potential in various domains, but its appearance in inventory control optimization settings remains rather limited. We propose a novel inventory cost minimization ...framework that exploits advanced decision-tree based models to approximate inventory performance at an item level, considering demand patterns and key replenishment policy parameters as input. The suggested approach enables data-driven approximations that are faster to perform compared to standard inventory simulations, while being flexible in terms of the methods used for forecasting demand or estimating inventory level, lost sales, and number of orders, among others. Moreover, such approximations can be based on knowledge extracted from different sets of items than the ones being optimized, thus providing more accurate proposals in cases where historical data are scarce or highly affected by stock-outs. The framework was evaluated using part of the M5 competition’s data. Our results suggest that the proposed framework, and especially its transfer learning variant, can result in significant improvements, both in terms of total inventory cost and realized service level.
•A data-driven approach is introduced for inventory policy optimization.•Machine learning methods are used to approximate inventory performance.•The approach results in fast and accurate approximations.•The approach is applicable in transfer leaning settings.
We present a machine learning approach for applying (multiple) temporal aggregation in time series forecasting settings. The method utilizes a classification model that can be used to either select ...the most appropriate temporal aggregation level for producing forecasts or to derive weights to properly combine the forecasts generated at various levels. The classifier consists a meta-learner that correlates key time series features with forecasting accuracy, thus enabling a dynamic, data-driven selection or combination. Our experiments, conducted in two large data sets of slow- and fast-moving series, indicate that the proposed meta-learner can outperform standard forecasting approaches.
Weather variables are an important driver of power generation from renewable energy sources. However, accurately predicting such variables is a challenging task, which has a significant impact on the ...accuracy of the power generation forecasts. In this study, we explore the impact of imperfect weather forecasts on two classes of forecasting methods (statistical and machine learning) for the case of wind power generation. We perform a stress test analysis to measure the robustness of different methods on the imperfect weather input, focusing on both the point forecasts and the 95% prediction intervals. The results indicate that different methods should be considered according to the uncertainty characterizing the weather forecasts.
Forecasters have been using various criteria to select the most appropriate model from a pool of candidate models. This includes measurements on the in-sample accuracy of the models, information ...criteria, and cross-validation, among others. Although the latter two options are generally preferred due to their ability to tackle overfitting, in univariate time-series forecasting settings, limited work has been conducted to confirm their superiority. In this study, we compared such popular criteria for the case of the exponential smoothing family of models using a large data set of real series. Our results suggest that there is significant disagreement between the suggestions of the examined criteria and that, depending on the approach used, models of different complexity may be favored, with possible negative effects on the forecasting accuracy. Moreover, we find that simple in-sample error measures can effectively select forecasting models, especially when focused on the most recent observations in the series.
PDF
