Academic research relies extensively on macroeconomic variables to forecast the U.S. equity risk premium, with relatively little attention paid to the technical indicators widely employed by ...practitioners. Our paper fills this gap by comparing the predictive ability of technical indicators with that of macroeconomic variables. Technical indicators display statistically and economically significant in-sample and out-of-sample predictive power, matching or exceeding that of macroeconomic variables. Furthermore, technical indicators and macroeconomic variables provide complementary information over the business cycle: technical indicators better detect the typical decline in the equity risk premium near business-cycle peaks, whereas macroeconomic variables more readily pick up the typical rise in the equity risk premium near cyclical troughs. Consistent with this behavior, we show that combining information from both technical indicators and macroeconomic variables significantly improves equity risk premium forecasts versus using either type of information alone. Overall, the substantial countercyclical fluctuations in the equity risk premium appear well captured by the combined information in technical indicators and macroeconomic variables.
Data, as supplemental material, are available at
http://dx.doi.org/10.1287/mnsc.2013.1838
.
This paper was accepted by Wei Jiang, finance.
Survey respondents who make point predictions and histogram forecasts of macro-variables reveal both how uncertain they believe the future to be, ex ante, as well as their ex post performance. ...Macroeconomic forecasters tend to be overconfident at horizons of a year or more, but overestimate (i.e., are underconfident regarding) the uncertainty surrounding their predictions at short horizons. Ex ante
uncertainty remains at a high level compared to the ex post measure as the forecast horizon shortens. There is little evidence of a link between individuals' ex post forecast accuracy and their ex ante
subjective assessments.
Understanding how the degree of information frictions varies among economic agents is of utmost importance for macroeconomic dynamics. We document and compare the frequency of forecast revisions and ...cross-sectional disagreement in inflation expectations among five categories of agents: households, firms, professional forecasters, policymakers and participants to laboratory experiments. First, we provide evidence of a heterogeneous frequency of forecast revisions across categories of agents, with policymakers revising more frequently their forecasts than firms and professional forecasters. Households revise less frequently. Second, all categories exhibit cross-sectional disagreement. There is however a strong heterogeneity: while policymakers and professional forecasters exhibit low disagreement, firms and households show strong disagreement. Our analysis suggests that the nature of information frictions is closer to noisy information model features. We also explore the external validity of experimental expectations.
•Interval and distributional forecast combinations at the state and national level.•Combining considered with frequent entry and exit of forecasting teams.•Weighted combining proposed, based on ...inverse of interval and distribution scores.•For early periods, the median of the available forecasts was most effective.•For later periods, weighting the forecasts was the most accurate combination.
The COVID-19 pandemic has placed forecasting models at the forefront of health policy making. Predictions of mortality, cases and hospitalisations help governments meet planning and resource allocation challenges. In this paper, we consider the weekly forecasting of the cumulative mortality due to COVID-19 at the national and state level in the U.S. Optimal decision-making requires a forecast of a probability distribution, rather than just a single point forecast. Interval forecasts are also important, as they can support decision making and provide situational awareness. We consider the case where probabilistic forecasts have been provided by multiple forecasting teams, and we combine the forecasts to extract the wisdom of the crowd. We use a dataset that has been made publicly available from the COVID-19 Forecast Hub. A notable feature of the dataset is that the availability of forecasts from participating teams varies greatly across the 40 weeks in our study. We evaluate the accuracy of combining methods that have been previously proposed for interval forecasts and predictions of probability distributions. These include the use of the simple average, the median, and trimming methods. In addition, we propose several new weighted combining methods. Our results show that, although the median was very useful for the early weeks of the pandemic, the simple average was preferable thereafter, and that, as a history of forecast accuracy accumulates, the best results can be produced by a weighted combining method that uses weights that are inversely proportional to the historical accuracy of the individual forecasting teams.
Making and Evaluating Point Forecasts Gneiting, Tilmann
Journal of the American Statistical Association,
06/2011, Volume:
106, Issue:
494
Journal Article
Peer reviewed
Open access
Typically, point forecasting methods are compared and assessed by means of an error measure or scoring function, with the absolute error and the squared error being key examples. The individual ...scores are averaged over forecast cases, to result in a summary measure of the predictive performance, such as the mean absolute error or the mean squared error. I demonstrate that this common practice can lead to grossly misguided inferences, unless the scoring function and the forecasting task are carefully matched. Effective point forecasting requires that the scoring function be specified ex ante, or that the forecaster receives a directive in the form of a statistical functional, such as the mean or a quantile of the predictive distribution. If the scoring function is specified ex ante, the forecaster can issue the optimal point forecast, namely, the Bayes rule. If the forecaster receives a directive in the form of a functional, it is critical that the scoring function be consistent for it, in the sense that the expected score is minimized when following the directive. A functional is elicitable if there exists a scoring function that is strictly consistent for it. Expectations, ratios of expectations and quantiles are elicitable. For example, a scoring function is consistent for the mean functional if and only if it is a Bregman function. It is consistent for a quantile if and only if it is generalized piecewise linear. Similar characterizations apply to ratios of expectations and to expectiles. Weighted scoring functions are consistent for functionals that adapt to the weighting in peculiar ways. Not all functionals are elicitable; for instance, conditional value-at-risk is not, despite its popularity in quantitative finance.
Weather regime forecasts are a prominent use case of sub‐seasonal prediction in the midlatitudes. A systematic evaluation and understanding of year‐round sub‐seasonal regime forecast performance is ...still missing, however. Here we evaluate the representation of and forecast skill for seven year‐round Atlantic–European weather regimes in sub‐seasonal reforecasts from the European Centre for Medium‐Range Weather Forecasts. Forecast calibration improves regime frequency biases and forecast skill most strongly in summer, but scarcely in winter, due to considerable large‐scale flow biases in summer. The average regime skill horizon in winter is about 5 days longer than in summer and spring, and 3 days longer than in autumn. The Zonal Regime and Greenland Blocking tend to have the longest year‐round skill horizon, which is driven by their high persistence in winter. The year‐round skill is lowest for the European Blocking, which is common for all seasons but most pronounced in winter and spring. For the related, more northern Scandinavian Blocking, the skill is similarly low in winter and spring but higher in summer and autumn. We further show that the winter average regime skill horizon tends to be enhanced following a strong stratospheric polar vortex (SPV), but reduced following a weak SPV. Likewise, the year‐round average regime skill horizon tends to be enhanced following phases 4 and 7 of the Madden–Julian Oscillation (MJO) but reduced following phase 2, driven by winter but also autumn and spring. Our study thus reveals promising potential for year‐round sub‐seasonal regime predictions. Further model improvements can be achieved by reduction of the considerable large‐scale flow biases in summer, better understanding and modeling of blocking in the European region, and better exploitation of the potential predictability provided by weak SPV states and specific MJO phases in winter and the transition seasons.
The overall sub‐seasonal forecast performance (biases and skill) for predicting seven year‐round Atlantic–European weather regimes is highest in winter and lowest in summer. The year‐round skill horizon is shortest for the European Blocking and longest for the Zonal Regime and Greenland Blocking (see figure). Furthermore, the winter skill horizon tends to be enhanced following a strong stratospheric polar vortex but reduced following a weak one. Madden–Julian Oscillation phases 4 and 7 tend to increase and phase 2 to decrease the year‐round skill horizon.
Five university-based research groups competed to recruit forecasters, elicit their predictions, and aggregate those predictions to assign the most accurate probabilities to events in a 2-year ...geopolitical forecasting tournament. Our group tested and found support for three psychological drivers of accuracy: training, teaming, and tracking. Probability training corrected cognitive biases, encouraged forecasters to use reference classes, and provided forecasters with heuristics, such as averaging when multiple estimates were available. Teaming allowed forecasters to share information and discuss the rationales behind their beliefs. Tracking placed the highest performers (top 2% from Year 1) in elite teams that worked together. Results showed that probability training, team collaboration, and tracking improved both calibration and resolution. Forecasting is often viewed as a statistical problem, but forecasts can be improved with behavioral interventions. Training, teaming, and tracking are psychological interventions that dramatically increased the accuracy of forecasts. Statistical algorithms (reported elsewhere) improved the accuracy of the aggregation. Putting both statistics and psychology to work produced the best forecasts 2 years in a row.
We consider two ways to aggregate expert opinions using simple averages: averaging probabilities and averaging quantiles. We examine analytical properties of these forecasts and compare their ability ...to harness the wisdom of the crowd. In terms of location, the two average forecasts have the same mean. The average quantile forecast is always sharper: it has lower variance than the average probability forecast. Even when the average probability forecast is overconfident, the shape of the average quantile forecast still offers the possibility of a better forecast. Using probability forecasts for gross domestic product growth and inflation from the Survey of Professional Forecasters, we present evidence that both when the average probability forecast is overconfident and when it is underconfident, it is outperformed by the average quantile forecast. Our results show that averaging quantiles is a viable alternative and indicate some conditions under which it is likely to be more useful than averaging probabilities.
This paper was accepted by Peter Wakker, decision analysis.
We explore some recent, and not so recent, developments concerning the use of probability forecasts and their combination in decision making. Despite these advances, challenges still exist. We expand ...on some important challenges influencing the “goodness” of combined probability forecasts such as miscalibration, dependence among forecasters, and selection of an appropriate evaluation measure while connecting the processes of aggregating and evaluating forecasts to decision making. Through three important applications from the domains of meteorology, economics, and political science, we illustrate state-of-the-art usage of probability forecasts: how they are combined, evaluated, and communicated to stakeholders. We expect to see greater use and aggregation of probability forecasts, especially given developments in statistical modeling, machine learning, and expert forecasting; the popularity of forecasting competitions; and the increased reporting of probabilities in the media. Our vision is that increased exposure to and improved visualizations of probability forecasts will enhance the public’s understanding of probabilities and how they can contribute to better decisions.
This paper examines the effects of macroeconomic and financial uncertainty on professional economic forecasts. Using the predictions of four variables sourced from the Bloomberg survey, we find that ...macroeconomic uncertainty, a proxy for the complexity of the forecasting task, is associated with high disagreement and low accuracy. By contrast, the results show that financial uncertainty is associated with low disagreement and high accuracy, which suggests that financial uncertainty encourages forecasters to adhere to the consensus to avoid large forecast errors. Furthermore, we find that the forecaster rank moderates the effect of macroeconomic uncertainty, indicating that high-ability forecasters can better navigate periods of high uncertainty. This study advances the understanding of forecasting behavior under uncertainty.
PDF
