For practical reasons, many forecasts of case, hospitalization and death counts in the context of the current COVID-19 pandemic are issued in the form of central predictive intervals at various ...levels. This is also the case for the forecasts collected in the COVID-19 Forecast Hub (https://covid19forecasthub.org/). Forecast evaluation metrics like the logarithmic score, which has been applied in several infectious disease forecasting challenges, are then not available as they require full predictive distributions. This article provides an overview of how established methods for the evaluation of quantile and interval forecasts can be applied to epidemic forecasts in this format. Specifically, we discuss the computation and interpretation of the weighted interval score, which is a proper score that approximates the continuous ranked probability score. It can be interpreted as a generalization of the absolute error to probabilistic forecasts and allows for a decomposition into a measure of sharpness and penalties for over- and underprediction.
The characteristics of influenza seasons varies substantially from year to year, posing challenges for public health preparation and response. Influenza forecasting is used to inform seasonal ...outbreak response, which can in turn potentially reduce the societal impact of an epidemic. The United States Centers for Disease Control and Prevention, in collaboration with external researchers, has run an annual prospective influenza forecasting exercise, known as the FluSight challenge. A subset of participating teams has worked together to produce a collaborative multi-model ensemble, the FluSight Network ensemble. Uniting theoretical results from the forecasting literature with domain-specific forecasts from influenza outbreaks, we applied parametric forecast combination methods that simultaneously optimize individual model weights and calibrate the ensemble via a beta transformation. We used the beta-transformed linear pool and the finite beta mixture model to produce ensemble forecasts retrospectively for the 2016/2017 to 2018/2019 influenza seasons in the U.S. We compared their performance to methods currently used in the FluSight challenge, namely the equally weighted linear pool and the linear pool. Ensemble forecasts produced from methods with a beta transformation were shown to outperform those from the equally weighted linear pool and the linear pool for all week-ahead targets across in the test seasons based on average log scores. We observed improvements in overall accuracy despite the beta-transformed linear pool or beta mixture methods' modest under-prediction across all targets and seasons. Combination techniques that explicitly adjust for known calibration issues in linear pooling should be considered to improve ensemble probabilistic scores in outbreak settings.
Accurate and reliable predictions of infectious disease dynamics can be valuable to public health organizations that plan interventions to decrease or prevent disease transmission. A great variety of ...models have been developed for this task, using different model structures, covariates, and targets for prediction. Experience has shown that the performance of these models varies; some tend to do better or worse in different seasons or at different points within a season. Ensemble methods combine multiple models to obtain a single prediction that leverages the strengths of each model. We considered a range of ensemble methods that each form a predictive density for a target of interest as a weighted sum of the predictive densities from component models. In the simplest case, equal weight is assigned to each component model; in the most complex case, the weights vary with the region, prediction target, week of the season when the predictions are made, a measure of component model uncertainty, and recent observations of disease incidence. We applied these methods to predict measures of influenza season timing and severity in the United States, both at the national and regional levels, using three component models. We trained the models on retrospective predictions from 14 seasons (1997/1998 - 2010/2011) and evaluated each model's prospective, out-of-sample performance in the five subsequent influenza seasons. In this test phase, the ensemble methods showed overall performance that was similar to the best of the component models, but offered more consistent performance across seasons than the component models. Ensemble methods offer the potential to deliver more reliable predictions to public health decision makers.
Forecasting has emerged as an important component of informed, data-driven decision-making in a wide array of fields. We introduce a new data model for probabilistic predictions that encompasses a ...wide range of forecasting settings. This framework clearly defines the constituent parts of a probabilistic forecast and proposes one approach for representing these data elements. The data model is implemented in Zoltar, a new software application that stores forecasts using the data model and provides standardized API access to the data. In one real-time case study, an instance of the Zoltar web application was used to store, provide access to, and evaluate real-time forecast data on the order of 10\(^7\) rows, provided by over 20 international research teams from academia and industry making forecasts of the COVID-19 outbreak in the US. Tools and data infrastructure for probabilistic forecasts, such as those introduced here, will play an increasingly important role in ensuring that future forecasting research adheres to a strict set of rigorous and reproducible standards.
The U.S. COVID-19 Forecast Hub aggregates forecasts of the short-term burden of COVID-19 in the United States from many contributing teams. We study methods for building an ensemble that combines ...forecasts from these teams. These experiments have informed the ensemble methods used by the Hub. To be most useful to policy makers, ensemble forecasts must have stable performance in the presence of two key characteristics of the component forecasts: (1) occasional misalignment with the reported data, and (2) instability in the relative performance of component forecasters over time. Our results indicate that in the presence of these challenges, an untrained and robust approach to ensembling using an equally weighted median of all component forecasts is a good choice to support public health decision makers. In settings where some contributing forecasters have a stable record of good performance, trained ensembles that give those forecasters higher weight can also be helpful.
Abstract
Aims
To objectively appraise evidence for possible adverse effects of long-term statin therapy on glucose homeostasis, cognitive, renal and hepatic function, and risk for haemorrhagic stroke ...or cataract.
Methods and results
A literature search covering 2000–2017 was performed. The Panel critically appraised the data and agreed by consensus on the categorization of reported adverse effects. Randomized controlled trials (RCTs) and genetic studies show that statin therapy is associated with a modest increase in the risk of new-onset diabetes mellitus (about one per thousand patient-years), generally defined by laboratory findings (glycated haemoglobin ≥6.5); this risk is significantly higher in the metabolic syndrome or prediabetes. Statin treatment does not adversely affect cognitive function, even at very low levels of low-density lipoprotein cholesterol and is not associated with clinically significant deterioration of renal function, or development of cataract. Transient increases in liver enzymes occur in 0.5–2% of patients taking statins but are not clinically relevant; idiosyncratic liver injury due to statins is very rare and causality difficult to prove. The evidence base does not support an increased risk of haemorrhagic stroke in individuals without cerebrovascular disease; a small increase in risk was suggested by the Stroke Prevention by Aggressive Reduction of Cholesterol Levels study in subjects with prior stroke but has not been confirmed in the substantive evidence base of RCTs, cohort studies and case–control studies.
Conclusion
Long-term statin treatment is remarkably safe with a low risk of clinically relevant adverse effects as defined above; statin-associated muscle symptoms were discussed in a previous Consensus Statement. Importantly, the established cardiovascular benefits of statin therapy far outweigh the risk of adverse effects.
The neuropeptide oxytocin (OT) is associated with a plethora of social behaviors, and is a key topic at the intersection of psychology and biology. However, tools for measuring OT are still not fully ...developed. We describe a robust nano liquid chromatography-mass spectrometry (nanoLC-MS) platform for measuring the total amount of OT in human plasma/serum. OT binds strongly to plasma proteins, but a reduction/alkylation (R/A) procedure breaks this bond, enabling ample detection of total OT. The method (R/A + robust nanoLC-MS) was used to determine total OT plasma/serum levels to startlingly high concentrations (high pg/mL-ng/mL). Similar results were obtained when combining R/A and ELISA. Compared to measuring free OT, measuring total OT can have advantages in e.g. biomarker studies.
PDF
