Spatially and temporally varying adaptive inflation algorithms have been developed to combat the loss of variance during the forecast due to various model and sampling errors. The adaptive Bayesian ...scheme of Anderson uses available observations to update the Gaussian inflation distribution assigned for every state variable. The likelihood function of the inflation is computed using model-minus-data innovation statistics. A number of enhancements for this inflation scheme are proposed. To prevent excessive deflation, an inverse gamma distribution for the prior inflation is considered. A non-Gaussian distribution offers a flexible framework for the inflation variance to evolve during the update. The innovations are assumed random variables, and a correction term is added to the mode of the likelihood distribution such that the observed inflation is slightly larger. This modification improves the stability of the adaptive scheme by limiting the occurrence of negative and physically intolerable inflations. The enhanced scheme is compared to the original one in twin experiments using the Lorenz-63 model, the Lorenz-96 model, and an idealized, high-dimensional atmospheric model. Results show that the proposed enhancements are capable of generating accurate and consistent state estimates. Allowing moderate deflation is shown to be useful.
Full text
Available for:
DOBA, IZUM, KILJ, NUK, PILJ, PNG, SAZU, UILJ, UKNU, UL, UM, UPUK
Abstract
An ensemble Kalman filter reanalysis has been archived in the Research Data Archive at the National Center for Atmospheric Research. It used a CAM6 configuration of the Community Earth ...System Model (CESM), several million observations per day, and the Data Assimilation Research Testbed (DART). The data saved from this global,
$$\sim 1^\circ $$
∼
1
∘
resolution, 80 member ensemble span 2011–2019. They include ensembles of: sub-daily, real world, atmospheric forcing for use by all of the nonatmospheric models of CESM; weekly, CAM6, restart file sets; 6 hourly, prior hindcast estimates of the assimilated observations; 6 hourly, land model, plant growth variables, and 6 hourly, ensemble mean, gridded, atmospheric analyses. This data can be used for hindcast studies and data assimilation using component models of CESM; CAM6, CLM5, CICE5, POP2. MOM6, MOSART, and CISM; and non-CESM Earth system models. This large dataset (~ 120 Tb) has a unique combination of a large ensemble, high frequency, and multiyear time span, which provides opportunities for robust statistical analysis and use as a machine learning training dataset.
Full text
Available for:
IZUM, KILJ, NUK, PILJ, PNG, SAZU, UL, UM, UPUK
Abstract
Model errors and sampling errors produce inaccurate sample covariances that limit the performance of ensemble Kalman filters. Linearly hybridizing the flow-dependent ensemble-based ...covariance with a time-invariant background covariance matrix gives a better estimate of the true error covariance. Previous studies have shown this, both in theory and in practice. How to choose the weight for each covariance remains an open question especially in the presence of model biases. This study assumes the weighting coefficient to be a random variable and then introduces a Bayesian scheme to estimate it using the available data. The scheme takes into account the discrepancy between the ensemble mean and the observations, the ensemble variance, the static background variance, and the uncertainties in the observations. The proposed algorithm is first derived for a spatially constant weight and then this assumption is relaxed by estimating a unique scalar weight for each state variable. Using twin experiments with the 40-variable Lorenz 96 system, it is shown that the proposed scheme is able to produce quality forecasts even in the presence of severe sampling errors. The adaptive algorithm allows the hybrid filter to switch between an EnKF and a simple EnOI depending on the statistics of the ensemble. In the presence of model errors, the adaptive scheme demonstrates additional improvements compared with standard enhancements alone, such as inflation and localization. Finally, the potential of the spatially varying variant to accommodate challenging sparse observation networks is demonstrated. The computational efficiency and storage of the proposed scheme, which remain an obstacle, are discussed.
Full text
Available for:
DOBA, IZUM, KILJ, NUK, PILJ, PNG, SAZU, UILJ, UKNU, UL, UM, UPUK
•Multivariate updates of physical–biogeochemical fields outperform isolated updates.•Success depends on close correlation between physical and biogeochemical properties.•Multivariate updates can ...preserve correlation between density and nutrient properties.
Effective data assimilation methods for incorporating observations into marine biogeochemical models are required to improve hindcasts, nowcasts and forecasts of the ocean's biogeochemical state. Recent assimilation efforts have shown that updating model physics alone can degrade biogeochemical fields while only updating biogeochemical variables may not improve a model's predictive skill when the physical fields are inaccurate. Here we systematically investigate whether multivariate updates of physical and biogeochemical model states are superior to only updating either physical or biogeochemical variables. We conducted a series of twin experiments in an idealized ocean channel that experiences wind-driven upwelling. The forecast model was forced with biased wind stress and perturbed biogeochemical model parameters compared to the model run representing the “truth”. Taking advantage of the multivariate nature of the deterministic Ensemble Kalman Filter (DEnKF), we assimilated different combinations of synthetic physical (sea surface height, sea surface temperature and temperature profiles) and biogeochemical (surface chlorophyll and nitrate profiles) observations. We show that when biogeochemical and physical properties are highly correlated (e.g., thermocline and nutricline), multivariate updates of both are essential for improving model skill and can be accomplished by assimilating either physical (e.g., temperature profiles) or biogeochemical (e.g., nutrient profiles) observations. In our idealized domain, the improvement is largely due to a better representation of nutrient upwelling, which results in a more accurate nutrient input into the euphotic zone. In contrast, assimilating surface chlorophyll improves the model state only slightly, because surface chlorophyll contains little information about the vertical density structure. We also show that a degradation of the correlation between observed subsurface temperature and nutrient fields, which has been an issue in several previous assimilation studies, can be reduced by multivariate updates of physical and biogeochemical fields.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UL, UM, UPCLJ, UPUK, ZRSKP
Ensemble-based Data Assimilation (EDA) has been effectively applied to estimate model parameters through inverse modeling in subsurface flow and transport problems. To facilitate the management of ...EDA workflow and lower the barriers for adopting EDA-based parameter estimation in subsurface science, we develop a software framework linking the Data Assimilation Research Testbed (DART) with a massively parallel subsurface FLOw and TRANsport code PFLOTRAN. DART-PFLOTRAN enables an iterative EDA workflow based on the Ensemble Smoother for Multiple Data Assimilation method (ES-MDA) to improve estimation accuracy for nonlinear forward problems. We verify the implementation of ES-MDA in DART-PFLOTRAN using two synthetic cases designed to estimate static permeability and dynamic exchange fluxes across the riverbed from continuous temperature measurements. Both cases yield accurate estimations of the parameters compared to their synthetic truth. With a code base in Python and Fortran, DART-PFLOTRAN paves the way for large-scale inverse modeling using the sequential ES-MDA.
•A new data assimilation software was developed by linking DART with PFLOTRAN.•Iterative ensemble smoother was enabled in the software for static and dynamic parameter estimations.•The software was verified using synthetic cases that estimate permeability and exchange flux from groundwater temperature.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
•We introduce an ensemble dual formulation for coupled states estimation.•An EnKF-based algorithm is derived for assimilation into one-way coupled models.•We implement the ensemble dual strategy with ...a subsurface contaminant model.•The dual estimation provides more accurate estimates than the joint strategy.•The dual estimation is consistent and computationally feasible technique.
Modeling the spread of subsurface contaminants requires coupling a groundwater flow model with a contaminant transport model. Such coupling may provide accurate estimates of future subsurface hydrologic states if essential flow and contaminant data are assimilated in the model. Assuming perfect flow, an ensemble Kalman filter (EnKF) can be used for direct data assimilation into the transport model. This is, however, a crude assumption as flow models can be subject to many sources of uncertainty. If the flow is not accurately simulated, contaminant predictions will likely be inaccurate even after successive Kalman updates of the contaminant model with the data. The problem is better handled when both flow and contaminant states are concurrently estimated using the traditional joint state augmentation approach. In this paper, we introduce a dual estimation strategy for data assimilation into a one-way coupled system by treating the flow and the contaminant models separately while intertwining a pair of distinct EnKFs, one for each model. The presented strategy only deals with the estimation of state variables but it can also be used for state and parameter estimation problems. This EnKF-based dual state-state estimation procedure presents a number of novel features: (i) it allows for simultaneous estimation of both flow and contaminant states in parallel; (ii) it provides a time consistent sequential updating scheme between the two models (first flow, then transport); (iii) it simplifies the implementation of the filtering system; and (iv) it yields more stable and accurate solutions than does the standard joint approach. We conducted synthetic numerical experiments based on various time stepping and observation strategies to evaluate the dual EnKF approach and compare its performance with the joint state augmentation approach. Experimental results show that on average, the dual strategy could reduce the estimation error of the coupled states by 15% compared with the joint approach. Furthermore, the dual estimation is proven to be very effective computationally, recovering accurate estimates at a reasonable cost.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UL, UM, UPCLJ, UPUK
Ensemble Kalman filtering (EnKF) is an efficient approach to addressing uncertainties in subsurface groundwater models. The EnKF sequentially integrates field data into simulation models to obtain a ...better characterization of the model's state and parameters. These are generally estimated following joint and dual filtering strategies, in which, at each assimilation cycle, a forecast step by the model is followed by an update step with incoming observations. The joint EnKF directly updates the augmented state-parameter vector, whereas the dual EnKF empirically employs two separate filters, first estimating the parameters and then estimating the state based on the updated parameters. To develop a Bayesian consistent dual approach and improve the state-parameter estimates and their consistency, we propose in this paper a one-step-ahead (OSA) smoothing formulation of the state-parameter Bayesian filtering problem from which we derive a new dual-type EnKF, the dual EnKFOSA. Compared with the standard dual EnKF, it imposes a new update step to the state, which is shown to enhance the performance of the dual approach with almost no increase in the computational cost. Numerical experiments are conducted with a two-dimensional (2-D) synthetic groundwater aquifer model to investigate the performance and robustness of the proposed dual EnKFOSA, and to evaluate its results against those of the joint and dual EnKFs. The proposed scheme is able to successfully recover both the hydraulic head and the aquifer conductivity, providing further reliable estimates of their uncertainties. Furthermore, it is found to be more robust to different assimilation settings, such as the spatial and temporal distribution of the observations, and the level of noise in the data. Based on our experimental setups, it yields up to 25 % more accurate state and parameter estimations than the joint and dual approaches.
Abstract
In this study, a one-step-ahead ensemble Kalman smoother (EnKS) is introduced for the purposes of parameter estimation. The potential for this system to provide new constraints on the ...surface-exchange coefficients of momentum (
C
d
) and enthalpy (
C
k
) is then explored using a series of observing system simulation experiments (OSSEs). The surface-exchange coefficients to be estimated within the data assimilation system are highly uncertain, especially at high wind speeds, and are well known to be important model parameters influencing the intensity and structure of tropical cyclones in numerical simulations. One major benefit of the developed one-step-ahead EnKS is that it allows for simultaneous updates of the rapidly evolving model state variables found in tropical cyclones using a short assimilation window and a long smoother window for the parameter updates, granting sufficient time for sensitivity to the parameters to develop. Overall, OSSEs demonstrate potential for this approach to accurately constrain parameters controlling the amplitudes of
C
d
and
C
k
, but the degree of success in recovering the truth model parameters varies throughout the tropical cyclone life cycle. During the rapid intensification phase, rapidly growing errors in the model state project onto the parameter updates and result in an overcorrection of the parameters. After the rapid intensification phase, however, the parameters are correctly adjusted back toward the truth values. Last, the relative success of parameter estimation in recovering the truth model parameter values also has sensitivity to the ensemble size and smoothing forecast length, each of which are explored.
Significance Statement
Large uncertainty in the surface-exchange coefficients of momentum and heat/moisture exists for hurricane conditions. This is a problem because the numerical weather model predictions of hurricane intensity and storm structure are sensitive to the surface-exchange coefficient values used. In this study we use data assimilation, or the relationships estimated between the surface-exchange coefficients and forecasted observations, to constrain uncertainty in the model’s surface-exchange coefficient values. More specifically, an approach to limit both the rapidly growing errors associated with the hurricane itself and the hurricane’s accumulated response to the surface-exchange coefficient values is presented. Overall, this approach has potential to accurately estimate the surface-exchange coefficients, but the success depends on the number of forecast realizations used and how rapidly the hurricane is changing.
Full text
Available for:
DOBA, IZUM, KILJ, NUK, PILJ, PNG, SAZU, UILJ, UKNU, UL, UM, UPUK
Abstract
Sampling errors and model errors are major drawbacks from which ensemble Kalman filters suffer. Sampling errors arise because of the use of a limited ensemble size, while model errors are ...deficiencies in the dynamics and underlying parameterizations that may yield biases in the model’s prediction. In this study, we propose a new time-adaptive posterior inflation algorithm in which the analyzed ensemble anomalies are locally inflated. The proposed inflation strategy is computationally efficient and is aimed at restoring enough spread in the analysis ensemble after assimilating the observations. The performance of this scheme is tested against the relaxation to prior spread (RTPS) and adaptive prior inflation. For this purpose, two model are used: the three-variable Lorenz 63 system and the Community Atmosphere Model (CAM). In CAM, global refractivity, temperature, and wind observations from several sources are incorporated to perform a set of assimilation experiments using the Data Assimilation Research Testbed (DART). The proposed scheme is shown to yield better quality forecasts than the RTPS. Assimilation results further suggest that when model errors are small, both prior and posterior inflation are able to mitigate sampling errors with a slight advantage to posterior inflation. When large model errors, such as wind and temperature biases, are present, prior inflation is shown to be more accurate than posterior inflation. Densely observed regions as in the Northern Hemisphere present numerous challenges to the posterior inflation algorithm. A compelling enhancement to the performance of the filter is achieved by combining both adaptive inflation schemes.
Full text
Available for:
DOBA, IZUM, KILJ, NUK, PILJ, PNG, SAZU, UILJ, UKNU, UL, UM, UPUK
Predicting major floods during extreme rainfall events remains an important challenge. Rapid changes in flows over short timescales, combined with multiple sources of model error, makes it difficult ...to accurately simulate intense floods. This study presents a general data assimilation framework that aims to improve flood predictions in channel routing models. Hurricane Florence, which caused catastrophic flooding and damages in the Carolinas in September 2018, is used as a case study. The National Water Model (NWM) configuration of the WRF-Hydro modeling framework is interfaced with the Data Assimilation Research Testbed (DART) to produce ensemble streamflow forecasts and analyses. Instantaneous streamflow observations from 107 United States Geological Survey (USGS) gauges are assimilated for a period of 1 month. The data assimilation (DA) system developed in this paper explores two novel contributions, namely (1) along-the-stream (ATS) covariance localization and (2) spatially and temporally varying adaptive covariance inflation. ATS localization aims to mitigate not only spurious correlations, due to limited ensemble size, but also physically incorrect correlations between unconnected and indirectly connected state variables in the river network. We demonstrate that ATS localization provides improved information propagation during the model update. Adaptive prior inflation is used to tackle errors in the prior, including large model biases which often occur in flooding situations. Analysis errors incurred during the update are addressed using posterior inflation. Results show that ATS localization is a crucial ingredient of our hydrologic DA system, providing at least 40 % more accurate (root mean square error) streamflow estimates than regular, Euclidean distance-based localization. An assessment of hydrographs indicates that adaptive inflation is extremely useful and perhaps indispensable for improving the forecast skill during flooding events with significant model errors. We argue that adaptive prior inflation is able to serve as a vigorous bias correction scheme which varies both spatially and temporally. Major improvements over the model's severely underestimated streamflow estimates are suggested along the Pee Dee River in South Carolina, and many other locations in the domain, where inflation is able to avoid filter divergence and, thereby, assimilate significantly more observations.
PDF
