The ensemble Kalman filter (EnKF) is a widely used methodology for state estimation in partially, noisily observed dynamical systems and for parameter estimation in inverse problems. Despite its ...widespread use in the geophysical sciences, and its gradual adoption in many other areas of application, analysis of the method is in its infancy. Furthermore, much of the existing analysis deals with the large ensemble limit, far from the regime in which the method is typically used. The goal of this paper is to analyze the method when applied to inverse problems with fixed ensemble size. A continuous time limit is derived and the long-time behavior of the resulting dynamical system is studied. Most of the rigorous analysis is confined to the linear forward problem, where we demonstrate that the continuous time limit of the EnKF corresponds to a set of gradient flows for the data misfit in each ensemble member, coupled through a common preconditioner which is the empirical covariance matrix of the ensemble. Numerical results demonstrate that the conclusions of the analysis extend beyond the linear inverse problem setting. Numerical experiments are also given which demonstrate the benefits of various extensions of the basic methodology.
The iterative ensemble Kalman filter (IEnKF) in a deterministic framework was introduced in Sakov et al. Mon. Wea. Rev. 140: 1988–2004 () to extend the ensemble Kalman filter (EnKF) and improve its ...performance in mildly up to strongly nonlinear cases. However, the IEnKF assumes that the model is perfect. This assumption simplified the update of the system at a time different from the observation time, which made it natural to apply the IEnKF for smoothing. In this study, we generalize the IEnKF to the case of an imperfect model with additive model error.
The new method called IEnKF‐Q conducts a Gauss–Newton minimization in ensemble space. It combines the propagated analysed ensemble anomalies from the previous cycle and model noise ensemble anomalies into a single ensemble of anomalies, and by doing so takes an algebraic form similar to that of the IEnKF. The performance of the IEnKF‐Q is tested in a number of experiments with the Lorenz'96 model, which show that the method consistently outperforms both the EnKF and the IEnKF naively modified to accommodate additive model noise.
The iterative ensemble Kalman filter (IEnKF) intended for perfect‐model chaotic systems is extended to the IEnKF‐Q which additionally handles imperfect models. The IEnKF‐Q is tested with the Lorenz'96 model in various noise and nonlinearity conditions and compared with the ensemble Kalmanfilter (EnKF) and straightforward but naive extensions of the IEnKF to noisy models. In line with the theory, the IEnKF‐Q is shown to systematically outperform the EnKF and IEnKF variants.
Research on particle filters has been progressing with the aim of applying them to high-dimensional systems, but alleviation of problems with ensemble Kalman filters (EnKFs) in nonlinear or ...non-Gaussian data assimilation is also an important issue. It is known that the deterministic EnKF is less robust than the stochastic EnKF in strongly nonlinear regimes. We prove that if the observation operator is linear the analysis ensemble perturbations of the local ensemble transform Kalman filter (LETKF) are uniform contractions of the forecast ensemble perturbations in observation space in each direction of the eigenvectors of a forecast error covariance matrix. This property approximately holds for a weakly nonlinear observation operator. These results imply that if the forecast ensemble is strongly non-Gaussian the analysis ensemble of the LETKF is also strongly non-Gaussian, and that strong non-Gaussianity therefore tends to persist in high-frequency assimilation cycles, leading to the degradation of analysis accuracy in nonlinear data assimilation. A hybrid EnKF that combines the LETKF and the stochastic EnKF is proposed to mitigate non-Gaussianity in nonlinear data assimilation with small additional computational cost. The performance of the hybrid EnKF is investigated through data assimilation experiments using a 40-variable Lorenz-96 model. Results indicate that the hybrid EnKF significantly improves analysis accuracy in high-frequency data assimilation with a nonlinear observation operator. The positive impact of the hybrid EnKF increases with the increase of the ensemble size.
•A new data assimilation filtering technique called unsupervised weak constrained ensemble Kalman filter (UWCEnKF) is proposed.•We assimilate GRACE and satellite soil moisture data ...simultaneously.•The uncertain water budget constraint is imposed in the filtering process.•Independent in-situ measurements are used to evaluate the results.•UWCEnKF provides more accurate water storage estimates, as well as smaller water budget.•imbalance error compared to standard EnKF.
The standard ensemble data assimilation schemes often violate the dynamical balances of hydrological models, in particular, the fundamental water balance equation, which relates water storage and water flux changes. The present study aims at extending the recently introduced Weak Constrained Ensemble Kalman Filter (WCEnKF) to a more general framework, namely unsupervised WCEnKF (UWCEnKF), in which the covariance of the water balance model is no longer known, thus requiring its estimation along with the model state variables. This extension is introduced because WCEnKF was found to be strongly sensitive to the (manual) choice of this covariance. The proposed UWCEnKF, on the other hand, provides a more general unsupervised framework that does not impose any (manual, thus heuristic) value of this covariance, but suggests an estimation of it, from the observations, along with the state. The new approach is tested based on numerical experiments of assimilating Terrestrial Water Storage (TWS) from Gravity Recovery and Climate Experiment (GRACE) and remotely sensed soil moisture data into a hydrological model. The experiments are conducted over different river basins, comparing WCEnKF, UWCEnKF, and the standard EnKF. In this setup, the UWCEnKF constrains the system state variables with TWS changes, precipitation, evaporation, and discharge data to balance the summation of water storage simulations. In-situ groundwater and soil moisture measurements are used to validate the results of the UWCEnKF and to evaluate its performances against the EnKF. Our numerical results clearly suggest that the proposed framework provides more accurate estimates of groundwater storage changes and soil moisture than WCEnKF and EnKF over the different studied basins.
Data assimilation has been demonstrated as the potential crop yield estimation approach. Accurate quantification of model and observation errors is the key to determining the success of a data ...assimilation system. However, the crop growth model error is not fully taken into account in most of the previous studies. The objective of this study is to better quantify the model uncertainty in the data assimilation system. Firstly, we calibrated a crop growth model and inferred its posterior uncertainty based on the Global LAnd Surface Satellite (GLASS) 250-m LAI product, regional statistical data, station observations, and field measurements with a Markov chain Monte Carlo (MCMC) method. Secondly, the model posterior uncertainty was used in the Ensemble Kalman Filter (EnKF) algorithm to better characterize the ensemble distribution of model errors. Our results indicated the proposed Bayesian posterior-based EnKF can improve the accuracy of winter wheat yield estimation at both the point scale (the coefficient of determination R 2 value increasing from 0.06 to 0.41, the mean absolute percentage error MAPE value decreasing from 12.65% to 7.82%, and the root mean square error RMSE value decreasing from 987 to 688 kg∙ha -1 ) and the regional scale (R 2 value from 0.30 to 0.57, MAPE value from 19.67% to 10.13%, and RMSE value from 1275 to 695 kg∙ha -1 ) compared with the open-loop estimation. Our analysis also indicated that the Bayesian posterior-based EnKF can perform better compared to the standard Gaussian perturbation-based EnKF. The proposed framework provides an important reference for crop yield estimation at the regional scale in similar agricultural landscapes worldwide.
The reconstruction of the dynamics of an observed physical system as a surrogate model has been brought to the fore by recent advances in machine learning. To deal with partial and noisy observations ...in that endeavor, machine learning representations of the surrogate model can be used within a Bayesian data assimilation framework. However, these approaches require to consider long time series of observational data, meant to be assimilated all together. This paper investigates the possibility to learn both the dynamics and the state online, i.e. to update their estimates at any time, in particular when new observations are acquired. The estimation is based on the ensemble Kalman filter (EnKF) family of algorithms using a rather simple representation for the surrogate model and state augmentation. We consider the implication of learning dynamics online through (ⅰ) a global EnKF, (ⅰ) a local EnKF and (ⅲ) an iterative EnKF and we discuss in each case issues and algorithmic solutions. We then demonstrate numerically the efficiency and assess the accuracy of these methods using one-dimensional, one-scale and two-scale chaotic Lorenz models.
The purpose of this paper is to provide a comprehensive presentation and interpretation of the Ensemble Kalman Filter (EnKF) and its numerical implementation. The EnKF has a large user group, and ...numerous publications have discussed applications and theoretical aspects of it. This paper reviews the important results from these studies and also presents new ideas and alternative interpretations which further explain the success of the EnKF. In addition to providing the theoretical framework needed for using the EnKF, there is also a focus on the algorithmic formulation and optimal numerical implementation. A program listing is given for some of the key subroutines. The paper also touches upon specific issues such as the use of nonlinear measurements, in situ profiles of temperature and salinity, and data which are available with high frequency in time. An ensemble based optimal interpolation (EnOI) scheme is presented as a cost-effective approach which may serve as an alternative to the EnKF in some applications. A fairly extensive discussion is devoted to the use of time correlated model errors and the estimation of model bias.
An iterative ensemble Kalman smoother Bocquet, M.; Sakov, P.
Quarterly journal of the Royal Meteorological Society,
July 2014 Part A, Letnik:
140, Številka:
682
Journal Article
Recenzirano
The iterative ensemble Kalman filter (IEnKF) was recently proposed in order to improve the performance of ensemble Kalman filtering with strongly nonlinear geophysical models. The IEnKF can be used ...as a lag‐one smoother and extended to a fixed‐lag smoother: the iterative ensemble Kalman smoother (IEnKS). The IEnKS is an ensemble variational method. It does not require the use of the tangent linear of the evolution and observation models, nor the adjoint of these models: the required sensitivities (gradient and Hessian) are obtained from the ensemble. Looking for optimal performance, out of the many possible extensions we consider a quasi‐static algorithm. The IEnKS is explored for the Lorenz '95 model and for a two‐dimensional turbulence model. As the logical extension of the IEnKF, the IEnKS significantly outperforms standard Kalman filters and smoothers in strongly nonlinear regimes. In mildly nonlinear regimes (typically synoptic‐scale meteorology), its filtering performance is marginally but clearly better than the standard ensemble Kalman filter and it keeps improving as the length of the temporal data assimilation window is increased. For long windows, its smoothing performance outranks the standard smoothers very significantly, a result that is believed to stem from the variational but flow‐dependent nature of the algorithm. For very long windows, the use of a multiple data assimilation variant of the scheme, where observations are assimilated several times, is advocated. This paves the way for finer reanalysis, freed from the static prior assumption of 4D‐Var but also partially freed from the Gaussian assumptions that usually impede standard ensemble Kalman filtering and smoothing.
Accurate determination of absolute positions for lunar landing holds a significant role in accomplishing diverse scientific and engineering mission objectives. In this article, we propose a ...LiDAR-inertial-based absolute positioning system concept for the approach phase of the lunar landing sequence. During the approach phase, since the lander is very close to the ground, the onboard LiDAR can acquire detailed terrain. On the other hand, the terrain maps constructed from data obtained by high-altitude reconnaissance orbiters are inherently sparse. Therefore, the resolution disparity makes it difficult to determine the absolute position by matching the two data, which is why existing systems primarily rely on relative navigation near the lunar surface. To address this challenge, this study employs an ensemble Kalman filter (EnKF) to mitigate the solution ambiguity caused by the sparseness of initial maps. Furthermore, we present a fractal terrain model and a two-step filtering framework to leverage the characteristics of terrains and EnKF effectively. The validity and accuracy of the proposed algorithm are evaluated through numerical simulations, while feasibility is confirmed through a scaled-down experiment. Both numerical simulation and experimental results, employing a map with a resolution of 60 m, demonstrate the potential to achieve absolute position determination with an error of approximately 10 m (1\sigma).
PDF
