Gaussian process emulators (GPE) are a machine learning approach that replicates computational demanding models using training runs of that model. Constructing such a surrogate is very challenging ...and, in the context of Bayesian inference, the training runs should be well invested. The current paper offers a fully Bayesian view on GPEs for Bayesian inference accompanied by Bayesian active learning (BAL). We introduce three BAL strategies that adaptively identify training sets for the GPE using information-theoretic arguments. The first strategy relies on Bayesian model evidence that indicates the GPE’s quality of matching the measurement data, the second strategy is based on relative entropy that indicates the relative information gain for the GPE, and the third is founded on information entropy that indicates the missing information in the GPE. We illustrate the performance of our three strategies using analytical- and carbon-dioxide benchmarks. The paper shows evidence of convergence against a reference solution and demonstrates quantification of post-calibration uncertainty by comparing the introduced three strategies. We conclude that Bayesian model evidence-based and relative entropy-based strategies outperform the entropy-based strategy because the latter can be misleading during the BAL. The relative entropy-based strategy demonstrates superior performance to the Bayesian model evidence-based strategy.
Various real world problems deal with data-driven uncertainty. In particular, in geophysical applications the amount of available data is often limited, posing a challenge in the construction of an ...appropriate stochastic discretization. Arbitrary polynomial chaos is an alternative to tackle this challenge. Approximating the dependence of model output on the uncertain model parameters by expansion in an orthogonal polynomial basis using data-driven principles. This type of global polynomial representation suffers often from Gibbs’ phenomena, especially if applied in non-linear convection dominated problems that require to deal with discontinuities. The multi-resolution or multi-element framework has been successfully used for reducing Gibbs’ phenomena in intrusive stochastic discretizations. In the present work, we introduce a multi-resolution extension of the arbitrary polynomial chaos expansion which is based on the construction of piecewise polynomial. Gaussian quadrature nodes and weights that are computed using only stochastic (localized) moments provided by the underlying raw data. We enhance our approach by a multi-wavelet based stochastic adaptivity that assures a significant reduction of the computational costs. Numerical experiments of increasing complexity demonstrate the performance of the non-intrusive implementation of the introduced methods in relevant scenarios. The use of a carbon dioxide storage benchmark scenario allows one to compare the presented methodology with other stochastic discretization techniques applied to this benchmark.
•Proposes a multi-resolution extension of the arbitrary polynomial chaos expansion.•Data-driven multi-wavelet based adaptivity is proposed.•Numerical experiments demonstrate the performance of the introduced methods.•Comparison with other techniques using CO2 storage benchmark.
Determination of relevant model parameters is crucial for accurate mathematical modelling and efficient numerical simulation of a wide spectrum of applications in geosciences. The conventional method ...of choice is the global sensitivity analysis (GSA). Unfortunately, at least the classical Monte-Carlo based GSA requires a high number of model runs. Response surfaces based techniques, e.g. arbitrary Polynomial Chaos (aPC) expansion, can reduce computational effort, however, they suffer from the Gibbs phenomena and high hardware requirements for higher accuracy. We introduce GSA for arbitrary Multi-Resolution Polynomial Chaos (aMR-PC) which is a localized aPC based data-driven polynomial discretization. The aMR-PC allows to reduce the Gibbs phenomena by construction and to achieve higher accuracy by means of localization also for lower polynomial degrees. We apply these techniques to perform the sensitivity analysis for the Stokes–Darcy problem which describes fluid flow in coupled free-flow and porous-medium systems. We consider the Stokes equations in the free-flow region, Darcy’s law in the porous-medium domain and the classical interface conditions across the fluid–porous interface including the conservation of mass, the balance of normal forces and the Beavers–Joseph condition for the tangential velocity. This coupled problem formulation contains four uncertain parameters: the exact location of the interface, the permeability, the Beavers–Joseph slip coefficient and the uncertainty in the boundary conditions. We carry out the sensitivity analysis of the coupled model with respect to these parameters using the Sobol indices on the aMR-PC expansion and conduct the corresponding numerical simulations.
Surrogate models are widely used to improve the computational efficiency in various geophysical simulation problems by reducing the number of model runs. Conventional one-layer surrogate ...representations are based on global (e.g. polynomial chaos expansion, PCE) or on local kernels (e.g., Gaussian process emulator, GPE). Global representations omit some details, while local kernels require more model runs. The existing multi-resolution PCE is a promising hybrid: it is a global representation with local refinement. However, it can not (yet) estimate the uncertainty of the resulting surrogate, which techniques like the GPE can do. We propose to join multi-resolution PCE and GPE s into a joint surrogate framework to get the best out of both worlds. By doing so, we correct the surrogate bias and assess the remaining uncertainty of the surrogate itself. The resulting multi-resolution emulator offers a pathway for several active learning strategies to improve the surrogate at acceptable computational costs, compared to the existing PCE-kriging approach it adds the multi-resolution aspect. We analyze the performance of a multi-resolution emulator and a plain GPE using didactic test cases and a CO
2
benchmark, that is representative of many alike problems in the geosciences. Both approaches show similar improvements during the active learning, but our multi-resolution emulator leads to much more stable results than the GPE. Overall, our suggested emulator can be seen as a generalization of multi-resolution PCE and GPE concepts that offers the possibility for active learning.
Polynomial chaos expansion (PCE) is a versatile tool widely used in uncertainty quantification and machine learning, but its successful application depends strongly on the accuracy and reliability of ...the resulting PCE-based response surface. High accuracy typically requires high polynomial degrees, demanding many training points especially in high-dimensional problems through the curse of dimensionality. So-called sparse PCE concepts work with a much smaller selection of basis polynomials compared to conventional PCE approaches and can overcome the curse of dimensionality very efficiently, but have to pay specific attention to their strategies of choosing training points. Furthermore, the approximation error resembles an uncertainty that most existing PCE-based methods do not estimate. In this study, we develop and evaluate a fully Bayesian approach to establish the PCE representation via joint shrinkage priors and Markov chain Monte Carlo. The suggested Bayesian PCE model directly aims to solve the two challenges named above: achieving a sparse PCE representation and estimating uncertainty of the PCE itself. The embedded Bayesian regularizing via the joint shrinkage prior allows using higher polynomial degrees for given training points due to its ability to handle underdetermined situations, where the number of considered PCE coefficients could be much larger than the number of available training points. We also explore multiple variable selection methods to construct sparse PCE expansions based on the established Bayesian representations, while globally selecting the most meaningful orthonormal polynomials given the available training data. We demonstrate the advantages of our Bayesian PCE and the corresponding sparsity-inducing methods on several benchmarks.
•Polynomial Chaos expansions are surrogates for computationally expensive models.•A key problem is the number of polynomial expansion terms in high dimensions.•We regularize the polynomial coefficients with a Bayesian joint shrinkage prior.•This is combined with statistical methods for Bayesian variable selection.•Our method outperforms existing methods in terms of accuracy and predictive performance.
Artificial Intelligence and Machine learning have been widely used in various fields of mathematical computing, physical modeling, computational science, communication science, and stochastic ...analysis. Approaches based on Deep Artificial Neural Networks (DANN) are very popular in our days. Depending on the learning task, the exact form of DANNs is determined via their multi-layer architecture, activation functions and the so-called loss function. However, for a majority of deep learning approaches based on DANNs, the kernel structure of neural signal processing remains the same, where the node response is encoded as a linear superposition of neural activity, while the non-linearity is triggered by the activation functions. In the current paper, we suggest to analyze the neural signal processing in DANNs from the point of view of homogeneous chaos theory as known from polynomial chaos expansion (PCE). From the PCE perspective, the (linear) response on each node of a DANN could be seen as a 1st degree multi-variate polynomial of single neurons from the previous layer, i.e. linear weighted sum of monomials. From this point of view, the conventional DANN structure relies implicitly (but erroneously) on a Gaussian distribution of neural signals. Additionally, this view revels that by design DANNs do not necessarily fulfill any orthogonality or orthonormality condition for a majority of data-driven applications. Therefore, the prevailing handling of neural signals in DANNs could lead to redundant representation as any neural signal could contain some partial information from other neural signals. To tackle that challenge, we suggest to employ the data-driven generalization of PCE theory known as arbitrary polynomial chaos (aPC) to construct a corresponding multi-variate orthonormal representations on each node of a DANN. Doing so, we generalize the conventional structure of DANNs to Deep arbitrary polynomial chaos neural networks (DaPC NN). They decompose the neural signals that travel through the multi-layer structure by an adaptive construction of data-driven multi-variate orthonormal bases for each layer. Moreover, the introduced DaPC NN provides an opportunity to go beyond the linear weighted superposition of single neurons on each node. Inheriting fundamentals of PCE theory, the DaPC NN offers an additional possibility to account for high-order neural effects reflecting simultaneous interaction in multi-layer networks. Introducing the high-order weighted superposition on each node of the network mitigates the necessity to introduce non-linearity via activation functions and, hence, reduces the room for potential subjectivity in the modeling procedure. Although the current DaPC NN framework has no theoretical restrictions on the use of activation functions. The current paper also summarizes relevant properties of DaPC NNs inherited from aPC as analytical expressions for statistical quantities and sensitivity indexes on each node. We also offer an analytical form of partial derivatives that could be used in various training algorithms. Technically, DaPC NNs require similar training procedures as conventional DANNs, and all trained weights determine automatically the corresponding multi-variate data-driven orthonormal bases for all layers of DaPC NN. The paper makes use of three test cases to illustrate the performance of DaPC NN, comparing it with the performance of the conventional DANN and also with plain aPC expansion. Evidence of convergence over the training data size against validation data sets demonstrates that the DaPC NN outperforms the conventional DANN systematically. Overall, the suggested re-formulation of the kernel network structure in terms of homogeneous chaos theory is not limited to any particular architecture or any particular definition of the loss function. The DaPC NN Matlab Toolbox is available online and users are invited to adopt it for own needs.
•Kernel structure reformulation of deep artificial neural networks in terms of homogeneous chaos theory.•Response on each node of the deep network represented through polynomial chaos expansion.•Orthonormal decomposition mitigates the non-optimal representation while processing the neural signal.•Accounting for high-order effects reflecting simultaneous neural impacts.•Analytical estimation of mean and variance of neural signal on each node is provided.
Strongly degenerate parabolic convection–diffusion equations arise as governing equations in a number of applications such as traffic flow with driver reaction and anticipation distance and ...sedimentation of solid–liquid suspensions in mineral processing and wastewater treatment. In these applications several parameters that define the convective flux function and the degenerating diffusion coefficient are subject to stochastic variability. A method to evaluate the variability of the solution of the governing partial differential equation in response to that of the parameters is presented. To this end, a generalized polynomial chaos (gPC) expansion of the solution is approximated by its projection onto a finite-dimensional space of piecewise polynomial functions defined on a suitable discretization of the stochastic domain, according to the basic principle of the hybrid stochastic Galerkin (HSG) approach. This approach is combined with a finite volume (FV) method, resulting in a so-called FV–HSG method, to compute the sought deterministic coefficient functions of the truncated polynomial-chaos-based expansion of the solution. Since the stochastic parameter space is now spanned by piecewise polynomial functions, one may employ the numerical result to compute the reconstruction of the numerical solution for arbitrary values of the random variables. The expectation, the variance or other stochastic quantities of the solution (as functions of time and position) can also be computed from these coefficient functions. The method is illustrated by a number of numerical examples.
Several environmental phenomena require monitoring time-dependent densities in porous media, e.g., clogging of river sediments, mineral dissolution/precipitation, or variably-saturated multiphase ...flow. Gamma-ray attenuation (GRA) can monitor time-dependent densities without being destructive or invasive under laboratory conditions. GRA sends gamma rays through a material, where they are attenuated by photoelectric absorption and then recorded by a photon detector. The attenuated intensity of the emerging beam relates to the density of the traversed material via Beer–Lambert’s law. An important parameter for designing time-variable GRA is the exposure time, the time the detector takes to gather and count photons before converting the recorded intensity to a density. Large exposure times capture the time evolution poorly (temporal raster error, inaccurate temporal discretization), while small exposure times yield imprecise intensity values (noise-related error, i.e. small signal-to-noise ratio). Together, these two make up the total error of observing time-dependent densities by GRA. Our goal is to provide an optimization framework for time-dependent GRA experiments with respect to exposure time and other key parameters, thus facilitating neater experimental data for improved process understanding. Experimentalists set, or iterate over, several experimental input parameters (e.g., Beer–Lambert parameters) and expectations on the yet unknown dynamics (e.g., mean and amplitude of density and characteristic time of density changes). We model the yet unknown dynamics as a random Gaussian Process to derive expressions for expected errors prior to the experiment as a function of key experimental parameters. Based on this, we provide an optimization framework that allows finding the optimal (minimal-total-error) setup and demonstrate its application on synthetic experiments.
Article Highlights
We study the influence of anticipated density changes and experimental setup on optimal designs for GRA measurements
We present a methodology that finds the optimal setup (minimum error) as a function of the exposure time and other parameters
We provide experimentalists with a quantitative understanding of unavoidable inaccuracies and how to minimize them
A variety of methods is available to quantify uncertainties arising within the modeling of flow and transport in carbon dioxide storage, but there is a lack of thorough comparisons. Usually, raw data ...from such storage sites can hardly be described by theoretical statistical distributions since only very limited data is available. Hence, exact information on distribution shapes for all uncertain parameters is very rare in realistic applications. We discuss and compare four different methods tested for data-driven uncertainty quantification based on a benchmark scenario of carbon dioxide storage. In the benchmark, for which we provide data and code, carbon dioxide is injected into a saline aquifer modeled by the nonlinear capillarity-free fractional flow formulation for two incompressible fluid phases, namely carbon dioxide and brine. To cover different aspects of uncertainty quantification, we incorporate various sources of uncertainty such as uncertainty of boundary conditions, of parameters in constitutive relations, and of material properties. We consider recent versions of the following non-intrusive and intrusive uncertainty quantification methods: arbitrary polynomial chaos, spatially adaptive sparse grids, kernel-based greedy interpolation, and hybrid stochastic Galerkin. The performance of each approach is demonstrated assessing expectation value and standard deviation of the carbon dioxide saturation against a reference statistic based on Monte Carlo sampling. We compare the convergence of all methods reporting on accuracy with respect to the number of model runs and resolution. Finally, we offer suggestions about the methods’ advantages and disadvantages that can guide the modeler for uncertainty quantification in carbon dioxide storage and beyond.
•A clarifier-thickener model with several random perturbations is proposed.•Uncertainty quantification for hyperbolic problems with several random perturbations.•Introduction to the hybrid stochastic ...Galerkin (HSG) finite volume method.•Discussion of the accuracy and computational efficiency of the HSG discretization.•Multivariate multi-wavelets provide stochastic adaptivity of the HSG method.
Continuous sedimentation processes in a clarifier-thickener unit can be described by a scalar nonlinear conservation law whose flux density function is discontinuous with respect to the spatial position. In the applications of this model, which include mineral processing and wastewater treatment, the rate and composition of the feed flow cannot be given deterministically. Efficient numerical simulation is required to quantify the effect of uncertainty in these control parameters in terms of the response of the clarifier-thickener system. Thus, the problem at hand is one of uncertainty quantification for nonlinear hyperbolic problems with several random perturbations. The presented hybrid stochastic Galerkin method is devised so as to extend the polynomial chaos approximation by multiresolution discretization in the stochastic space. This approach leads to a deterministic hyperbolic system, which is partially decoupled and therefore suitable for efficient parallelisation. Stochastic adaptivity reduces the computational effort. Several numerical experiments are presented.
PDF
