Abstract : Predictions of the state or parameter of a system of interest that is subject to some type of stochastic noise are typically achieved by estimating the associated density. Yet this approach becomes highly challenging for extended state and parameter spaces and if the unknown density is non-parametric. State of the art methods are designed to use a Monte Carlo type empirical estimation often referred to as an ensemble or particle filter. Here we will explore theoretical and algorithmic advances of these methods in the context of data assimilation and classical inverse problems.
Abstract : Sampling-based inference and learning techniques, especially Bayesian inference, provide an essential approach to handling uncertainty in machine learning (ML). As these techniques are increasingly used in daily life, it becomes essential to safeguard the ML systems with various trustworthyrelated constraints, such as fairness, safety, interpretability. We propose a family of constrained sampling algorithms which generalize Langevin Dynamics (LD) and Stein Variational Gradient Descent (SVGD) to incorporate a moment constraint or a level set specified by a general nonlinear function. By exploiting the gradient flow structure of LD and SVGD, we derive algorithms for handling constraints, including a primal-dual gradient approach and the constraint controlled gradient descent approach.We investigate the continuous-time mean-field limit of these algorithms and show that they have O(1/t) convergence under mild conditions.
[03107] Subsampling in ensemble kalman inversion
Format : Talk at Waseda University
Author(s) :
Matei Hanu (Free University of Berlin)
Jonas Latz (Heriot-Watt-University)
Claudia Schillings (Free University of Berlin)
Abstract : The Ensemble Kalman Inversion (EKI) is an efficient, gradient-free optimisation method to estimate unknown parameters in an inverse setting. For large data sets, the EKI becomes computationally infeasible as the data misfit needs to be evaluated for each particle in each iteration.
Randomised algorithms can successfully overcome this issue by using only a random subset of the data in each iteration, so-called subsampling techniques.
In this talk we present subsampling-techniques within Ensemble Kalman Inversion.
[03842] Ensemble Inference Methods for Models with Noisy and Expensive Likelihoods
Format : Online Talk on Zoom
Author(s) :
Marie-Therese Wolfram (University of Warwick)
Andrew Stuart (California Institute of Technology)
Andrew Duncan (Imperial College London)
Oliver Dunbar (California Institute of Technology )
Abstract : This talk concerns interacting particle systems to solve inverse problems where the forward model evaluations present rapid fluctuations over the smoothly varying quantity of interest. After comparing the performance of ensemble Kalman methods (EKS) and Langevin-based methods (ELS) using formal multiscale analysis, we introduce a new class of algorithms, named ensemble Gaussian process samplers, which combine the main benefits of both approaches while avoiding their flaws.
[04379] Projected ensemble data assimilation
Format : Talk at Waseda University
Author(s) :
Svetlana Dubinkina (VU Amsterdam)
Jana de Wiljes (University of Potsdam)
Abstract : Ensemble data assimilation is unable to reduce the error estimate for high-dimensional systems when used with a small ensemble. A typical remedy is dimesion reduction by localization. Though localization reduces the error substantially for both linear and nonlinear data-assimilation methods, the former ones considerably outperform the latter ones in quasi-linear regimes. We propose a further dimension reduction based on projection and show numerically considerable error decrease when used with small ensemble.
[04587] Alternatives to Monte Carlo based sampling and high dimensional integration
Format : Talk at Waseda University
Author(s) :
Sahani Pathiraja (University of New South Wales)
Abstract : Monte Carlo is a fundamental component of popular methods for uncertainty quantification and Bayesian inference. However, Monte Carlo based techniques are often inefficient in high dimensions, representing distributional tails and in complex time-dependent systems. This is in part due to the reliance on points (i.e. delta functions) to approximate distributions. We numerically and analytically investigate the performance of various sampling techniques that make use of alternatives to delta functions and compare to standard Monte Carlo.
[05018] Mixtures of Gaussian Process Experts with SMC^2
Format : Talk at Waseda University
Author(s) :
Lassi Roininen (LUT University)
Abstract : Gaussian processes are a key component of many flexible statistical and machine learning models. However, they exhibit cubic computational complexity and high memory constraints due to the need of inverting and storing a full covariance matrix. To circumvent this, mixtures of Gaussian process experts have been considered where data points are assigned to independent experts, reducing the complexity by allowing inference based on smaller, local covariance matrices. Moreover, mixtures of Gaussian process experts substantially enrich the model's flexibility, allowing for behaviors such as non-stationarity, heteroscedasticity, and discontinuities. In this work, we construct a novel inference approach based on nested sequential Monte Carlo samplers to simultaneously infer both the gating network and Gaussian process expert parameters. This greatly improves inference compared to importance sampling, particularly in settings when a stationary Gaussian process is inappropriate, while still being thoroughly parallelizable.
[05073] Eulerian calibration for stochastic transport models
Format : Talk at Waseda University
Author(s) :
Oana Andrea Lang (Imperial College London)
Abstract : In this talk I will talk about a new probabilistic approach for calibrating a general class of stochastic nonlinear fluid dynamics models. A key step for ensuring the successful application of the combined stochastic parameterisation and data assimilation procedure is the “correct” calibration of stochastic model parameters. Currently, most methodologies are based on Lagrangian particle trajectories which are simulated starting from each point on both the physical grid and its refined version. Then the differences between the particle positions are used to calibrate the noise. This is computationally expensive and not fully justified from a theoretical perspective. We currently explore an Eulerian approach based on calibrating the amplitude of the individual noises to obtain an approximate representation of uncertainty that uses a finite set of individual noises, and in this talk I will report on the current advances.
This is joint work with Prof Dan Crisan and Dr Alexander Lobbe (Imperial College London).
[05209] Can possibility theory help with uncertainty quantification for neural networks?
Format : Talk at Waseda University
Author(s) :
Jeremie Houssineau (University of Warwick)
Abstract : We consider an alternative to the Gaussian version of the stochastic weight averaging method (SWAG), an approach to uncertainty quantification in neural networks. It is well accepted that the uncertainty in the parameters of the network is epistemic rather than being induced by randomness, which motivates the use of possibility theory. We will see how possibility theory helps to overcome difficulties with standard Bayesian neural networks and how it leads to an alternative to SWAG.
