[00966] Theoretical and computational advances in measure transport
Session Date & Time :
00966 (1/3) : 1C (Aug.21, 13:20-15:00)
00966 (2/3) : 1D (Aug.21, 15:30-17:10)
00966 (3/3) : 1E (Aug.21, 17:40-19:20)
Type : Proposal of Minisymposium
Abstract : Transportation of measures is an important topic in applied mathematics based on constructing invertible transformations between random variables. These transformations can include deterministic maps, plans and stochastic processes. In recent years, this broad topic has seen wide applications for generative modeling, inference, and comparing probability distributions. Despite these successes, efficiently constructing these transformations remains challenging, especially in high-dimensional problems with complex data manifolds. This minisymposium will present novel statistical analysis and computational methods that widen the breadth of transport methods in statistics and scientific computing applications.
Shiying Li (University of North Carolina at Chapel Hill)
Sergey Dolgov (University of Bath)
Martin Eigel (WIAS)
Jonathan Niles-Weed (New York University)
Marco Cuturi (Apple)
Talks in Minisymposium :
[04063] Conditional simulation through the data-driven optimal transport barycenter problem
Author(s) :
Esteban Gregorio Tabak (New York University, Courant Institute)
Abstract : A methodology is proposed to generate samples from a conditional probability distribution, with factors that are either known explicitly, up to discovery or only by association.The procedure pushes forward the conditional distribution to its barycenter through particle flows, whose inverse provides the simulation sought. Idiosyncratic factors are included through subsampling. The methodology serves as a conditional generator, to eliminate batch effects, to uncover hidden factors and to predict and optimize trajectories under treatment.
[04077] Diffusion Schrödinger Bridge Matching
Author(s) :
Yuyang Shi (Oxford university)
Valentin De Bortoli (ENS Ulm)
Andrew Campbell (Oxford University)
Arnaud Doucet (Oxford University)
Abstract : Solving transport problems, i.e. finding a map transporting one given distribution to another, has numerous applications in machine learning. Novel mass transport methods motivated by generative modeling have recently been proposed, e.g. Denoising Diffusion Models (DDMs) and Flow Matching Models (FMMs) implement such a transport through a Stochastic Differential Equation (SDE) or an Ordinary Differential Equation (ODE). However, while it is desirable in many applications to approximate the deterministic dynamic Optimal Transport (OT) map which admits attractive properties, DDMs and FMMs are not guaranteed to provide transports close to the OT map. In contrast, Schrödinger bridges (SBs) compute stochastic dynamic mappings which recover entropy-regularized versions of OT. Unfortunately, existing numerical methods approximating SBs either scale poorly with dimension or accumulate errors across iterations. In this work, we introduce Iterative Markovian Fitting, a new methodology for solving SB problems, and Diffusion Schrödinger Bridge Matching (DSBM), a novel numerical algorithm for computing IMF iterates. DSBM significantly improves over previous SB numerics and recovers as special/limiting cases various recent transport methods. We demonstrate the performance of DSBM on a variety of problems.
[04080] TBA
Author(s) :
Augusto Gerolin (Canada Research Chair and University of Ottawa)
Abstract : TBA
[04165] Efficient subspace modeling via transport transforms
Author(s) :
Shiying Li (University of North Carolina - Chapel Hill)
Gustavo Rohde (University of Virginia)
Akram Aldroubi (Vanderbilt University)
Abu Hasnat Rubaiyat (University of Virginia)
Yan Zhuang (NIH)
Mohammad Shifat-E-Rabbi (University of Virginia)
Xuwang Yin (Center for AI Safety )
Abstract : When data is generated though deformations of certain template distributions, transport-based transforms often linearize data clusters which are nonlinear in the original domain. We will describe convexification properties of several transport transforms under various generative modeling assumptions, enabling efficient modeling of data classes as subspaces in the transform domain. Such subspace representations also give rise to accurate machine learning algorithms with low computational cost. We will show applications for image and signal classification.
[04231] Tensor-train methods for sequential state and parameter learning in state-space models
Author(s) :
Tiangang Cui (Monash University)
Yiran Zhao (Monash University)
Abstract : We consider sequential state and parameter learning in state-space models with intractable state transition and observation processes. By exploiting low-rank tensor-train (TT) decompositions, we propose new sequential learning methods for joint parameter and state estimation under the Bayesian framework. Our key innovation is the introduction of scalable function approximation tools such as TT for recursively learning the sequentially updated posterior distributions. The function approximation perspective of our methods offers tractable error analysis and potentially alleviates the particle degeneracy faced by many particle-based methods. In addition to the new insights into algorithmic design, our methods complement conventional particle-based methods. Our TT-based approximations naturally define conditional Knothe-Rosenblatt (KR) rearrangements that lead to filtering, smoothing, and path estimation accompanying our sequential learning algorithms, which open the door to removing potential approximation bias. We also explore several preconditioning techniques based on either linear or nonlinear KR rearrangements to enhance the approximation power of TT for practical problems. We demonstrate the efficacy and efficiency of our proposed methods on several state-space models, in which our methods achieve state-of-the-art estimation accuracy and computational performance.
[04311] On the Monge gap and the MBO feature-sparse transport estimator.
Author(s) :
marco cuturi (Apple)
Abstract : This talk will cover two recent works aimed at estimating Monge maps from samples. In the first part (in collaboration with Théo Uscidda) I will present a novel approach to train neural networks so that they mimic Monge maps for the squared-Euclidean cost. In that field, a popular approach has been to parameterize dual potentials using input convex neural networks, and estimate their parameters using SGD and a convex conjugate approximation. We present in this work a regularizer for that task that is conceptually simpler (as it does not require any assumption on the architecture) and which extends to non-Euclidean costs. In the second part (in collaboration with Michal Klein and Pierre Ablin), I will show that when adding to the squared-Euclidean distance an extra translation-invariant cost, the Brenier theorem translates into the application of the proximal mapping of that extra term to the derivative of the dual potential. Using an entropic map to parameterize that potential, we obtain the Monge-Bregman-Occam (MBO) estimator, which has the definOn the Monge gap and the MBO feature-sparse transport estimator.ing property that its displacement vectors $T(x) - x$ are sparse, resulting in interpretable OT maps in high dimensions.
[04473] Tensor train approximation of deep transport maps for Bayesian inverse problems.
Author(s) :
Tiangang Cui (Monash University)
Sergey Dolgov (University of Bath)
Robert Scheichl (Heidelberg University)
Olivier Zahm (Universite Grenoble Alpes, Inria)
Abstract : We develop a deep transport map for sampling concentrated distributions defined by an unnormalised density function. We approximate the target distribution as the pushforward of a reference distribution under a composition of transport maps formed by tensor-train approximations of bridging densities. We propose two bridging strategies: tempering the target density, and smoothing of an indicator function with a sigmoids. The latter opens the door to efficient computation of rare event probabilities in Bayesian inference problems.
Abstract : Wasserstein barycenters are desirable objects, but they suffer from the curse of dimensionality (statistically and computationally). In this talk, I will propose a new look at the entropic regularization of Wasserstein barycenters. I will show that, via a double regularization of the problem, one obtains a notion of barycenter with none of these drawbacks. In addition, and perhaps counter-intuitively, with well-chosen regularization strengths, this double regularization approximates the true Wasserstein better than a single regularization.
[05065] Accelerated Interacting Particle Transport for Bayesian Inversion
Author(s) :
Martin Eigel (WIAS)
Robert Gruhlke (FU Berlin)
David Sommer (WIAS)
Abstract : Ensemble methods have become ubiquitous for solving Bayesian inference problems. State-of-the-art Langevin samplers such as the Ensemble Kalman Sampler (EKS) and Affine Invariant Langevin Dynamics (ALDI) rely on many evaluations of the forward model, which we try to improve. First, adaptive ensemble enrichment strategies are discussed. Second, analytical consistency guarantees of the ensemble enrichment for linear forward models are presented. Third, a homotopy formalism for involved distributions is introduced.
[05226] Optimal transport map estimation in general function spaces
Author(s) :
Vincent Divol (Université PSL)
Jonathan Niles-Weed (New York University)
Aram Pooladian (New York University)
Abstract : We consider the problem of estimating the optimal transport map between a (fixed) source distribution P and an unknown target distribution Q, based on samples from Q. The estimation of such optimal transport maps has become increasingly relevant in modern statistical applications, such as generative modeling. At present, estimation rates are only known in a few settings (e.g. when P and Q have densities bounded above and below and when the transport map lies in a Hölder class), which are often not reflected in practice. We present a unified methodology for obtaining rates of estimation of optimal transport maps in general function spaces. Our assumptions are significantly weaker than those appearing in the literature: we require only that the source measure P satisfies a Poincaré inequality and that the optimal map be the gradient of a smooth convex function that lies in a space whose metric entropy can be controlled. As a special case, we recover known estimation rates for bounded densities and Hölder transport maps, but also obtain nearly sharp results in many settings not covered by prior work. For example, we provide the first statistical rates of estimation when P is the normal distribution and the transport map is given by an infinite-width shallow neural network.
[05300] Simulation-Free Generative Modeling with Neural ODEs
Author(s) :
Ricky Tian Qi Chen (Meta AI)
Abstract : Standard diffusion models offer a simulation-free method of training continuous-time transport maps but are typically restricted to linear stochastic processes. In this talk, I will discuss Flow Matching, a training objective that allows regressing onto the generating vector field instead of the score vector field. This allows more flexibility in the design of probability paths, extends seamlessly to general manifolds, and brings the model closer to optimal transport solutions.
