Abstract : Transportation of measures is an important topic in applied mathematics based on constructing invertible transformations between random variables. These transformations can include deterministic maps, plans and stochastic processes. In recent years, this broad topic has seen wide applications for generative modeling, inference, and comparing probability distributions. Despite these successes, efficiently constructing these transformations remains challenging, especially in high-dimensional problems with complex data manifolds. This minisymposium will present novel statistical analysis and computational methods that widen the breadth of transport methods in statistics and scientific computing applications.
[05226] Optimal transport map estimation in general function spaces
Format : Talk at Waseda University
Author(s) :
Vincent Divol (Université PSL)
Jonathan Niles-Weed (New York University)
Aram Pooladian (New York University)
Abstract : We consider the problem of estimating the optimal transport map between a (fixed) source distribution P and an unknown target distribution Q, based on samples from Q. The estimation of such optimal transport maps has become increasingly relevant in modern statistical applications, such as generative modeling. At present, estimation rates are only known in a few settings (e.g. when P and Q have densities bounded above and below and when the transport map lies in a Hölder class), which are often not reflected in practice. We present a unified methodology for obtaining rates of estimation of optimal transport maps in general function spaces. Our assumptions are significantly weaker than those appearing in the literature: we require only that the source measure P satisfies a Poincaré inequality and that the optimal map be the gradient of a smooth convex function that lies in a space whose metric entropy can be controlled. As a special case, we recover known estimation rates for bounded densities and Hölder transport maps, but also obtain nearly sharp results in many settings not covered by prior work. For example, we provide the first statistical rates of estimation when P is the normal distribution and the transport map is given by an infinite-width shallow neural network.
[04080] TBA
Author(s) :
Augusto Gerolin (Canada Research Chair and University of Ottawa)
Abstract : TBA
[04165] Efficient subspace modeling via transport transforms
Format : Online Talk on Zoom
Author(s) :
Shiying Li (University of North Carolina - Chapel Hill)
Gustavo Rohde (University of Virginia)
Akram Aldroubi (Vanderbilt University)
Abu Hasnat Rubaiyat (University of Virginia)
Yan Zhuang (NIH)
Mohammad Shifat-E-Rabbi (University of Virginia)
Xuwang Yin (Center for AI Safety )
Abstract : When data is generated though deformations of certain template distributions, transport-based transforms often linearize data clusters which are nonlinear in the original domain. We will describe convexification properties of several transport transforms under various generative modeling assumptions, enabling efficient modeling of data classes as subspaces in the transform domain. Such subspace representations also give rise to accurate machine learning algorithms with low computational cost. We will show applications for image and signal classification.
[04063] Conditional simulation through the data-driven optimal transport barycenter problem
Format : Talk at Waseda University
Author(s) :
Esteban Gregorio Tabak (New York University, Courant Institute)
Abstract : A methodology is proposed to generate samples from a conditional probability distribution, with factors that are either known explicitly, up to discovery or only by association.The procedure pushes forward the conditional distribution to its barycenter through particle flows, whose inverse provides the simulation sought. Idiosyncratic factors are included through subsampling. The methodology serves as a conditional generator, to eliminate batch effects, to uncover hidden factors and to predict and optimize trajectories under treatment.
[04311] On the Monge gap and the MBO feature-sparse transport estimator.
Format : Talk at Waseda University
Author(s) :
marco cuturi (Apple)
Abstract : This talk will cover two recent works aimed at estimating Monge maps from samples. In the first part (in collaboration with Théo Uscidda) I will present a novel approach to train neural networks so that they mimic Monge maps for the squared-Euclidean cost. In that field, a popular approach has been to parameterize dual potentials using input convex neural networks, and estimate their parameters using SGD and a convex conjugate approximation. We present in this work a regularizer for that task that is conceptually simpler (as it does not require any assumption on the architecture) and which extends to non-Euclidean costs. In the second part (in collaboration with Michal Klein and Pierre Ablin), I will show that when adding to the squared-Euclidean distance an extra translation-invariant cost, the Brenier theorem translates into the application of the proximal mapping of that extra term to the derivative of the dual potential. Using an entropic map to parameterize that potential, we obtain the Monge-Bregman-Occam (MBO) estimator, which has the definOn the Monge gap and the MBO feature-sparse transport estimator.ing property that its displacement vectors $T(x) - x$ are sparse, resulting in interpretable OT maps in high dimensions.
[05300] Simulation-Free Generative Modeling with Neural ODEs
Format : Talk at Waseda University
Author(s) :
Ricky Tian Qi Chen (Meta AI)
Abstract : Standard diffusion models offer a simulation-free method of training continuous-time transport maps but are typically restricted to linear stochastic processes. In this talk, I will discuss Flow Matching, a training objective that allows regressing onto the generating vector field instead of the score vector field. This allows more flexibility in the design of probability paths, extends seamlessly to general manifolds, and brings the model closer to optimal transport solutions.
[04077] Diffusion Schrödinger Bridge Matching
Format : Talk at Waseda University
Author(s) :
Yuyang Shi (Oxford university)
Valentin De Bortoli (ENS Ulm)
Andrew Campbell (Oxford University)
Arnaud Doucet (Oxford University)
Abstract : Solving transport problems, i.e. finding a map transporting one given distribution to another, has numerous applications in machine learning. Novel mass transport methods motivated by generative modeling have recently been proposed, e.g. Denoising Diffusion Models (DDMs) and Flow Matching Models (FMMs) implement such a transport through a Stochastic Differential Equation (SDE) or an Ordinary Differential Equation (ODE). However, while it is desirable in many applications to approximate the deterministic dynamic Optimal Transport (OT) map which admits attractive properties, DDMs and FMMs are not guaranteed to provide transports close to the OT map. In contrast, Schrödinger bridges (SBs) compute stochastic dynamic mappings which recover entropy-regularized versions of OT. Unfortunately, existing numerical methods approximating SBs either scale poorly with dimension or accumulate errors across iterations. In this work, we introduce Iterative Markovian Fitting, a new methodology for solving SB problems, and Diffusion Schrödinger Bridge Matching (DSBM), a novel numerical algorithm for computing IMF iterates. DSBM significantly improves over previous SB numerics and recovers as special/limiting cases various recent transport methods. We demonstrate the performance of DSBM on a variety of problems.
Abstract : The dynamic Schrödinger bridge problem seeks a stochastic process that defines a transport between two target probability measures, while optimally satisfying the criteria of being closest, in terms of Kullback-Leibler divergence, to a reference process.
We propose a novel sampling-based iterative algorithm, the iterated diffusion bridge mixture (IDBM) procedure, aimed at solving the dynamic Schrödinger bridge problem.
The IDBM procedure exhibits the attractive property of realizing a valid transport between the target probability measures at each iteration.
We perform an initial theoretical investigation of the IDBM procedure, and carry out numerical experiments illustrating the competitive performance of the IDBM procedure.
Recent advancements in generative modeling employ the time-reversal of a diffusion process to define a generative process that approximately transports a simple distribution to the data distribution.
As an alternative, we propose utilizing the first iteration of the IDBM procedure as an approximation-free method for realizing this transport.
This approach offers greater flexibility in selecting the generative process dynamics and exhibits accelerated training and superior sample quality over larger discretization intervals.
[05484] Neural Optimal Transport for Single-Cell Biology
Format : Online Talk on Zoom
Author(s) :
Charlotte Bunne (ETH Zurich)
Abstract : To accurately predict the responses of a patient’s tumor cells to a cancer drug, it is vital to recover the underlying population dynamics and fate decisions of single cells. However, measuring molecular properties of single cells requires destroying them. As a result, a cell population can only be monitored with sequential snapshots, obtained by sampling a few particles that are sacrificed in exchange for measurements. In order to reconstruct individual cell fate trajectories, as well as the overall dynamics, one needs to re-align these unpaired snapshots, in order to guess for each cell what it might have become at the next step. Optimal transport theory can provide such maps, and reconstruct these incremental changes in cell states over time. This celebrated theory provides the mathematical link that unifies the several contributions to model cellular dynamics that we present here: Inference from data of an energy potential best able to describe the evolution of differentiation processes (Bunne et al., 2022), building on the Jordan-Kinderlehrer-Otto (JKO) flow; recovery of differential equations modeling the stochastic transitions between cell fates in developmental processes (Bunne et al., 2023) through Schrödinger bridges; as well as zero-sum game theory models parameterizing distribution shifts upon interventions, which we employ to model heterogeneous responses of tumor cells to cancer drugs (Bunne et al., 2022, 2023).
[04473] Tensor train approximation of deep transport maps for Bayesian inverse problems.
Format : Talk at Waseda University
Author(s) :
Tiangang Cui (Monash University)
Sergey Dolgov (University of Bath)
Robert Scheichl (Heidelberg University)
Olivier Zahm (Universite Grenoble Alpes, Inria)
Abstract : We develop a deep transport map for sampling concentrated distributions defined by an unnormalised density function. We approximate the target distribution as the pushforward of a reference distribution under a composition of transport maps formed by tensor-train approximations of bridging densities. We propose two bridging strategies: tempering the target density, and smoothing of an indicator function with a sigmoids. The latter opens the door to efficient computation of rare event probabilities in Bayesian inference problems.
[04231] Tensor-train methods for sequential state and parameter learning in state-space models
Format : Talk at Waseda University
Author(s) :
Tiangang Cui (Monash University)
Yiran Zhao (Monash University)
Abstract : We consider sequential state and parameter learning in state-space models with intractable state transition and observation processes. By exploiting low-rank tensor-train (TT) decompositions, we propose new sequential learning methods for joint parameter and state estimation under the Bayesian framework. Our key innovation is the introduction of scalable function approximation tools such as TT for recursively learning the sequentially updated posterior distributions. The function approximation perspective of our methods offers tractable error analysis and potentially alleviates the particle degeneracy faced by many particle-based methods. In addition to the new insights into algorithmic design, our methods complement conventional particle-based methods. Our TT-based approximations naturally define conditional Knothe-Rosenblatt (KR) rearrangements that lead to filtering, smoothing, and path estimation accompanying our sequential learning algorithms, which open the door to removing potential approximation bias. We also explore several preconditioning techniques based on either linear or nonlinear KR rearrangements to enhance the approximation power of TT for practical problems. We demonstrate the efficacy and efficiency of our proposed methods on several state-space models, in which our methods achieve state-of-the-art estimation accuracy and computational performance.
[05065] Accelerated Interacting Particle Transport for Bayesian Inversion
Format : Talk at Waseda University
Author(s) :
Martin Eigel (WIAS)
Robert Gruhlke (FU Berlin)
David Sommer (WIAS)
Abstract : Ensemble methods have become ubiquitous for solving Bayesian inference problems. State-of-the-art Langevin samplers such as the Ensemble Kalman Sampler (EKS) and Affine Invariant Langevin Dynamics (ALDI) rely on many evaluations of the forward model, which we try to improve. First, adaptive ensemble enrichment strategies are discussed. Second, analytical consistency guarantees of the ensemble enrichment for linear forward models are presented. Third, a homotopy formalism for involved distributions is introduced.
