Abstract : Mean field control problems have attracted massive interest and provide a promising approach dealing with multi-agent systems. The aim of this mini-symposium is to share the new trends of both theory and applications of this area. We would like to invite the frontier scholars to talk about recent developments in various learning methods for the mean field control problems under different application aspects, as well as analyzing HJB equation in the infinite dimensional spaces for the theory prospective.
Organizer(s) : Xin Guo, Jiacheng Zhang
Sponsor : This session is sponsored by the SIAM Activity Group on Financial Mathematics and Engineering.
[04658] Actor-critic learning for mean-field control in continuous time
Format : Talk at Waseda University
Author(s) :
HUYEN PHAM (Université Paris Cité )
Noufel Frikha (Université Paris 1)
Maximilien Germain (Morgan Stanley )
Mathieu Laurière (NYU Shanghai)
Xuanye Song (Université Paris Cité)
Abstract : We study policy gradient for mean-field control in continuous time in a reinforcement learning setting. By considering
randomised policies with entropy regularisation, we derive a gradient expectation representation of the value function, which is amenable to actor-critic type algorithms, where the value functions and the policies are learnt alternately
based on observation samples of the state and model-free estimation of the population state distribution, either by
offline or online learning.
[04674] Mean-field singular control problem: regularitiy and related mean-field reflected diffusion
Format : Talk at Waseda University
Author(s) :
Jodi Dianetti (Bielefeld University)
Xin Guo (UC Berkeley)
Jiacheng Zhang (UC Berkeley)
Huyên Pham (Université Paris Cité )
Abstract : We study a class of mean-field control problems with singular controls. Such a model represents the limit of the control problems in which a controller can adjust, through a bounded variation process, an underlying diffusion, which in turn affects an n-particle system. Adopting appropriate notions of convexities, we are able to establish the regularity of the value function of the problem and to show the existence of the optimal control. The regularity of the value function allows to characterize the solution of the problem in terms of a related mean-field Skorokhod problem. This consists in keeping the optimally controlled state process in a region prescribed by the derivative of the value function, by using the optimal control in order to reflect the state at its boundary.
[04752] A non-asymptotic perspective on mean field control
Format : Talk at Waseda University
Author(s) :
Lane Chun Yeung (Columbia University)
Daniel Lacker (Columbia University)
Sumit Mukherjee (Columbia University)
Abstract : We study a class of stochastic control problems in which a large number of players cooperatively choose their drifts to maximize an expected reward minus a quadratic running cost. For a broad class of potentially asymmetric rewards, we show that there exist approximately optimal controls which are decentralized, in the sense that each player's control depends only on its own state and not the states of the other players.
[05079] Signature SDEs with jumps and their tractability properties
Format : Talk at Waseda University
Author(s) :
Christa Cuchiero (University of Vienna)
Francesca Primavera (University of Vienna)
Sara Svaluto Ferro (University of Verona)
Abstract : Signature-based models have recently entered the field of Mathematical Finance. Relying on recent advances on the signature of càdlàg paths, we introduce here a generic class of jump-diffusion models via so-called signature SDEs with jumps. We elaborate on their tractability properties and show that the signature-based models for asset prices proposed so far can be embedded in this framework. As a special case, we focus on jump-diffusions with entire characteristics, leading to a far-reaching extension of the class of polynomial processes.
[05439] The convergence problem in mean field control
Format : Online Talk on Zoom
Author(s) :
Joe Jackson (The University of Texas at Austin)
Samuel Daudin (Université Côte d'Azur)
François Delarue (Université Côte d'Azur)
Abstract : This talk will be about a recent joint work with Samuel Daudin and François Delarue concerning the convergence problem in mean field control. When the data is convex and sufficiently smooth, the (optimal) rate of convergence (of the $N$-player value function towards the limiting value function) is known to be $1/N$. The goal of our work is to identify the optimal rate of convergence in the more subtle non-convex setting.
[05419] Markov $alpha$-Potential Game
Format : Online Talk on Zoom
Author(s) :
Xinyu Li (UC Berkeley)
Xin Guo (UC Berkeley)
Chinmay Maheshwari (UC Berkeley)
Manxi Wu (Cornell University)
Shankar Sastry (UC Berkeley)
Abstract : We propose a new framework to study multi-agent interaction in Markov games: Markov α-potential games. Markov potential games are special cases of Markov α-potential games, so are two important and practically significant classes of games: Markov congestion games and perturbed Markov team games. In this paper, α-potential functions for both games are provided and the gap α is characterized with respect to game parameters. Two algorithms – the projected gradient-ascent algorithm and the sequential maximum improvement smoothed best response dynamics – are introduced for approximating the stationary Nash equilibrium in Markov α-potential games. The Nash-regret for each algorithm is shown to scale sub-linearly in time horizon. Our analysis and numerical experiments demonstrates that simple algorithms are capable of finding approximate equilibrium in Markov α-potential games.
[05421] MF-OMO: An Optimization Formulation of Mean-Field Games
Format : Online Talk on Zoom
Author(s) :
Xin Guo (UC Berkeley)
Anran Hu (University of Oxford)
Junzi Zhang (Citadel Securities)
Abstract : The literature on theory and computation of mean-field games (MFGs) has grown exponentially recently, but current approaches are limited to contractive or monotone settings, or with an a priori assumption of the uniqueness of the Nash equilibrium (NE). In this talk, we present MF-OMO (Mean-Field Occupation Measure Optimization), a mathematical framework that analyzes MFGs without these restrictions. MF-OMO reformulates the problem of finding NE solutions in MFGs as a single optimization problem. This formulation thus allows for directly utilizing various optimization tools, algorithms and solvers to find NE solutions of MFGs in practice. We also provide convergence guarantees for finding (multiple) NE solutions using popular algorithms like projected gradient descent. For MFGs with linear rewards and mean-field independent dynamics, solving MF-OMO can be reduced to solving a finite number of linear programs, hence solved in finite time.
