# Registered Data

## [01167] Recent development in mean field control and learning

**Session Date & Time**:- 01167 (1/2) : 1E (Aug.21, 17:40-19:20)
- 01167 (2/2) : 2C (Aug.22, 13:20-15:00)

**Type**: Proposal of Minisymposium**Abstract**: Mean field control problems have attracted massive interest and provide a promising approach dealing with multi-agent systems. The aim of this mini-symposium is to share the new trends of both theory and applications of this area. We would like to invite the frontier scholars to talk about recent developments in various learning methods for the mean field control problems under different application aspects, as well as analyzing HJB equation in the infinite dimensional spaces for the theory prospective.**Organizer(s)**: Xin Guo, Jiacheng Zhang**Sponsor**: This session is sponsored by the SIAM Activity Group on Financial Mathematics and Engineering.**Classification**:__49N80__,__35Q89__**Speakers Info**:- Halil Mete Soner (Princeton University)
- Huyên Pham (Université Paris Cité)
- Janka Möller (University of Vienna)
- Jodi Dianetti ( Bielefeld University)
- Lane Chun Yeung (Columbia University)

**Talks in Minisymposium**:**[04658] Actor-critic learning for mean-field control in continuous time****Author(s)**:**HUYEN PHAM**(Université Paris Cité )- Noufel Frikha (Université Paris 1)
- Maximilien Germain (Morgan Stanley )
- Mathieu Laurière (NYU Shanghai)
- Xuanye Song (Université Paris Cité)

**Abstract**: We study policy gradient for mean-field control in continuous time in a reinforcement learning setting. By considering randomised policies with entropy regularisation, we derive a gradient expectation representation of the value function, which is amenable to actor-critic type algorithms, where the value functions and the policies are learnt alternately based on observation samples of the state and model-free estimation of the population state distribution, either by offline or online learning.

**[04674] Mean-field singular control problem: regularitiy and related mean-field reflected diffusion****Author(s)**:**Jodi Dianetti**(Bielefeld University)- Xin Guo (UC Berkeley)
- Jiacheng Zhang (UC Berkeley)
- Huyên Pham (Université Paris Cité )

**Abstract**: We study a class of mean-field control problems with singular controls. Such a model represents the limit of the control problems in which a controller can adjust, through a bounded variation process, an underlying diffusion, which in turn affects an n-particle system. Adopting appropriate notions of convexities, we are able to establish the regularity of the value function of the problem and to show the existence of the optimal control. The regularity of the value function allows to characterize the solution of the problem in terms of a related mean-field Skorokhod problem. This consists in keeping the optimally controlled state process in a region prescribed by the derivative of the value function, by using the optimal control in order to reflect the state at its boundary.

**[04752] A non-asymptotic perspective on mean field control****Author(s)**:**Lane Chun Yeung**(Columbia University)- Daniel Lacker (Columbia University)
- Sumit Mukherjee (Columbia University)

**Abstract**: We study a class of stochastic control problems in which a large number of players cooperatively choose their drifts to maximize an expected reward minus a quadratic running cost. For a broad class of potentially asymmetric rewards, we show that there exist approximately optimal controls which are decentralized, in the sense that each player's control depends only on its own state and not the states of the other players.

**[04923] Synchronization in a Kuramoto Mean Field Game****Author(s)**:**Halil Mete Soner**(Princeton University)- Rene Carmona (Princeton University)
- Quentin Cormier (Inria Saclay)

**Abstract**: Originally motivated by systems of chemical and biological oscillators, the classical Kuramoto model has found an amazing range of applications from neuroscience to Josephson junctions in superconductors, and has become a key mathematical model to describe self organization in complex systems. These autonomous oscillators are coupled through a nonlinear interaction term which plays a central role in the long term behavior of the system. While the system is not synchronized when this term is not sufficiently strong, fascinatingly, they exhibit an abrupt transition to a full synchronization above a critical value of the interaction parameter. We explore this system in the mean field formalism. We treat the system of oscillators as an infinite particle system, but instead of positing the dynamics of the particles, we let the individual particles determine endogenously their behaviors by minimizing a cost functional and eventually, settling in a Nash equilibrium. The mean field game also exhibits a bifurcation from incoherence to self-organization. This approach has found interesting applications including circadian rhythms and jet-lag recovery.

**[05079] Signature SDEs with jumps and their tractability properties****Author(s)**:- Christa Cuchiero (University of Vienna)
**Francesca Primavera**(University of Vienna)- Sara Svaluto Ferro (University of Verona)

**Abstract**: Signature-based models have recently entered the field of Mathematical Finance. Relying on recent advances on the signature of càdlàg paths, we introduce here a generic class of jump-diffusion models via so-called signature SDEs with jumps. We elaborate on their tractability properties and show that the signature-based models for asset prices proposed so far can be embedded in this framework. As a special case, we focus on jump-diffusions with entire characteristics, leading to a far-reaching extension of the class of polynomial processes.

**[05419] Markov $alpha$-Potential Game****Author(s)**:**Xinyu Li**(UC Berkeley)- Xin Guo (UC Berkeley)
- Chinmay Maheshwari (UC Berkeley)
- Manxi Wu (Cornell University)
- Shankar Sastry (UC Berkeley)

**Abstract**: We propose a new framework to study multi-agent interaction in Markov games: Markov α-potential games. Markov potential games are special cases of Markov α-potential games, so are two important and practically significant classes of games: Markov congestion games and perturbed Markov team games. In this paper, α-potential functions for both games are provided and the gap α is characterized with respect to game parameters. Two algorithms – the projected gradient-ascent algorithm and the sequential maximum improvement smoothed best response dynamics – are introduced for approximating the stationary Nash equilibrium in Markov α-potential games. The Nash-regret for each algorithm is shown to scale sub-linearly in time horizon. Our analysis and numerical experiments demonstrates that simple algorithms are capable of finding approximate equilibrium in Markov α-potential games.

**[05421] MF-OMO: An Optimization Formulation of Mean-Field Games****Author(s)**:- Xin Guo (UC Berkeley)
**Anran Hu**(University of Oxford)- Junzi Zhang (Citadel Securities)

**Abstract**: The literature on theory and computation of mean-field games (MFGs) has grown exponentially recently, but current approaches are limited to contractive or monotone settings, or with an a priori assumption of the uniqueness of the Nash equilibrium (NE). In this talk, we present MF-OMO (Mean-Field Occupation Measure Optimization), a mathematical framework that analyzes MFGs without these restrictions. MF-OMO reformulates the problem of finding NE solutions in MFGs as a single optimization problem. This formulation thus allows for directly utilizing various optimization tools, algorithms and solvers to find NE solutions of MFGs in practice. We also provide convergence guarantees for finding (multiple) NE solutions using popular algorithms like projected gradient descent. For MFGs with linear rewards and mean-field independent dynamics, solving MF-OMO can be reduced to solving a finite number of linear programs, hence solved in finite time.

**[05439] The convergence problem in mean field control****Author(s)**:**Joe Jackson**(The University of Texas at Austin)- Samuel Daudin (Université Côte d'Azur)
- François Delarue (Université Côte d'Azur)

**Abstract**: This talk will be about a recent joint work with Samuel Daudin and François Delarue concerning the convergence problem in mean field control. When the data is convex and sufficiently smooth, the (optimal) rate of convergence (of the $N$-player value function towards the limiting value function) is known to be $1/N$. The goal of our work is to identify the optimal rate of convergence in the more subtle non-convex setting.