Deep Reinforcement Learning for Infinite Horizon Mean Field Problems in Continuous Spaces
Year: 2025
Author: Andrea Angiuli, Jean-Pierre Fouque, Ruimeng Hu, Alan Raydan
Journal of Machine Learning, Vol. 4 (2025), Iss. 1 : pp. 11–47
Abstract
We present the development and analysis of a reinforcement learning algorithm designed to solve continuous-space mean field game (MFG) and mean field control (MFC) problems in a unified manner. The proposed approach pairs the actor-critic (AC) paradigm with a representation of the mean field distribution via a parameterized score function, which can be efficiently updated in an online fashion, and uses Langevin dynamics to obtain samples from the resulting distribution. The AC agent and the score function are updated iteratively to converge, either to the MFG equilibrium or the MFC optimum for a given mean field problem, depending on the choice of learning rates. A straightforward modification of the algorithm allows us to solve mixed mean field control games. The performance of our algorithm is evaluated using linear-quadratic benchmarks in the asymptotic infinite horizon framework.
Journal Article Details
Publisher Name: Global Science Press
Language: English
DOI: https://doi.org/10.4208/jml.230919
Journal of Machine Learning, Vol. 4 (2025), Iss. 1 : pp. 11–47
Published online: 2025-01
AMS Subject Headings:
Copyright: COPYRIGHT: © Global Science Press
Pages: 37
Keywords: Actor-critic Linear-quadratic control Mean field game Mean field control Mixed mean field control game Score matching Reinforcement learning Timescales.
Author Details
Andrea Angiuli Email
Jean-Pierre Fouque Email
Ruimeng Hu Email
Alan Raydan Email