Shaping Large Population Agent Behaviors Through Entropy-Regularized Mean-Field Games
Shaping Large Population Agent Behaviors Through Entropy-Regularized Mean-Field Games
复制标题
DOI：
10.23919/acc53348.2022.9867358
复制
发表时间：
2021-10
期刊：
影响因子：
--
通讯作者：
Yue Guan;Michael X. Zhou;A. Pakniyat;P. Tsiotras
中科院分区：
文献类型：
--
作者：
Yue Guan;Michael X. Zhou;A. Pakniyat;P. Tsiotras
平均场博弈（MFG）被引入到有效地分析近似纳什均衡在大人口设置。在这项工作中，我们考虑熵正则化的平均场游戏与一个有限的状态动作空间中的离散时间设置。我们表明，熵正则化提供了必要的正则性条件，缺乏标准的有限平均场游戏。这样的正则性条件使我们能够设计不动点迭代算法来找到唯一的平均场平衡（MFE）。此外，在正则化中使用的参考策略提供了一个额外的参数，通过它可以控制种群的行为。我们首先考虑一个随机对策与一个大的人口N齐次代理。我们建立了在N趋于无穷大的极限情况下存在纳什均衡的条件，并且我们证明了无限人口情况下的纳什均衡也是N-代理系统的Nash-纳什均衡，其中次最优解的阶为${\mathcal{O}}\left（{1/\sqrt N }\right）$。最后，我们通过一个资源分配的例子验证了理论保证，并证明了使用参考策略来控制大量人口的行为的有效性。
Mean-field games (MFG) were introduced to efficiently analyze approximate Nash equilibria in large population settings. In this work, we consider entropy-regularized mean-field games with a finite state-action space in a discrete time setting. We show that entropy regularization provides the necessary regularity conditions, that are lacking in the standard finite mean field games. Such regularity conditions enable us to design fixed-point iteration algorithms to find the unique mean-field equilibrium (MFE). Furthermore, the reference policy used in the regularization provides an extra parameter, through which one can control the behavior of the population. We first consider a stochastic game with a large population of N homogeneous agents. We establish conditions for the existence of a Nash equilibrium in the limiting case as N tends to infinity, and we demonstrate that the Nash equilibrium for the infinite population case is also an ϵ-Nash equilibrium for the N-agent system, where the sub-optimality ϵ is of order ${\mathcal{O}}\left({1/\sqrt N }\right)$. Finally, we verify the theoretical guarantees through a resource allocation example and demonstrate the efficacy of using a reference policy to control the behavior of a large population.