Reinforcement Learning for Mean Field Games with Strategic Complementarities
Reinforcement Learning for Mean Field Games with Strategic Complementarities
复制标题
DOI：
--
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Kiyeob Lee;Desik Rengarajan;D. Kalathil;S. Shakkottai
中科院分区：
文献类型：
--
作者：
Kiyeob Lee;Desik Rengarajan;D. Kalathil;S. Shakkottai
平均现场游戏（MFG）是具有大量代理的游戏类别，而标准的平衡概念是平均均衡（MFE）（MFE）。具有称为战略补充的单调性属性（MFG-SC）。颤抖的手法MFE（T-MFE）可以在考虑到这种随机化对其payo的影响时采用随机化的测量引入一种基于模型的学习T-MFE的方法，并提供两种算法的样本复杂性。最后，我们通过由现实世界应用激励的示例从经验上评估所提出的算法的性能。
Mean Field Games (MFG) are the class of games with a very large number of agents and the standard equilibrium concept is a Mean Field Equilibrium (MFE). Algorithms for learning MFE in dynamic MFGs are unknown in general. Our focus is on an important subclass that possess a monotonicity property called Strategic Complementar-ities (MFG-SC). We introduce a natural re-finement to the equilibrium concept that we call Trembling-Hand-Perfect MFE (T-MFE), which allows agents to employ a measure of randomization while accounting for the impact of such randomization on their payoffs. We propose a simple algorithm for computing T-MFE under a known model. We also introduce a model-free and a model-based approach to learning T-MFE and provide sample complexities of both algorithms. We also develop a fully online learning scheme that obviates the need for a simulator. Finally, we empirically evaluate the performance of the proposed algorithms via examples motivated by real-world applications.