Deep Fictitious Play for Finding Markovian Nash Equilibrium in Multi-Agent Games
Deep Fictitious Play for Finding Markovian Nash Equilibrium in Multi-Agent Games
复制标题
DOI：
--
复制
发表时间：
2019-12
期刊：
影响因子：
--
通讯作者：
Jiequn Han;Ruimeng Hu
中科院分区：
文献类型：
--
作者：
Jiequn Han;Ruimeng Hu
我们提出了一种基于深度神经网络的算法来识别一般大$N$参与人随机微分对策的马尔可夫纳什均衡。根据虚拟游戏的理念，我们将N个玩家的游戏重新定义为N个解耦决策问题（每个玩家一个），并迭代地解决它们。个体决策问题具有半线性Hamilton-Jacobi-Bellman方程的特征，我们采用了最近发展的深度BSDE方法来求解该方程。所得到的算法可以求解大量的N人博弈，而传统的数值方法会受到维数的困扰。多个涉及相同或异质代理，具有风险中性或风险敏感目标的数值示例进行了测试，以验证所提出算法在大型群体博弈中的准确性。即使对于存在共同噪声的50人博弈，所提出的算法仍能准确地找到近似纳什均衡，据我们所知，这是其他数值算法难以实现的。
We propose a deep neural network-based algorithm to identify the Markovian Nash equilibrium of general large $N$-player stochastic differential games. Following the idea of fictitious play, we recast the $N$-player game into $N$ decoupled decision problems (one for each player) and solve them iteratively. The individual decision problem is characterized by a semilinear Hamilton-Jacobi-Bellman equation, to solve which we employ the recently developed deep BSDE method. The resulted algorithm can solve large $N$-player games for which conventional numerical methods would suffer from the curse of dimensionality. Multiple numerical examples involving identical or heterogeneous agents, with risk-neutral or risk-sensitive objectives, are tested to validate the accuracy of the proposed algorithm in large group games. Even for a fifty-player game with the presence of common noise, the proposed algorithm still finds the approximate Nash equilibrium accurately, which, to our best knowledge, is difficult to achieve by other numerical algorithms.