Convergence Analysis of Gradient-Based Learning in Continuous Games
Convergence Analysis of Gradient-Based Learning in Continuous Games
复制标题
DOI：
--
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
Benjamin J. Chasnov;L. Ratliff;Eric V. Mazumdar;Samuel A. Burden
中科院分区：
文献类型：
--
作者：
Benjamin J. Chasnov;L. Ratliff;Eric V. Mazumdar;Samuel A. Burden
考虑一类基于梯度的多智能体学习算法在非合作设置，我们提供收敛保证的一个稳定的纳什均衡的邻域。特别是，我们考虑连续游戏，其中智能体在1）确定性设置中学习，其中Oracle可以访问其个人梯度，2）随机设置中具有其个人梯度的无偏估计。我们还研究了非均匀学习率的影响，这会导致向量场的扭曲，从而改变智能体收敛的均衡和学习路径。我们通过数值示例支持分析，这些示例提供了对如何合成游戏以实现理想均衡的见解。
Considering a class of gradient-based multi-agent learning algorithms in non-cooperative settings, we provide convergence guarantees to a neighborhood of a stable Nash equilibrium. In particular, we consider continuous games where agents learn in 1) deterministic settings with oracle access to their individual gradient and 2) stochastic settings with an unbiased estimator of their individual gradient. We also study the effects of non-uniform learning rates, which cause a distortion of the vector field that can alter the equilibrium to which the agents converge and the learning path. We support the analysis with numerical examples that provide insight into how games may be synthesized to achieve desirable equilibria.