Disturbance Decoupling for Gradient-Based Multi-Agent Learning With Quadratic Costs
Disturbance Decoupling for Gradient-Based Multi-Agent Learning With Quadratic Costs
复制标题
DOI：
10.1109/lcsys.2020.3001240
复制
发表时间：
2020-07
影响因子：
3
通讯作者：
Sarah H. Q. Li;L. Ratliff;Behçet Açikmese
中科院分区：
文献类型：
--
作者：
Sarah H. Q. Li;L. Ratliff;Behçet Açikmese
受多智能体学习在噪声环境中应用的启发，本文研究了基于梯度的学习动态对干扰的鲁棒性。虽然沿着对应于任何单个玩家的动作的坐标注入的干扰总是会影响整体学习动态，但玩家的子集可以是干扰分离的--即，这样的玩家的动作完全不受注入的干扰的影响。对于具有二次代价函数的对策，包括二次一次连续对策、有限水平线性二次(LQ)动态对策和双线性对策，我们给出了保证这一性质的充要条件。具体地说，扰动解耦是由学习动力学的代数条件和图论条件来描述的，后者是通过构造基于玩家成本梯度的博弈图来获得的。对于LQ对策，我们证明了干扰解耦对玩家的可控子空间和不可观测子空间施加了约束。对于两个参与者双线性对策，我们证明了在参与者的行动坐标内的干扰解耦对支付矩阵施加了约束。文中给出了算例。
Motivated by applications of multi-agent learning in noisy environments, this letter studies the robustness of gradient-based learning dynamics with respect to disturbances. While disturbances injected along a coordinate corresponding to any individual player’s actions can always affect the overall learning dynamics, a subset of players can be disturbance decoupled—i.e., such players’ actions are completely unaffected by the injected disturbance. We provide necessary and sufficient conditions to guarantee this property for games with quadratic cost functions, which encompass quadratic one-shot continuous games, finite-horizon linear quadratic (LQ) dynamic games, and bilinear games. Specifically, disturbance decoupling is characterized by both algebraic and graph-theoretic conditions on the learning dynamics, the latter is obtained by constructing a game graph based on gradients of players’ costs. For LQ games, we show that disturbance decoupling imposes constraints on the controllable and unobservable subspaces of players. For two player bilinear games, we show that disturbance decoupling within a player’s action coordinates imposes constraints on the payoff matrices. Illustrative numerical examples are provided.