RI: Small: Understanding Value-based Multiagent Learning and Its Applications

这个项目探索了基于价值的学习方法在多智能体环境中的行为。基于价值的方法通过经验来估计替代方案的效用影响，并选择具有较高预测值的方案来做出决策。因为它们评估行为的组成部分，而不是将行为视为原子单位，所以它们在计算和统计上都是有效的。虽然这些方法已经在计算实验中使用了很多年，但研究人员直到最近才开始正式描述它们的行为。我们自己的初步工作是发现一些基于值的方法表现出超级Nash行为，使它们特别值得研究。更具体地说，我们正在从数学和实验上分析基于值的算法在人工智能社区、无线网络领域的多智能体工程应用以及与认知神经科学家合作的人类和动物决策模型中的各种复杂程度的模拟游戏中的性能。在可能的情况下，我们正在改进现有的基于值的算法，以便比现有的算法更有效、更健壮、更普遍地工作。我们还在设计教育推广活动，包括制作有趣的教学视频，讲述如何在现实生活中的社会困境中促进合作行为。

This project explores the behavior of value-based learning methods in multi-agent environments. Value-based methods make decisions by using experience to estimate the utility impact of alternatives and choosing those with high predicted value. Because they evaluate components of behavior instead of treating behaviors as atomic units, they are computationally and statistically efficient. While these methods have been used in computational experiments for many years, only recently have researchers begun to formally characterize their behavior. Our own preliminary work is finding that some value-based methods exhibit super-Nash behavior, making them particularly worthy of study.More specifically, we are analyzing, mathematically and experimentally, how value-based algorithms perform in several classes of simulated games of varying complexity from the artificial intelligence community, multi-agent engineering applications drawn from the wireless networking area, and as models of human and animal decision making in collaboration with cognitive neuroscientists. Where possible, we are refining existing value-based algorithms to work more efficiently, robustly, and generally than existing algorithms. We are also designing educational outreach activities, including creating entertaining instructional videos on how to promote cooperative behavior in real-life social dilemmas.