Trustworthy Reinforcement Learning for Multi-Agent Systems: Foundations of Robust and Accountable Decision Making

强化学习是一种在不确定条件下进行序列决策建模和自动化的计算方法。强化学习的最新进展突出了在高风险领域（如推荐系统、交通或教育）利用这种方法的惊人潜力。然而，关于最先进的强化学习技术在高风险领域的适用性，人们提出了许多担忧，因为它们无法解释现实世界场景中存在的复杂性，包括它们的多智能体结构。在这个项目中，我们提出了一个框架来设计可信和可靠的强化学习算法。我们确定了设计这样一个框架的两个重要组成部分，代理设计（设计强化学习算法本身）和系统设计（设计改进代理学习过程的支持工具）。在这些组件中，我们将研究这样一个框架应该具有的两个重要属性，以便被认为是值得信赖的：鲁棒性（处理对手和不确定性的能力）和可问责性（为一个人的行为提供解释的能力）。在鲁棒性的背景下，该建议阐述了对其他代理（包括对手）的存在具有鲁棒性的代理的设计，以及通过引导可信信息支持鲁棒学习的系统的设计。在问责制的背景下，本建议阐述了可以为其行为提供解释的代理的设计，以及通过为结果分配责任来支持问责制的系统的设计。该提案进一步概述了解决与这四个研究方向相关的技术挑战的议程，并提出了依赖于多智能体学习、强化学习和博弈论的方法。提出的议程侧重于稳健和负责任决策的理论和算法方面，为多智能体系统的可信强化学习提供基础步骤。

Reinforcement learning is a computational approach to modeling and automating sequential decision making under uncertainty. Recent advancements in reinforcement learning have highlighted the incredible potential of utilizing this approach in high-stake domains, such as recommendation systems, transportation, or education. However, many concerns have been raised regarding the applicability of the state-of-the-art reinforcement learning techniques to high stake domains, as they fail to account for complexities present in real-world scenarios, including their multi-agent structure.In this project, we propose a framework for designing trustworthy and reliable reinforcement learning algorithms. We identify two important components in designing such a framework, agent design (designing reinforcement learning algorithms themselves) and system design (designing supporting tools that improve the learning processes of agents). Within each of these components, we will study two important properties that such a framework ought to have in order to be deemed trustworthy: robustness (ability to deal with adversaries and uncertainty) and accountability (ability to provide an account for one’s behavior). In the context of robustness, this proposal explicates the design of agents that are robust to the presence of other agents, including adversaries, as well as the design of systems that support robust learning through channeling trusted information. In the context of accountability, this proposal explicates the design of agents that can provide explanations for their actions, as well as the design of systems that support accountability by assigning responsibility for the outcomes. The proposal further outlines the agenda for resolving technical challenges related to these four research directions, and proposes approaches that rely on multi-agent learning, reinforcement learning, and game theory. The proposed agenda focuses on theoretical and algorithmic aspects of robust and accountable decision making that provide foundational steps toward trustworthy reinforcement learning for multi-agent systems.