Trustworthy Reinforcement Learning for Multi-Agent Systems: Foundations of Robust and Accountable Decision Making

强化学习是一种在不确定条件下对序贯决策进行建模和自动化的计算方法。强化学习的最新进展突显了在高风险领域使用这种方法的令人难以置信的潜力，例如推荐系统、交通或教育。然而，最新的强化学习技术在高风险领域的适用性受到了许多关注，因为它们没有考虑到现实世界场景中存在的复杂性，包括其多智能体结构。在这个项目中，我们提出了一个框架来设计可信和可靠的强化学习算法。我们确定了设计这样一个框架的两个重要组成部分，代理设计(设计强化学习算法本身)和系统设计(设计改进代理学习过程的支持工具)。在每个组成部分中，我们将研究这样一个框架被认为值得信任应该具备的两个重要属性：健壮性(处理对手和不确定性的能力)和责任(为自己的行为提供解释的能力)。在稳健性方面，这一建议详细说明了对包括对手在内的其他代理的存在具有健壮性的代理的设计，以及通过引导可信信息来支持健壮学习的系统的设计。在问责制方面，该提案详细说明了能够为其行为提供解释的代理人的设计，以及通过为结果分配责任来支持问责制的制度的设计。该提案进一步概述了解决与这四个研究方向相关的技术挑战的议程，并提出了依赖多代理学习、强化学习和博弈论的方法。拟议的议程集中在稳健和负责任的决策制定的理论和算法方面，这些决策为多代理系统提供了值得信赖的强化学习的基础步骤。

Reinforcement learning is a computational approach to modeling and automating sequential decision making under uncertainty. Recent advancements in reinforcement learning have highlighted the incredible potential of utilizing this approach in high-stake domains, such as recommendation systems, transportation, or education. However, many concerns have been raised regarding the applicability of the state-of-the-art reinforcement learning techniques to high stake domains, as they fail to account for complexities present in real-world scenarios, including their multi-agent structure.In this project, we propose a framework for designing trustworthy and reliable reinforcement learning algorithms. We identify two important components in designing such a framework, agent design (designing reinforcement learning algorithms themselves) and system design (designing supporting tools that improve the learning processes of agents). Within each of these components, we will study two important properties that such a framework ought to have in order to be deemed trustworthy: robustness (ability to deal with adversaries and uncertainty) and accountability (ability to provide an account for one’s behavior). In the context of robustness, this proposal explicates the design of agents that are robust to the presence of other agents, including adversaries, as well as the design of systems that support robust learning through channeling trusted information. In the context of accountability, this proposal explicates the design of agents that can provide explanations for their actions, as well as the design of systems that support accountability by assigning responsibility for the outcomes. The proposal further outlines the agenda for resolving technical challenges related to these four research directions, and proposes approaches that rely on multi-agent learning, reinforcement learning, and game theory. The proposed agenda focuses on theoretical and algorithmic aspects of robust and accountable decision making that provide foundational steps toward trustworthy reinforcement learning for multi-agent systems.