Enabling Relational Reasoning in Multi-Agent Deep Reinforcement Learning

强化学习的目的是教会人工智能体如何在不确定的环境中采取最优的顺序决策来完成任务。该领域的最新进展利用深度学习方法作为函数逼近器，从而在从游戏到生物信息学的各个领域实现了复杂的应用。尽管取得了这些进展，但大多数工作都集中在单个代理与环境相互作用的情况下。然而，许多现实世界的应用涉及多个协作代理共同决策；一些突出的例子包括自动驾驶汽车、制造机器人和网络安全机器人。在这种情况下，主体间的通信对于实现协作行为至关重要，该领域最近的发展一直关注于促进整个学习过程中自发出现的通信协议。在这个项目中，我们将开发一个建模框架，除了学习如何沟通之外，代理还可以开发执行关系推理的能力，即它们将能够推断环境中行为的实体如何相互关联，并对这些关系进行编码，以改进决策过程。在此过程中，我们将大量借鉴几何深度学习领域，其中关系图神经网络目前被用于从网络值数据中学习关系模式。我们的目标是为多智能体系统开发一种统一的关系强化学习方法，该方法既分散又可扩展。将考虑几个日益复杂的应用程序，以展示我们的算法在实际用例中的潜在用途。

The aim of reinforcement learning is to teach an artificial agent how to take optimal sequential decisions in an uncertain environment to complete a task. Recent advancements in the field have leveraged deep learning methods as function approximators thus enabling complex applications in various areas, from gaming to bioinformatics. Despite these advances, most of the work has focused on the case of a single agent interacting with the environment. However, many real-world applications involve multiple cooperative agents taking joint decisions; some prominent examples include autonomous vehicles, manufacturing robotics and cyber-security bots. In such settings, inter-agent communication becomes essential to achieve collaborative behaviour, and recent developments in the fields have been concerned with facilitating the spontaneous emergence of communication protocols throughout the learning process. In this project, we will develop a modelling framework where, in addition to learning how to communicate, the agents can also develop the ability to perform relational reasoning, i.e. they'll be able to infer how the entities acting in the environment are related to one another and encode those relationships in order to improve the decision-making process. In doing so, we will draw heavily from the field of geometric deep learning where relational graph neural networks are currently employed to learn relational patterns from network-valued data. Our aim is to develop a unified relational reinforcement learning approach for multi agent systems that is both decentralised and scalable. Several applications of increasing complexity will be considered to showcase the potential use of our algorithms in real-world use cases.