Enabling Relational Reasoning in Multi-Agent Deep Reinforcement Learning

强化学习的目的是教人工智能体如何在不确定的环境中采取最优的顺序决策来完成任务。该领域的最新进展利用深度学习方法作为函数逼近器，从而使从游戏到生物信息学的各种领域的复杂应用成为可能。尽管取得了这些进展，但大多数工作都集中在单个代理人与环境相互作用的情况下。然而，许多现实世界的应用涉及多个合作代理共同做出决定；一些突出的例子包括自动驾驶汽车、制造机器人和网络安全机器人。在这种情况下，代理间的通信对于实现协作行为变得至关重要，该领域的最新发展一直关注于促进在整个学习过程中自发出现通信协议。在这个项目中，我们将开发一个建模框架，在其中，除了学习如何通信外，代理还可以发展执行关系推理的能力，即他们将能够推断在环境中行为的实体如何相互关联并对这些关系进行编码，以改进决策过程。在这样做的过程中，我们将大量借鉴几何深度学习领域，在该领域中，关系图神经网络目前被用于从网络值数据中学习关系模式。我们的目标是为多智能体系统开发一种既分散又可扩展的统一关系强化学习方法。将考虑几个日益复杂的应用程序，以展示我们的算法在现实世界用例中的潜在使用。

The aim of reinforcement learning is to teach an artificial agent how to take optimal sequential decisions in an uncertain environment to complete a task. Recent advancements in the field have leveraged deep learning methods as function approximators thus enabling complex applications in various areas, from gaming to bioinformatics. Despite these advances, most of the work has focused on the case of a single agent interacting with the environment. However, many real-world applications involve multiple cooperative agents taking joint decisions; some prominent examples include autonomous vehicles, manufacturing robotics and cyber-security bots. In such settings, inter-agent communication becomes essential to achieve collaborative behaviour, and recent developments in the fields have been concerned with facilitating the spontaneous emergence of communication protocols throughout the learning process. In this project, we will develop a modelling framework where, in addition to learning how to communicate, the agents can also develop the ability to perform relational reasoning, i.e. they'll be able to infer how the entities acting in the environment are related to one another and encode those relationships in order to improve the decision-making process. In doing so, we will draw heavily from the field of geometric deep learning where relational graph neural networks are currently employed to learn relational patterns from network-valued data. Our aim is to develop a unified relational reinforcement learning approach for multi agent systems that is both decentralised and scalable. Several applications of increasing complexity will be considered to showcase the potential use of our algorithms in real-world use cases.