MIMIc: Multimodal Imitation Learning in MultI-Agent Environments

在英国，我们要到17岁才能开车。这是因为驾驶是一项复杂且对安全至关重要的活动，需要许多高级认知技能，如识别可能的威胁，预测其他道路使用者的行为以及对新情况的敏捷反应。想象一个足球运动员在场上做决定。优秀的玩家能够通过预测其他玩家的行动来感知机会，并选择能够提高得分几率的行动。人类需要很长时间才能发展出这些先进的认知技能，才能在如此复杂的现实世界任务中成为专家。人工智能在过去十年中取得了重大进展，在癌症检测、计算机击败围棋大师和智能机器人方面取得了突破。然而，如果人工智能要实现其科幻小说中的承诺，帮助人类甚至取代人类的智能，它至少应该具备人类所拥有的认知技能。该项目旨在开发突破性的算法，使自主系统具备在现实世界环境中茁壮成长所需的人类认知技能。我们专注于需要自主代理（例如机器人或无人驾驶汽车）与环境中的多个智能代理交互以完成任务的应用程序（称为多代理环境：MAEs）。这样的应用程序需要一个代理来预测其他代理的行为，并选择最合适的操作过程。赋予智能体这种自主决策能力被称为策略学习。与单智能体领域的政策学习（教机器人走路或教计算机玩视频游戏）相比，MAEs中政策学习的最新进展相当有限。这是由于多种原因造成的：1)由于智能体的行为，环境是动态的；2)多智能体策略学习受到称为维度诅咒（CoD）的理论限制；3)捕获智能体目标的效用函数难以定义；4)严重缺乏足够的多智能体数据集，无法进行有意义的研究。本项目拟通过解决上述局限性，对MAEs的政策学习进行研究。我们在MAEs中独特的政策学习方法是由人类如何在类似的环境中茁壮成长所激发的。首先，我们通过多种感官（即视觉、听觉、触觉）感知世界，从而对世界有丰富的感知。其次，当在MAE中行动时，人类不会关注所有的刺激，而只关注关键的刺激，例如，当一名足球运动员在进攻时，他只关注能够进球的队友和关键的后卫。最后，我们采用的学习范式，即模仿学习，是一种新兴的通过观察专家来学习的方法，这是一种我们用来学习新技能的有效方法。因此，我们建议利用多模态数据融合和选择性注意建模，通过模仿学习来学习MAEs中的现实策略。多模态数据融合可以捕获现实世界的高维上下文，选择性注意模型可以减轻CoD问题。一个精英足球俱乐部为我们提供了一个独特的多模式多代理数据集，并使用了最先进的设施来捕获数据，以促进这个雄心勃勃的研究项目。项目产出将被主观验证为回答与足球比赛相关的“假设”问题的工具，协助教练组可视化推测的比赛策略，并作为量化足球运动员认知技能的计算基准。计划中的影响活动将确保该项目通过在无人驾驶汽车、视频游戏和辅助机器人等多个高增长领域做出重大贡献，为英国PLC的人工智能发展留下遗产。

In UK, we are not allowed to drive a vehicle until we are 17. It is because, driving is a complex and safety critical activity that requires many advanced cognitive skills like recognition of possible threats, anticipation of behavior of other road users and agile reaction to emerging situations. Think about a football player making decisions on field. A good player can sense the opportunities, through anticipating what other players will do, and select an action that will increase the odds of scoring. It takes a long time for humans to develop these advanced cognitive skills, to become an expert at such complex real-world tasks. Artificial Intelligence has made significant progress during the last decade, demonstrated by breakthroughs in cancer detection, computers beating 'Go' masters and intelligent robotics. However, if AI is to live up to its science fictional promises to assist humanity or even supersede human intelligence, it should at least be equipped with cognitive skills such as those possessed by humans. This project aims to develop ground breaking algorithms that equip autonomous systems with human like cognitive skills required to thrive in real world environments.We are focused on applications that require autonomous agents (e.g. Robot or Driverless car) to interact with multiple intelligent agents in the environment to accomplish a task (known as Multi-Agent Environments: MAEs). Such applications require an agent to anticipate the behaviour of other agents and to select the most appropriate course of actions. Equipping agents with such autonomous decision-making capability is known as policy learning. Compared to policy learning in single agent domains (teaching a robot to walk or a computer to play a video game), the recent progress of policy learning in MAEs has been quite modest. This is due to multiple reasons: 1)Due to agent actions the environment is dynamic 2)multi-agent policy learning suffers from a theoretical limitation known as curse of dimensionality (CoD) 3)Utility functions that capture agent objectives are difficult to define 4)there is a significant lack of adequate multi-agent datasets that allow meaningful research. This project proposes to undertake research in to policy learning in MAEs, by addressing the above limitations. Our unique approach to policy learning in MAEs is motivated by how humans thrive in similar settings. Firstly, we perceive the world through multiple senses, (i.e. vision, audition, touch) enabling a rich perception of the world. Secondly, when acting in a MAE, humans do not pay attention to all the stimuli but only to key stimuli e.g. when a football player is attacking the ball, the player pays attention only to the teammates capable of effecting a goal and the key defenders. Finally, the learning paradigm we employ known as imitation learning is an emerging methodology to learn by observing experts, which is a productive approach that we use to learn new skills. Accordingly, we propose to learn realistic policies in MAEs through imitation learning by leveraging multimodal data fusion and selective-attention modelling. Multimodal data fusion allows to capture high dimensional context of the real world and selective attention model allows for allaying the issue of CoD. We have been provided a unique multimodal multi-agent dataset and access to state-of-the-art facilities to capture data, by an elite football club facilitating this ambitious research project.The project outputs will be subjectively validated as a tool to answer "what-if" questions related to game play in football assisting coaching staff to visualize speculative game strategies, and as a computational benchmark to quantify cognitive skills of football players. The planned impact activities will ensure the project will leave a legacy in AI development benefiting UK PLC through significant contribution in multiple high growth areas, such as driverless vehicles, video gaming, and assistive robots.