Multi-Agent Reinforcement Learning for Autonoumous Vehicles

这项研究的长期目标是创建一个由机器和设备组成的系统，该系统可以有效地学习如何在不断变化的环境中协同工作。我们正在为多机器人应用提出这样的系统。这个想法是让许多无人驾驶车辆和传感器一起工作，学习如何适应他们的环境。应用包括工业设施和边境地区的安全，以及可以在不危及人类生命的情况下确保领土安全的自动驾驶车辆团队。在这些情况下，视觉系统和各种类型的无人驾驶车辆的组合将学习如何协同工作，以确保该地区的安全并解决任何危险。人工智能与多个车辆和设备的交互将是人工智能发展的巨大飞跃。这项工作对于安全和国防工业、机器人工业以及从事无人驾驶飞行器（无人机）和自动驾驶汽车开发的人员尤为重要。这项工作对工业的影响将是深远的。本研究将集中在多机器人系统的自适应和学习方面。研究了追逃博弈和守域博弈。这项研究的独特之处在于开发学习算法，使机器人有能力学习如何玩这些游戏。我们建议开发学习算法，这样机器人团队就可以学习如何一起玩和如何竞争。在一种情况下，许多机器人将被定义为守卫，被命令防止另一组机器人的入侵。目标是使守卫机器人尽可能远离“目标区域”拦截入侵机器人，并使入侵机器人尽可能靠近“目标区域”。在第二种情况下，机器人将学习如何玩逃避追赶游戏。学习算法的目标是为所有参与者找到“最优”策略。机器人将学习如何考虑自己的能力和其他机器人的能力。我们在开发一种学习算法方面取得了重大进展，这种算法适用于一种逃避者和一种追捕者以及一种守卫者和一种入侵者，每种情况都具有恒定的速度。实验工作表明，我们的算法将如何适应实时情况，并利用另一个机器人的糟糕表现。然后我们发展到多个追击者和多个守卫追逐和防御一个速度更快的入侵者和逃避者的情况。我们开发了一种实验设备，可以使用多达三个移动机器人一起工作。此外，我们正在与金斯顿皇家军事学院的研究人员合作，我们也可以使用他们的实验性移动机器人。

The long term objective of this research is to create a system of machines and devices that can effectively learn how to work together in a changing environment. We are proposing such systems for the multi-robot application. The idea is to have many unmanned vehicles and sensors working together and learning how to adapt to their environment. Applications include the security of industrial facilities and border regions and for teams of autonomous vehicles that can secure territory without endangering human life. In these cases combinations of vision systems and various types of unmanned vehicles will learn how to work together to secure the region and address any dangers. The interaction of artificial intelligence with actions of multiple vehicles and devices will be a huge leap forward in the development of artificial intelligence. This work is specifically important for those in the security and defence industries, the robotics industry and for those working on the development of unmanned aerial vehicles (drones) and self-driving cars. The industrial impact of this work will be far reaching. This research will focus on the adaptation and learning aspects of multi-robot systems. We will investigate the pursuer evader game and the guarding a territory game. The unique aspect of this research is to develop learning algorithms such that the robots have the ability to learn how to play these games. We propose to develop learning algorithms so that teams of robots can learn how to play together and how to compete. In one case, a number of robots will be defined as guards commanded to guard against invasion by another set of robots. The goal is for the guarding set of robots to intercept the invading robots as far as possible from the “target region” and for the invading robots to get as close as possible to the “target region”. In the second case the robots will learn how to play the evader pursuer game. The objective of the learning algorithms is to find the “optimal” strategy for all the players. The robots will learn how to take into consideration their own capabilities and the capabilities of the other robots as well. We have made significant progress in developing learning algorithms for the case of one evader and one pursuer and the case of one guard and one invader, each having constant speed. Experimental work shows how our algorithms will adapt to real time situations and to take advantage of another robot's poor performance. We then progressed to the case of multiple pursuers and multiple guards chasing and defending against a higher speed invader and evader. We have developed an experimental facility that uses up to three mobile robots working together. Furthermore, we are collaborating with researchers at the Royal Military College in Kingston and we have access to their experimental mobile robots as well.