Cooperative pursuit of unauthorized UAVs in urban airspace via Multi-agent reinforcement learning
Cooperative pursuit of unauthorized UAVs in urban airspace via Multi-agent reinforcement learning
复制标题
DOI：
10.1016/j.trc.2021.103122
复制
发表时间：
2021-07
期刊：
影响因子：
--
通讯作者：
Wenbo Du;Guo Tong;Jun Chen;Li Biyue;Guangxiang Zhu;Xianbin Cao
中科院分区：
文献类型：
--
作者：
Wenbo Du;Guo Tong;Jun Chen;Li Biyue;Guangxiang Zhu;Xianbin Cao
城市空中交通（UAM）是未来航空运输的一个新兴概念。通过城市空中交通，货物和乘客将在城市空域按需运输。城市空中交通在缓解地面拥堵和为人们提供替代出行选择方面展现出了广阔的前景。然而，城市空域中未经授权的无人机（UAV）对UAM的安全构成了重大威胁，最近引起了研究界的高度关注。在所有解决方案中，无人机团队协同追击是针对城市空域未经授权的无人机的有效对策。在本文中，我们将合作追击建模为追击逃避游戏问题（PEG），并提出一种基于多智能体强化学习（MARL）的方法来有效地解决该问题。所提出的方法结合了新颖的蜂窝参数共享和课程学习方案，以增强追击者无人机在城市空域捕获更快的未经授权无人机的能力。为了评估所提出方法的性能，已经使用模拟城市空域进行了广泛的实验。实验结果表明，通过结合参数共享方案，所提出的方法在更短的时间内提供了更高的捕获率。当通信限制更加严格和/或未经授权的无人机速度更快时，这种优势更加明显。
Urban Air Mobility (UAM) is an emergent concept for future air transportation. With UAM, cargo and passengers will be transported on-demand in urban airspace. UAM has shown a promising prospect in mitigating ground congestion and providing people with an alternative mobility option. However, unauthorized unmanned aerial vehicles (UAVs) in urban airspace present a significant threat to safety of UAM, drawing significant attention from research communities recently. Among all solutions, cooperative pursuit using a team of UAVs is an effective countermeasure for unauthorized UAVs in urban airspace. In this paper, we model cooperative pursuit as a pursuit-evasion game problem (PEG) and propose a multi-agent reinforcement learning (MARL) based approach to solve the problem efficiently. The proposed approach incorporates novel cellular-enabled parameter sharing and curriculum learning schemes to enhance the capability of pursuer UAVs in capturing faster unauthorized UAVs in urban airspace. Extensive experiments have been conducted using simulated urban airspace in order to evaluate the performance of the proposed method. Experimental results demonstrate that by incorporating the parameter sharing scheme, the proposed methods provide much higher capturing rates in a shorter time. Such superiority is more evident when communication constraints are more stringent and/or unauthorized UAVs are faster.