Altruistic Maneuver Planning for Cooperative Autonomous Vehicles Using Multi-agent Advantage Actor-Critic
Altruistic Maneuver Planning for Cooperative Autonomous Vehicles Using Multi-agent Advantage Actor-Critic
复制标题
DOI：
--
复制
发表时间：
2021-07
期刊：
影响因子：
--
通讯作者：
Behrad Toghi;Rodolfo Valiente;Dorsa Sadigh;Ramtin Pedarsani;Y. P. Fallah
中科院分区：
文献类型：
--
作者：
Behrad Toghi;Rodolfo Valiente;Dorsa Sadigh;Ramtin Pedarsani;Y. P. Fallah
随着自动驾驶汽车在道路上的应用，我们将见证一个混合自动驾驶环境，自动驾驶汽车和人类驾驶汽车必须学会通过共享相同的道路基础设施来共存。为了实现社会期望的行为，必须指示自动驾驶车辆在决策过程中考虑周围其他车辆的效用。特别是，我们研究自动驾驶车辆的机动规划问题，并研究去中心化的奖励结构如何诱导其行为中的利他主义，并激励它们考虑其他自动驾驶和人类驾驶车辆的利益。由于人类驾驶员与自动驾驶车辆合作的意愿不明确，这是一个具有挑战性的问题。因此，与依赖人类驾驶员行为模型的现有工作相比，我们采取端到端的方法，让自主代理仅从经验中隐式学习人类驾驶员的决策过程。我们引入了同步 Advantage Actor-Critic (A2C) 算法的多智能体变体，并训练相互协调并可以影响人类驾驶员行为的智能体，以改善交通流量和安全。
With the adoption of autonomous vehicles on our roads, we will witness a mixed-autonomy environment where autonomous and human-driven vehicles must learn to co-exist by sharing the same road infrastructure. To attain socially-desirable behaviors, autonomous vehicles must be instructed to consider the utility of other vehicles around them in their decision-making process. Particularly, we study the maneuver planning problem for autonomous vehicles and investigate how a decentralized reward structure can induce altruism in their behavior and incentivize them to account for the interest of other autonomous and human-driven vehicles. This is a challenging problem due to the ambiguity of a human driver's willingness to cooperate with an autonomous vehicle. Thus, in contrast with the existing works which rely on behavior models of human drivers, we take an end-to-end approach and let the autonomous agents to implicitly learn the decision-making process of human drivers only from experience. We introduce a multi-agent variant of the synchronous Advantage Actor-Critic (A2C) algorithm and train agents that coordinate with each other and can affect the behavior of human drivers to improve traffic flow and safety.