Social Coordination and Altruism in Autonomous Driving
Social Coordination and Altruism in Autonomous Driving
复制标题
DOI：
10.1109/tits.2022.3207872
复制
发表时间：
2021-07
影响因子：
8.5
通讯作者：
Behrad Toghi;Rodolfo Valiente;Dorsa Sadigh;Ramtin Pedarsani;Y. P. Fallah
中科院分区：
文献类型：
--
作者：
Behrad Toghi;Rodolfo Valiente;Dorsa Sadigh;Ramtin Pedarsani;Y. P. Fallah
尽管自动驾驶领域取得了进展，但自动驾驶汽车在相互合作或与人类驾驶的车辆协调方面仍然效率低下，受到限制。一组自动驾驶和人工驾驶的车辆(HV)可以无缝共存，确保道路上的安全和效率，它们共同努力优化利他主义的社会效用。在没有代理之间明确协调的情况下实现这一任务是具有挑战性的，主要是因为很难预测混合自治环境中具有不同偏好的人的行为。在形式上，我们将车辆在混合自主交通中的机动规划建模为部分可观测的随机博弈，并尝试使用多智能体强化学习框架(MARL)来获得导致社会期望结果的最优策略，并针对我们的MAIL问题提出了一种半顺序多智能体训练和策略传播算法。我们引入了社会偏好的量化表示，并设计了一种分布式奖励结构，将利他主义引入他们的决策过程。利他主义的AVs能够结成联盟，引导交通，并影响HV的行为，以应对竞争的驾驶场景。我们将利己主义的AVs与我们的利他的自主代理在高速公路合并环境中进行了比较，并展示了新出现的行为导致成功合并的数量以及整体交通流量和安全性的改善。
Despite the advances in the autonomous driving domain, autonomous vehicles (AVs) are still inefficient and limited in terms of cooperating with each other or coordinating with vehicles operated by humans. A group of autonomous and human-driven vehicles (HVs) which work together to optimize an altruistic social utility can co-exist seamlessly and assure safety and efficiency on the road. Achieving this mission without explicit coordination among agents is challenging, mainly due to the difficulty of predicting the behavior of humans with heterogeneous preferences in mixed-autonomy environments. Formally, we model an AV’s maneuver planning in mixed-autonomy traffic as a partially-observable stochastic game and attempt to derive optimal policies that lead to socially-desirable outcomes using a multi-agent reinforcement learning framework (MARL), and propose a semi-sequential multi-agent training and policy dissemination algorithm for our MARL problem. We introduce a quantitative representation of the AVs’ social preferences and design a distributed reward structure that induces altruism into their decision-making process. Altruistic AVs are able to form alliances, guide the traffic, and affect the behavior of the HVs to handle competitive driving scenarios. We compare egoistic AVs to our altruistic autonomous agents in a highway merging setting and demonstrate the emerging behaviors that lead to improvement in the number of successful merges and the overall traffic flow and safety.