CPS: Small: NSF-DST: Safety-Aware Behaviour-Driven Reinforcement Learning Based Autonomous Driving Solution for Urban Areas

NSF网络物理系统（CPS）项目将支持旨在增强自动驾驶车辆（AV）群在各种交通环境中的运行的研究，包括结构化十字路口（如美国的十字路口）和非结构化十字路口（如印度的十字路口）。该项目将研究“公共悲剧”（ToC），即自动驾驶汽车虽然本身很聪明，但当它们都使用相同的逻辑时，可能会造成严重的交通振荡和混乱。该研究还将研究复杂交通情况下类似人类的“紧急合作行为（ECB）”，旨在将这种合作驾驶行为纳入自动驾驶控制系统，并希望在多智能体系统中实现高效运行。该项目可能会在自动驾驶汽车的各种交通环境中实现协同控制和管理的尖端技术，这一点至关重要，因为它们在我们的街道上变得越来越普遍。此外，从理解自动驾驶汽车中的“公地悲剧”和“紧急合作行为”中获得的见解可以应用于更广泛的多智能体系统，可能影响金融市场算法或机器人等各种领域。美国和印度在这方面的合作不仅为改善全球道路性能铺平了道路，而且树立了新的国际基准。本研究的主要技术目标是在不同的交通环境中将自动驾驶汽车的控制和交互模式从ToC过渡到ECB。自动驾驶汽车控制显示ToC的假设将首先进行测试，即使单个自动驾驶汽车表现较好（例如，稳定性较低），但一组自动驾驶汽车的性能可能较差（例如，导致交通中断）。与该假设相对应的是，人类驾驶车辆（HV）的行为表明，尽管单个HV可能表现不佳，但即使在不利条件下，一系列HV也可能保持合理的表现。这一假设将通过研究AVs和hv的马尔可夫性和均匀性来验证。马尔可夫性质表明车辆的行为是由当前的交通状态决定的，与之前的经验无关，而均匀性意味着无论交通环境如何，车辆都遵循相似的行为规则。假设自动驾驶汽车具有这些特性，而hv没有，将使用现场数据进行评估。这些特性将如何导致自动驾驶汽车的TOC和hv的ECB，并使用分析建模和仿真进行研究。这些基本特性的发现将用于构建一种基于元学习的自动驾驶控制方法，适用于美国和印度的十字路口，旨在扭转现有自动驾驶车辆的ToC现象，并使用ECB系统培养新的自动驾驶控制。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This NSF Cyber-Physical Systems (CPS) project will support research that intends to enhance the operation of automated vehicles (AV) swarms in various traffic environments, including structured intersections (e.g., intersections in the US) and unstructured intersections (e.g., intersections in India). The project will examine the 'Tragedy of the Commons (ToC)'— a situation where AVs, while smart on their own, might cause significant traffic oscillation and disorder when they all use the same logic. The research will also examine human-like 'Emergent Cooperative Behavior (ECB)' in complex traffic situations, aiming to incorporate such cooperative driving behaviors into AV control systems and hoping to achieve efficient operation in multi-agent systems. This project could lead to cutting-edge advancements in collaborative control and management in various traffic environments with AVs, which is crucial as they become more common on our streets. Moreover, insights gained from understanding the 'Tragedy of the Commons' and 'Emergent Cooperative Behavior' in AVs could apply to a broader range of multi-agent systems, potentially influencing fields as varied as financial market algorithms or robotics. The collaboration between the U.S. and India in this endeavor not only paves the way for improved global road performance but also sets new international benchmarks. The primary technical objective of this research is to transition the control and interaction patterns of AVs from the ToC to ECB in diverse traffic environments. The hypothesis that AV control exhibits the ToC such that a stream of AVs may have inferior performance (e.g., causing traffic breakdown) even when an individual AV performs superior (e.g., with less instability) will first be tested. The counterpart of the hypothesis is that human-driven vehicle (HV) behavior manifests the ECB such that while an individual HV may not perform as well, a stream of HVs may maintain reasonable performance even in adverse conditions. This hypothesis will be tested by investigating the Markovian and uniformity properties of AVs and HVs. The Markovian property indicates that the vehicle action is determined by the current traffic state independent of its previous experience, while uniformity implies that vehicles follow a similar behavior rule regardless of the traffic environment. The conjecture that AVs have these properties while HVs do not will be evaluated using field data. How these properties would lead to TOC for AVs yet ECB for HVs with the be examined using analytical modeling and simulation. The finding of these fundamental properties will be used to construct a meta-learning-based AV control approach applicable to both US and Indian intersections, with the intention of reversing the ToC phenomenon for existing AVs and cultivating new AV control using an ECB system.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.