CPS: Small: NSF-DST: Safety-Aware Behaviour-Driven Reinforcement Learning Based Autonomous Driving Solution for Urban Areas

这个NSF网络物理系统（CPS）项目将支持旨在增强自动车辆（AV）群在各种交通环境中的运行的研究，包括结构化交叉路口（例如，美国的交叉口）和非结构化交叉口（例如，印度的十字路口）。该项目将研究“公共资源悲剧”（ToC）-一种情况，即自动驾驶汽车虽然本身很聪明，但当它们都使用相同的逻辑时，可能会导致显著的交通振荡和混乱。该研究还将研究在复杂交通情况下类似人类的“紧急合作行为（ECB）”，旨在将这种合作驾驶行为纳入AV控制系统，并希望在多智能体系统中实现高效运行。该项目可能会导致在各种交通环境中使用自动驾驶汽车的协同控制和管理方面取得前沿进展，这一点至关重要，因为它们在我们的街道上变得越来越普遍。此外，从理解AV中的“公共资源悲剧”和“紧急合作行为”中获得的见解可以应用于更广泛的多智能体系统，可能影响金融市场算法或机器人等领域。美国和印度在这奋进的合作不仅为改善全球道路性能铺平了道路，而且还设定了新的国际基准。本研究的主要技术目标是在不同的交通环境中将AV的控制和交互模式从ToC过渡到ECB。AV控制展现ToC使得AV流可具有较差性能（例如，导致交通中断）即使当单个AV执行上级（例如，具有较少的不稳定性）将首先被测试。假设的对应物是，人类驾驶的车辆（HV）行为表明ECB，使得虽然单个HV可能表现不佳，但即使在不利条件下，HV流也可以保持合理的性能。这一假设将通过调查的马尔可夫和均匀性属性的AV和HV进行测试。马尔可夫性质表明车辆行为是由当前交通状态决定的，与其先前的经验无关，而一致性意味着车辆遵循类似的行为规则，而不管交通环境如何。将使用现场数据评估AV具有这些特性而HV不具有这些特性的推测。这些属性将如何导致AV的TOC，而ECB的HV与使用分析建模和模拟进行检查。这些基本特性的发现将用于构建一个基于元学习的AV控制方法，适用于美国和印度的交叉路口，旨在扭转现有AV的ToC现象，并使用ECB系统培养新的AV控制。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

This NSF Cyber-Physical Systems (CPS) project will support research that intends to enhance the operation of automated vehicles (AV) swarms in various traffic environments, including structured intersections (e.g., intersections in the US) and unstructured intersections (e.g., intersections in India). The project will examine the 'Tragedy of the Commons (ToC)'— a situation where AVs, while smart on their own, might cause significant traffic oscillation and disorder when they all use the same logic. The research will also examine human-like 'Emergent Cooperative Behavior (ECB)' in complex traffic situations, aiming to incorporate such cooperative driving behaviors into AV control systems and hoping to achieve efficient operation in multi-agent systems. This project could lead to cutting-edge advancements in collaborative control and management in various traffic environments with AVs, which is crucial as they become more common on our streets. Moreover, insights gained from understanding the 'Tragedy of the Commons' and 'Emergent Cooperative Behavior' in AVs could apply to a broader range of multi-agent systems, potentially influencing fields as varied as financial market algorithms or robotics. The collaboration between the U.S. and India in this endeavor not only paves the way for improved global road performance but also sets new international benchmarks. The primary technical objective of this research is to transition the control and interaction patterns of AVs from the ToC to ECB in diverse traffic environments. The hypothesis that AV control exhibits the ToC such that a stream of AVs may have inferior performance (e.g., causing traffic breakdown) even when an individual AV performs superior (e.g., with less instability) will first be tested. The counterpart of the hypothesis is that human-driven vehicle (HV) behavior manifests the ECB such that while an individual HV may not perform as well, a stream of HVs may maintain reasonable performance even in adverse conditions. This hypothesis will be tested by investigating the Markovian and uniformity properties of AVs and HVs. The Markovian property indicates that the vehicle action is determined by the current traffic state independent of its previous experience, while uniformity implies that vehicles follow a similar behavior rule regardless of the traffic environment. The conjecture that AVs have these properties while HVs do not will be evaluated using field data. How these properties would lead to TOC for AVs yet ECB for HVs with the be examined using analytical modeling and simulation. The finding of these fundamental properties will be used to construct a meta-learning-based AV control approach applicable to both US and Indian intersections, with the intention of reversing the ToC phenomenon for existing AVs and cultivating new AV control using an ECB system.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.