CPS: Small: NSF-DST: Safety-Aware Behaviour-Driven Reinforcement Learning Based Autonomous Driving Solution for Urban Areas

该NSF网络物理系统(CPS)项目将支持旨在增强自动车辆群在各种交通环境中的运行的研究，包括结构化十字路口(例如，美国的十字路口)和非结构化十字路口(例如，印度的十字路口)。该项目将研究“公地悲剧(TOC)”--在这种情况下，虽然自动取款机本身是智能的，但当它们都使用相同的逻辑时，可能会导致严重的交通振荡和混乱。这项研究还将研究复杂交通情况下的类似人类的紧急合作行为(ECB)，旨在将这种合作驾驶行为纳入视听控制系统，并希望在多智能体系统中实现高效运行。该项目可能导致在各种交通环境中使用自动驾驶系统进行协作控制和管理方面的尖端进步，这一点至关重要，因为它们在我们的街道上变得越来越常见。此外，从理解AVs中的“公地悲剧”和“紧急合作行为”中获得的洞察力可能适用于更广泛的多智能体系统，可能会影响金融市场算法或机器人等各种领域。美国和印度在这方面的合作不仅为改善全球道路性能铺平了道路，还设定了新的国际基准。这项研究的主要技术目标是在不同的交通环境下将自动驾驶系统的控制和交互模式从TOC过渡到ECB。将首先测试这样的假设，即AV控制表现出TOC，使得即使当单个AV表现优异(例如，具有较少不稳定性)时，AV流也可能具有较差的性能(例如，导致业务中断)。与这一假设相对应的是，人驾驶车辆(HV)的行为体现了欧洲央行的行为，即虽然单个HV的表现可能不佳，但一系列HV即使在不利条件下也可能保持合理的表现。这一假设将通过研究AVS和HV的马尔可夫性质和一致性属性来检验。马尔科夫属性表明车辆的行为由当前的交通状态决定，而与其先前的经验无关，而一致性意味着无论交通环境如何，车辆都遵循类似的行为规则。AVs具有这些特性而Hv没有的猜想将使用现场数据进行评估。这些特性将如何导致用于AVS的TOC，以及用于HV的ECB，将使用分析建模和模拟来研究。这些基本性质的发现将被用来构建一种适用于美国和印度交叉口的基于元学习的AV控制方法，目的是扭转现有AVs的TOC现象，并利用欧洲央行系统培养新的AV控制。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This NSF Cyber-Physical Systems (CPS) project will support research that intends to enhance the operation of automated vehicles (AV) swarms in various traffic environments, including structured intersections (e.g., intersections in the US) and unstructured intersections (e.g., intersections in India). The project will examine the 'Tragedy of the Commons (ToC)'— a situation where AVs, while smart on their own, might cause significant traffic oscillation and disorder when they all use the same logic. The research will also examine human-like 'Emergent Cooperative Behavior (ECB)' in complex traffic situations, aiming to incorporate such cooperative driving behaviors into AV control systems and hoping to achieve efficient operation in multi-agent systems. This project could lead to cutting-edge advancements in collaborative control and management in various traffic environments with AVs, which is crucial as they become more common on our streets. Moreover, insights gained from understanding the 'Tragedy of the Commons' and 'Emergent Cooperative Behavior' in AVs could apply to a broader range of multi-agent systems, potentially influencing fields as varied as financial market algorithms or robotics. The collaboration between the U.S. and India in this endeavor not only paves the way for improved global road performance but also sets new international benchmarks. The primary technical objective of this research is to transition the control and interaction patterns of AVs from the ToC to ECB in diverse traffic environments. The hypothesis that AV control exhibits the ToC such that a stream of AVs may have inferior performance (e.g., causing traffic breakdown) even when an individual AV performs superior (e.g., with less instability) will first be tested. The counterpart of the hypothesis is that human-driven vehicle (HV) behavior manifests the ECB such that while an individual HV may not perform as well, a stream of HVs may maintain reasonable performance even in adverse conditions. This hypothesis will be tested by investigating the Markovian and uniformity properties of AVs and HVs. The Markovian property indicates that the vehicle action is determined by the current traffic state independent of its previous experience, while uniformity implies that vehicles follow a similar behavior rule regardless of the traffic environment. The conjecture that AVs have these properties while HVs do not will be evaluated using field data. How these properties would lead to TOC for AVs yet ECB for HVs with the be examined using analytical modeling and simulation. The finding of these fundamental properties will be used to construct a meta-learning-based AV control approach applicable to both US and Indian intersections, with the intention of reversing the ToC phenomenon for existing AVs and cultivating new AV control using an ECB system.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.