CAREER: Distributionally Robust Learning, Control, and Benefits Analysis of Information Sharing for Connected and Autonomous Vehicles

职业：互联和自动驾驶车辆信息共享的分布式鲁棒学习、控制和效益分析

• 批准号: 2047354
• 负责人: Fei Miao
• 金额: $ 50.96万
• 依托单位: University of Connecticut
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-06-01 至 2026-05-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2047354&HistoricalAwards=false
• 关键词: CAREER; Distributionally; Robust; Learning; Control

The rapid evolution of ubiquitous sensing, communication, and computation technologies has contributed to the revolution of cyber-physical systems (CPS). Learning-based methodologies are integrated to the control of physical systems and demonstrating impressive performance in many CPS domains and connected and autonomous vehicles (CAVs) system is one such example with the development of vehicle-to-everything communication technologies. However, existing literature still lacks understanding of the tridirectional relationship among communication, learning, and control. The main challenges to be solved include (1) how to model dynamic system state and state uncertainties with shared information, (2) how to make robust learning and control decisions under model uncertainties, (3) how to integrate learning and control to guarantee the safety of networked CPS, and (4) how to quantify the benefits of communication.To address these challenges, this CAREER proposal aims to design integrated communication, learning, and control rules that are robust to hybrid system model uncertainties for safe operation and system efficiency of CAVs. The key intellectual merit is the design of integrated distributionally robust multi-agent reinforcement learning (DRMARL) and control framework with rigorous safety guarantees, considering hybrid system state uncertainties predicted with shared information, and the development of scientific foundation for analyzing and quantifying the benefits of communication. The fundamental theory and algorithm principles will be validated using simulators, small-scale testbeds, and full-scale CAVs field demonstrations, to form a new framework for future connectivity, learning, and control of CAVs and networked CPS. The technical contributions are as follows. (1). With shared information, we will design a cooperative prediction algorithm to improve hybrid system state and model uncertainty representations needed by learning and control. (2). Given enhanced prediction, we will design an integrated DRMARL and control framework with rigorous safety guarantee, and a computationally tractable algorithm to calculate the hybrid system decision-making policy. This integrates the strengths of both learning and control to improve system safety and efficiency. (3). We will define formally and quantify the value of communication given and propose a novel learn to communicate approach, to utilize learning and control to improve the communication actions. This project will also integrate an educational plan with the research goals by developing a learning platform of ``ssCAVs'' as an education tool and new interdisciplinary courses on “learning and control”, undertaking outreach to the general public and K-12 students and teachers, and directly involving high-school scholars, undergraduate and graduate students in research. This project is in response to the NSF CAREER 20-525 solicitation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

无处不在的传感、通信和计算技术的快速发展促进了信息物理系统（CPS）的革命。基于学习的方法论被集成到物理系统的控制中，并在许多CPS领域中表现出令人印象深刻的性能，互联和自动驾驶汽车（CAVs）系统就是随着车联网通信技术发展的一个例子。然而，现有的文献仍然缺乏对沟通，学习和控制之间的三方关系的理解。需要解决的主要挑战包括（1）如何利用共享信息对动态系统状态和状态不确定性建模，（2）如何在模型不确定性下做出鲁棒的学习和控制决策，（3）如何集成学习和控制以保证网络化CPS的安全性，以及（4）如何量化通信的好处。该CAREER建议旨在设计集成的通信、学习和控制规则，这些规则对于混合系统模型的不确定性具有鲁棒性，以实现CAV的安全操作和系统效率。主要的智能优点是设计了集成的分布式鲁棒多智能体强化学习（DRMARL）和具有严格安全保证的控制框架，考虑了用共享信息预测的混合系统状态不确定性，并为分析和量化通信的好处奠定了科学基础。基本理论和算法原理将使用模拟器、小规模测试台和全尺寸通用航空飞行器现场演示进行验证，以形成未来通用航空飞行器和网络CPS的连接、学习和控制的新框架。 技术贡献如下。（一）.通过共享信息，我们将设计一个协同预测算法，以改善混合系统的学习和控制所需的状态和模型的不确定性表示。（二）、考虑到增强的预测，我们将设计一个集成的DRMARL和控制框架，具有严格的安全保证，和一个计算易处理的算法来计算混合系统的决策政策。这集成了学习和控制的优势，以提高系统的安全性和效率。（三）、我们将正式定义和量化的沟通的价值，并提出了一种新的学习沟通的方法，利用学习和控制，以改善沟通的行动。该项目还将把教育计划与研究目标结合起来，开发一个"ssCAV“学习平台，作为一种教育工具和关于“学习与控制”的新的跨学科课程，向公众和K-12学生和教师开展外联活动，并直接让高中学者、本科生和研究生参与研究。该项目是对NSF CAREER 20-525征集的回应。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Stable and Efficient Shapley Value-Based Reward Reallocation for Multi-Agent Reinforcement Learning of Autonomous Vehicles

• DOI: 10.48550/arxiv.2203.06333
• 发表时间: 2022-03
• 期刊: 2022 International Conference on Robotics and Automation (ICRA)
• 作者: Songyang Han;He Wang;Sanbao Su;Yuanyuan Shi;Fei Miao

Spatial-Temporal-Aware Safe Multi-Agent Reinforcement Learning of Connected Autonomous Vehicles in Challenging Scenarios

• DOI: 10.1109/icra48891.2023.10161216
• 发表时间: 2022-10
• 期刊: 2023 IEEE International Conference on Robotics and Automation (ICRA)
• 作者: Zhili Zhang;Songyang Han;Jiangwei Wang;Fei Miao

Robust Multi-Agent Reinforcement Learning with Adversarial State Uncertainties

具有对抗性状态不确定性的鲁棒多智能体强化学习

• 发表时间: 2023
• 期刊: Transactions on Machine Learning Research
• 作者: He, Sihong;Han, Songyang;Su, Sanbao;Han, Shuo;Zou, Shaofeng;Miao, Fei.
• 通讯作者: Miao, Fei.

A Multi-Agent Reinforcement Learning Approach for Safe and Efficient Behavior Planning of Connected Autonomous Vehicles

• DOI: 10.1109/tits.2023.3336670
• 发表时间: 2020-03
• 期刊: IEEE Transactions on Intelligent Transportation Systems
• 影响因子: 8.5
• 作者: Songyang Han;Shangli Zhou;Jiangwei Wang;Lynn Pepin;Caiwen Ding;Jie Fu;Fei Miao

Surrogate Lagrangian Relaxation: A Path to Retrain-Free Deep Neural Network Pruning

• DOI: 10.1145/3624476
• 发表时间: 2023-04
• 期刊: ACM Transactions on Design Automation of Electronic Systems
• 影响因子: 1.4
• 作者: Shangli Zhou;Mikhail A. Bragin;Lynn Pepin;Deniz Gurevin;Fei Miao;Caiwen Ding