Collaborative Research: Enabling Machine Learning based Cooperative Perception with mmWave Communication for Autonomous Vehicle Safety

从机器学习的角度来看，通过了解自动驾驶汽车之间交换什么以及如何交换数据，可以实现对自动驾驶汽车的精确协同感知，使大量传感器信息能够在汽车之间共享。这种进步对于扩展自动驾驶车辆的视线和视野非常有用，否则会受到盲点和遮挡的影响。当存在阻碍对环境的完整感知的遮挡时，自动驾驶车辆上的扩展视野将是有益的。这种态势感知的增加在狭窄的范围内促进安全驾驶，并在扩展的范围内提高交通流量效率。拟议中的研究工作不仅将改变人们对自动驾驶汽车感知系统的看法，还可能为设计以前无法想象的新系统提供机会。该项目提供了各种各样的研究活动，从数据收集，算法设计，系统开发，并在现场评估，这将是有吸引力的学生具有不同的背景和目标。本科生和研究生将直接参与研究活动，作为不同层次的助理。该项目的主要研究目标是了解实现自动驾驶车辆之间的协作感知所面临的传感和通信挑战，并将由此获得的见解用于指导设计合适的数据交换格式、数据融合算法、和高效的毫米波车载通信。该项目的成果将包括一个基于机器学习的协作感知框架，该框架将有助于以分布式方式有效地组合从自动驾驶汽车的机器学习模型中获得的特征图。由此产生的特征地图压缩和特征地图选择方法将显著减少车辆之间交换的数据量，从而实现对联网和自动驾驶车辆的敏捷和精确的协作感知。所提出的可扩展特征地图传输机制联合考虑了毫米波链路的应用需求、链路和物理层特性，使传感器数据能够在自动驾驶车辆之间大规模共享。实施的系统和评估平台将作为所提出的解决方案的令人信服的概念验证，从而打开了通过毫米波通信在未来车辆网络中广泛采用协作感知应用的大门。该项目收集的数据集将被公开，作为合作目标检测、车辆边缘计算和机器学习等创新研究的催化剂。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

By understanding what and how data are exchanged among autonomous vehicles, from a machine learning perspective, it is possible to realize precise cooperative perception on autonomous vehicles, enabling massive amounts of sensor information to be shared amongst vehicles. Such an advance can be extremely useful to extend the line of sight and field of view of autonomous vehicles, which otherwise suffers from blind spots and occlusions. The extended field of view on autonomous vehicles will be beneficial at times when there are occlusions preventing a complete perception of the environment. This increase in situational awareness promotes safe driving over a narrow scope and improves traffic flow efficiency over an extended scope. The proposed research work will not only change the way people think about the perception system on autonomous vehicles but could also open up opportunities to design novel systems that were previously inconceivable. This project offers a wide variety of research activities from data collection, algorithm design, system development, and in-the-field evaluation, which will be attractive to students with various backgrounds and goals. Undergraduate and graduate students will be involved directly in the research activities as assistants at different levels. The expected research outcomes from this project will also enhance the current curricula related to machine learning, Internet of things, and wireless communications.The main research objective of this project is to understand the sensing and communication challenges to achieving cooperative perception among autonomous vehicles, and to use the insights thus gained to guide the design of suitable data exchange format, data fusion algorithms, and efficient millimeter wave vehicular communications. Results from this project will include a machine learning based cooperative perception framework, which will shed light on effectively combining feature maps, derived from machine learning models on autonomous vehicles, in a distributed manner. The resulted feature map compression and feature map selection approaches will significantly reduce the amount of data exchanged among vehicles, enabling agile and precise cooperative perception on connected and autonomous vehicles. The proposed scalable feature map transmission mechanism jointly considers the application requirements, link and physical layer characteristics of millimeter wave links, enabling sensor data sharing on a massive scale among autonomous vehicles. The implemented system and evaluation platform will serve as a convincing proof-of-concept for the proposed solution, thus opening the door to widespread adoption of cooperative perception applications via millimeter wave communications in future vehicle networks. The collected dataset from this project will be made publicly available, serving as a catalyst for enabling innovative research on cooperative object detection, vehicular edge computing, and machine learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.