Collaborative Research: Enabling Machine Learning based Cooperative Perception with mmWave Communication for Autonomous Vehicle Safety

通过从机器学习的角度了解自动驾驶车辆之间交换什么以及如何交换数据，就有可能实现对自动驾驶车辆的精确协作感知，从而使大量的传感器信息能够在车辆之间共享。这样的进步对于扩大自动驾驶车辆的视线和视野非常有用，否则自动驾驶车辆会出现盲区和遮挡。在自动驾驶汽车上的扩展视野将是有益的，有时会有遮挡阻碍对环境的完全感知。这种情景感知的增加促进了在狭窄范围内的安全驾驶，并在更大范围内提高了交通流量效率。这项拟议的研究工作不仅将改变人们对自动驾驶汽车感知系统的看法，还可能为设计以前不可想象的新系统打开机会。该项目提供了广泛的研究活动，从数据收集，算法设计，系统开发，以及实地评估，将吸引不同背景和目标的学生。本科生和研究生将作为不同级别的助理直接参与研究活动。该项目的预期研究成果还将加强与机器学习、物联网和无线通信相关的现有课程。本项目的主要研究目标是了解自主车辆之间实现协作感知所面临的传感和通信挑战，并利用所获得的见解来指导设计合适的数据交换格式、数据融合算法和高效的毫米波车载通信。该项目的成果将包括一个基于机器学习的协作感知框架，该框架将有助于以分布式方式有效地组合从自动驾驶车辆的机器学习模型派生的特征地图。由此产生的特征地图压缩和特征地图选择方法将显著减少车辆之间交换的数据量，使互联和自动驾驶车辆能够灵活而精确地协作感知。提出的可扩展特征地图传输机制综合考虑了毫米波链路的应用需求、链路和物理层特性，使得传感器数据能够在自主车辆之间实现大规模共享。实施的系统和评估平台将作为拟议解决方案的令人信服的概念验证，从而为在未来的车辆网络中通过毫米波通信广泛采用合作感知应用打开大门。从该项目收集的数据集将被公开，作为促进合作对象检测、车辆边缘计算和机器学习的创新研究的催化剂。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

By understanding what and how data are exchanged among autonomous vehicles, from a machine learning perspective, it is possible to realize precise cooperative perception on autonomous vehicles, enabling massive amounts of sensor information to be shared amongst vehicles. Such an advance can be extremely useful to extend the line of sight and field of view of autonomous vehicles, which otherwise suffers from blind spots and occlusions. The extended field of view on autonomous vehicles will be beneficial at times when there are occlusions preventing a complete perception of the environment. This increase in situational awareness promotes safe driving over a narrow scope and improves traffic flow efficiency over an extended scope. The proposed research work will not only change the way people think about the perception system on autonomous vehicles but could also open up opportunities to design novel systems that were previously inconceivable. This project offers a wide variety of research activities from data collection, algorithm design, system development, and in-the-field evaluation, which will be attractive to students with various backgrounds and goals. Undergraduate and graduate students will be involved directly in the research activities as assistants at different levels. The expected research outcomes from this project will also enhance the current curricula related to machine learning, Internet of things, and wireless communications.The main research objective of this project is to understand the sensing and communication challenges to achieving cooperative perception among autonomous vehicles, and to use the insights thus gained to guide the design of suitable data exchange format, data fusion algorithms, and efficient millimeter wave vehicular communications. Results from this project will include a machine learning based cooperative perception framework, which will shed light on effectively combining feature maps, derived from machine learning models on autonomous vehicles, in a distributed manner. The resulted feature map compression and feature map selection approaches will significantly reduce the amount of data exchanged among vehicles, enabling agile and precise cooperative perception on connected and autonomous vehicles. The proposed scalable feature map transmission mechanism jointly considers the application requirements, link and physical layer characteristics of millimeter wave links, enabling sensor data sharing on a massive scale among autonomous vehicles. The implemented system and evaluation platform will serve as a convincing proof-of-concept for the proposed solution, thus opening the door to widespread adoption of cooperative perception applications via millimeter wave communications in future vehicle networks. The collected dataset from this project will be made publicly available, serving as a catalyst for enabling innovative research on cooperative object detection, vehicular edge computing, and machine learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.