CPS: Medium: GOALI: Real-Time Computer Vision in Autonomous Vehicles: Real Fast Isn't Good Enough

在汽车上部署自动驾驶功能的努力正在以惊人的速度进行。半自动驾驶功能正变得越来越普遍，而大众市场规模的全自动驾驶汽车即将问世。摄像头是一种成本低廉的传感器，因此计算机视觉技术在实现自动驾驶功能方面显得尤为重要。在车辆中，这些技术必须“实时”发挥作用。不幸的是，这一要求是一个重大脱节的核心：当计算机视觉研究人员提到“实时”时，他们通常指的是“非常快”；相比之下，可认证的汽车系统必须是“实时的”，即能够在规定的期限内对输入信息（如检测到的行人）做出可预测的反应，从而可证明地排除不利结果（如撞到行人）。这个项目的目标是消除这种脱节。它将通过几个方面的研究来实现这一目标。首先，将通过扩展OpenVX创建实时计算机视觉编程框架，OpenVX是最近批准的用于开发嵌入式系统计算机视觉应用程序的标准。其次，利用这种编程框架的特点的新的计算机视觉算法将被创建，并且将开发方法来转换现有算法，使其在可预测性意义上“实时”。第三，对“real-fast”vs。“实时”计算机视觉将通过驾驶模拟器、次规模自动驾驶汽车和通用汽车的先进测试基础设施进行。虽然业界正在大力推进自动驾驶领域，但如果没有实时安全认证的方法，自动驾驶汽车永远不会成为普遍的交通方式。该项目将重点关注认证的一个关键方面：验证计算机视觉应用程序的实时正确性。所产生的结果将通过开源软件向全世界提供。该软件将包括将要生成的新编程框架，以及用于验证使用该框架开发的应用程序的实时正确性的工具。在这个项目中，将特别强调向女孩和妇女伸出援手，因为将有三名女研究生参与该项目。这种拓展将包括：涉及北卡罗来纳大学计算机科学研究生女性（GWiCS）小组的活动，该小组每年举办一次针对本科生女性和其他代表性不足的少数民族的研究研讨会；Tar Heel Hack，一个针对当地初高中女生的黑客马拉松；北卡罗来纳大学编程女孩俱乐部，为当地6-12年级的女孩提供学习计算机科学的社区；以及北卡罗来纳大学计算机科学系的年度开放日和科学博览会。这些活动将包括黑客马拉松项目，以及驾驶模拟器和小型自动驾驶汽车的演示。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The push towards deploying autonomous-driving capabilities in vehicles is happening at breakneck speed. Semi-autonomous features are becoming increasingly common, and fully autonomous vehicles at mass-market scales are on the horizon. Cameras are cost-effective sensors, so computer-vision techniques have loomed large in implementing autonomous features. In a vehicle, these techniques must function "in real time." Unfortunately, this requirement lies at the heart of a significant disconnect: when computer-vision researchers refer to "real time," they usually mean "real fast"; in contrast, certifiable automotive systems must be "real time" in the sense of being able to predictably react to input information (such as a detected pedestrian) within specified deadlines so that adverse outcomes (such as striking a pedestrian) are provably precluded. The goal of this project is to eliminate this disconnect. It will do so through research on several fronts. First, a real-time computer-vision programming framework will be created by extending OpenVX, which is a recently ratified standard intended for developing computer-vision applications for embedded systems. Second, new computer-vision algorithms that exploit the features of this programming framework will be created, and methods will be developed to transform existing algorithms to make them "real time" in a predictability sense. Third, an experimental evaluation of "real-fast" vs. "real-time" computer vision will be conducted using driving simulators, sub-scale autonomous vehicles, and advanced testing infrastructure at General Motors.While industry is pushing hard in the area of autonomous driving, autonomous vehicles will never become a common mode of transportation unless methods for certifying real-time safety are produced. This project will focus on a key aspect of certification: validating the real-time correctness of computer-vision applications. The results that are produced will be made available to the world at large through open-source software. This software will include the new programming framework to be produced as well as tools for validating the real-time correctness of applications developed using this framework. In this project, a special emphasis will be placed on outreach to girls and women, as three female graduate students will be involved in the project. Such outreach will include: events involving the Graduate Women in Computer Science (GWiCS) group at the University of North Carolina (UNC), which hosts an annual research symposium targeted toward undergraduate women and other under-represented minorities; Tar Heel Hack, a hackathon for local middle and high school girls; the UNC Girls Who Code Club, which provides local girls in grades 6-12 with a community for learning about computer science; and the UNC Computer Science Department's annual Open House and Science Expo. These events will include hackathon projects as well as demos of a driving simulator and a sub-scale autonomous car.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.