Prophet: Realizing a Predictable Real-time Perception Pipeline for Autonomous Vehicles
Prophet: Realizing a Predictable Real-time Perception Pipeline for Autonomous Vehicles
复制标题
DOI：
10.1109/rtss55097.2022.00034
复制
发表时间：
2022-12
期刊：
影响因子：
--
通讯作者：
Liangkai Liu;Zheng-hong Dong;Yanzhi Wang;Weisong Shi
中科院分区：
文献类型：
--
作者：
Liangkai Liu;Zheng-hong Dong;Yanzhi Wang;Weisong Shi
我们见证了深度神经网络 (DNN) 在自动驾驶汽车 (AV) 中的广泛采用。作为一个安全关键的系统，基于期限的调度用于保证AV系统的可预测性。然而，对于 AV 系统中的大多数 DNN 模型来说，即使整个系统只运行一个模型，也存在不可忽略的时间变化。多个 DNN 在同一平台上运行的事实使得时间变化问题更加严重。然而，现有的工作还没有彻底研究时变问题的根本原因。在论文的第一部分，我们进行了全面的实证研究。我们发现单个 DNN 模型的推理时间变化主要是由 DNN 的多阶段/多分支结构引起的，该结构具有动态数量的提案或原始点。此外，我们发现不协调的竞争和合作是多租户 DNN 推理时间变化的根源。其次，基于这些见解，我们提出了 Prophet 系统，分两步解决 AV 感知系统中的时间变化问题。第一步是根据提案和原始点等中间结果预测时间变化。第二步是协调多租户 DNN，以确保执行进度彼此接近。从KITTI数据集上的评估结果来看，Faster R-CNN、LaneNet、PINet的单一模型时间预测准确率均达到91%以上。此外，感知融合时延限制在150ms，融合丢包率从5.4%降低到1%以下。
We have witnessed the broad adoption of Deep Neu-ral Networks (DNNs) in autonomous vehicles (AV). As a safety-critical system, deadline-based scheduling is used to guarantee the predictability of the AV system. However, non-negligible time variations exist for most DNN models in an AV system, even when the whole system is just running one model. The fact that multiple DNNs are running on the same platform makes the time variations issue even more severe. However, none of the existing works have thoroughly studied the root cause of the time variation issue. In the first part of the paper, we conducted a comprehensive empirical study. We found that the inference time variations for a single DNN model are mainly caused by the DNN's multi-stage/multi-branch structure, which has a dynamic number of proposals or raw points. In addition, we found that the uncoordinated contention and cooperation are the roots of the time variations for multi-tenant DNNs inference. Second, based on these insights, we proposed the Prophet system that addresses the time variations in the AV perception system in two steps. The first step is to predict the time variations based on the intermediate results like proposals and raw points. The second step is coordinating the multi-tenant DNNs to ensure the execution progress is close to each other. From the evaluation results on the KITTI dataset, the time prediction of a single model all achieve higher than 91% accuracy for Faster R-CNN, LaneNet, and PINet. Besides, the perception fusion delay is bounded to 150ms, and the fusion drop ratio is reduced from 5.4% to less than 1 percent.