CNS Core: Small: Towards Timing-Predictable Autonomy in DNN-driven Embedded Systems

机器学习技术，特别是深度神经网络（dnn），正在机器人和交通等重要领域实现更好的自主性。例如，在汽车系统中，dnn用于将车载摄像头的原始像素映射到转向控制决策。最近的端到端自动驾驶框架甚至使深度神经网络能够从有限的人类驾驶数据集中学习自动驾驶。英伟达（NVIDIA）和奥迪（Audi）最近宣布了推出基于dnn的自动驾驶汽车的计划。在任何安全关键型嵌入式系统（例如汽车）中安全可靠地采用dnn的主要挑战是需要确保时间可预测性（即，使时间约束能够在设计时进行分析验证），这是此类安全关键系统所需认证中最重要的原则之一。例如，汽车的功能正确性在很大程度上取决于时间正确性，因为控制操作依赖于在特定时间限制内处理某些环境感知和计算任务。不幸的是，由于dnn可能造成的资源瓶颈，在这样的系统中实现时间可预测性并不简单。本研究的目标是在深度神经网络驱动的自主嵌入式系统中实现时间可预测性。利用由丰田公司率先开发的成熟的生产均衡方法——平实化（hejunka），将建立一个新的系统模型。新的dnn感知、实时资源分配方法和验证时间约束的相关分析技术将被开发出来，可应用于dnn驱动的嵌入式系统。此外，将实现异构硬件架构下具有高效内存管理的开源生态系统。该项目的成果将为dnn驱动的解决方案在许多嵌入式领域中被安全、自信地采用铺平道路，在这些领域中，时间可预测性是一种自然需求。该项目还将培养一批精通深度神经网络驱动嵌入式系统跨学科性质的计算机工程师和科学家，并提高各级学生对实时和自主系统设计概念的认识。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning techniques, particularly deep neural networks (DNNs), are enabling dramatically better autonomy in important domains, such as robotics and transportation. For instance, in automotive systems, DNNs are used to map raw pixels from on-vehicle cameras to steering control decisions. Recent end-to-end self-driving frameworks even make it possible for DNNs to learn to self-steer from limited human driving datasets. NVIDIA and Audi recently announced their plans to deliver DNN-based automated vehicles. A major challenge of safely and reliably adopting DNNs in any safety-critical embedded systems (e.g., cars) is the need to ensure timing predictability (i.e., enabling timing constraints to be analytically validated at design time), which is one of the most important tenets in the certification required for such safety-critical systems. For example, the functional correctness of an automobile hinges crucially upon temporal correctness, as the control operations depend on the processing of certain environmental sensing and computation tasks within specific time constraints. Unfortunately, it is not straightforward to achieve timing predictability in such systems, due to the resource bottlenecks that DNNs can impose. The goal of this research is to achieve timing predictability in DNN-driven autonomous embedded systems. A novel system model leveraging Heijunka, a mature production leveling approach first developed by Toyota, will be established. New DNN-aware, real-time resource allocation methods and associated analysis techniques for validating timing constraints will be developed that can be applied in DNN-driven embedded systems. Moreover, an open-source ecosystem with efficient memory management under heterogeneous hardware architectures will be implemented. The outcome of this project will pave the way to enable DNN-driven solutions to be safely and confidently adopted in many embedded domains in which timing predictability is a natural requirement. This project will also result in a pipeline of computer engineers and scientists who are skilled in the interdisciplinary nature of DNN-driven embedded systems, as well as increase awareness of real-time and autonomous system design concepts among students at all academic levels.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.