RTML: Large: Real-Time Autonomic Decision Making on Sparsity-Aware Accelerated Hardware via Online Machine Learning and Approximation

实时智能和自主决策包括两个主要阶段，感知（传感器数据，然后转化为可操作的知识）和规划（利用这些知识做出决策）。这两个阶段发生在智能物理系统（IPS）的内部和外部操作中。对于内部操作，感知是指从板载传感器读取数据，规划是指IPS上运行的固件的智能执行。在外部操作的情况下，传感是指外部安装的传感器的传感数据，规划是指执行构成应用程序的软件。在传感阶段，IPS应该能够应对不同形式的不确定性，特别是数据和模型的不确定性。本研究项目的目标是通过实时机器学习（RTML）和一组ips（如执行数据收集和/或多目标跟踪/分类的无人机）的近似，实现在稀疏感知加速硬件上的在线自主决策目标，并在难以建模的高度动态环境中运行。值得注意的是，本项目采用的技术具有很好的通用性，可以应用于各种IPS域，包括自然灾害、人为灾害和恐怖袭击。基于无人机的分布式多目标跟踪/分类将使公民、政府机构、救援机构和行业等利益攸关方能够了解损害程度，并制定更有效的减灾政策。该研究还将培训学生，包括该领域的少数民族和代表性不足的学生。这个项目有三个具体的任务。在Task 1中，将提出一种基于在线深度强化学习的实时决策方法，该方法具有固有的分布式训练能力；流媒体视频中的时间和空间相关性将被用于实时多目标跟踪/检测。在任务2中，将设计新的硬件架构来支持稀疏卷积神经网络（CNN）。考虑到稀疏性对深度神经网络（DNN）模型的低计算复杂度和空间复杂度的双重好处，稀疏性感知的CNN加速器可以在延迟、吞吐量和能源效率方面实现显着的硬件性能改进。最后，在任务3中，将研究硬件感知的软件工程解决方案以加速执行。为了优化执行性能，我们将研究利用编译器优化和底层硬件特性相结合的想法；然后，将介绍数据驱动建模技术，用等效的数据驱动模型（即微神经网络）取代ML软件包中耗时的部分。一旦这三个研究任务根据其既定目标通过原则实验分别得到验证，它们将被整合到一个统一的框架中，该框架将通过在互补的现场场景中进行多次试验进行彻底研究。该项目还将与DARPA的一个协同项目合作进行相关硬件开发。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Real-time smart and autonomic decision making involves two major stages, sensing (of sensor data and then transformation into actionable knowledge) and planning (taking decisions using this knowledge). These two stages happen in both internal and external operations of an Intelligent Physical System (IPS). In case of internal operations, sensing refers to reading data from on-board sensors and planning refers to smart execution of the firmware running on the IPS. In case of external operations, sensing refers to sensing data from externally-mounted sensors and planning refers to executing the software that constitutes an application. In the sensing stage, an IPS should be able to cope with different forms of uncertainty, especially data and model uncertainties. The goal of this research project is to achieve the objectives of online autonomic decision making on sparsity-aware accelerated hardware via Real-Time Machine Learning (RTML) and approximation for a group of IPSs such as drones performing data collection and/or multi-object tracking/classification and operating in a highly dynamic environment that is difficult to model. Remarkably, the techniques adopted in this project generalize well as they can be applied to a variety of IPS domains including natural calamities, man-made disasters, and terrorist attacks. The drone-based distributed multi-object tracking/classification will enable stakeholders such as citizens, government bodies, rescue agencies, and industries to comprehend the extent of damage, and to develop more effective mitigation policies. The research will also train students including minority and underrepresented students in the field.There are three specific tasks in this project. In Task 1, a real-time decision-making approach will be proposed via online deep reinforcement learning with inherent distributed training capability; temporal and spatial correlation in streaming video will then be exploited towards real-time multi-object tracking/detection. In Task 2, novel hardware architectures will be designed to support sparse Convolution Neural Networks (CNN). Considering the dual benefits of sparsity on both lower computational and space complexity for Deep Neural Network (DNN) models, a sparsity-aware CNN accelerator can achieve significant hardware performance improvements in term of latency, throughput, and energy efficiency over non-sparsity-aware techniques. Finally, in Task 3, hardware-aware software engineering solutions will be studied for accelerated execution. The idea of leveraging compiler optimization and the underlying hardware features in combination will be investigated in order to optimize execution performance; then, data-driven modeling techniques will be presented to replace the time-consuming segments of the ML software packages with their equivalent data-driven models, namely micro-neural networks. Once these three research tasks are validated individually via principled experimentation in terms of their stated goals, they will be integrated into a unified framework, which will be thoroughly studied via multiple trials on complementary field scenarios. The project will also collaborate with a synergistic DARPA program for related hardware development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.