RTML: Large: Real-Time Autonomic Decision Making on Sparsity-Aware Accelerated Hardware via Online Machine Learning and Approximation

实时智能和自主决策包括两个主要阶段，感知(传感器数据，然后转换为可操作的知识)和规划(使用这些知识做出决策)。这两个阶段发生在智能物理系统(IPS)的内部和外部操作中。在内部操作的情况下，传感是指从车载传感器读取数据，规划是指智能执行在IPS上运行的固件。在外部操作的情况下，感测是指从外部安装的传感器感测数据，规划是指执行构成应用程序的软件。在感知阶段，IPS应该能够处理不同形式的不确定性，特别是数据和模型的不确定性。本研究项目的目标是在稀疏性感知的加速硬件上，通过实时机器学习(RTML)和近似实现一组IPS的在线自主决策，例如执行数据收集和/或多目标跟踪/分类的无人机，并在难以建模的高度动态环境中操作。值得注意的是，该项目中采用的技术具有很好的通用性，因为它们可以应用于各种IPS领域，包括自然灾害、人为灾难和恐怖袭击。基于无人机的分布式多目标跟踪/分类将使公民、政府机构、救援机构和行业等利益攸关方能够了解损害的程度，并制定更有效的缓解政策。这项研究还将培训该领域的学生，包括少数民族和代表性不足的学生。该项目有三个具体任务。在任务1中，将通过具有固有的分布式训练能力的在线深度强化学习来提出一种实时决策方法；然后利用流视频中的时间和空间相关性来进行实时多目标跟踪/检测。在任务2中，将设计新的硬件体系结构来支持稀疏卷积神经网络(CNN)。考虑到稀疏性对深度神经网络(DNN)模型计算和空间复杂度的双重好处，稀疏性感知的CNN加速器可以在延迟、吞吐量和能源效率方面比非稀疏性感知技术获得显著的硬件性能改进。最后，在任务3中，将研究硬件感知的软件工程解决方案，以加速执行。为了优化执行性能，将研究利用编译器优化和底层硬件特性相结合的思想；然后，将提出数据驱动建模技术，以将ML软件包中耗时的部分替换为其等效的数据驱动模型，即微神经网络。一旦这三项研究任务通过原则性实验就其所述目标进行了单独验证，它们将被整合到一个统一的框架中，并将通过在互补的实地情景下进行的多项试验进行彻底研究。该项目还将与DARPA相关硬件开发的协同计划合作。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Real-time smart and autonomic decision making involves two major stages, sensing (of sensor data and then transformation into actionable knowledge) and planning (taking decisions using this knowledge). These two stages happen in both internal and external operations of an Intelligent Physical System (IPS). In case of internal operations, sensing refers to reading data from on-board sensors and planning refers to smart execution of the firmware running on the IPS. In case of external operations, sensing refers to sensing data from externally-mounted sensors and planning refers to executing the software that constitutes an application. In the sensing stage, an IPS should be able to cope with different forms of uncertainty, especially data and model uncertainties. The goal of this research project is to achieve the objectives of online autonomic decision making on sparsity-aware accelerated hardware via Real-Time Machine Learning (RTML) and approximation for a group of IPSs such as drones performing data collection and/or multi-object tracking/classification and operating in a highly dynamic environment that is difficult to model. Remarkably, the techniques adopted in this project generalize well as they can be applied to a variety of IPS domains including natural calamities, man-made disasters, and terrorist attacks. The drone-based distributed multi-object tracking/classification will enable stakeholders such as citizens, government bodies, rescue agencies, and industries to comprehend the extent of damage, and to develop more effective mitigation policies. The research will also train students including minority and underrepresented students in the field.There are three specific tasks in this project. In Task 1, a real-time decision-making approach will be proposed via online deep reinforcement learning with inherent distributed training capability; temporal and spatial correlation in streaming video will then be exploited towards real-time multi-object tracking/detection. In Task 2, novel hardware architectures will be designed to support sparse Convolution Neural Networks (CNN). Considering the dual benefits of sparsity on both lower computational and space complexity for Deep Neural Network (DNN) models, a sparsity-aware CNN accelerator can achieve significant hardware performance improvements in term of latency, throughput, and energy efficiency over non-sparsity-aware techniques. Finally, in Task 3, hardware-aware software engineering solutions will be studied for accelerated execution. The idea of leveraging compiler optimization and the underlying hardware features in combination will be investigated in order to optimize execution performance; then, data-driven modeling techniques will be presented to replace the time-consuming segments of the ML software packages with their equivalent data-driven models, namely micro-neural networks. Once these three research tasks are validated individually via principled experimentation in terms of their stated goals, they will be integrated into a unified framework, which will be thoroughly studied via multiple trials on complementary field scenarios. The project will also collaborate with a synergistic DARPA program for related hardware development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.