SHF: Small: Content-Aware Mapping of Streaming AI Workloads on Heterogeneous Edge Devices

人类可以从各种复杂的数据源中无缝地检测和分类对象，并进行推理以做出预测和决策。被称为深度神经网络（DNN）的新型算法正在开发中，以赋予计算机相同的功能。 目前将所有数据传输到远程数据中心并在那里执行算法的方法是不可持续的，因为生成的数据量呈指数级增长，速度太慢，并且可能危及隐私和安全。该项目的目标是在数据采集地点或附近执行复杂的DNN算法。被称为“边缘的人工智能”，几乎所有的领先行业都在开发各种新的“边缘设备”，以部署在该领域。该项目将开发一个由技术不可知的软件工具组成的框架，该框架将在边缘设备的异构网络上优化部署DNN算法，以最大限度地提高其性能和能效。将从该项目的成果中受益的领域包括零售，安全，运输和物流，工厂自动化，医疗保健等项目团队还将包括研究生和本科生。将大力从代表性不足的群体中招收学生。该团队还将积极寻求各种商业化途径。该项目的目的是使用DNN算法实现“边缘AI”，该算法可以在任何类型的数据上进行训练，在任何维度上，然后用于提取有价值的信息，用于自动预测，分类和决策。复杂的DNN模型可能涉及数百个层和数千万个参数。 因为训练是计算和内存密集型的，所以它是在服务器上执行的。然而，为了在边缘执行推理，业界正在构建硬件加速器，在硅中实现DNN，将其与部署在边缘的移动的片上系统（SoC）集成，每个都有自己的架构，内存组织和神经形态引擎。 此外，复杂的ML应用程序将被表示为在流数据上运行的DNN的异构模型网络（NoM）。 该项目中要解决的关键挑战是确定如何最佳地将NoM映射到异构边缘计算设备的网络上，NoM的结构根据数据的内容不断变化。优化将涉及复制和管道化DNN模型，并决定在哪个边缘计算设备上部署模型的每个实例，所有这些都在运行时进行。此外，该确定将基于数据流的内容、可用资源、通信介质的特性以及模型到设备的当前分配。该项目的成果将包括技术不可知的算法和软件工具，用于执行此映射。该奖项反映了NSF的法定使命，并已被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

Humans can seamlessly detect and classify objects from a wide range of complex data sources, and draw inferences to make predictions and decisions. New types of algorithms known as deep neural networks (DNN), are being developed to endow computers with very same capabilities. The present approach of transferring all the data to a remote datacenter and have the algorithms executed there is not sustainable because the amount of data being generated is growing exponentially, is too slow, and can compromise privacy and security. The aim of this project is to enable the execution of complex DNN algorithms at or near the place of data acquisition. Referred to as "AI at the edge", nearly all the leading industries are developing varieties of new "edge devices" to be deployed in the field. This project will develop a framework consisting of technology agnostic software tools that will optimally deploy the DNN algorithms on heterogenous networks of edge devices to maximize their performance and energy efficiency. Domains that will benefit from the outcomes of this project, include retail, security, transportation and logistics, factory automation, healthcare etc. The project team will also include graduate and undergraduate students. Strong effort to recruit students from underrepresented groups will be made. The team will also vigorously pursue various avenues for commercialization. The aim of this project is to enable "AI at the Edge" using DNN algorithms, which can be trained on any kind of data, in any number of dimensions, and then used to extract valuable information for automated prediction, classification, and decision making. Sophisticated DNN models can involve 100s of layers and tens of millions of parameters. Because training is computation and memory intensive, it is performed on servers. However, for performing inference at the edge, industry is building hardware accelerators that implement DNNs in silicon, integrating them with their mobile Systems on Chips (SoC)s to be deployed at the edge, each with their own architectures, memory organization and neuromorphic engines. Furthermore, complex ML applications will be expressed as heterogeneous Networks of Models (NoMs) of DNNs operating on streaming data. The key challenges to be addressed in this project are to determine how to optimally map NoMs, whose structure keeps changing depending the content of the data, onto a network of heterogeneous edge computing devices. The optimization will involve replicating and pipelining DNN models and deciding on which edge computing device to deploy each instance of a model, all at run-time. Furthermore, this determination will be based on the content of the data stream, the available resources, the characteristics of the communication medium, as well as the present allocation of models to devices. The outcomes of this project will include technology agnostic algorithms and software tools for performing this mapping.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.