SHF: Small: Content-Aware Mapping of Streaming AI Workloads on Heterogeneous Edge Devices

SHF：小型：异构边缘设备上流式 AI 工作负载的内容感知映射

• 批准号: 2008244
• 负责人: Sarma Vrudhula
• 金额: $ 49.83万
• 依托单位: Arizona State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-07-01 至 2024-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2008244&HistoricalAwards=false
• 关键词: SHF; Small; Content; Aware; Mapping

Humans can seamlessly detect and classify objects from a wide range of complex data sources, and draw inferences to make predictions and decisions. New types of algorithms known as deep neural networks (DNN), are being developed to endow computers with very same capabilities. The present approach of transferring all the data to a remote datacenter and have the algorithms executed there is not sustainable because the amount of data being generated is growing exponentially, is too slow, and can compromise privacy and security. The aim of this project is to enable the execution of complex DNN algorithms at or near the place of data acquisition. Referred to as "AI at the edge", nearly all the leading industries are developing varieties of new "edge devices" to be deployed in the field. This project will develop a framework consisting of technology agnostic software tools that will optimally deploy the DNN algorithms on heterogenous networks of edge devices to maximize their performance and energy efficiency. Domains that will benefit from the outcomes of this project, include retail, security, transportation and logistics, factory automation, healthcare etc. The project team will also include graduate and undergraduate students. Strong effort to recruit students from underrepresented groups will be made. The team will also vigorously pursue various avenues for commercialization. The aim of this project is to enable "AI at the Edge" using DNN algorithms, which can be trained on any kind of data, in any number of dimensions, and then used to extract valuable information for automated prediction, classification, and decision making. Sophisticated DNN models can involve 100s of layers and tens of millions of parameters. Because training is computation and memory intensive, it is performed on servers. However, for performing inference at the edge, industry is building hardware accelerators that implement DNNs in silicon, integrating them with their mobile Systems on Chips (SoC)s to be deployed at the edge, each with their own architectures, memory organization and neuromorphic engines. Furthermore, complex ML applications will be expressed as heterogeneous Networks of Models (NoMs) of DNNs operating on streaming data. The key challenges to be addressed in this project are to determine how to optimally map NoMs, whose structure keeps changing depending the content of the data, onto a network of heterogeneous edge computing devices. The optimization will involve replicating and pipelining DNN models and deciding on which edge computing device to deploy each instance of a model, all at run-time. Furthermore, this determination will be based on the content of the data stream, the available resources, the characteristics of the communication medium, as well as the present allocation of models to devices. The outcomes of this project will include technology agnostic algorithms and software tools for performing this mapping.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人类可以从广泛的复杂数据源中无缝地检测和分类对象，并进行推断以做出预测和决策。被称为深度神经网络（DNN）的新型算法正在开发中，以赋予计算机完全相同的能力。目前将所有数据传输到远程数据中心并在那里执行算法的方法是不可持续的，因为生成的数据量呈指数级增长，速度太慢，并且可能危及隐私和安全性。该项目的目的是使复杂的深度神经网络算法能够在数据采集处或附近执行。被称为“边缘人工智能”的几乎所有领先行业都在开发各种新的“边缘设备”，以部署在该领域。该项目将开发一个由技术无关软件工具组成的框架，该框架将在边缘设备的异构网络上优化部署DNN算法，以最大限度地提高其性能和能源效率。从该项目的成果中受益的领域包括零售、安全、运输和物流、工厂自动化、医疗保健等。项目团队还将包括研究生和本科生。将大力招收来自代表性不足群体的学生。该团队还将积极寻求各种商业化途径。该项目的目的是使用深度神经网络算法实现“边缘人工智能”，该算法可以在任何类型的数据、任何数量的维度上进行训练，然后用于提取有价值的信息，用于自动预测、分类和决策。复杂的深度神经网络模型可以涉及100个层和数千万个参数。因为训练需要大量的计算和内存，所以需要在服务器上执行。然而，为了在边缘执行推理，业界正在构建硬件加速器，在硅中实现dnn，将它们与部署在边缘的移动系统芯片（SoC）集成，每个都有自己的架构、内存组织和神经形态引擎。此外，复杂的机器学习应用将被表示为在流数据上运行的dnn的异构模型网络（NoMs）。在这个项目中需要解决的关键挑战是确定如何将NoMs（其结构根据数据内容不断变化）最佳地映射到异构边缘计算设备网络上。优化将涉及复制和流水线DNN模型，并决定在哪个边缘计算设备上部署模型的每个实例，所有这些都在运行时进行。此外，这种确定将基于数据流的内容、可用资源、通信媒介的特征以及目前对设备的模型分配。这个项目的结果将包括技术不可知算法和执行此映射的软件工具。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

EdgeWise: Energy-efficient CNN Computation on Edge Devices under Stochastic Communication Delays

• DOI: 10.1145/3530908
• 发表时间: 2022-04
• 期刊: ACM Transactions on Embedded Computing Systems (TECS)
• 作者: Mehdi Ghasemi;Daler N. Rakhmatov;Carole-Jean Wu;S. Vrudhula

A Novel ASIC Design Flow Using Weight-Tunable Binary Neurons as Standard Cells

使用权重可调二元神经元作为标准单元的新型 ASIC 设计流程

• DOI: 10.1109/tcsi.2022.3164995
• 发表时间: 2022
• 期刊: IEEE Transactions on Circuits and Systems I: Regular Papers
• 作者: Wagle, Ankit;Singh, Gian;Khatri, Sunil;Vrudhula, Sarma
• 通讯作者: Vrudhula, Sarma

CAMDNN: Content-Aware Mapping of a Network of Deep Neural Networks on Edge MPSoCs

CAMDNN：边缘 MPSoC 上深度神经网络的内容感知映射

• DOI: 10.1109/tc.2022.3207137
• 发表时间: 2022
• 期刊: IEEE Transactions on Computers
• 影响因子: 3.7
• 作者: Heidari, Soroush;Ghasemi, Mehdi;Kim, Young Geun;Wu, Carole-Jean;Vrudhula, Sarma
• 通讯作者: Vrudhula, Sarma

Energy-Efficient Mapping for a Network of DNN Models at the Edge

边缘 DNN 模型网络的节能映射

• DOI: 10.1109/smartcomp52413.2021.00024
• 发表时间: 2021
• 期刊: 2021 IEEE International Conference on Smart Computing (SMARTCOMP
• 作者: Ghasemi, Mehdi;Heidari, Soroush;Kim, Young Geun;Lamb, Aaron;Wu, Carole-Jean;Vrudhula, Sarma
• 通讯作者: Vrudhula, Sarma