CNS Core: Small: Ultra Low Power Hardware AI Accelerator for Training at the Edge

CNS 核心：小型：用于边缘训练的超低功耗硬件 AI 加速器

• 批准号: 2106237
• 负责人: Sanghamitra Roy
• 金额: $ 49.93万
• 依托单位: Utah State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2106237&HistoricalAwards=false
• 关键词: CNS; Core; Small; Ultra; Low

Advances in the Artificial Intelligence (AI) domain have enabled a plethora of applications in recent years, many of which were well beyond imagination even a decade ago. A typical use-case scenario involves training a Deep Neural Network (DNN) on a server farm involving powerful processor systems, and then using a trained model for inference at the edge of the network. To enable efficient and widespread use of inference, there is an emergence of domain-specific architectures (e.g. Google Tensor Processing Unit). The current training at the cloud---inference at the edge model can expose the data to privacy breaches, suffer large data transfer bottlenecks, and scale poorly with our growing smart ecosystem. With an expected growth to trillions of connected devices by 2035, the cost of data transfer, as well as, server energy expenditures, are set to explode. This research project will realize retraining capabilities at the edge, in a hardware accelerator with limited power and resource constraints. The project will allow AI edge devices to largely operate as their own standalone engines, breaking down their dependency on the cloud. Specifically, the project will explore problems and solutions in the following directions: (1) managing timing related errors at ultra-low power operation while preserving training convergence; (2) exploiting diverse utilization of hardware components during training for energy efficiency; and (3) developing an open sourced AI hardware edge simulation environment that will spawn further research on low power training at the edge. This research project will establish a foundation for incremental training at the edge, thereby reshaping the ecosystem of AI computation.The proposed AI edge platform, if successfully developed, can facilitate newcapabilities in our ubiquitous interconnected world. For example, the proposed framework will allow a hand-held fitness device to retrain itself using the personalized traits and behavior of a user, incrementally updating its base model that was trained over huge datasets. The project will engage in an extensive outreach program through three closely-related activities: (i) high school girls will be introduced to hands-on engineering exercises through active participation in the Engineering Extravaganza event at Utah State University; (ii) AI learning modules will be disseminated to K-12 classrooms in Utah; and (iii) the edge AI simulation platform developed in this project will be shared in an open source Github repository, allowing academic and industrial researchers to explore AI hardware design techniques beyond those in this project.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

近年来，人工智能（AI）领域的进步已经实现了大量的应用，其中许多甚至在十年前都远远超出了想象。一个典型的用例场景涉及在包含强大处理器系统的服务器群上训练深度神经网络（DNN），然后在网络边缘使用经过训练的模型进行推理。为了能够有效和广泛地使用推理，出现了特定领域的架构（例如Google Tensor Processing Unit）。目前在云端的训练-边缘模型的推理可能会使数据暴露于隐私泄露，遭受大的数据传输瓶颈，并且随着我们不断增长的智能生态系统的扩展性很差。预计到2035年，连接设备将增长到数万亿台，数据传输成本以及服务器能源支出将呈爆炸式增长。该研究项目将在具有有限功率和资源约束的硬件加速器中实现边缘再培训功能。该项目将允许AI边缘设备在很大程度上作为自己的独立引擎运行，打破它们对云的依赖。具体而言，该项目将在以下方向探索问题和解决方案：（1）在超低功耗操作下管理与时序相关的错误，同时保持训练收敛;（2）在训练过程中利用硬件组件的多样性来提高能效;以及（3）开发一个开源的AI硬件边缘仿真环境，这将促进对边缘低功耗训练的进一步研究。该研究项目将为边缘增量训练奠定基础，从而重塑人工智能计算的生态系统。如果成功开发，拟议的人工智能边缘平台将在我们无处不在的互联世界中促进新功能。例如，所提出的框架将允许手持健身设备使用用户的个性化特征和行为来重新训练自己，逐步更新其在巨大数据集上训练的基础模型。该项目将通过三项密切相关的活动开展广泛的外联方案：（i）通过积极参与犹他州州立大学的工程盛会活动，向高中女生介绍动手工程练习;（ii）将向犹他州的K-12教室传播人工智能学习模块;以及（iii）本项目开发的边缘AI仿真平台将在开源Github存储库中共享，允许学术和工业研究人员探索AI硬件设计技术，超越本项目。该奖项反映了NSF的法定基金会的使命是履行其使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评价，被认为值得支持。

项目成果

STRIVE: Enabling Choke Point Detection and Timing Error Resilience in a Low-Power Tensor Processing Unit

• DOI: 10.1109/dac56929.2023.10247879
• 发表时间: 2023-07
• 期刊: 2023 60th ACM/IEEE Design Automation Conference (DAC)
• 作者: N. D. Gundi;Z. Mowri;Andrew Chamberlin;Sanghamitra Roy;Koushik Chakraborty

Implementing a Timing Error-Resilient and Energy-Efficient Near-Threshold Hardware Accelerator for Deep Neural Network Inference

• DOI: 10.3390/jlpea12020032
• 发表时间: 2022-06
• 期刊: Journal of Low Power Electronics and Applications
• 影响因子: 2.1
• 作者: N. D. Gundi;Pramesh Pandey;Sanghamitra Roy;Koushik Chakraborty

UPTPU: Improving Energy Efficiency of a Tensor Processing Unit through Underutilization Based Power-Gating

• DOI: 10.1109/dac18074.2021.9586224
• 发表时间: 2021-12
• 期刊: 2021 58th ACM/IEEE Design Automation Conference (DAC)
• 作者: Pramesh Pandey;N. D. Gundi;Koushik Chakraborty;Sanghamitra Roy