CNS Core: Small: Ultra Low Power Hardware AI Accelerator for Training at the Edge

近年来，人工智能(AI)领域的进步使大量应用成为可能，其中许多应用甚至在十年前都远远超出想象。典型的用例场景涉及在涉及强大处理器系统的服务器群上训练深度神经网络(DNN)，然后使用训练好的模型在网络边缘进行推理。为了能够高效和广泛地使用推理，出现了特定于领域的体系结构(例如Google张量处理单元)。目前在云端的培训-边缘模型的推理可能会使数据暴露在隐私泄露的风险之下，遭受巨大的数据传输瓶颈，并且与我们不断增长的智能生态系统的可扩展性很差。预计到2035年，联网设备将增长到数万亿台，数据传输成本以及服务器能源支出将呈爆炸式增长。这项研究项目将在边缘实现再培训能力，在功率和资源有限的硬件加速器中。该项目将允许AI边缘设备在很大程度上作为自己的独立引擎运行，打破它们对云的依赖。具体地说，该项目将在以下方向探索问题和解决方案：(1)在保持训练收敛的同时，管理超低功率操作下的计时相关错误；(2)在训练期间利用硬件组件的不同利用来提高能效；以及(3)开发一个开源的AI硬件边缘模拟环境，这将催生对边缘低功率训练的进一步研究。这一研究项目将为边缘增量训练奠定基础，从而重塑人工智能计算的生态系统。如果成功开发，拟议的AI边缘平台可以在我们无处不在的互联世界中促进新的能力。例如，拟议的框架将允许手持健身设备使用用户的个性化特征和行为来重新训练自己，增量地更新其在海量数据集上训练的基本模型。该项目将通过三个密切相关的活动开展广泛的外联计划：(I)通过积极参与犹他州州立大学的工程活动，向高中女生介绍动手工程练习；(Ii)人工智能学习模块将传播到犹他州的K-12教室；(Iii)在本项目中开发的EDGE AI模拟平台将在开源的Github资源库中共享，允许学术和工业研究人员探索本项目以外的人工智能硬件设计技术。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Advances in the Artificial Intelligence (AI) domain have enabled a plethora of applications in recent years, many of which were well beyond imagination even a decade ago. A typical use-case scenario involves training a Deep Neural Network (DNN) on a server farm involving powerful processor systems, and then using a trained model for inference at the edge of the network. To enable efficient and widespread use of inference, there is an emergence of domain-specific architectures (e.g. Google Tensor Processing Unit). The current training at the cloud---inference at the edge model can expose the data to privacy breaches, suffer large data transfer bottlenecks, and scale poorly with our growing smart ecosystem. With an expected growth to trillions of connected devices by 2035, the cost of data transfer, as well as, server energy expenditures, are set to explode. This research project will realize retraining capabilities at the edge, in a hardware accelerator with limited power and resource constraints. The project will allow AI edge devices to largely operate as their own standalone engines, breaking down their dependency on the cloud. Specifically, the project will explore problems and solutions in the following directions: (1) managing timing related errors at ultra-low power operation while preserving training convergence; (2) exploiting diverse utilization of hardware components during training for energy efficiency; and (3) developing an open sourced AI hardware edge simulation environment that will spawn further research on low power training at the edge. This research project will establish a foundation for incremental training at the edge, thereby reshaping the ecosystem of AI computation.The proposed AI edge platform, if successfully developed, can facilitate newcapabilities in our ubiquitous interconnected world. For example, the proposed framework will allow a hand-held fitness device to retrain itself using the personalized traits and behavior of a user, incrementally updating its base model that was trained over huge datasets. The project will engage in an extensive outreach program through three closely-related activities: (i) high school girls will be introduced to hands-on engineering exercises through active participation in the Engineering Extravaganza event at Utah State University; (ii) AI learning modules will be disseminated to K-12 classrooms in Utah; and (iii) the edge AI simulation platform developed in this project will be shared in an open source Github repository, allowing academic and industrial researchers to explore AI hardware design techniques beyond those in this project.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.