SHF: Small: Dynamic Gating and Adaptation of Deep Neural Networks for Efficient Inference and Training

在过去的五年里，深度神经网络(DNN)的机器学习出现了前所未有的增长，它现在代表了许多人工智能应用中的最先进技术。然而，现有的DNN模型需要大量的内存和计算能力，这极大地限制了它们在移动和物联网设备等资源受限系统中的使用。该项目将开发新的算法和硬件，以显著提高DNN的效率，并代表着即使在资源有限的环境中也能够快速和自适应地执行DNN的重要一步。从这个意义上说，这个项目有可能使机器学习得到更广泛的部署，这将在未来智能社会的许多方面发挥关键作用。该研究项目将为学生提供研究培训机会，并通过利用康奈尔大学的现有资源，如夏令营和包括女性在内的高中生推广方案，开发新的课程。该项目旨在通过共同开发算法优化和高效的硬件加速器架构，显著提高DNN的效率，同时保持高精度。虽然在降低DNN执行成本方面存在许多工作，但这些技术中的大多数主要是为了改进推理和执行静态优化而设计的，这些静态优化可以统一地减少所有输入的计算，或者只利用有限形式的动态稀疏，即零。该项目旨在通过利用运行时特定于每个输入的通用形式的动态稀疏性，为DNN实现新的性能-精度折衷点，这在今天是不可能的。更具体地说，该项目计划调查特定于输入的门控技术，该技术可以消除训练和推理的冗余计算，开发不需要训练数据的动态量化技术，并设计一个高效和统一的硬件加速器架构，提供真实世界的性能和能量改进。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The past half-decade has seen unprecedented growth in machine learning with deep neural networks (DNNs), which now represent the state-of-the-art in many AI applications. However, existing DNN models require substantial memory and computing power, which greatly limit their use in resource-constrained systems such as mobile and IoT devices. This project will develop new algorithms and hardware to significantly improve the efficiency of DNNs, and represents an important step towards enabling fast and adaptive DNN executions even in resource-limited environments. In that sense, this project has the potential to enable a wider deployment of machine learning, which will play a critical role in many aspects of the future smart society. The research project will provide research training opportunities to the students as well as new curriculum development by leveraging existing resources at Cornell, e.g., summer camps as well as an outreach programs for high-school students including women.This project aims to significantly improve the efficiency of DNNs, while maintaining high accuracy, by co-developing algorithm optimizations and an efficient hardware-accelerator architecture. While there exist many lines of work on reducing DNN execution costs, the majority of these techniques are designed primarily to improve inference and perform static optimizations that reduce computation uniformly for all inputs or only exploit a limited form of dynamic sparsity, namely zeros. This project aims to enable new performance-accuracy trade-off points for DNNs that are not possible today by exploiting general forms of dynamic sparsity that are specific to each input at run-time. More specifically, the project plans to investigate input-specific gating techniques that can remove redundant computations for both training and inference, develop dynamic quantization techniques that do not require training data, and design an efficient and unified hardware accelerator architecture that provides both real-world performance and energy improvements.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.