SHF: Small: Dynamic Gating and Adaptation of Deep Neural Networks for Efficient Inference and Training

在过去的五年里，深度神经网络（dnn）的机器学习取得了前所未有的发展，在许多人工智能应用中代表了最先进的技术。然而，现有的深度神经网络模型需要大量的内存和计算能力，这极大地限制了它们在移动和物联网设备等资源受限系统中的使用。该项目将开发新的算法和硬件，以显着提高深度神经网络的效率，并代表了在资源有限的环境中实现快速和自适应深度神经网络执行的重要一步。从这个意义上说，这个项目有可能使机器学习得到更广泛的应用，这将在未来智能社会的许多方面发挥关键作用。该研究项目将为学生提供研究培训机会，并利用康奈尔大学现有的资源开发新课程，例如夏令营以及面向包括女性在内的高中生的外展项目。该项目旨在通过共同开发算法优化和高效的硬件加速器架构，在保持高精度的同时显著提高深度神经网络的效率。虽然在降低深度神经网络执行成本方面存在许多工作，但这些技术中的大多数主要是为了提高推理和执行静态优化，从而统一减少所有输入的计算，或者只利用有限形式的动态稀疏性，即零。该项目旨在通过利用运行时特定于每个输入的一般形式的动态稀疏性，为dnn提供新的性能-精度折衷点，这在今天是不可能的。更具体地说，该项目计划研究可以消除训练和推理冗余计算的输入特定门通技术，开发不需要训练数据的动态量化技术，并设计一个高效和统一的硬件加速器架构，提供真实世界的性能和能源改进。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The past half-decade has seen unprecedented growth in machine learning with deep neural networks (DNNs), which now represent the state-of-the-art in many AI applications. However, existing DNN models require substantial memory and computing power, which greatly limit their use in resource-constrained systems such as mobile and IoT devices. This project will develop new algorithms and hardware to significantly improve the efficiency of DNNs, and represents an important step towards enabling fast and adaptive DNN executions even in resource-limited environments. In that sense, this project has the potential to enable a wider deployment of machine learning, which will play a critical role in many aspects of the future smart society. The research project will provide research training opportunities to the students as well as new curriculum development by leveraging existing resources at Cornell, e.g., summer camps as well as an outreach programs for high-school students including women.This project aims to significantly improve the efficiency of DNNs, while maintaining high accuracy, by co-developing algorithm optimizations and an efficient hardware-accelerator architecture. While there exist many lines of work on reducing DNN execution costs, the majority of these techniques are designed primarily to improve inference and perform static optimizations that reduce computation uniformly for all inputs or only exploit a limited form of dynamic sparsity, namely zeros. This project aims to enable new performance-accuracy trade-off points for DNNs that are not possible today by exploiting general forms of dynamic sparsity that are specific to each input at run-time. More specifically, the project plans to investigate input-specific gating techniques that can remove redundant computations for both training and inference, develop dynamic quantization techniques that do not require training data, and design an efficient and unified hardware accelerator architecture that provides both real-world performance and energy improvements.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.