SHF: Small: Energy Efficient Learning on Chip with Quantized Representations

机器学习模型，特别是深度神经网络，已经被应用于许多实时应用中，如语音和图像识别或目标检测。这些应用程序通常必须是快速和节能的，才能在现场使用。该项目为神经网络开发量化表示和算法转换，这些神经网络表示大型模型，存储、能量和面积显著减少，并且精度几乎没有下降-换句话说，能够在移动设备上使用高效的神经网络。该项目的结果将改变设计者处理大规模深度神经网络的建模、分析和优化方法的方式，更广泛地说，任何依赖昂贵的计算操作、占用大量内存的统计学习应用程序。该项目不仅将产生技术影响，还将成为重要的教育和指导组成部分，可能会改变工程师接受多学科培训的方式，以应对下一代技术进步，特别是硅领域的高能效机器学习问题。这项工作将利用卡内基梅隆大学团队在涉及培训不同学生群体的代表性不足群体方面的丰富经验，同时进一步扩大该项目对高中生和中学生的影响。该项目利用了这样一个事实，即使用量化表示进行芯片上培训和推理可以减少相关硬件实施的能源消耗和存储需求。这是实现实时学习系统更高吞吐量和更低延迟的关键特征。该项目通过为机器学习模型开发新的量化方法来应对这些挑战，这些方法减少了数据移动和计算，从而降低了整体能源消耗。此外，本文所追求的研究依赖于新的算法方法来训练量化学习模型，以便在不损害其准确性的情况下加速其训练过程，并且在硬件(即，现场可编程门阵列)上学习量化模型以验证其益处并加速其采用。为了证明可行性，该项目以基于硬件加速器的测试平台为特色，以量化的方式进行推理和培训模型。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning models, especially deep neural networks, have been adopted in many real-time applications, such as speech and image recognition or object detection. These applications typically must be fast and energy efficient so they are usable in the field. This project develops quantized representations and algorithmic transformations for neural networks that represent large models with significantly less storage, energy and area, and with little or no accuracy degradation - in other words, enabling the use of efficient neural networks on mobile devices. The results of this project are poised to change how designers approach modeling, analysis, and optimization methodologies for large-scale deep neural networks, and more generally, any statistical learning applications that rely on expensive compute operations, with large memory footprint. The project will have not only a technical impact, but also an important educational and mentoring component by potentially changing how engineers are trained in a multidisciplinary fashion for dealing with next generation technological advances in general, and the problem of energy efficient machine learning in silicon in particular. The extensive experience of the Carnegie Mellon team in outreach to underrepresented groups involving training a diverse student body will be leveraged in this work, while further expanding the project's outreach to high-school and middle-school students.This project exploits the fact that using a quantized representation for on-chip training and inference can reduce the energy consumption and storage requirements of the associated hardware implementation. This is a crucial feature in achieving higher throughput and lower latency for real-time learning systems. This project addresses these challenges by developing novel quantization approaches for machine learning models that reduce both data movement and computation, and therefore reduce overall energy consumption. Furthermore, the research pursued herein relies on new algorithmic approaches for training quantized learning models so as to accelerate their training process without harming their accuracy, and learning quantized models on hardware (i.e., field programmable gate arrays) to verify their benefits and accelerate their adoption. To demonstrate feasibility, the project features a hardware accelerator-based testbed for inference and training models in a quantized fashion.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.