SPX: Collaborative Research: FASTLEAP: FPGA based compact Deep Learning Platform

近年来，随着人工智能的兴起，深度神经网络(DNN)以其高精度、良好的可扩展性和自适应特性得到了广泛的应用。许多应用都采用DNN作为核心技术，如人脸检测、语音识别、场景分析等。为了满足各种应用对高精度的要求，DNN模型正在变得越来越深入和庞大，并且正在以快速的速度发展。它们是计算和存储密集型的，并且对传统的用于计算的冯·诺伊曼体系结构构成了密集的挑战。该项目解决的关键问题是如何加速深度学习，不仅是推理，而且是训练和模型压缩，这在以往的研究中没有得到足够的重视。这一努力有可能使设计快速、节能的深度学习系统成为可能，其应用在我们的日常生活中-从自动驾驶到移动设备，再到物联网系统，从而造福于整个社会。该项目的成果是FASTLEAP-一个基于现场可编程门阵列(FPGA)的加速深度学习的平台。该平台接受一个数据集作为输入，并输出一个模型，该模型经过训练、修剪和映射到现场可编程门阵列上，优化了快速推理。该项目将利用新兴的可访问高带宽存储器(HBM)并由浮点数字信号处理器单元组成的现场可编程门阵列技术。从纵向角度来看，FASTLEAP集成了从整个系统堆栈算法、体系结构到高效的FPGA硬件实现的多个层面的创新。从水平的角度来看，它包括系统的DNN模型压缩和相关的基于FPGA的训练，以及基于FPGA的压缩DNN模型的推理加速。该平台将作为一个完整的解决方案交付，包括软件工具链和硬件实施，以确保易用性。在FASTLEAP的算法级，提出的神经网络乘子交替方向方法(ADMM-NN)框架将在给定训练数据、目标精度和目标FPGA平台特性(性能模型、加速器间通信)的情况下执行统一的权重剪枝和量化。ADMM-NN中的训练过程是在一个具有多个FPGA加速器的平台上执行的，这取决于体系结构级的通信和并行性优化。最后，基于训练好的带压缩的DNN模型生成优化的FPGA推理设计，为FPGA性能建模提供了依据。该项目将解决以下SPX研究领域：1)算法：弥合深度学习理论发展与其系统实现之间的差距，认识到平台的性能模型。2)应用：深度学习在图像处理等领域的伸缩。3)架构和系统：自动生成关于FPGA优化面积、能效、延迟和吞吐量的深度学习设计。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

With the rise of artificial intelligence in recent years, Deep Neural Networks (DNNs) have been widely used because of their high accuracy, excellent scalability, and self-adaptiveness properties. Many applications employ DNNs as the core technology, such as face detection, speech recognition, scene parsing. To meet the high accuracy requirement of various applications, DNN models are becoming deeper and larger, and are evolving at a fast pace. They are computation and memory intensive and pose intensive challenges to the conventional Von Neumann architecture used in computing. The key problem addressed by the project is how to accelerate deep learning, not only inference, but also training and model compression, which have not received enough attention in the prior research. This endeavor has the potential to enable the design of fast and energy-efficient deep learning systems, applications of which are found in our daily lives -- ranging from autonomous driving, through mobile devices, to IoT systems, thus benefiting the society at large.The outcome of this project is FASTLEAP - an Field Programmable Gate Array (FPGA)-based platform for accelerating deep learning. The platform takes in a dataset as an input and outputs a model which is trained, pruned, and mapped on FPGA, optimized for fast inferencing. The project will utilize the emerging FPGA technologies that have access to High Bandwidth Memory (HBM) and consist of floating-point DSP units. In a vertical perspective, FASTLEAP integrates innovations from multiple levels of the whole system stack algorithm, architecture and down to efficient FPGA hardware implementation. In a horizontal perspective, it embraces systematic DNN model compression and associated FPGA-based training, as well as FPGA-based inference acceleration of compressed DNN models. The platform will be delivered as a complete solution, with both the software tool chain and hardware implementation to ensure the ease of use. At algorithm level of FASTLEAP, the proposed Alternating Direction Method of Multipliers for Neural Networks (ADMM-NN) framework, will perform unified weight pruning and quantization, given training data, target accuracy, and target FPGA platform characteristics (performance models, inter-accelerator communication). The training procedure in ADMM-NN is performed on a platform with multiple FPGA accelerators, dictated by the architecture-level optimizations on communication and parallelism. Finally, the optimized FPGA inference design is generated based on the trained DNN model with compression, accounting for FPGA performance modeling. The project will address the following SPX research areas: 1) Algorithms: Bridging the gap between deep learning developments in theory and their system implementations cognizant of performance model of the platform. 2) Applications: Scaling of deep learning for domains such as image processing. 3) Architecture and Systems: Automatic generation of deep learning designs on FPGA optimizing area, energy-efficiency, latency, and throughput.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.