SHF: Small: Sparsity-Aware Hardware Accelerators for Natural Language Processing with Transformers

自然语言处理（NLP）使人们能够以与彼此相同的方式与机器进行交互。更重要的是，它为机器提供了访问书籍、文章和各种非结构化文档中随时可用的信息和知识的能力。由于NLP驱动的服务的质量和可用性主要取决于系统能够处理的文本数量，因此高级NLP应用程序的计算需求远远超过通用计算机的能力，并继续增长。该项目旨在大大提高基于transformers的NLP应用程序的性能，transformers是大多数最先进的NLP技术中使用的一类神经网络。该项目将显著提高NLP应用程序的性能和效率，使其能够在新兴的数据中心中广泛部署，从而提高人类与机器以及彼此之间的交互质量。（硬件和编译器），主要关注大型多层自注意模型中的稀疏感知推理，到目前为止，它只受到建筑界的有限关注。该项目还推进了稀疏注意力函数的NLP知识，研究了允许重新利用预训练模型以更快运行的设计技术，并提高了与其训练设置不同的应用程序的有效性。调查的重点是关键的观察，即计算复杂性的大规模增长可以通过动态识别模型中的固有稀疏性和无效计算来缓解，改装模型以诱导稀疏性，目标是近似或完全避免对模型结果影响有限的计算部分。这项调查将证明这些技术所获得的性能改善，利用稀疏性和动态预测的一种新的稀疏意识的硬件加速框架内，实现现场可编程门阵列（FPGA）。该奖项反映了NSF的法定使命，并已被认为是值得通过评估使用基金会的智力价值和更广泛的影响审查标准的支持。

Natural Language Processing (NLP) enables people to interact with machines in the same manner as with each other. More importantly, it provides machines with the ability to access the information and knowledge that are readily available in books, articles, and various unstructured documents. Because the quality and usability of NLP-powered services depends primarily on the quantity of text the system is able to process, the computational demands of advanced NLP applications far exceed the capabilities of general-purpose computers and continue to grow. This project aims to greatly improve the performance of NLP applications based on transformers, a class of neural networks used in most state-of-the-art NLP technology. This project will significantly improve performance and efficiency for NLP applications, enabling their widespread deployment in emerging datacenters and thus enhancing the quality of human interactions with machines and each other.This project advances the state of the art of accelerators (hardware and compilers) for natural language processing, focusing primarily on sparsity-aware inference in large multi-layered self-attention based models, which have so far received limited attention from the architecture community. The project also advances NLP knowledge of sparse attention functions, studies design techniques that allow for repurposing pre-trained models to run faster, and improves the effectiveness in applications which diverge from its training setting. The investigation focuses on the key observation that the massive growth in computational complexity can be mitigated by dynamically identifying inherent sparsity and ineffectual computation in models, refitting the model to induce sparsity with the goal of either approximating or entirely avoiding parts of the computation that have limited impact on the model results. This investigation will demonstrate the performance improvement obtained by these techniques, leveraging sparsity and dynamic predictions within a novel sparsity-aware hardware acceleration framework, implemented on a field-programmable gate array (FPGA).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.