SHF: Small: Sparsity-Aware Hardware Accelerators for Natural Language Processing with Transformers

SHF：小型：使用 Transformer 进行自然语言处理的稀疏感知硬件加速器

• 批准号: 2007362
• 负责人: Peter Milder
• 金额: $ 50万
• 依托单位: SUNY at Stony Brook
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2007362&HistoricalAwards=false
• 关键词: SHF; Small; Sparsity; Aware; Hardware

Natural Language Processing (NLP) enables people to interact with machines in the same manner as with each other. More importantly, it provides machines with the ability to access the information and knowledge that are readily available in books, articles, and various unstructured documents. Because the quality and usability of NLP-powered services depends primarily on the quantity of text the system is able to process, the computational demands of advanced NLP applications far exceed the capabilities of general-purpose computers and continue to grow. This project aims to greatly improve the performance of NLP applications based on transformers, a class of neural networks used in most state-of-the-art NLP technology. This project will significantly improve performance and efficiency for NLP applications, enabling their widespread deployment in emerging datacenters and thus enhancing the quality of human interactions with machines and each other.This project advances the state of the art of accelerators (hardware and compilers) for natural language processing, focusing primarily on sparsity-aware inference in large multi-layered self-attention based models, which have so far received limited attention from the architecture community. The project also advances NLP knowledge of sparse attention functions, studies design techniques that allow for repurposing pre-trained models to run faster, and improves the effectiveness in applications which diverge from its training setting. The investigation focuses on the key observation that the massive growth in computational complexity can be mitigated by dynamically identifying inherent sparsity and ineffectual computation in models, refitting the model to induce sparsity with the goal of either approximating or entirely avoiding parts of the computation that have limited impact on the model results. This investigation will demonstrate the performance improvement obtained by these techniques, leveraging sparsity and dynamic predictions within a novel sparsity-aware hardware acceleration framework, implemented on a field-programmable gate array (FPGA).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

自然语言处理(NLP)使人们能够以与彼此相同的方式与机器交互。更重要的是，它为机器提供了访问书籍、文章和各种非结构化文档中随时可用的信息和知识的能力。由于NLP支持的服务的质量和可用性主要取决于系统能够处理的文本数量，因此高级NLP应用程序的计算需求远远超过通用计算机的能力，而且还在继续增长。该项目旨在极大地提高基于变压器的NLP应用程序的性能，变压器是大多数最先进的NLP技术中使用的一类神经网络。这个项目将显著提高自然语言处理应用程序的性能和效率，使它们能够在新兴的数据中心得到广泛的部署，从而提高人类与机器和彼此之间交互的质量。这个项目促进了自然语言处理加速器(硬件和编译器)的发展，主要集中在基于大型多层自我注意的模型中的稀疏感知推理，到目前为止，建筑界对这些模型的关注有限。该项目还提高了稀疏注意函数的NLP知识，研究了允许重新调整预先训练的模型以更快运行的设计技术，并提高了与其训练设置背道而驰的应用程序的有效性。研究的重点是，通过动态识别模型中固有的稀疏性和无效计算，重新调整模型以引入稀疏性，目标是近似或完全避免对模型结果影响有限的部分计算，可以缓解计算复杂性的大规模增长。这项调查将展示这些技术在现场可编程门阵列(FPGA)上实施的新型稀疏性感知硬件加速框架内利用稀疏性和动态预测获得的性能改进。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

IrEne: Interpretable Energy Prediction for Transformers

• DOI: 10.18653/v1/2021.acl-long.167
• 发表时间: 2021-06
• 期刊: ArXiv
• 作者: Qingqing Cao;Yash Kumar Lal;H. Trivedi;A. Balasubramanian;Niranjan Balasubramanian

On the Distribution, Sparsity, and Inference-time Quantization of Attention Values in Transformers

• DOI: 10.18653/v1/2021.findings-acl.363
• 发表时间: 2021-06
• 期刊: ArXiv
• 作者: Tianchu Ji;Shraddhan Jain;M. Ferdman;Peter Milder;H. A. Schwartz;Niranjan Balasubramanian