CAREER: Efficient Large Language Model Inference Through Codesign: Adaptable Software Partitioning and FPGA-based Distributed Hardware

人工智能(AI)已经进入了“规模化时代”。大量的训练数据正被用来在大型计算机上训练巨大的深度神经网络(DNN)，大型语言模型(LLM)的兴起就是一个缩影。对这项技术的极高需求是显而易见的，最近的例子是ChatGPT：一个LLM聊天机器人，在发布两个月后就获得了1亿活跃用户，创造了新的世界纪录。然而，部署LLM的成本可能相当高，因为它们的内存占用可能会扩展到TB级的数据，同时还需要很高的计算资源。因此，大规模分布式计算机变得必不可少，尤其是为了满足交互应用所需的性能。为了提高效率，该项目解决了特定于LLM的新挑战，包括它们的巨大内存占用、不同的计算需求和分布式计算。这对于使低收入者更容易获得和可持续地广泛使用至关重要。同时，该奖项旨在培养一支精通算法、硬件和软件的多元化人工智能劳动力队伍，通过公立大学针对不同学生群体的大规模人工智能课程、全面的课程整合以及研究生和本科生水平的学生指导来实现。该项目将使LLMS和分布式计算平台能够协同设计，分为三个主要推力，对应于计算堆栈的三个级别：软件、硬件和算法。最初，该项目将重点放在自动分区和映射算法上，因为这些算法构成了在现有和新的分布式计算平台上部署和优化LLM的基础。这项研究的关键是开发一种可扩展的硬件性能估计器，它可以与新的分布式计算方法一起对当前基于GPU的系统进行建模。具体地说，第二个推力研究了在分布式系统中使用网络内和近存储的现场可编程门阵列来加速LLM推理。最后研究了LLMS的平台感知压缩，包括混合精度量化和低阶近似。除了提高整个计算堆栈的LLM效率外，该项目还将开发一个研究框架，以协同地共同优化LLM和分布式硬件平台，从而产生新的优化的LLM计算系统和实施方法。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Artificial intelligence (AI) has entered the "age of scale". Huge amounts of training data are being used to train enormous deep neural networks (DNNs) on large-scale computers as epitomized by the rise of large language models (LLMs). The extremely high demand for this technology is clearly evident, as recently exemplified by ChatGPT: an LLM chatbot that garnered 100 million active users merely two months post-release, setting a new world record. However, deploying LLMs can be quite costly, given that their memory footprint can extend to terabytes of data while also demanding high computational resources. Consequently, large-scale distributed computers have become essential, particularly to meet the performance required for interactive applications. To improve efficiency, this project tackles new challenges that are specific to LLMs, including their large memory footprint, varying computational demands, and distributed computing. This is critical to make LLMs more accessible and sustainable for widespread use. Concurrently, this award seeks to develop a diverse AI workforce proficient in algorithms, hardware, and software, achieved through a large-scale AI course for diverse student population at public universities, comprehensive curriculum integration, and student mentorship at both graduate and undergraduate levels.This project will enable the codesign of LLMs and distributed computing platforms, divided into three major thrusts that correspond to three levels of the computing stack: software, hardware, and algorithms. Initially, the project will focus on automated partitioning and mapping algorithms, as these form the foundations by which LLMs can be deployed and optimized on both existing and new distributed computing platforms. Key to this research thrust is the development of an extensible hardware performance estimator that can model current GPU-based systems alongside new distributed computing approaches. In particular, the second thrust investigates the use of in-network and near-storage FPGAs within distributed systems to speed up LLM inference. The final thrust investigates platform-aware compression for LLMs, including mixed-precision quantization and low-rank approximation. In addition to improving LLM efficiency across the computing stack, this project will develop a research framework to synergistically co-optimize LLMs and distributed hardware platforms, resulting in new optimized LLM computing systems and implementation methodologies.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.