SHF: Small: ML Accelerator Cohort Architecture

SHF：小型：ML 加速器群组架构

• 批准号: 2224319
• 负责人: Murali Annavaram
• 金额: $ 60万
• 依托单位: University of Southern California
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2224319&HistoricalAwards=false
• 关键词: SHF; Small; ML; Accelerator; Cohort

Machine learning (ML) models play an increasingly crucial role in people's daily lives: from autonomous driving, health care, data center management, to machine translation. To deal with the range of usages, model diversity has been growing: from convolutional neural networks, to embedding table-based recommendation systems, to graph neural networks to natural language processing. As model usage embeds in sensitive domains, such as speech- and emotion-recognition engines, data privacy is also a key requirement for the next generation ML hardware. As the model diversity has grown, the compute and communication demands also vary widely, both across models and within a model. And new privacy protecting execution models such as multi-party computing (MPC) demand fundamentally different hardware support. Hence, there is a need to rethink the design of ML hardware accelerators for the new era of privacy-preserving heterogeneous model usage. To address these concerns, this project focuses on the development of the ML Accelerator Cohort Architecture (MLACA). MLACA is collection of heterogeneous ML accelerator tiles that work in unison to adapt to the dynamically changing ML execution demands. MLACA uses multiple research thrusts to achieve its goals: the first thrust focuses on building a heterogeneous MLACA compute fabric that supports a wide range of dense and sparse execution paradigms, including novel support for private computing. The second thrust focuses on MLACA's memory and acceleration fabric that performs in-memory indexing acceleration for embedding tables, prefetching that uses perfect future knowledge of training data. MLACA's communication thrust focuses on distributed training acceleration, using techniques such as dynamic tensor decomposition that tradeoff computation and communication costs. The runtime system thrust manages MLACA fabric allocation across competing ML models to maximize the resource utilization and improve power efficiency. Technology transition is planned through strong industry collaborations with the USC-Meta center and Intel's Private AI Institute. This research uses NSF’s research experience for undergraduate funding, and USC's internal SURE and K-12 SHINE programs to engage high school students and teachers, and undergraduate students in preparing them for a career in ML systems design.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习(ML)模型在人们的日常生活中发挥着越来越重要的作用：从自动驾驶、医疗保健、数据中心管理到机器翻译。为了应对使用的范围，模型的多样性一直在增长：从卷积神经网络，到嵌入式基于表格的推荐系统，再到图形神经网络，再到自然语言处理。随着模型使用嵌入到语音和情感识别引擎等敏感领域，数据隐私也是下一代ML硬件的关键要求。随着模型多样性的增长，计算和通信需求也有很大的不同，无论是在模型之间还是在模型内。而新的隐私保护执行模式，如多方计算(MPC)，需要完全不同的硬件支持。因此，有必要重新考虑ML硬件加速器的设计，以适应保护隐私的异类模型使用的新时代。为了解决这些问题，本项目侧重于ML加速器队列体系结构(MLACA)的开发。MLACA是异类ML加速器块的集合，它们协同工作以适应动态变化的ML执行需求。MLACA使用多个研究推力来实现其目标：第一个推力专注于构建支持广泛的密集和稀疏执行范例的异类MLACA计算结构，包括对私有计算的新支持。第二个重点是MLACA的内存和加速结构，它为嵌入表执行内存索引加速，预取使用训练数据的完美未来知识。MLACA的通信重点是分布式训练加速，使用动态张量分解等技术来权衡计算和通信成本。运行时系统推力管理跨竞争ML型号的MLACA交换矩阵分配，以最大限度地提高资源利用率和提高能效。技术过渡是通过与南加州大学Meta中心和英特尔的私人人工智能研究所进行强有力的行业合作来计划的。这项研究利用NSF在本科资助方面的研究经验，以及南加州大学的内部SURE和K-12 SLAW计划，让高中生和教师以及本科生为他们在ML系统设计方面的职业生涯做好准备。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

LAORAM: A Look Ahead ORAM Architecture for Training Large Embedding Tables

• DOI: 10.1145/3579371.3589111
• 发表时间: 2021-07
• 期刊: Proceedings of the 50th Annual International Symposium on Computer Architecture
• 作者: Rachit Rajat;Yongqin Wang;M. Annavaram

Characterization of MPC-based Private Inference for Transformer-based Models

• DOI: 10.1109/ispass55109.2022.00025
• 发表时间: 2022-05
• 期刊: 2022 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS)
• 作者: Yongqin Wang;G. Suh;Wenjie Xiong;Benjamin Lefaudeux;Brian Knott;M. Annavaram;Hsien-Hsin S. Lee