SHF: Small: ML Accelerator Cohort Architecture

机器学习（ML）模型在人们的日常生活中发挥着越来越重要的作用：从自动驾驶、医疗保健、数据中心管理到机器翻译。为了处理各种用法，模型的多样性一直在增长：从卷积神经网络，到嵌入基于表的推荐系统，再到图神经网络，再到自然语言处理。由于模型的使用嵌入到敏感领域，如语音和情感识别引擎，数据隐私也是下一代机器学习硬件的关键要求。随着模型多样性的增长，计算和通信需求也在模型之间和模型内变化很大。新的隐私保护执行模型，如多方计算（MPC），需要完全不同的硬件支持。因此，有必要重新考虑机器学习硬件加速器的设计，以适应保护隐私的异构模型使用的新时代。为了解决这些问题，该项目专注于ML加速器队列架构（MLACA）的开发。MLACA是异构ML加速块的集合，它们协同工作以适应动态变化的ML执行需求。MLACA使用多个研究重点来实现其目标：第一个重点是构建一个异构的MLACA计算结构，该结构支持广泛的密集和稀疏执行范例，包括对私有计算的新支持。第二个重点是MLACA的内存和加速结构，该结构为嵌入表执行内存索引加速，预取使用完美的训练数据的未来知识。MLACA的通信推力侧重于分布式训练加速，使用动态张量分解等技术来权衡计算和通信成本。运行时系统推力管理跨竞争ML模型的MLACA结构分配，以最大限度地提高资源利用率并提高功率效率。技术转型是通过与南加州大学meta中心和英特尔私人人工智能研究所的强大行业合作来规划的。本研究利用美国国家科学基金会的研究经验为本科生提供资金，并利用南加州大学的内部SURE和K-12 SHINE项目，让高中生、教师和本科生为ML系统设计的职业生涯做好准备。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning (ML) models play an increasingly crucial role in people's daily lives: from autonomous driving, health care, data center management, to machine translation. To deal with the range of usages, model diversity has been growing: from convolutional neural networks, to embedding table-based recommendation systems, to graph neural networks to natural language processing. As model usage embeds in sensitive domains, such as speech- and emotion-recognition engines, data privacy is also a key requirement for the next generation ML hardware. As the model diversity has grown, the compute and communication demands also vary widely, both across models and within a model. And new privacy protecting execution models such as multi-party computing (MPC) demand fundamentally different hardware support. Hence, there is a need to rethink the design of ML hardware accelerators for the new era of privacy-preserving heterogeneous model usage. To address these concerns, this project focuses on the development of the ML Accelerator Cohort Architecture (MLACA). MLACA is collection of heterogeneous ML accelerator tiles that work in unison to adapt to the dynamically changing ML execution demands. MLACA uses multiple research thrusts to achieve its goals: the first thrust focuses on building a heterogeneous MLACA compute fabric that supports a wide range of dense and sparse execution paradigms, including novel support for private computing. The second thrust focuses on MLACA's memory and acceleration fabric that performs in-memory indexing acceleration for embedding tables, prefetching that uses perfect future knowledge of training data. MLACA's communication thrust focuses on distributed training acceleration, using techniques such as dynamic tensor decomposition that tradeoff computation and communication costs. The runtime system thrust manages MLACA fabric allocation across competing ML models to maximize the resource utilization and improve power efficiency. Technology transition is planned through strong industry collaborations with the USC-Meta center and Intel's Private AI Institute. This research uses NSF’s research experience for undergraduate funding, and USC's internal SURE and K-12 SHINE programs to engage high school students and teachers, and undergraduate students in preparing them for a career in ML systems design.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.