Collaborative Research: PPoSS: Planning: Model-Driven Compiler Optimization and Algorithm-Architecture Co-Design for Scalable Machine Learning

协作研究：PPoSS：规划：用于可扩展机器学习的模型驱动编译器优化和算法架构协同设计

• 批准号: 2119677
• 负责人: Ponnuswamy Sadayappan
• 金额: $ 18.7万
• 依托单位: University of Utah
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-08-01 至 2022-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2119677&HistoricalAwards=false
• 关键词: Collaborative; Research; PPoSS; Planning; Model

There is an inexorable need for increased computational performance and improved energy efficiency in the development and use of machine-learning (ML) models. Currently available frameworks for ML have limitations in developing non-traditional models, e.g., using tensor networks, as well as in developing and using models that are too large to fit in the physical memory of processors. The design of efficient hardware accelerators and the mapping of ML algorithms to them is another challenge. This planning project presents a plan of action to address these needs via advances to model-driven compiler optimization. The research conducted in this project is enhancing productivity, performance, and portability in developing software for ML. It is enabling new ML applications to be developed with high productivity, with high achieved performance, and performance-portability over a diverse set of hardware platforms. It is enabling greater "democratization of ML", permitting researchers who only have access to low-end hardware platforms to be able to run the largest models -- infeasible today due to limitations of existing ML frameworks. The project involves training activities tailored for K-12 students, undergraduate students, and graduate students. In this planning project, the following primary technical directions are explored: (1) ML Algorithms: flexible new ML models, offering trade-offs between model size, model execution time, model accuracy, and energy efficiency; (2) Optimizing Compilers: advances in polyhedral compiler optimization to enable parametric tilesize optimization and code generation for diverse target platforms, including CPUs, GPUs, and accelerators; (3) ML Accelerators: new ML accelerator designs for sparse and dense operators, optimized for multiple criteria via comprehensive design space exploration. Broader impact aims of the project include the "Democratization of AI”, to enable state-of-the-art ML models to be used by all, on widely available non-state-of-the-art hardware. To achieve these goals, the project integrates expertise in computer architecture, optimizing compilers, ML algorithms, and high-performance computing.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在开发和使用机器学习(ML)模型时，不可避免地需要提高计算性能和提高能源效率。目前可用的ML框架在开发非传统模型方面具有局限性，例如使用张量网络，以及在开发和使用太大而不适合处理器的物理存储器的模型方面。高效硬件加速器的设计和ML算法到它们的映射是另一个挑战。这个计划项目提出了一项行动计划，通过推进模型驱动的编译器优化来满足这些需求。本项目进行的研究是为了提高ML软件开发的生产率、性能和可移植性。它使新的ML应用程序能够以高生产率、高性能和在不同硬件平台上的性能可移植性进行开发。它实现了更大程度的“ML民主化”，允许只能访问低端硬件平台的研究人员能够运行最大的模型--由于现有ML框架的限制，这在今天是不可行的。该项目包括为K-12学生、本科生和研究生量身定做的培训活动。在这个规划项目中，探索了以下主要技术方向：(1)ML算法：灵活的新ML模型，提供模型大小、模型执行时间、模型精度和能源效率之间的权衡；(2)优化编译器：多面体编译器优化方面的进展，以支持针对不同目标平台(包括CPU、GPU和加速器)的参数块大小优化和代码生成；(3)ML加速器：针对稀疏和密集操作符的新ML加速器设计，通过全面的设计空间探索针对多个标准进行优化。该项目更广泛的影响目标包括“人工智能的民主化”，使最先进的ML模型能够在广泛可用的非最先进硬件上被所有人使用。为了实现这些目标，该项目整合了计算机体系结构、优化编译器、ML算法和高性能计算方面的专业知识。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

CANDLES: Channel-Aware Novel Dataflow-Microarchitecture Co-Design for Low Energy Sparse Neural Network Acceleration

• DOI: 10.1109/hpca53966.2022.00069
• 发表时间: 2022-04
• 期刊: 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA)
• 作者: Sumanth Gudaparthi;Sarabjeet Singh;Surya Narayanan;R. Balasubramonian;V. Sathe

Comprehensive Accelerator-Dataflow Co-design Optimization for Convolutional Neural Networks

• DOI: 10.1109/cgo53902.2022.9741281
• 发表时间: 2022-04
• 期刊: 2022 IEEE/ACM International Symposium on Code Generation and Optimization (CGO)
• 作者: Miheer Vaidya;Aravind Sukumaran-Rajam;A. Rountev;P. Sadayappan