Collaborative Research: PPoSS: Planning: Model-Driven Compiler Optimization and Algorithm-Architecture Co-Design for Scalable Machine Learning

协作研究：PPoSS：规划：用于可扩展机器学习的模型驱动编译器优化和算法架构协同设计

• 批准号: 2118737
• 负责人: Atanas Rountev
• 金额: $ 6.3万
• 依托单位: Ohio State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-08-01 至 2022-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2118737&HistoricalAwards=false
• 关键词: Collaborative; Research; PPoSS; Planning; Model

There is an inexorable need for increased computational performance and improved energy efficiency in the development and use of machine-learning (ML) models. Currently available frameworks for ML have limitations in developing non-traditional models, e.g., using tensor networks, as well as in developing and using models that are too large to fit in the physical memory of processors. The design of efficient hardware accelerators and the mapping of ML algorithms to them is another challenge. This planning project presents a plan of action to address these needs via advances to model-driven compiler optimization. The research conducted in this project is enhancing productivity, performance, and portability in developing software for ML. It is enabling new ML applications to be developed with high productivity, with high achieved performance, and performance-portability over a diverse set of hardware platforms. It is enabling greater "democratization of ML", permitting researchers who only have access to low-end hardware platforms to be able to run the largest models -- infeasible today due to limitations of existing ML frameworks. The project involves training activities tailored for K-12 students, undergraduate students, and graduate students. In this planning project, the following primary technical directions are explored: (1) ML Algorithms: flexible new ML models, offering trade-offs between model size, model execution time, model accuracy, and energy efficiency; (2) Optimizing Compilers: advances in polyhedral compiler optimization to enable parametric tilesize optimization and code generation for diverse target platforms, including CPUs, GPUs, and accelerators; (3) ML Accelerators: new ML accelerator designs for sparse and dense operators, optimized for multiple criteria via comprehensive design space exploration. Broader impact aims of the project include the "Democratization of AI”, to enable state-of-the-art ML models to be used by all, on widely available non-state-of-the-art hardware. To achieve these goals, the project integrates expertise in computer architecture, optimizing compilers, ML algorithms, and high-performance computing.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在开发和使用机器学习模型（ML）模型中，需要提高计算性能和提高能源效率。当前可用的ML框架在开发非传统模型（例如使用张量网络）以及开发和使用太大而无法适合处理器物理内存的模型方面有局限性。有效的硬件加速器的设计和向其映射ML算法是另一个挑战。该计划项目提出了一项行动计划，可以通过模型驱动的编译器优化来满足这些需求。该项目进行的研究是提高为ML开发软件的生产率，性能和可移植性。它使新的ML应用程序能够以较高的生产率，高度达到的性能以及对潜水员一组硬件平台的性能 - 通货性的开发。它正在实现更大的“ ML民主化”，使他们只能使用低端硬件平台的研究人员能够运行最大的模型 - 由于现有ML框架的局限性，如今是不可行的。该项目涉及针对K-12学生，本科生和研究生量身定制的培训活动。在此计划项目中，探讨了以下主要技术方向：（1）ML算法：灵活的新ML模型，在模型大小，模型执行时间，模型准确性和能源效率之间提供权衡； （2）优化编译器：多面体编译器优化的进步，以启用参数图块的优化和代码生成，包括CPU，GPU和加速器在内的潜水员目标平台； （3）ML加速器：针对稀疏和密集操作员的新ML加速器设计，通过全面的设计空间探索针对多个标准进行了优化。该项目的更广泛影响的目的包括“ AI的民主化”，以使所有人都能使用最先进的ML模型，并在广泛可用的非国家硬件上。为了实现这些目标，该项目将计算机架构，优化编译器，ML算法和高性能计算的专业知识融为一体。该奖项反映了NSF的法定任务，并被认为是通过基金会的智力优点和更广泛的影响来通过评估来支持的。

项目成果

Comprehensive Accelerator-Dataflow Co-design Optimization for Convolutional Neural Networks

• DOI: 10.1109/cgo53902.2022.9741281
• 发表时间: 2022-04
• 期刊: 2022 IEEE/ACM International Symposium on Code Generation and Optimization (CGO)
• 作者: Miheer Vaidya;Aravind Sukumaran-Rajam;A. Rountev;P. Sadayappan