Collaborative Research: PPoSS: Planning: Model-Driven Compiler Optimization and Algorithm-Architecture Co-Design for Scalable Machine Learning

在机器学习（ML）模型的开发和使用中，迫切需要提高计算性能和提高能源效率。目前可用的ML框架在开发非传统模型方面存在局限性，例如，使用张量网络，以及开发和使用太大而无法容纳在处理器的物理内存中的模型。高效硬件加速器的设计以及ML算法到它们的映射是另一个挑战。这个规划项目提出了一个行动计划，以解决这些需求，通过模型驱动的编译器优化的进步。在这个项目中进行的研究正在提高ML软件开发的生产力，性能和可移植性。它使新的机器学习应用程序能够以高生产力，高性能和在各种硬件平台上的性能可移植性进行开发。它正在实现更大的“ML民主化”，允许只能访问低端硬件平台的研究人员能够运行最大的模型-由于现有ML框架的限制，这在今天是不可行的。该项目涉及为K-12学生，本科生和研究生量身定制的培训活动。在这个计划项目中，探索了以下主要技术方向：（1）ML算法：灵活的新ML模型，提供模型大小，模型执行时间，模型准确性和能源效率之间的权衡;（2）优化编译器：多面体编译器优化的进步，以实现参数瓦片大小优化和代码生成，用于各种目标平台，包括CPU，GPU和加速器;（3）ML加速器：针对稀疏和密集运算符的新ML加速器设计，通过全面的设计空间探索针对多个标准进行优化。该项目更广泛的影响目标包括“人工智能的民主化”，使所有人都能在广泛可用的非最先进的硬件上使用最先进的ML模型。为了实现这些目标，该项目整合了计算机体系结构、优化编译器、ML算法和高性能计算方面的专业知识。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

There is an inexorable need for increased computational performance and improved energy efficiency in the development and use of machine-learning (ML) models. Currently available frameworks for ML have limitations in developing non-traditional models, e.g., using tensor networks, as well as in developing and using models that are too large to fit in the physical memory of processors. The design of efficient hardware accelerators and the mapping of ML algorithms to them is another challenge. This planning project presents a plan of action to address these needs via advances to model-driven compiler optimization. The research conducted in this project is enhancing productivity, performance, and portability in developing software for ML. It is enabling new ML applications to be developed with high productivity, with high achieved performance, and performance-portability over a diverse set of hardware platforms. It is enabling greater "democratization of ML", permitting researchers who only have access to low-end hardware platforms to be able to run the largest models -- infeasible today due to limitations of existing ML frameworks. The project involves training activities tailored for K-12 students, undergraduate students, and graduate students. In this planning project, the following primary technical directions are explored: (1) ML Algorithms: flexible new ML models, offering trade-offs between model size, model execution time, model accuracy, and energy efficiency; (2) Optimizing Compilers: advances in polyhedral compiler optimization to enable parametric tilesize optimization and code generation for diverse target platforms, including CPUs, GPUs, and accelerators; (3) ML Accelerators: new ML accelerator designs for sparse and dense operators, optimized for multiple criteria via comprehensive design space exploration. Broader impact aims of the project include the "Democratization of AI”, to enable state-of-the-art ML models to be used by all, on widely available non-state-of-the-art hardware. To achieve these goals, the project integrates expertise in computer architecture, optimizing compilers, ML algorithms, and high-performance computing.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.