Automating Matrix Code Optimization for Performance and Portability

许多大规模科学和数据分析代码在机器学习和物理模拟中的应用程序的可扩展性和性能在很大程度上依赖于用于操作大型矩阵的优化和并行实现。具有惊人海量数据的新应用程序的出现，使得传统的优化矩阵计算的方法，如使用专门库和经典数学方法，在许多情况下都不适用。数学方法通常本身也是不可伸缩的，并在矩阵计算中引入数据依赖关系。手工编写的专门库应用有限的优化来保持通用性，必须手动移植到新的体系结构，并且可能会随着体系结构的进步而停滞不前。此外，矩阵算法中复杂的依赖结构限制了编译器可以应用于这些代码的优化。 该项目采用了一种综合的方法，结合了数学重构、高性能算法设计、编译器和系统设计的努力，构建了用于大规模仿真的高性能和可扩展的软件框架。对于稀疏矩阵计算，我们计划分析非零模式，即符号信息，并结合数值算法，以便我们的框架可以检测稀疏矩阵方法中的计算模式。因此，通过将符号分析与数值计算完全解耦，我们的框架将自动生成高性能的稀疏矩阵码。对于处理大量密集矩阵输入的大数据应用，我们的框架将支持近似矩阵计算。近似矩阵算法降低了矩阵计算的计算和存储复杂性，其目标是在精度、速度和稳健性方面击败确定性算法。我们还将为大数据机器学习中使用的大型优化模型制定可伸缩的矩阵算法，并构建用于这些算法的分布式实现的集群计算引擎。正如我们的产业界和学术界合作者的兴趣所证明的那样，我们相信这项研究将被领域专家广泛使用，以取代手工优化的库代码，并显著提高大规模模拟中矩阵计算的性能。

The scalability and performance of many large-scale scientific and data analytics codes for applications in machine learning and physics simulations depend heavily on the optimizations and parallel implementations used to operate on large matrices. The emergence of new applications with stupendously large data has rendered the classical approaches to optimizing matrix computations, such as using specialized libraries and classical mathematical methods, inadequate in many situations. Mathematical methods are also often inherently unscalable and introduce data dependencies in matrix computations. Hand-written specialized libraries apply limited optimizations to maintain generality, must be manually ported to new architectures, and may stagnate with architectural advances. Also, complex dependence structures in matrix algorithms limit the optimizations that a compiler can apply to these codes. The project takes an integrated approach that combines efforts in mathematical reformulation, high-performance algorithm design, and compiler and system design to build high-performance and scalable software frameworks for large-scale simulations. For sparse matrix computations, we plan to analyze the non-zero patterns, i.e. symbolic information, along with the numerical algorithm so that our framework can detect computation patterns in the sparse matrix methods. As a result, our framework will automatically generate high-performance sparse matrix codes by fully decoupling the symbolic analysis from numeric computation. For big data applications that manipulate large dense matrix inputs, our framework will support approximate matrix computations. Approximate matrix algorithms reduce the computation and storage complexity of matrix computations with the objective of beating deterministic algorithms in terms of accuracy, speed, and robustness. We will also formulate scalable matrix algorithms for large optimization models used in “big data” machine learning and build a cluster-computing engine to be used for distributed implementations of these algorithms. As evidenced by interest from our industrial and academic collaborators, we believe this research will be broadly used by domain experts to replace hand-optimized library codes and significantly improve the performance of matrix computations in large-scale simulations.