Automating Matrix Code Optimization for Performance and Portability

许多用于机器学习和物理模拟应用的大规模科学和数据分析代码的可扩展性和性能在很大程度上取决于用于对大型矩阵进行操作的优化和并行实现。大量数据的新应用程序的出现使得优化矩阵计算的经典方法（例如使用专门的库和经典的数学方法）在许多情况下都不适用。数学方法通常也是固有的不可扩展性，并在矩阵计算中引入数据依赖性。手工编写的专用库应用有限的优化来保持通用性，必须手动移植到新的体系结构，并且可能随着体系结构的发展而停滞不前。此外，矩阵算法中的复杂依赖结构限制了编译器可以应用于这些代码的优化。** 该项目采用综合方法，将数学重构、高性能算法设计以及编译器和系统设计方面的工作结合起来，为大规模模拟构建高性能和可扩展的软件框架。对于稀疏矩阵计算，我们计划分析非零模式，即符号信息，沿着数值算法，以便我们的框架可以检测稀疏矩阵方法中的计算模式。因此，我们的框架将自动生成高性能的稀疏矩阵代码完全解耦的符号分析从数值计算。对于处理大型密集矩阵输入的大数据应用程序，我们的框架将支持近似矩阵计算。近似矩阵算法降低了矩阵计算的计算和存储复杂度，其目标是在准确性、速度和鲁棒性方面击败确定性算法。我们还将制定用于“大数据”机器学习的大型优化模型的可扩展矩阵算法，并构建一个集群计算引擎，用于这些算法的分布式实现。正如我们的工业和学术合作者的兴趣所证明的那样，我们相信这项研究将被领域专家广泛用于取代手工优化的库代码，并显着提高大规模模拟中矩阵计算的性能。

The scalability and performance of many large-scale scientific and data analytics codes for applications in machine learning and physics simulations depend heavily on the optimizations and parallel implementations used to operate on large matrices. The emergence of new applications with stupendously large data has rendered the classical approaches to optimizing matrix computations, such as using specialized libraries and classical mathematical methods, inadequate in many situations. Mathematical methods are also often inherently unscalable and introduce data dependencies in matrix computations. Hand-written specialized libraries apply limited optimizations to maintain generality, must be manually ported to new architectures, and may stagnate with architectural advances. Also, complex dependence structures in matrix algorithms limit the optimizations that a compiler can apply to these codes. ******The project takes an integrated approach that combines efforts in mathematical reformulation, high-performance algorithm design, and compiler and system design to build high-performance and scalable software frameworks for large-scale simulations. For sparse matrix computations, we plan to analyze the non-zero patterns, i.e. symbolic information, along with the numerical algorithm so that our framework can detect computation patterns in the sparse matrix methods. As a result, our framework will automatically generate high-performance sparse matrix codes by fully decoupling the symbolic analysis from numeric computation. For big data applications that manipulate large dense matrix inputs, our framework will support approximate matrix computations. Approximate matrix algorithms reduce the computation and storage complexity of matrix computations with the objective of beating deterministic algorithms in terms of accuracy, speed, and robustness. We will also formulate scalable matrix algorithms for large optimization models used in “big data” machine learning and build a cluster-computing engine to be used for distributed implementations of these algorithms. As evidenced by interest from our industrial and academic collaborators, we believe this research will be broadly used by domain experts to replace hand-optimized library codes and significantly improve the performance of matrix computations in large-scale simulations.**