Automating Matrix Code Optimization for Performance and Portability

点击翻译按钮获取中文摘要

The scalability and performance of many large-scale scientific and data analytics codes for applications in machine learning and physics simulations depend heavily on the optimizations and parallel implementations used to operate on large matrices. The emergence of new applications with stupendously large data has rendered the classical approaches to optimizing matrix computations, such as using specialized libraries and classical mathematical methods, inadequate in many situations. Mathematical methods are also often inherently unscalable and introduce data dependencies in matrix computations. Hand-written specialized libraries apply limited optimizations to maintain generality, must be manually ported to new architectures, and may stagnate with architectural advances. Also, complex dependence structures in matrix algorithms limit the optimizations that a compiler can apply to these codes. The project takes an integrated approach that combines efforts in mathematical reformulation, high-performance algorithm design, and compiler and system design to build high-performance and scalable software frameworks for large-scale simulations. For sparse matrix computations, we plan to analyze the non-zero patterns, i.e. symbolic information, along with the numerical algorithm so that our framework can detect computation patterns in the sparse matrix methods. As a result, our framework will automatically generate high-performance sparse matrix codes by fully decoupling the symbolic analysis from numeric computation. For big data applications that manipulate large dense matrix inputs, our framework will support approximate matrix computations. Approximate matrix algorithms reduce the computation and storage complexity of matrix computations with the objective of beating deterministic algorithms in terms of accuracy, speed, and robustness. We will also formulate scalable matrix algorithms for large optimization models used in "big data" machine learning and build a cluster-computing engine to be used for distributed implementations of these algorithms. As evidenced by interest from our industrial and academic collaborators, we believe this research will be broadly used by domain experts to replace hand-optimized library codes and significantly improve the performance of matrix computations in large-scale simulations.