AutoHOOT: Automatic High-Order Optimization for Tensors
AutoHOOT: Automatic High-Order Optimization for Tensors
复制标题
DOI：
10.1145/3410463.3414647
复制
发表时间：
2020-05
期刊：
影响因子：
--
通讯作者：
Linjian Ma;Jiayu Ye;Edgar Solomonik
中科院分区：
文献类型：
--
作者：
Linjian Ma;Jiayu Ye;Edgar Solomonik
高阶优化方法，包括牛顿方法及其变体以及交替最小化方法，主导了张量分解和张量网络的优化算法。这些张量方法用于量子系统的数据分析和模拟。在这项工作中，我们介绍了AutoHOOT，第一个自动微分（AD）框架，目标是高阶优化张量计算。AutoHOOT采用输入张量计算表达式并生成优化的导数表达式。特别是，AutoHOOT包含一个新的显式Jacobian / Hessian表达式生成内核，其输出保持输入张量的粒度并且易于优化。然后通过传统的编译器优化技术和特定的张量代数变换来优化表达式。实验结果表明，与现有的AD软件和其他具有手动编写内核的张量计算库相比，AutoHOOT在张量分解和张量网络应用中实现了具有竞争力的CPU和GPU性能。AutoHOOT生成的张量方法也是很好的并行化，我们表现出良好的可扩展性上的分布式内存超级计算机。
High-order optimization methods, including Newton's method and its variants as well as alternating minimization methods, dominate the optimization algorithms for tensor decompositions and tensor networks. These tensor methods are used for data analysis and simulation of quantum systems. In this work, we introduce AutoHOOT, the first automatic differentiation (AD) framework targeting at high-order optimization for tensor computations. AutoHOOT takes input tensor computation expressions and generates optimized derivative expressions. In particular, AutoHOOT contains a new explicit Jacobian / Hessian expression generation kernel whose outputs maintain the input tensors' granularity and are easy to optimize. The expressions are then optimized by both the traditional compiler optimization techniques and specific tensor algebra transformations. Experimental results show that AutoHOOT achieves competitive CPU and GPU performance for both tensor decomposition and tensor network applications compared to existing AD software and other tensor computation libraries with manually written kernels. The tensor methods generated by AutoHOOT are also well-parallelizable, and we demonstrate good scalability on a distributed memory supercomputer.