Investigating the Benefit of FP16-Enabled Mixed-Precision Solvers for Symmetric Positive Definite Matrices Using GPUs
Investigating the Benefit of FP16-Enabled Mixed-Precision Solvers for Symmetric Positive Definite Matrices Using GPUs
复制标题
DOI：
10.1007/978-3-030-50417-5_18
复制
发表时间：
2020-06-15
期刊：
影响因子：
--
通讯作者：
Dongarra J
中科院分区：
文献类型：
--
作者：
Abdelfattah A;Tomov S;Dongarra J
半精度计算是指以16位格式执行浮点运算。虽然半精度在很大程度上是由机器学习应用驱动的，但最近在数值线性代数中的算法进步已经发现了半精度在加速更高精度线性方程组解方面的有益用例。在本文中，我们提出了一个高性能，混合精度的线性求解器（）的对称正定系统在双精度使用图形处理单元（gpu）。该求解器基于混合精度的Cholesky分解，利用支持cuda的gpu中的高性能张量核心单元。由于Cholesky因子受到低精度的影响，需要迭代细化求解器将解恢复到双精度精度。两种不同类型的红外求解器讨论了广泛的测试矩阵。还开发了预处理步骤，必要时缩放和移动矩阵，以便在较低精度下保持其正确定性。我们在V100 GPU上的实验表明，与直接双精度求解器相比，性能加速高达4.7。然而，条件数和特征值分布等矩阵性质会影响收敛速度，从而影响整体性能。
Half-precision computation refers to performing floating-point operations in a 16-bit format. While half-precision has been driven largely by machine learning applications, recent algorithmic advances in numerical linear algebra have discovered beneficial use cases for half precision in accelerating the solution of linear systems of equations at higher precisions. In this paper, we present a high-performance, mixed-precision linear solver () for symmetric positive definite systems in double-precision using graphics processing units (GPUs). The solver is based on a mixed-precision Cholesky factorization that utilizes the high-performance tensor core units in CUDA-enabled GPUs. Since the Cholesky factors are affected by the low precision, an iterative refinement (IR) solver is required to recover the solution back to double-precision accuracy. Two different types of IR solvers are discussed on a wide range of test matrices. A preprocessing step is also developed, which scales and shifts the matrix, if necessary, in order to preserve its positive-definiteness in lower precisions. Our experiments on the V100 GPU show that performance speedups are up to 4.7 against a direct double-precision solver. However, matrix properties such as the condition number and the eigenvalue distribution can affect the convergence rate, which would consequently affect the overall performance.
登录
查看更多内容
影响因子：
6.3
作者：
Baboulin, Marc;Buttari, Alfredo;Tomov, Stanimire
通讯作者：
Tomov, Stanimire
影响因子：
3.1
作者：
Carson, Erin;Higham, Nicholas J.
通讯作者：
Higham, Nicholas J.
影响因子：
3.1
作者：
Carson, Erin;Higham, Nicholas J.
通讯作者：
Higham, Nicholas J.
DOI：
10.1137/s0895479895284944
发表时间：
1997-07-01
影响因子：
1.5
作者：
Greenbaum, A
通讯作者：
Greenbaum, A
DOI：
10.1137/0907058
发表时间：
1986-07-01
期刊：
SIAM JOURNAL ON SCIENTIFIC AND STATISTICAL COMPUTING
影响因子：
--
作者：
SAAD, Y;SCHULTZ, MH
通讯作者：
SCHULTZ, MH