Mixed Precision Arithmetic for Large Scale Linear Inverse Problems

游戏行业、机器学习（ML）和人工智能（AI）是需要大量计算资源和/或需要非常快速计算的领域，但在某些计算问题中并不总是需要高精度。这促使公司制造计算机芯片硬件，例如图形处理单元（GPU），可以使用低精度计算机算术格式执行非常快速的计算。与典型科学应用中使用的高精度算法相比，这可以导致4倍的加速。在过去的十年中，更快的计算潜力推动了人们对使用强大的GPU服务器进行科学应用的兴趣日益增长，特别是使用混合精度算法来解决需要高精度的问题。也就是说，在可能的情况下，使用低精度来提高速度，但在需要保持准确性时，混合使用高精度计算。虽然以前的工作已经完成了某些核心线性代数计算，相对较少的已经做了利用和理解的影响，使用混合精度算法更具有挑战性的一类不适定问题。 这项工作的重点是为这一前沿领域开发方法，以便高效的求解器可以利用具有混合精度计算能力的现代GPU。 需要考虑一些特殊的考虑，这些考虑通常在解决良态问题时不会出现。 机器学习、图像恢复和图像重建（包括乳腺成像）中的应用将被用作目标测试问题，但这项工作的一个重要目的是构建一个计算平台，该平台可用于有效地计算各种应用中大规模不适定逆问题的近似解。通过开发一个灵活和适应性强的计算平台，该项目产生的工作旨在对应用产生广泛的科学影响，这些应用需要计算具有正则化的大规模逆问题的解决方案，包括天文学，宇宙学，生物物理学，机器学习，显微镜和医学成像。学生和博士后将作为该项目的一部分接受培训。 在过去的十年中，更快的计算速度的潜力推动了人们对使用强大的GPU服务器进行科学应用的兴趣日益增长，特别是使用混合精度算法来解决需要高精度的问题;也就是说，在可能的情况下，使用低精度来提高速度，但混合高精度计算来提高精度。最近以前的工作，以开发混合精度计算方法的科学应用集中在一般的，良好的条件下的线性系统，包括迭代精化，Cholesky分解和最小二乘问题，QR分解，和GMRES。该项目的目的是专注于发展更具有挑战性的大型不适定反问题的混合精度计算。 该方法将使用算子近似的组合，使用低精度截断奇异值分解与利用混合精度格式的迭代细化，以确保足够的精度，或作为Krylov子空间迭代方法中的预处理器。此外，混合精度，可能与迭代细化，将适用于灵活和/或不精确的混合Krylov子空间方法，以减少存储要求和计算成本，增加在每次迭代。该项目中开发的方法可用作获得不适定逆问题近似解的工具，或用作机器学习中的求解器。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The gaming industry, machine learning (ML), and artificial intelligence (AI) are areas that require substantial computational resources and/or require very fast computations, but do not always require high accuracy in certain computational problems. This has motivated companies to manufacture computer chip hardware, such as graphical processing units (GPUs) that can perform very fast computations using low precision computer arithmetic formats. This can result in a 4-times speedup compared to high precision arithmetic used in typical scientific applications. The potential for much faster computations has fueled a growing interest in the last decade to use powerful GPU servers for scientific applications, and in particular to use mixed precision algorithms for problems that require high accuracy. That is, when possible, use low precision for speed, but mix in high precision computations when needed to maintain accuracy. Although previous work has been done for certain core linear algebra computations, relatively little has been done to exploit and understand the implications of using mixed precision arithmetic for the more challenging class of ill-posed problems. The focus of this work is on developing methods for this frontier, so that efficient solvers can take advantage of modern GPUs with mixed precision computing capabilities. Special considerations, which normally do not arise when solving well-conditioned problems, need to be considered. Applications in machine learning, image restoration and image reconstruction, including breast imaging, will be used as target test problems, but an important aim of this work is to construct a computational platform that can be used to efficiently compute approximate solutions of large scale ill-posed inverse problems in a variety of applications. By developing a flexible and adaptable computational platform, the work produced from this project aims to have a broad scientific impact for applications where it is necessary to compute solutions of large-scale inverse problems with regularization, including astronomy, cosmology, geophysics, machine learning, microscopy, and medical imaging. Students and postdocs will be trained as part of this project. The potential for much faster computations has fueled a growing interest in the last decade to use powerful GPU servers for scientific applications, and in particular to use mixed precision algorithms for problems that require high accuracy; that is, when possible, use low precision for speed, but mix in high precision computations to improve accuracy. Recent previous work to develop mixed precision computational approaches for scientific applications have focused on general, well-conditioned linear systems, including iterative refinement, Cholesky factorization and least squares problems, QR factorization, and GMRES. The aim of this project is to focus on the development of mixed precision computations for the more challenging class of large-scale ill-posed inverse problems. The approach will use a combination of operator approximation, using a low precision truncated singular value decomposition with iterative refinement exploiting mixed precision formats to ensure sufficient accuracy, or as preconditioners in Krylov subspace iterative methods. In addition, mixed precision, possibly with iterative refinement, will be applied within flexible and/or inexact hybrid Krylov subspace methods to reduce storage requirements and computational costs that increase at each iteration. The methods developed in this project can be used as tools to obtain approximate solutions of ill-posed inverse problems, or as solvers in machine learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.