Mixed Precision Arithmetic for Large Scale Linear Inverse Problems

游戏行业、机器学习（ML）和人工智能（AI）是需要大量计算资源和/或需要非常快速的计算的领域，但在某些计算问题上并不总是要求高精度。这促使公司制造计算机芯片硬件，例如图形处理单元（gpu），它可以使用低精度的计算机算术格式执行非常快的计算。与典型科学应用中使用的高精度算法相比，这可以使速度提高4倍。在过去的十年里，更快计算的潜力激发了人们对使用强大的GPU服务器进行科学应用的兴趣，特别是在需要高精度的问题上使用混合精度算法。也就是说，在可能的情况下，使用低精度的速度，但在需要保持精度时混合使用高精度计算。虽然以前的工作已经完成了某些核心线性代数计算，但相对较少的工作是开发和理解使用混合精度算法解决更具挑战性的不适定问题的含义。这项工作的重点是开发这一前沿的方法，以便高效的求解器可以利用具有混合精确计算能力的现代gpu。需要考虑在解决条件良好的问题时通常不会出现的特殊考虑。在机器学习、图像恢复和图像重建（包括乳房成像）中的应用将被用作目标测试问题，但本工作的一个重要目的是构建一个计算平台，可用于在各种应用中有效地计算大规模不适定逆问题的近似解。通过开发一个灵活和适应性强的计算平台，该项目产生的工作旨在对需要计算正则化大规模逆问题解决方案的应用产生广泛的科学影响，包括天文学、宇宙学、地球物理学、机器学习、显微镜和医学成像。作为该项目的一部分，将对学生和博士后进行培训。在过去十年中，更快计算的潜力激发了人们对使用强大的GPU服务器进行科学应用的兴趣，特别是在需要高精度的问题上使用混合精度算法；也就是说，在可能的情况下，使用低精度来提高速度，但混合使用高精度计算来提高精度。最近为科学应用开发混合精度计算方法的工作主要集中在一般的、条件良好的线性系统上，包括迭代细化、Cholesky分解和最小二乘问题、QR分解和GMRES。该项目的目的是专注于混合精度计算的发展，以解决更具挑战性的大规模病态逆问题。该方法将结合算子近似，使用低精度截断奇异值分解和迭代细化，利用混合精度格式来确保足够的精度，或者作为Krylov子空间迭代方法的前置条件。此外，混合精度（可能与迭代细化）将应用于灵活和/或不精确的混合Krylov子空间方法中，以减少每次迭代增加的存储需求和计算成本。在这个项目中开发的方法可以作为工具来获得病态逆问题的近似解，或者作为机器学习中的求解器。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The gaming industry, machine learning (ML), and artificial intelligence (AI) are areas that require substantial computational resources and/or require very fast computations, but do not always require high accuracy in certain computational problems. This has motivated companies to manufacture computer chip hardware, such as graphical processing units (GPUs) that can perform very fast computations using low precision computer arithmetic formats. This can result in a 4-times speedup compared to high precision arithmetic used in typical scientific applications. The potential for much faster computations has fueled a growing interest in the last decade to use powerful GPU servers for scientific applications, and in particular to use mixed precision algorithms for problems that require high accuracy. That is, when possible, use low precision for speed, but mix in high precision computations when needed to maintain accuracy. Although previous work has been done for certain core linear algebra computations, relatively little has been done to exploit and understand the implications of using mixed precision arithmetic for the more challenging class of ill-posed problems. The focus of this work is on developing methods for this frontier, so that efficient solvers can take advantage of modern GPUs with mixed precision computing capabilities. Special considerations, which normally do not arise when solving well-conditioned problems, need to be considered. Applications in machine learning, image restoration and image reconstruction, including breast imaging, will be used as target test problems, but an important aim of this work is to construct a computational platform that can be used to efficiently compute approximate solutions of large scale ill-posed inverse problems in a variety of applications. By developing a flexible and adaptable computational platform, the work produced from this project aims to have a broad scientific impact for applications where it is necessary to compute solutions of large-scale inverse problems with regularization, including astronomy, cosmology, geophysics, machine learning, microscopy, and medical imaging. Students and postdocs will be trained as part of this project. The potential for much faster computations has fueled a growing interest in the last decade to use powerful GPU servers for scientific applications, and in particular to use mixed precision algorithms for problems that require high accuracy; that is, when possible, use low precision for speed, but mix in high precision computations to improve accuracy. Recent previous work to develop mixed precision computational approaches for scientific applications have focused on general, well-conditioned linear systems, including iterative refinement, Cholesky factorization and least squares problems, QR factorization, and GMRES. The aim of this project is to focus on the development of mixed precision computations for the more challenging class of large-scale ill-posed inverse problems. The approach will use a combination of operator approximation, using a low precision truncated singular value decomposition with iterative refinement exploiting mixed precision formats to ensure sufficient accuracy, or as preconditioners in Krylov subspace iterative methods. In addition, mixed precision, possibly with iterative refinement, will be applied within flexible and/or inexact hybrid Krylov subspace methods to reduce storage requirements and computational costs that increase at each iteration. The methods developed in this project can be used as tools to obtain approximate solutions of ill-posed inverse problems, or as solvers in machine learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.