Elements: Software: NSCI: A high performance suite of SVD related solvers for machine learning

海量数据的积累是我们这个世纪的特征之一。在计算机的帮助下，科学家们利用这些数据来提出和检验假设，得出推论，预测复杂的现象，并做出明智的政策决定。机器学习（ML）是计算机科学的一个领域，它使用统计方法让计算机在有或没有人类监督的情况下从数据中“学习”。机器学习方法应用的核心是非常大维度矩阵的奇异值分解（SVD）的数值计算，通常大于一百万甚至十亿。然而，由于“现成的”算法和SVD软件无法处理非常大维度的矩阵，因此科学计算中使用的迭代方法更为合适。然而，它们严格的近似质量要求通常对下游应用程序来说是过高的，并导致缓慢的执行时间。最近，基于随机化的方法改进了执行时间，但它们的实现放松了近似质量，通常达到有害的水平。该项目建议开发一个软件包，该软件包将随机和迭代方法统一起来，特别关注各种ML应用程序的特定要求，并为现代计算平台提供高性能优化。这将使科学家能够分析更大的数据集，机器学习研究人员可以研究以前无法解决的大型模型，机器学习服务提供商可以使用新的求解器来降低运营成本。该项目建议开发一个软件包，该软件包将随机和迭代方法统一起来，特别关注各种ML应用程序的特定要求，并为现代计算平台提供高性能优化。这将使科学家能够分析更大的数据集，机器学习研究人员可以研究以前无法解决的大型模型，机器学习服务提供商可以使用新的求解器来降低运营成本。具体来说，该软件包建立在最先进的特征值/奇异值软件包primeme之上，该软件包集成了尖端的迭代方法和高性能实现。该软件包的开发包括两个重点：（T1）统一最先进的算法技术，包括随机化、流化和迭代方法，为具有不同质量要求、硬件平台和精度以及编程环境的各种矩阵提供一致的体验。（T2）开发软件设备，使下游系统和SVD求解器能够互操作，这样用户就可以调整和定制求解器，而无需成为数字线性代数的专家。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The accrual of vast amounts of data is one of the defining characteristics of our century. With the help of computers, scientists use this data to make and test hypotheses, draw inferences, predict complex phenomena, and make educated policy decisions. Machine learning (ML) is an area in computer science that uses statistical methods to allow computers to "learn" from data, with and without human supervision. Central to the application of machine learning methods is the numerical computation of the Singular Value Decomposition (SVD) of matrices of very large dimension, often larger than a million or even a billion. Since "off-the-shelf" algorithms and SVD software, however, cannot handle matrices of very large dimension, iterative methods used in scientific computing are more appropriate. Yet their stringent approximation quality requirements are often excessive for downstream applications, and result in slow execution times. Recently, methods based on randomization have improved execution times, but their implementations relax the approximation quality, often to detrimental levels. This project proposes to develop a software package that unifies randomized and iterative methods with a particular focus on the specific requirements of various ML applications and with high performance optimizations for modern computing platforms. This will allow scientists to analyze significantly larger datasets, ML researchers to study large models that could not be tackled before, and ML service providers to use the new solvers to reduce their operational cost. This project proposes to develop a software package that unifies randomized and iterative methods with a particular focus on the specific requirements of various ML applications and with high performance optimizations for modern computing platforms. This will allow scientists to analyze significantly larger datasets, ML researchers to study large models that could not be tackled before, and ML service providers to use the new solvers to reduce their operational cost. Specifically, the software package builds upon the state-of-the-art eigenvalue/singular value software package PRIMME that integrates cutting-edge iterative methods and high-performance implementations. The development of the package consists of two thrusts: (T1) Unifying state-of-the-art algorithmic techniques including randomized, streaming, and iterative methods, to deliver consistent experience for a diverse range of matrices with different quality requirements, hardware platforms and precisions, and programming environments. (T2) Developing software devices that enable downstream systems and SVD solvers to interoperate so that users can tune and customize solvers without being experts in numeric linear algebra.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.