SHF: Small: Algorithms and Software for Scalable Kernel Methods

SHF：小型：可扩展核方法的算法和软件

• 批准号: 1817048
• 负责人: George Biros
• 金额: $ 47.62万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-07-01 至 2021-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1817048&HistoricalAwards=false
• 关键词: SHF; Small; Algorithms; Software; Scalable

Scientists and engineers are increasingly interested in using machine learning methods on huge datasets that cannot be processed on a single workstation. At the same time public and private institutions are making significant investments on high-performance computing (HPC) clusters equipped with thousands of leading edge processors and network connectivity. However, despite the availability of such HPC systems, data analysis tasks are mostly restricted to a single or a few workstations. The reason is that, with few exceptions, existing machine learning software does not scale efficiently on HPC systems. The need to process in-situ large scientific and engineering datasets is not met with current software and significant downsampling is required in order to use existing tools. A serious bottleneck in current artificial intelligence (AI) workflows is the significant cost of training for large scale problems. The slow convergence of existing methods and the large number of calibration hyper-parameters (learning rate, batch size, and other knobs that control the performance of the AI system) make training extremely expensive. Design and analysis of scalable optimization algorithms for faster training, that is the fitting of the machine learning (ML) model parameters to the data, are needed for analytics in real time and at scale, which is the goal of this project.The proposed research will introduce novel numerical methods and parallel algorithms for second-order/Newton methods that will be tailored to machine learning (ML) models and will be many orders of magnitude faster than the existing state of-the-art (first-order methods like steepest descent). The researchers plan to design, analyze, and implement robust approximations for covariance matrices, a class of matrices in AI and computational statistics, used in statistical analysis (e.g., sampling, risk assessment, and uncertainty quantification). The investigators plan to design, analyze, and implement scalable fast algorithms in the context of high-performance computing for the so called nearest-neighbor problem, a particular method in ML, data analysis, and information retrieval. The resulting software library will provide a means for end-to-end tools for discovery and innovation and provide new capabilities in the NSF XSEDE infrastructure project. Along with research activities, an educational and dissemination program is designed to communicate the results of this work to both students and researchers, as well as a more general audience of computational and application scientists.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

科学家和工程师对在单个工作站无法处理的庞大数据集上使用机器学习方法越来越感兴趣。与此同时，公共和私人机构正在对高性能计算（HPC）集群进行大量投资，这些集群配备了数千个领先的处理器和网络连接。然而，尽管这种高性能计算系统的可用性，数据分析任务大多局限于单个或几个工作站。原因是，除了少数例外，现有的机器学习软件不能在高性能计算系统上有效地扩展。目前的软件无法满足现场处理大型科学和工程数据集的需求，为了使用现有的工具，需要进行大量的降采样。当前人工智能（AI）工作流程的一个严重瓶颈是大规模问题的培训成本高昂。现有方法的缓慢收敛和大量的校准超参数（学习率、批处理大小和控制人工智能系统性能的其他旋钮）使得训练非常昂贵。设计和分析可扩展的优化算法，以实现更快的训练，即机器学习（ML）模型参数与数据的拟合，这是实时和大规模分析所需要的，这是本项目的目标。拟议的研究将为二阶/牛顿方法引入新的数值方法和并行算法，这些方法将为机器学习（ML）模型量身定制，并且将比现有最先进的方法（如最陡下降的一阶方法）快许多个数量级。研究人员计划设计、分析和实现协方差矩阵的鲁棒近似，协方差矩阵是人工智能和计算统计学中的一类矩阵，用于统计分析（例如，抽样、风险评估和不确定性量化）。研究人员计划在高性能计算的背景下设计、分析和实现可扩展的快速算法，以解决所谓的最近邻问题，这是ML、数据分析和信息检索中的一种特定方法。由此产生的软件库将为发现和创新提供端到端工具，并为NSF XSEDE基础设施项目提供新功能。在开展研究活动的同时，还设计了一个教育和传播计划，将这项工作的结果传达给学生和研究人员，以及更广泛的计算和应用科学家受众。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Distributed O(N) Linear Solver for Dense Symmetric Hierarchical Semi-Separable Matrices

密集对称分层半可分离矩阵的分布式 O(N) 线性求解器

• DOI: 10.1109/mcsoc.2019.00008
• 发表时间: 2019
• 期刊: 2019 IEEE 13th International Symposium on Embedded Multicore/Many-core Systems-on-Chip (MCSoC
• 作者: Yu, Chenhan D.;Reiz, Severin;Biros, George
• 通讯作者: Biros, George

Hardware Accelerator Integration Tradeoffs for High-Performance Computing: A Case Study of GEMM Acceleration in N-Body Methods

高性能计算的硬件加速器集成权衡：N 体方法中 GEMM 加速的案例研究

• DOI: 10.1109/tpds.2021.3056045
• 发表时间: 2021
• 期刊: IEEE Transactions on Parallel and Distributed Systems
• 影响因子: 5.3
• 作者: Asri, Mochamad;Malhotra, Dhairya;Wang, Jiajun;Biros, George;John, Lizy K;Gerstlauer, Andreas
• 通讯作者: Gerstlauer, Andreas

ANODE: Unconditionally Accurate Memory-Efficient Gradients for Neural ODEs

• DOI: 10.24963/ijcai.2019/103
• 发表时间: 2019-02
• 期刊: ArXiv
• 作者: A. Gholami;K. Keutzer;G. Biros

RCHOL: Randomized Cholesky Factorization for Solving SDD Linear Systems

RCHOL：用于求解 SDD 线性系统的随机 Cholesky 分解

• DOI: 10.1137/20m1380624
• 发表时间: 2021
• 期刊: SIAM Journal on Scientific Computing
• 影响因子: 3.1
• 作者: Chen, Chao;Liang, Tianyu;Biros, George
• 通讯作者: Biros, George

ANODEV2: A Coupled Neural ODE Framework

• 发表时间: 2019
• 期刊: ArXiv
• 作者: Tianjun Zhang;Z. Yao;A. Gholami;Joseph Gonzalez;K. Keutzer;Michael W. Mahoney;G. Biros