Distributed-Memory Parallel JointNMF
Distributed-Memory Parallel JointNMF
复制标题
DOI：
10.1145/3577193.3593733
复制
发表时间：
2023-06
期刊：
影响因子：
--
通讯作者：
Srinivas Eswar;Benjamin Cobb;Koby Hayashi;R. Kannan;Grey Ballard;R. Vuduc;Haesun Park
中科院分区：
文献类型：
--
作者：
Srinivas Eswar;Benjamin Cobb;Koby Hayashi;R. Kannan;Grey Ballard;R. Vuduc;Haesun Park
联合非负矩阵分解(JointNMF)是一种从包含特征和连接信息的数据集中挖掘信息的混合方法。提出了基于交替非负最小二乘、投影梯度下降和投影高斯-牛顿的三种求解JointNMF问题的算法的分布式存储并行化。我们将使用单处理器网格的众所周知的避免通信的算法扩展到我们在两个处理器网格上的耦合情况。我们展示了算法在多达960个核(40个节点)上的可扩展性以及60%的并行效率。更复杂的交替非负最小二乘(ANLS)和高斯-牛顿变种在处理大规模问题时优于一阶梯度下降法。我们在包含3700多万篇论文摘要和近10亿条引文关系的大型学术论文语料库上进行了主题建模任务，证明了该方法的实用性和可扩展性。
Joint Nonnegative Matrix Factorization (JointNMF) is a hybrid method for mining information from datasets that contain both feature and connection information. We propose distributed-memory parallelizations of three algorithms for solving the JointNMF problem based on Alternating Nonnegative Least Squares, Projected Gradient Descent, and Projected Gauss-Newton. We extend well-known communication-avoiding algorithms using a single processor grid case to our coupled case on two processor grids. We demonstrate the scalability of the algorithms on up to 960 cores (40 nodes) with 60% parallel efficiency. The more sophisticated Alternating Nonnegative Least Squares (ANLS) and Gauss-Newton variants outperform the first-order gradient descent method in reducing the objective on large-scale problems. We perform a topic modelling task on a large corpus of academic papers that consists of over 37 million paper abstracts and nearly a billion citation relationships, demonstrating the utility and scalability of the methods.