Distributed Relational Algebra at Scale
Distributed Relational Algebra at Scale
复制标题
大规模分布式关系代数
DOI：
--
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
Sidharth Kumar
中科院分区：
文献类型：
--
作者：
Thomas Gilray;Sidharth Kumar
关系代数构成了适用于图形和网络中应用程序，程序分析，演绎数据库和约束逻辑编程的原始操作的基础。尽管具有表现力的能力，但关系代数在高性能计算研究中并未与更常见的原始词相同的关注，例如模板计算，浮点操作，数值集成和稀疏线性代数。此外，在解决关系的分布部分之间的表示和通信方面，尤其是对于固有的不平衡关系，以前已经阻止了将关系代数应用程序成功地扩展到超级计算机。在本文中，我们提出了一系列有效的算法，以有效地平行和扩展关键关系代数原始算法。我们介绍了一种混合Hash-Tree方法来表示分布式不平衡关系并允许有效的沟通。最后，我们使用定点算法在32,768个过程上使用定点算法（生成2760亿个边缘生成2760亿个边缘）的传递闭合来证明实现的可扩展性。
Relational algebra forms a basis of primitive operations suitable for applications in graphs and networks, program analysis, deductive databases, and constraint logic programming. Despite its expressive power, relational algebra has not received the same attention in high-performance-computing research as more common primitives like stencil computations, floating-point operations, numerical integration, and sparse linear algebra. Furthermore, specific challenges in addressing representation and communication among distributed portions of a relation, especially for inherently imbalanced relations, have previously thwarted successful scaling of relational algebra applications to supercomputers. In this paper, we present a set of efficient algorithms to effectively parallelize and scale key relational algebra primitives. We introduce a hybrid hash-tree approach to representing distributed imbalanced relations and permitting efficient communication. Finally, we demonstrate the scalability of our implementation with a fixed-point algorithm computing the transitive closure of a large graph (generating over 276 billion edges) on 32,768 processes.
DOI：
--
发表时间：
2018-10
期刊：
--
影响因子：
--
作者：
Kai Wang;Zhiqiang Zuo;John Thorpe;Tien Quang Nguyen;G. Xu
通讯作者：
Kai Wang;Zhiqiang Zuo;John Thorpe;Tien Quang Nguyen;G. Xu