SHMEMGraph: Efficient and Balanced Graph Processing Using One-Sided Communication
SHMEMGraph: Efficient and Balanced Graph Processing Using One-Sided Communication
复制标题
DOI：
10.1109/ccgrid.2018.00078
复制
发表时间：
2018-05
期刊：
影响因子：
--
通讯作者：
Huansong Fu;Manjunath Gorentla Venkata;Shaeke Salman;N. Imam;Weikuan Yu
中科院分区：
文献类型：
--
作者：
Huansong Fu;Manjunath Gorentla Venkata;Shaeke Salman;N. Imam;Weikuan Yu
最先进的同步图形处理框架面临着低效和不平衡的问题，导致它们的性能不是最优的。这些问题包括通信效率低下和迭代中不平衡的图计算/通信成本。我们建议用单边通信模式取代传统的双边通信模式。因此，我们设计了SHMEMGraph，这是一个高效和平衡的图形处理框架，它在全局内存空间中制定，并利用单侧通信的灵活性和效率进行图形处理。通过高效的单侧通信通道，SHMEMGraph利用了RDMA的高性能操作，同时最大限度地减少了计算机节点内的资源争用。此外，SHMEMGraph综合了许多优化来解决计算不平衡和通信不平衡。通过使用10亿个边的图，我们的评估表明，与最先进的Gemini框架相比，SHMEMGraph在五个代表性图算法的任务完成时间方面平均提高了35.5%。
State-of-the-art synchronous graph processing frameworks face both inefficiency and imbalance issues that cause their performance to be suboptimal. These issues include the inefficiency of communication and the imbalanced graph computation/communication costs in an iteration. We propose to replace their conventional two-sided communication model with the one-sided counterpart. Accordingly, we design SHMEMGraph, an efficient and balanced graph processing framework that is formulated across a global memory space and takes advantage of the flexibility and efficiency of one-sided communication for graph processing. Through an efficient one-sided communication channel, SHMEMGraph utilizes the high-performance operations with RDMA while minimizing the resource contention within a computer node. In addition, SHMEMGraph synthesizes a number of optimizations to address both computation imbalance and communication imbalance. By using a graph of 1 billion edges, our evaluation shows that compared to the state-of-the-art Gemini framework, SHMEMGraph achieves an average improvement of 35.5% in terms of job completion time for five representative graph algorithms.