Rethinking graph data placement for graph neural network training on multiple GPUs
Rethinking graph data placement for graph neural network training on multiple GPUs
复制标题
DOI：
10.1145/3524059.3532384
复制
发表时间：
2022-03
期刊：
影响因子：
--
通讯作者：
Shihui Song;Peng Jiang
中科院分区：
文献类型：
--
作者：
Shihui Song;Peng Jiang
图形分区通常用于将图形数据用于并行处理。尽管它们在传统的图形处理算法方面取得了良好的性能，但现有的图形分区方法对于GPU的数据并行GNN培训不令人满意。在这项工作中，我们重新考虑了多个GPU的大规模GNN培训的图数据放置问题。我们发现，加载输入功能是用于在GPU上无法存储的大图上GNN训练的性能瓶颈。为了减少数据加载开销，我们首先提出了GNN培训中CPU和GPU之间数据移动的性能模型。然后，基于性能模型，我们提供了一种有效的算法，以将图形数据分配和分布到多个GPU上，以便将数据加载时间最小化。对于仅数据放置无法实现良好性能的情况，我们提出了一种局部感知的邻居抽样技术，以进一步减少数据移动开销而不会失去准确性。我们在不同数量的GPU上使用不同尺寸的图表的实验表明，我们的技术不仅达到了较小的数据加载时间，而且要比现有的图形分配方法所产生的预处理开销要少得多。
Graph partitioning is commonly used for dividing graph data for parallel processing. While they achieve good performance for the traditional graph processing algorithms, the existing graph partitioning methods are unsatisfactory for data-parallel GNN training on GPUs. In this work, we rethink the graph data placement problem for large-scale GNN training on multiple GPUs. We find that loading input features is a performance bottleneck for GNN training on large graphs that cannot be stored on GPU. To reduce the data loading overhead, we first propose a performance model of data movement among CPU and GPUs in GNN training. Then, based on the performance model, we provide an efficient algorithm to divide and distribute the graph data onto multiple GPUs so that the data loading time is minimized. For cases where data placement alone cannot achieve good performance, we propose a locality-aware neighbor sampling technique to further reduce the data movement overhead without losing accuracy. Our experiments with graphs of different sizes on different numbers of GPUs show that our techniques not only achieve smaller data loading time but also incur much less preprocessing overhead than the existing graph partitioning methods.