NeutronStar: Distributed GNN Training with Hybrid Dependency Management
NeutronStar: Distributed GNN Training with Hybrid Dependency Management
复制标题
DOI：
10.1145/3514221.3526134
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
Qiange Wang;Yanfeng Zhang;Hao Wang;Chao Chen;Xiaodong Zhang;Geoffrey X. Yu
中科院分区：
文献类型：
--
作者：
Qiange Wang;Yanfeng Zhang;Hao Wang;Chao Chen;Xiaodong Zhang;Geoffrey X. Yu
GNN的训练需要解决顶点依赖性的问题，即，每个顶点表示的更新取决于它的邻居。现有的分布式GNN系统采用依赖缓存的方法或依赖通信的方法。经过深入的实验和分析，我们发现选择一种或另一种方法以获得最佳性能的决定是由一组因素决定的，包括图形输入，模型配置和底层计算集群环境。如果各种GNN训练仅由一种方法支持，则性能结果通常是次优的。我们在执行之前研究每个GNN训练的相关因素，以选择最适合的方法。我们提出了一种混合依赖处理方法，自适应地在运行时采取的两种方法的优点。基于混合方法，我们进一步开发了一个分布式GNN训练系统NeutronStar，它以自动的方式进行高性能的GNN训练。NeutronStar还在CPU-GPU计算和数据处理方面进行了有效的优化。我们在16节点阿里云集群上的实验结果表明，NeutronStar比现有的GNN系统（包括DistDGL和ROC）实现了1.81 - 14.25倍的加速比。
GNN's training needs to resolve issues of vertex dependencies, i.e., each vertex representation's update depends on its neighbors. Existing distributed GNN systems adopt either a dependencies-cached approach or a dependencies-communicated approach. Having made intensive experiments and analysis, we find that a decision to choose one or the other approach for the best performance is determined by a set of factors, including graph inputs, model configurations, and an underlying computing cluster environment. If various GNN trainings are supported solely by one approach, the performance results are often suboptimal. We study related factors for each GNN training before its execution to choose the best-fit approach accordingly. We propose a hybrid dependency-handling approach that adaptively takes the merits of the two approaches at runtime. Based on the hybrid approach, we further develop a distributed GNN training system called NeutronStar, which makes high performance GNN trainings in an automatic way. NeutronStar is also empowered by effective optimizations in CPU-GPU computation and data processing. Our experimental results on 16-node Aliyun cluster demonstrate that NeutronStar achieves 1.81X-14.25X speedup over existing GNN systems including DistDGL and ROC.