TLPGNN: A Lightweight Two-Level Parallelism Paradigm for Graph Neural Network Computation on GPU
TLPGNN: A Lightweight Two-Level Parallelism Paradigm for Graph Neural Network Computation on GPU
复制标题
DOI：
10.1145/3502181.3531467
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
Qiang Fu;Yuede Ji;Huimin Huang
中科院分区：
文献类型：
--
作者：
Qiang Fu;Yuede Ji;Huimin Huang
图神经网络（gnn）是一类新兴的基于图的深度学习模型，有许多成功的应用，如推荐系统、药物发现和社会网络分析。GNN的计算既包括正则神经网络运算，也包括一般的图卷积运算，这两种运算占总计算时间的大部分。虽然最近已经提出了一些加速gnn计算的工作，但它们面临着繁重的预处理、低效率的原子操作和不必要的内核启动的限制。本文设计了用于GNN计算的轻量级两级并行范式TLPGNN。首先，系统分析了GNN工作负载的硬件资源使用情况，深入了解了GNN工作负载的特点。根据这些深刻的观察，我们将GNN计算分为两个层次，即第一级的顶点并行性和第二级的特征等位性。其次，我们采用了一种新的混合动态工作负载分配方法来解决工作负载分配不平衡的问题。此外，我们融合内核以减少内核启动次数，并将频繁访问的数据缓存到寄存器中以避免不必要的内存流量。总之，TLPGNN能够显著优于现有的GNN计算系统，如DGL、GNNAdivsor和FeatGraph，平均分别高出5.6倍、7.7倍和3.3倍。
Graph Neural Networks (GNNs) are an emerging class of deep learning models on graphs, with many successful applications, such as, recommendation systems, drug discovery, and social network analysis. The GNN computation includes both regular neural network operations and general graph convolution operations, which take the majority of the total computation time. Though several recent works have been proposed to accelerate the computation for GNNs, they face the limitations of heavy pre-processing, low efficient atomic operations, and unnecessary kernel launches. In this paper, we design TLPGNN, a lightweight two-level parallelism paradigm for GNN computation. First, we conduct a systematic analysis on the hardware resource usage of GNN workloads to deeply understand the specialties of GNN workloads. With the insightful observations, we then divide the GNN computation into two levels, i.e., vertex parallelism for the first level and feature par- allelism for the second. Next, we employ a novel hybrid dynamic workload assignment to address the imbalanced workload distribution. Furthermore, we fuse the kernels to reduce the number of kernel launches and cache the frequently accessed data into registers to avoid unnecessary memory traffics. Together, TLPGNN is able to significantly outperform existing GNN computation systems, such as DGL, GNNAdivsor, and FeatGraph, by 5.6×, 7.7×, and 3.3×, respectively, on the average.