Low-latency Mini-batch GNN Inference on CPU-FPGA Heterogeneous Platform
Low-latency Mini-batch GNN Inference on CPU-FPGA Heterogeneous Platform
复制标题
DOI：
10.1109/hipc56025.2022.00015
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
Bingyi Zhang;Hanqing Zeng;V. Prasanna
中科院分区：
文献类型：
--
作者：
Bingyi Zhang;Hanqing Zeng;V. Prasanna
图神经网络的小批量推理是许多实际应用中的关键问题。在本文中，我们开发了一个计算效率高的映射GNN到CPU-FPGA异构平台，以实现低延迟的小批量推理。虽然GNN的轻量级预处理算法可以有效地映射到CPU平台上，但在FPGA平台上，我们设计了一种具有自适应数据路径的新型GNN硬件加速器，表示为自适应计算内核（ACK），可以低延迟执行GNN的各种计算内核：（1）对于表示为矩阵乘法的密集计算内核，ACK作为具有完全局部连接的脉动阵列工作，（2）对于稀疏计算内核，ACK遵循分散-聚集范式，并作为多个并行管道来支持图的不规则连接。所提出的任务调度隐藏了CPU-FPGA的数据通信开销，以减少推理延迟。我们开发了一个快速的设计空间探索算法，为多个目标GNN模型生成一个加速器。我们实现了我们的加速器在一个国家的最先进的CPU-FPGA平台，并使用三个代表性的模型（GCN，GraphSAGE，GAT）的性能进行评估。结果表明，与仅CPU、CPU-GPU和CPU-FPGA平台上的最新实现相比，我们的CPU-FPGA实现了21.4−50.8×、2.9 − 21.6×和4.7×的延迟减少。
Mini-batch inference of Graph Neural Networks (GNNs) is a key problem in many real-world applications. In this paper, we develop a computationally efficient mapping of GNNs onto CPU-FPGA heterogeneous platforms to achieve low-latency mini-batch inference. While the lightweight preprocessing algorithm of GNNs can be efficiently mapped onto the CPU platform, on the FPGA platform, we design a novel GNN hardware accelerator with an adaptive datapath denoted as Adaptive Computation Kernel (ACK) that can execute various computation kernels of GNNs with low-latency: (1) for dense computation kernels expressed as matrix multiplication, ACK works as a systolic array with fully localized connections, (2) for sparse computation kernels, ACK follows the scatter-gather paradigm and works as multiple parallel pipelines to support the irregular connectivity of graphs. The proposed task scheduling hides the CPU-FPGA data communication overhead to reduce the inference latency. We develop a fast design space exploration algorithm to generate a single accelerator for multiple target GNN models. We implement our accelerator on a state-of-the-art CPU-FPGA platform and evaluate the performance using three representative models (GCN, GraphSAGE, GAT). Results show that our CPU-FPGA implementation achieves 21.4−50.8×, 2.9 − 21.6×, 4.7× latency reduction compared with state-of-the-art implementations on CPU-only, CPU-GPU and CPU-FPGA platforms.