Graph Neural Network Inference on Multi-FPGA Clusters

神经网络已被广泛部署，以在各种领域的任务中实现最先进的性能，例如图像分类，机器翻译和文本生成。这些模型通常在图形处理单元（GPU）上执行，GPU在商业上广泛使用，并且由于其深度并行化的架构，其性能比通用计算机有很大的提高。随着尖端模型复杂性的增加，GPU由于昂贵的数据管理机制而表现出性能限制。特别是，高能物理或自动驾驶汽车等低延迟应用需要定制硬件来实现亚微秒计算。现场可编程门阵列（FPGA）是一类集成电路，由于其可重新配置的结构，能够很好地满足这些要求，并且已被证明与GPU同行相比，可以实现高达10倍的延迟和吞吐量改进，功耗降低了几个数量级。此外，FPGA还具有可重构性，可以灵活地在网络实现中执行细粒度优化。近年来，图神经网络（GNN）因其在非欧几里德数据上的分类性能而引起了极大的关注，例如在社交网络、药物发现和推荐系统中。FPGA加速被证明对GNN特别有益，因为它们的不规则内存访问模式是由图的稀疏结构引起的。这些独特的计算要求已经解决了几个FPGA加速器在文献中。尽管可重构逻辑推理的好处，高端FPGA仍然受到片上资源可用性的限制。通过高速互连连接多个器件的FPGA集群可以解决这一挑战。这提供了随着网络中连接的设备数量近似线性地扩展推理性能的能力。这种方法已经在文献中进行了探索，通过探索专用的层划分方法来加速卷积神经网络（CNN）。尽管这种方法已被证明对CNN加速有效，但GNNs提供了一个未探索的问题设置。GNN已经显示出比CNN更浅的固有结构，因为层数对应于特征传播所通过的邻居的数量。因此，我的研究旨在证明FPGA集群上的GNN推理最受益于图中的分区，而不是层维度。文献中已经提出了几种图分区方法;一种简单的方法涉及将邻接矩阵拆分为规则的节点间隔。或者，基于动态滑动窗口的方法考虑图形数据，导致更密集的分区和更高的空间局部性。在实时应用中，需要权衡此预处理步骤的延迟与每层节点特征转换中增加的吞吐量。对于任何给定的分区方案，分布式节点转换引擎都需要仔细考虑数据一致性，这是计算机体系结构中的一个经典问题。跨具有专用存储器组件的多个设备的特征更新的分布显示了设备之间的“剩余”连接的需要，使得可以计算消息。总之，随着对高效硬件加速需求的增长超出了传统GPU的范围，FPGA提供了一个引人注目的解决方案。然而，高端FPGA的可扩展性挑战促使FPGA集群的探索。对于GNN，在FPGA集群中从层分区转向图分区的提议显示出了希望，但细化分区策略和解决数据一致性对于释放全部潜力至关重要

Neural networks have been widely deployed to achieve state-of-the-art performance in tasks within various domains, such as in image classification, machine translation, and text generation. Such models are typically executed on Graphical Processing Units (GPU), which are widely commercially available, and offer large performance improvements over general-purpose computers due to their deeply parallelized architecture.With increasing complexity in cutting edge models, GPUs have shown a performance limitation due to expensive data management mechanisms. In particular, low-latency applications such as in high-energy physics or autonomous vehicles show the need for custom hardware to achieve sub-microsecond computation. Field-Programmable Gate Arrays (FPGA) are a class of integrated circuit which are well capable of meeting these requirements due to their reconfigurable fabric, and have been shown to achieve up to 10x latency and throughput improvements over GPU counterparts, with orders of magnitude lower power consumption. Additionally, FPGAs provide the flexibility to perform fine-grained optimizations in the network implementation, due to their reconfigurability.In recent times, Graph Neural Networks (GNNs) have attracted great attention due to their classification performance on non-Euclidean data, such as in social networks, drug discovery and recommendation systems. FPGA acceleration proves particularly beneficial for GNNs given their irregular memory access patterns, resulting from the sparse structure of graphs. These unique compute requirements have been addressed by several FPGA accelerators in the literature. Despite the benefits of inference on reconfigurable logic, high-end FPGAs are still limited by resource availability on-chip. This challenge can be addressed by FPGA clusters connecting multiple devices through high-speed interconnects. This offers the ability to scale inference performance approximately linearly with the number of devices connected in the network. This approach has been explored in the literature to accelerate Convolutional Neural Networks (CNN), through an exploration of dedicated layer partitioning approaches.Although this method has proved effective for CNN acceleration, GNNs offer an unexplored problem setting. GNNs have shown an inherently shallower structure than CNNs since the number of layers corresponds to the number of neighbours through which features propagate. As such, my research aims to demonstrate that GNN inference on FPGA clusters benefits most from partitioning in the graph rather than layer dimension.Several graph partitioning approaches have been proposed in the literature; a naïve approach involves splitting the adjacency matrix into regular node intervals. Alternatively, dynamic sliding-window based approaches consider the graph data, leading to denser partitions and higher spatial locality. In real-time applications, the latency of this pre-processing step needs to be traded-off against the added throughput in node feature transformations per layer. With any given partitioning scheme, a distributed node transformation engine requires careful consideration of data coherency, a classic problem in computer architecture. The distribution of feature updates across several devices with dedicated memory components shows the need for "residual" connections between devices such that messages can be computed. Various hardware optimisations could then be explored to limit the overhead of intra-device communication.In conclusion, as the demand for efficient hardware acceleration grows beyond traditional GPUs, FPGAs present a compelling solution. However, scalability challenges in high-end FPGAs prompt the exploration of FPGA clusters. For GNNs, the proposal to shift from layer to graph partitioning in FPGA clusters shows promise, but refining partitioning strategies and addressing data coherency are critical for unlocking the full potential