Graph Neural Network Inference on Multi-FPGA Clusters

神经网络已经被广泛应用于在各种领域的任务中实现最先进的性能，例如在图像分类、机器翻译和文本生成中。这类模型通常在商业上广泛使用的图形处理单元(GPU)上执行，由于其高度并行化的体系结构，与通用计算机相比性能有了很大的提高。随着尖端模型的日益复杂，由于昂贵的数据管理机制，GPU表现出了性能限制。特别是，低延时应用，如高能物理或自动驾驶车辆，表明需要定制硬件来实现亚微秒级计算。现场可编程门阵列(现场可编程门阵列)是一类集成电路，由于其可重构的结构，能够很好地满足这些要求，并已被证明实现了高达10倍的延迟和吞吐量改进，与GPU同类产品相比，功耗降低了数量级。此外，由于其可重构性，在网络实现中提供了执行细粒度优化的灵活性。近年来，图神经网络(GNN)由于其在非欧几里德数据上的分类性能而引起了人们的极大关注，例如在社会网络、药物发现和推荐系统中。考虑到GNN由于图的稀疏结构而产生的不规则的存储器访问模式，事实证明，对于GNN来说，FPGA加速特别有益。这些独特的计算要求已经在文献中的几个FPGA加速器中得到解决。尽管可重构逻辑推理具有优势，但高端现场可编程门阵列仍然受到片上资源可用性的限制。这一挑战可以通过通过高速互连连接多个设备的FPGA集群来解决。这提供了根据网络中连接的设备数量近似线性地扩展推理性能的能力。这种加速卷积神经网络(CNN)的方法已经在文献中被探索过，通过探索专用的层划分方法。尽管这种方法被证明对CNN加速是有效的，但GNN提供了一个未被探索的问题背景。GNN显示出比CNN更浅的固有结构，因为层的数量对应于特征传播所通过的邻居的数量。因此，我的研究旨在证明，在FPGA集群上进行GNN推理最大的好处是在图中进行划分，而不是在层维度上进行划分。文献中已经提出了几种图划分方法，其中一个朴素的方法是将邻接矩阵划分为规则的节点区间。或者，基于动态滑动窗口的方法考虑了图数据，从而导致更密集的分区和更高的空间局部性。在实时应用中，此预处理步骤的延迟需要与每层节点特征转换中增加的吞吐量进行权衡。对于任何给定的分区方案，分布式节点转换引擎都需要仔细考虑数据一致性，这是计算机体系结构中的一个经典问题。功能更新在具有专用存储组件的几个设备上的分布表明，需要在设备之间建立“剩余”连接，以便可以计算消息。然后，可以探索各种硬件优化来限制设备内通信的开销。总而言之，随着对高效硬件加速的需求超出了传统的GPU，现场可编程门阵列提供了一个引人注目的解决方案。然而，高端现场可编程门阵列的可扩展性挑战推动了对现场可编程门阵列集群的探索。对于GNN来说，在FPGA集群中从层分区转换为图分区的提议显示出了希望，但完善分区策略和解决数据一致性对于释放全部潜力至关重要

Neural networks have been widely deployed to achieve state-of-the-art performance in tasks within various domains, such as in image classification, machine translation, and text generation. Such models are typically executed on Graphical Processing Units (GPU), which are widely commercially available, and offer large performance improvements over general-purpose computers due to their deeply parallelized architecture.With increasing complexity in cutting edge models, GPUs have shown a performance limitation due to expensive data management mechanisms. In particular, low-latency applications such as in high-energy physics or autonomous vehicles show the need for custom hardware to achieve sub-microsecond computation. Field-Programmable Gate Arrays (FPGA) are a class of integrated circuit which are well capable of meeting these requirements due to their reconfigurable fabric, and have been shown to achieve up to 10x latency and throughput improvements over GPU counterparts, with orders of magnitude lower power consumption. Additionally, FPGAs provide the flexibility to perform fine-grained optimizations in the network implementation, due to their reconfigurability.In recent times, Graph Neural Networks (GNNs) have attracted great attention due to their classification performance on non-Euclidean data, such as in social networks, drug discovery and recommendation systems. FPGA acceleration proves particularly beneficial for GNNs given their irregular memory access patterns, resulting from the sparse structure of graphs. These unique compute requirements have been addressed by several FPGA accelerators in the literature. Despite the benefits of inference on reconfigurable logic, high-end FPGAs are still limited by resource availability on-chip. This challenge can be addressed by FPGA clusters connecting multiple devices through high-speed interconnects. This offers the ability to scale inference performance approximately linearly with the number of devices connected in the network. This approach has been explored in the literature to accelerate Convolutional Neural Networks (CNN), through an exploration of dedicated layer partitioning approaches.Although this method has proved effective for CNN acceleration, GNNs offer an unexplored problem setting. GNNs have shown an inherently shallower structure than CNNs since the number of layers corresponds to the number of neighbours through which features propagate. As such, my research aims to demonstrate that GNN inference on FPGA clusters benefits most from partitioning in the graph rather than layer dimension.Several graph partitioning approaches have been proposed in the literature; a naïve approach involves splitting the adjacency matrix into regular node intervals. Alternatively, dynamic sliding-window based approaches consider the graph data, leading to denser partitions and higher spatial locality. In real-time applications, the latency of this pre-processing step needs to be traded-off against the added throughput in node feature transformations per layer. With any given partitioning scheme, a distributed node transformation engine requires careful consideration of data coherency, a classic problem in computer architecture. The distribution of feature updates across several devices with dedicated memory components shows the need for "residual" connections between devices such that messages can be computed. Various hardware optimisations could then be explored to limit the overhead of intra-device communication.In conclusion, as the demand for efficient hardware acceleration grows beyond traditional GPUs, FPGAs present a compelling solution. However, scalability challenges in high-end FPGAs prompt the exploration of FPGA clusters. For GNNs, the proposal to shift from layer to graph partitioning in FPGA clusters shows promise, but refining partitioning strategies and addressing data coherency are critical for unlocking the full potential