Graph Neural Network Inference on Multi-FPGA Clusters

点击翻译按钮获取中文摘要

Neural networks have been widely deployed to achieve state-of-the-art performance in tasks within various domains, such as in image classification, machine translation, and text generation. Such models are typically executed on Graphical Processing Units (GPU), which are widely commercially available, and offer large performance improvements over general-purpose computers due to their deeply parallelized architecture.With increasing complexity in cutting edge models, GPUs have shown a performance limitation due to expensive data management mechanisms. In particular, low-latency applications such as in high-energy physics or autonomous vehicles show the need for custom hardware to achieve sub-microsecond computation. Field-Programmable Gate Arrays (FPGA) are a class of integrated circuit which are well capable of meeting these requirements due to their reconfigurable fabric, and have been shown to achieve up to 10x latency and throughput improvements over GPU counterparts, with orders of magnitude lower power consumption. Additionally, FPGAs provide the flexibility to perform fine-grained optimizations in the network implementation, due to their reconfigurability.In recent times, Graph Neural Networks (GNNs) have attracted great attention due to their classification performance on non-Euclidean data, such as in social networks, drug discovery and recommendation systems. FPGA acceleration proves particularly beneficial for GNNs given their irregular memory access patterns, resulting from the sparse structure of graphs. These unique compute requirements have been addressed by several FPGA accelerators in the literature. Despite the benefits of inference on reconfigurable logic, high-end FPGAs are still limited by resource availability on-chip. This challenge can be addressed by FPGA clusters connecting multiple devices through high-speed interconnects. This offers the ability to scale inference performance approximately linearly with the number of devices connected in the network. This approach has been explored in the literature to accelerate Convolutional Neural Networks (CNN), through an exploration of dedicated layer partitioning approaches.Although this method has proved effective for CNN acceleration, GNNs offer an unexplored problem setting. GNNs have shown an inherently shallower structure than CNNs since the number of layers corresponds to the number of neighbours through which features propagate. As such, my research aims to demonstrate that GNN inference on FPGA clusters benefits most from partitioning in the graph rather than layer dimension.Several graph partitioning approaches have been proposed in the literature; a naïve approach involves splitting the adjacency matrix into regular node intervals. Alternatively, dynamic sliding-window based approaches consider the graph data, leading to denser partitions and higher spatial locality. In real-time applications, the latency of this pre-processing step needs to be traded-off against the added throughput in node feature transformations per layer. With any given partitioning scheme, a distributed node transformation engine requires careful consideration of data coherency, a classic problem in computer architecture. The distribution of feature updates across several devices with dedicated memory components shows the need for "residual" connections between devices such that messages can be computed. Various hardware optimisations could then be explored to limit the overhead of intra-device communication.In conclusion, as the demand for efficient hardware acceleration grows beyond traditional GPUs, FPGAs present a compelling solution. However, scalability challenges in high-end FPGAs prompt the exploration of FPGA clusters. For GNNs, the proposal to shift from layer to graph partitioning in FPGA clusters shows promise, but refining partitioning strategies and addressing data coherency are critical for unlocking the full potential