DARe: DropLayer-Aware Manycore ReRAM architecture for Training Graph Neural Networks
DARe: DropLayer-Aware Manycore ReRAM architecture for Training Graph Neural Networks
复制标题
DOI：
10.1109/iccad51958.2021.9643511
复制
发表时间：
2021-11
期刊：
影响因子：
--
通讯作者：
Aqeeb Iqbal Arka;B. K. Joardar;J. Doppa;P. Pande;K. Chakrabarty
中科院分区：
文献类型：
--
作者：
Aqeeb Iqbal Arka;B. K. Joardar;J. Doppa;P. Pande;K. Chakrabarty
图神经网络（GNN）是在图上操作的深度神经网络（DNN）的变体。GNN具有DNN和图计算的属性。然而，在众核架构上训练GNN是一项具有挑战性的任务，因为它涉及到大量的通信，从而阻碍了性能。DropEdge和Dropout，我们统称为DropLayer，是可以提高GNN预测精度的正则化技术。此外，当在众核架构上实现时，DropEdge和Dropout能够减少片上流量。在本文中，我们提出了一种基于ReRAM的3D众核架构，称为DARe，专门用于加速GNN的片上训练。DARe架构的关键组件是片上网络（NoC），它减少了使用DropLayer的通信量。减少的通信量防止了通信热点，并带来了更好的性能。我们证明，DARe在执行时间方面比传统GPU高出6.7倍（平均5.6倍），而GNN训练的能效高出30倍（平均23倍）。
Graph Neural Networks (GNNs) are a variant of Deep Neural Networks (DNNs) operating on graphs. GNNs have attributes of both DNNs and graph computation. However, training GNNs on manycore architectures is a challenging task because it involves heavy communication that bottlenecks performance. DropEdge and Dropout, which we collectively refer to as DropLayer, are regularization techniques that can improve the predictive accuracy of GNNs. Moreover, when implemented on a manycore architecture, DropEdge and Dropout are capable of reducing the on-chip traffic. In this paper, we present a ReRAM-based 3D manycore architecture called DARe, tailored for accelerating on-chip training of GNNs. The key component of the DARe architecture is a Network-on-Chip (NoC) that reduces the amount of communication using DropLayer. The reduced traffic prevents communication hotspots and leads to better performance. We demonstrate that DARe outperforms conventional GPUs by up to 6.7X (5.6X on average) in terms of execution time, while being up to 30X (23X on average) more energy efficient for GNN training.