Not Half Bad: Exploring Half-Precision in Graph Convolutional Neural Networks
Not Half Bad: Exploring Half-Precision in Graph Convolutional Neural Networks
复制标题
DOI：
10.1109/bigdata50022.2020.9378263
复制
发表时间：
2020-10
期刊：
影响因子：
--
通讯作者：
John Brennan;Stephen Bonner;Amir Atapour-Abarghouei;Philip T. G. Jackson;B. Obara;A. Mcgough
中科院分区：
文献类型：
--
作者：
John Brennan;Stephen Bonner;Amir Atapour-Abarghouei;Philip T. G. Jackson;B. Obara;A. Mcgough
随着图形在众多应用中作为数据的有效表示的重要性日益增加，使用现代机器学习的高效图形分析正在受到越来越多的关注。深度学习方法通常在整个邻接矩阵上运行-因为输入和中间网络层都是按邻接矩阵的大小成比例设计的-随着图形大小的增加，导致密集的计算和大量的内存需求。因此，希望确定有效的措施，以减少运行时间和内存的要求，允许分析最大的图形可能。在深度神经网络的向前和向后传递中使用降低精度的操作，沿着现代GPU中的新型专用硬件，可以为提高效率提供有希望的途径。在本文中，我们深入探讨了降低精度操作的使用，可以轻松集成到非常流行的PyTorch框架中，并分析了Tensor Cores对图卷积神经网络的影响。我们使用知名的基准测试和综合生成的数据集对三种GPU架构和两种广泛使用的图分析任务（顶点分类和链接预测）进行了广泛的实验评估。从而使我们能够对降低精度的操作和张量核心对图卷积神经网络的计算和内存使用的影响进行重要的观察-这在文献中经常被忽视。
With the growing significance of graphs as an effective representation of data in numerous applications, efficient graph analysis using modern machine learning is receiving a growing level of attention. Deep learning approaches often operate over the entire adjacency matrix – as the input and intermediate network layers are all designed in proportion to the size of the adjacency matrix – leading to intensive computation and large memory requirements as the graph size increases. It is therefore desirable to identify efficient measures to reduce both run-time and memory requirements allowing for the analysis of the largest graphs possible. The use of reduced precision operations within the forward and backward passes of a deep neural network along with novel specialised hardware in modern GPUs could offer promising avenues towards efficiency. In this paper, we provide an in-depth exploration of the use of reduced-precision operations, easily integrable into the highly popular PyTorch framework, and an analysis of the effects of Tensor Cores on graph convolutional neural networks. We perform an extensive experimental evaluation of three GPU architectures and two widely-used graph analysis tasks (vertex classification and link prediction) using well-known benchmark and synthetically generated datasets. Thus allowing us to make important observations on the effects of reduced-precision operations and Tensor Cores on computational and memory usage of graph convolutional neural networks – often neglected in the literature.