Binary Graph Neural Networks
Binary Graph Neural Networks
复制标题
DOI：
10.1109/cvpr46437.2021.00937
复制
发表时间：
2020-12
期刊：
影响因子：
--
通讯作者：
Mehdi Bahri;Gaétan Bahl;S. Zafeiriou
中科院分区：
文献类型：
--
作者：
Mehdi Bahri;Gaétan Bahl;S. Zafeiriou
图神经网络（GNN）已经成为一个强大而灵活的框架，用于不规则数据的表示学习。由于它们将经典CNN在网格上的操作推广到任意拓扑结构，因此GNN也带来了许多欧几里德同行的实现挑战。模型大小、内存占用和能耗是许多实际应用中常见的问题。网络二进制化将单个位分配给参数和激活，从而大幅降低内存需求（与单精度浮点数相比高达32倍），并最大限度地发挥现代硬件上快速SIMD指令的优势，实现可测量的加速。然而，尽管在经典CNN的二值化方面有大量的工作，但在几何深度学习中，这一领域在很大程度上仍未被探索。在本文中，我们提出并评估了不同的策略，图神经网络的二值化。我们表明，通过精心设计的模型，和训练过程的控制，二进制图神经网络可以在具有挑战性的基准测试中以适中的精度进行训练。特别是，我们在汉明空间中提出了第一个动态图神经网络，能够利用二进制向量上的有效k-NN搜索来加速动态图的构建。我们进一步验证，二进制模型提供了显着的节省嵌入式设备。我们的代码在Github1上公开。
Graph Neural Networks (GNNs) have emerged as a powerful and flexible framework for representation learning on irregular data. As they generalize the operations of classical CNNs on grids to arbitrary topologies, GNNs also bring much of the implementation challenges of their Euclidean counterparts. Model size, memory footprint, and energy consumption are common concerns for many real-world applications. Network binarization allocates a single bit to parameters and activations, thus dramatically reducing the memory requirements (up to 32x compared to single-precision floating-point numbers) and maximizing the benefits of fast SIMD instructions on modern hardware for measurable speedups. However, in spite of the large body of work on binarization for classical CNNs, this area remains largely unexplored in geometric deep learning. In this paper, we present and evaluate different strategies for the binarization of graph neural networks. We show that through careful design of the models, and control of the training process, binary graph neural networks can be trained at only a moderate cost in accuracy on challenging benchmarks. In particular, we present the first dynamic graph neural network in Hamming space, able to leverage efficient k-NN search on binary vectors to speed-up the construction of the dynamic graph. We further verify that the binary models offer significant savings on embedded devices. Our code is publicly available on Github1.