Multi-hop Attention Graph Neural Networks
Multi-hop Attention Graph Neural Networks
复制标题
DOI：
10.24963/ijcai.2021/425
复制
发表时间：
2020-09
期刊：
影响因子：
--
通讯作者：
Guangtao Wang;Rex Ying;Jing Huang;J. Leskovec
中科院分区：
文献类型：
--
作者：
Guangtao Wang;Rex Ying;Jing Huang;J. Leskovec
图神经网络（GNN）中的自注意力机制在许多图表示学习任务上取得了最先进的性能。目前，在每一层，注意力都是在连接的节点对之间计算的，并且仅取决于两个节点的表示。然而，这种注意力机制并没有考虑到那些不直接连接但提供重要网络上下文的节点。在这里，我们提出了多跳注意力图神经网络（MAGNA），这是一种将多跳上下文信息合并到注意力计算的每一层中的原理方法。 MAGNA 将注意力分数分散到整个网络中，从而增加了 GNN 每层的感受野。与以前的方法不同，MAGNA 在注意力值上使用扩散先验，以有效地解释一对断开连接的节点之间的所有路径。我们在理论和实验中证明，MAGNA 可以捕获每一层的大规模结构信息，并具有低通效应，可以消除图数据中的噪声高频信息。节点分类以及知识图谱完成基准测试的实验结果表明，MAGNA 取得了最先进的结果：与之前在 Cora、Citeseer 和 Pubmed 上最先进的技术相比，MAGNA 的相对误差降低了高达 5.7%。 MAGNA 还在大规模 Open Graph Benchmark 数据集上获得了最佳性能。在知识图完成方面，MAGNA 在四种不同的性能指标上推进了 WN18RR 和 FB15k-237 的最先进技术。
Self-attention mechanism in graph neural networks (GNNs) led to state-of-the-art performance on many graph representation learning tasks. Currently, at every layer, attention is computed between connected pairs of nodes and depends solely on the representation of the two nodes. However, such attention mechanism does not account for nodes that are not directly connected but provide important network context. Here we propose Multi-hop Attention Graph Neural Network (MAGNA), a principled way to incorporate multi-hop context information into every layer of attention computation. MAGNA diffuses the attention scores across the network, which increases the receptive field for every layer of the GNN. Unlike previous approaches, MAGNA uses a diffusion prior on attention values, to efficiently account for all paths between the pair of disconnected nodes. We demonstrate in theory and experiments that MAGNA captures large-scale structural information in every layer, and has a low-pass effect that eliminates noisy high-frequency information from graph data. Experimental results on node classification as well as the knowledge graph completion benchmarks show that MAGNA achieves state-of-the-art results: MAGNA achieves up to 5.7% relative error reduction over the previous state-of-the-art on Cora, Citeseer, and Pubmed. MAGNA also obtains the best performance on a large-scale Open Graph Benchmark dataset. On knowledge graph completion MAGNA advances state-of-the-art on WN18RR and FB15k-237 across four different performance metrics.