Multi-Level Attention Pooling for Graph Neural Networks: Unifying Graph Representations with Multiple Localities
Multi-Level Attention Pooling for Graph Neural Networks: Unifying Graph Representations with Multiple Localities
复制标题
DOI：
10.1016/j.neunet.2021.11.001
复制
发表时间：
2021-03
期刊：
影响因子：
--
通讯作者：
Takeshi D. Itoh;Takatomi Kubo;K. Ikeda
中科院分区：
文献类型：
--
作者：
Takeshi D. Itoh;Takatomi Kubo;K. Ikeda
图神经网络（GNN）已被广泛用于学习图结构数据的向量表示，并取得了比传统方法更好的任务性能。GNN的基础是消息传递过程，它将节点中的信息传播到其邻居。由于该过程每层进行一个步骤，所以节点之间的信息传播的范围在较低层中较小，并且其向较高层扩展。因此，GNN模型必须足够深入，以捕获图中的全局结构信息。另一方面，众所周知，深度GNN模型会遭受性能下降，因为它们通过许多消息传递步骤丢失了节点的本地信息，这对于良好的模型性能至关重要。在这项研究中，我们提出了多层次的注意力池（MPEG4）的图级分类任务，它可以适应在一个图的局部和全局结构信息。它为每个消息传递步骤提供了一个注意力池层，并通过统一逐层图形表示来计算最终的图形表示。MPEG4架构允许模型利用具有多个位置级别的图的结构信息，因为它在由于过度平滑而丢失它们之前保留了分层信息。我们的实验结果表明，与基线架构相比，该架构提高了图分类性能。此外，对逐层图形表示的分析表明，从多个级别的地方聚集信息确实有可能提高学习图形表示的可辨别性。
Graph neural networks (GNNs) have been widely used to learn vector representation of graph-structured data and achieved better task performance than conventional methods. The foundation of GNNs is the message passing procedure, which propagates the information in a node to its neighbors. Since this procedure proceeds one step per layer, the range of the information propagation among nodes is small in the lower layers, and it expands toward the higher layers. Therefore, a GNN model has to be deep enough to capture global structural information in a graph. On the other hand, it is known that deep GNN models suffer from performance degradation because they lose nodes’ local information, which would be essential for good model performance, through many message passing steps. In this study, we propose multi-level attention pooling (MLAP) for graph-level classification tasks, which can adapt to both local and global structural information in a graph. It has an attention pooling layer for each message passing step and computes the final graph representation by unifying the layer-wise graph representations. The MLAP architecture allows models to utilize the structural information of graphs with multiple levels of localities because it preserves layer-wise information before losing them due to oversmoothing. Results of our experiments show that the MLAP architecture improves the graph classification performance compared to the baseline architectures. In addition, analyses on the layer-wise graph representations suggest that aggregating information from multiple levels of localities indeed has the potential to improve the discriminability of learned graph representations.