Transfer Learning of Graph Neural Networks with Ego-graph Information Maximization
Transfer Learning of Graph Neural Networks with Ego-graph Information Maximization
复制标题
DOI：
--
复制
发表时间：
2020-09
期刊：
影响因子：
--
通讯作者：
Qi Zhu;Yidan Xu;Haonan Wang-;Chao Zhang;Jiawei Han;Carl Yang
中科院分区：
文献类型：
--
作者：
Qi Zhu;Yidan Xu;Haonan Wang-;Chao Zhang;Jiawei Han;Carl Yang
图形神经网络（GNN）在各种应用中都具有出色的性能，但是对于大规模图，培训专用的GNN可能是昂贵的。最近的一些工作开始研究GNN的预培训。但是，它们都没有提供有关其框架设计的理论见解，或者对GNN的可转让性的明确要求和保证。在这项工作中，我们为GNN的转移学习建立了一个理论上扎根且实际上有用的框架。首先，我们对基本图表信息提出了一种新颖的看法，并主张将其作为可转让GNN培训的目标，这激发了我们的设计，这是一个基于Ego-Graph信息最大化的新型GNN框架，以分析实现这一目标。其次，我们将探索结构节点特征的要求作为GNN输入，并根据源和目标图的本地图Laplacians之间的差异得出了GNN可传递性的严格界限。最后，我们进行受控的合成实验，以直接证明我们的理论结论是合理的。在现实世界网络上进行角色识别的广泛实验在严格分析的直接转移设置中显示出一致的结果，而朝向大规模关系预测的实验表现出了有希望的结果，在更概括的实用环境中通过微调转移。
Graph neural networks (GNNs) have been shown with superior performance in various applications, but training dedicated GNNs can be costly for large-scale graphs. Some recent work started to study the pre-training of GNNs. However, none of them provide theoretical insights into the design of their frameworks, or clear requirements and guarantees towards the transferability of GNNs. In this work, we establish a theoretically grounded and practically useful framework for the transfer learning of GNNs. Firstly, we propose a novel view towards the essential graph information and advocate the capturing of it as the goal of transferable GNN training, which motivates the design of Ours, a novel GNN framework based on ego-graph information maximization to analytically achieve this goal. Secondly, we specify the requirement of structure-respecting node features as the GNN input, and derive a rigorous bound of GNN transferability based on the difference between the local graph Laplacians of the source and target graphs. Finally, we conduct controlled synthetic experiments to directly justify our theoretical conclusions. Extensive experiments on real-world networks towards role identification show consistent results in the rigorously analyzed setting of direct-transfering, while those towards large-scale relation prediction show promising results in the more generalized and practical setting of transfering with fine-tuning.