Self-Supervised Graph Structure Refinement for Graph Neural Networks
Self-Supervised Graph Structure Refinement for Graph Neural Networks
复制标题
DOI：
10.1145/3539597.3570455
复制
发表时间：
2022-11
期刊：
影响因子：
--
通讯作者：
Jianan Zhao;Qianlong Wen;Mingxuan Ju;Chuxu Zhang;Yanfang Ye
中科院分区：
文献类型：
--
作者：
Jianan Zhao;Qianlong Wen;Mingxuan Ju;Chuxu Zhang;Yanfang Ye
图结构学习（GSL），旨在学习图神经网络（GNNs）的邻接矩阵，在提高GNNs的性能方面显示出巨大的潜力。大多数现有的GSL作品应用联合学习框架，其中估计的邻接矩阵和GNN参数针对下游任务进行优化。然而，由于GSL本质上是一个链接预测任务，其目标可能与下游任务的目标有很大不同。这两个目标的不一致性限制了GSL方法学习潜在的最优图结构。此外，在邻接矩阵的估计和优化过程中，联合学习框架在时间和空间方面存在可扩展性问题。为了缓解这些问题，我们提出了一个具有预训练-微调管道的图结构细化（GSR）框架。具体来说，预训练阶段旨在通过具有视图内和视图间链接预测任务的多视图对比学习框架来全面估计底层图结构。然后，通过根据由预训练模型估计的边缘概率添加和移除边缘来细化图结构。最后，微调GNN由预训练的模型初始化，并针对下游任务进行优化。由于改进后的图结构在微调空间中保持静态，GSR避免了在微调阶段估计和优化图结构，具有很好的可扩展性和效率。此外，通过迁移知识和细化图来促进GNN的微调。进行了大量的实验，以评估所提出的模型的有效性（在六个基准数据集上的最佳性能），效率和可扩展性（与Cora上的最佳GSL基线相比，使用32.8%GPU内存时快13.8倍）。
Graph structure learning (GSL), which aims to learn the adjacency matrix for graph neural networks (GNNs), has shown great potential in boosting the performance of GNNs. Most existing GSL works apply a joint learning framework where the estimated adjacency matrix and GNN parameters are optimized for downstream tasks. However, as GSL is essentially a link prediction task, whose goal may largely differ from the goal of the downstream task. The inconsistency of these two goals limits the GSL methods to learn the potential optimal graph structure. Moreover, the joint learning framework suffers from scalability issues in terms of time and space during the process of estimation and optimization of the adjacency matrix. To mitigate these issues, we propose a graph structure refinement (GSR) framework with a pretrain-finetune pipeline. Specifically, The pre-training phase aims to comprehensively estimate the underlying graph structure by a multi-view contrastive learning framework with both intra- and inter-view link prediction tasks. Then, the graph structure is refined by adding and removing edges according to the edge probabilities estimated by the pre-trained model. Finally, the fine-tuning GNN is initialized by the pre-trained model and optimized toward downstream tasks. With the refined graph structure remaining static in the fine-tuning space, GSR avoids estimating and optimizing graph structure in the fine-tuning phase which enjoys great scalability and efficiency. Moreover, the fine-tuning GNN is boosted by both migrating knowledge and refining graphs. Extensive experiments are conducted to evaluate the effectiveness (best performance on six benchmark datasets), efficiency, and scalability (13.8 times faster using 32.8% GPU memory compared to the best GSL baseline on Cora) of the proposed model.