Spectral Augmentation for Self-Supervised Learning on Graphs
Spectral Augmentation for Self-Supervised Learning on Graphs
复制标题
DOI：
10.48550/arxiv.2210.00643
复制
发表时间：
2022-10
期刊：
影响因子：
--
通讯作者：
Lu Lin;Jinghui Chen;Hongning Wang
中科院分区：
文献类型：
--
作者：
Lu Lin;Jinghui Chen;Hongning Wang
图对比学习（GCL）作为一种新兴的图自监督学习技术，旨在通过实例辨别来学习表示。其性能在很大程度上依赖于图增强来反映对小扰动具有鲁棒性的不变模式；然而，目前还不清楚 GCL 应该捕获什么图不变性。最近的研究主要在空间域中以均匀随机的方式进行拓扑增强，忽略了其对嵌入谱域的内在结构特性的影响。在这项工作中，我们的目标是通过从谱角度探索图的不变性来找到拓扑增强的原则方法。我们开发了光谱增强，它通过最大化光谱变化来指导拓扑增强。对图和节点分类任务的广泛实验证明了我们的方法在自监督表示学习中的有效性。该方法还为迁移学习带来了良好的泛化能力，并在对抗性攻击下具有令人感兴趣的鲁棒性。我们的研究揭示了图拓扑增强的一般原理。
Graph contrastive learning (GCL), as an emerging self-supervised learning technique on graphs, aims to learn representations via instance discrimination. Its performance heavily relies on graph augmentation to reflect invariant patterns that are robust to small perturbations; yet it still remains unclear about what graph invariance GCL should capture. Recent studies mainly perform topology augmentations in a uniformly random manner in the spatial domain, ignoring its influence on the intrinsic structural properties embedded in the spectral domain. In this work, we aim to find a principled way for topology augmentations by exploring the invariance of graphs from the spectral perspective. We develop spectral augmentation which guides topology augmentations by maximizing the spectral change. Extensive experiments on both graph and node classification tasks demonstrate the effectiveness of our method in self-supervised representation learning. The proposed method also brings promising generalization capability in transfer learning, and is equipped with intriguing robustness property under adversarial attacks. Our study sheds light on a general principle for graph topology augmentation.