Adversarial Caching Training: Unsupervised Inductive Network Representation Learning on Large-Scale Graphs
Adversarial Caching Training: Unsupervised Inductive Network Representation Learning on Large-Scale Graphs
复制标题
对抗性缓存训练：大规模图上的无监督归纳网络表示学习
DOI：
10.1109/tnnls.2021.3084195
复制
发表时间：
2021-06-09
影响因子：
10.4
通讯作者：
Liu, Weiwen
中科院分区：
文献类型：
--
作者：
Chen, Junyang;Gong, Zhiguo;Liu, Weiwen
网络表示学习（NRL）对数据挖掘研究具有深远的影响，显示出其在许多实际应用中的重要性。 NRL，也称为网络嵌入，旨在在低维空间中保留图结构。这些学习到的表示可用于后续的机器学习任务，例如顶点分类、链接预测和数据可视化。最近，基于图卷积网络（GCN）的模型，例如 GraphSAGE，因其在归纳 NRL 方面的成功而引起了广泛关注。在大规模图上进行无监督学习时，其中一些模型采用负采样（NS）进行优化，这会鼓励目标顶点靠近其邻居，同时远离其负样本。然而，NS 通过随机模式或基于顶点的度数来绘制负顶点。因此，生成的样本可能与目标顶点高度相关或完全无关。而且，随着训练的进行，用不相关的负样本与目标顶点的内积计算出的NS目标的梯度可能会变为零，这将导致学习较差的表示。为了解决这些问题，我们提出了一种针对大型网络上的无监督归纳 NRL 量身定制的对抗性训练方法。为了有效地跟踪高质量的负样本，我们设计了一种具有采样和更新策略的缓存方案，该方案在考虑训练成本的同时对顶点邻近度进行了广泛的探索。此外，所提出的方法适用于各种现有的基于 GCN 的模型，而不会显着使其优化过程复杂化。大量的实验表明，与最先进的模型相比，我们提出的方法可以实现更好的性能。
Network representation learning (NRL) has far-reaching effects on data mining research, showing its importance in many real-world applications. NRL, also known as network embedding, aims at preserving graph structures in a low-dimensional space. These learned representations can be used for subsequent machine learning tasks, such as vertex classification, link prediction, and data visualization. Recently, graph convolutional network (GCN)-based models, e.g., GraphSAGE, have drawn a lot of attention for their success in inductive NRL. When conducting unsupervised learning on large-scale graphs, some of these models employ negative sampling (NS) for optimization, which encourages a target vertex to be close to its neighbors while being far from its negative samples. However, NS draws negative vertices through a random pattern or based on the degrees of vertices. Thus, the generated samples could be either highly relevant or completely unrelated to the target vertex. Moreover, as the training goes, the gradient of NS objective calculated with the inner product of the unrelated negative samples and the target vertex may become zero, which will lead to learning inferior representations. To address these problems, we propose an adversarial training method tailored for unsupervised inductive NRL on large networks. For efficiently keeping track of high-quality negative samples, we design a caching scheme with sampling and updating strategies that has a wide exploration of vertex proximity while considering training costs. Besides, the proposed method is adaptive to various existing GCN-based models without significantly complicating their optimization process. Extensive experiments show that our proposed method can achieve better performance compared with the state-of-the-art models.