A Simple Yet Effective Layered Loss for Pre-Training of Network Embedding
A Simple Yet Effective Layered Loss for Pre-Training of Network Embedding
复制标题
用于网络嵌入预训练的简单而有效的分层损失
DOI：
10.1109/tnse.2022.3153643
复制
发表时间：
2022-05-01
影响因子：
6.6
通讯作者：
Leung, Victor C. M.
中科院分区：
文献类型：
--
作者：
Chen, Junyang;Li, Xueliang;Leung, Victor C. M.
网络嵌入预训练的目的是将未标记的节点接近性编码到一个低维空间中，在这个空间中，节点与相邻节点接近，同时远离负样本。近年来，图神经网络在节点分类和链路预测任务的半监督学习方面取得了突破性的进展。然而，由于其固有的信息聚集模式，几乎所有这些方法在未标记节点的预训练中都只能得到较差的嵌入结果。在节点消息聚合过程中，目标节点与其多跳邻居之间的边界很难区分。为了解决这个问题，我们提出了一种简单而有效的分层损失方法，将其与一个被称为LlossNet的图注意网络结合起来进行预训练。我们将目标节点与其两跳邻居的接近度视为一个单位（称为单位图），其中目标节点需要比其两跳邻居更接近其直接邻居。因此，LlossNet将能够在学习的嵌入空间中保留节点的边缘。包括分类和聚类在内的各种下游任务的实验结果证明了我们的方法在学习判别节点表示方面的有效性。
Pre-training of network embedding aims to encode unlabeled node proximity into a low-dimensional space, where nodes are close to their neighbors while being far from negative samples. In recent years, Graph Neural Networks have shown groundbreaking performance in semi-supervised learning on the node classification and link prediction tasks. However, because of their inherent information aggregation pattern, almost all these methods can only obtain inferior embedding results in the pre-training of the unlabeled nodes. The margins between a target node and its multi-hop neighbors become hard distinguishable during node message aggregation. To address this problem, we propose a simple yet effective layered loss to combine with a graph attention network, dubbed as LlossNet, for pre-training. We regard the proximity of a target node and its two-hop neighbors as a unit (called a unit graph), where a target node is needed to be more closer to its direct neighbor than its two-hop neighbors. As such, LlossNet would be able to preserve the margins of nodes in the learned embedding space. Experimental results of various downstream tasks including classification and clustering demonstrate the effectiveness of our method on learning discriminative node representations.