Investigating and Mitigating Degree-Related Biases in Graph Convoltuional Networks
Investigating and Mitigating Degree-Related Biases in Graph Convoltuional Networks
复制标题
DOI：
10.1145/3340531.3411872
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
Xianfeng Tang;Huaxiu Yao;Yiwei Sun;Yiqi Wang;Jiliang Tang;C. Aggarwal;P. Mitra;Suhang Wang
中科院分区：
文献类型：
--
作者：
Xianfeng Tang;Huaxiu Yao;Yiwei Sun;Yiqi Wang;Jiliang Tang;C. Aggarwal;P. Mitra;Suhang Wang
图卷积网络(GCNS)在图的半监督学习任务中显示出良好的效果，因此与其他方法相比具有很大的优势。尽管GCNS取得了显著的成功，但在监管不足的情况下，很难对GCNS进行培训。当标签数据有限时，对于低度节点，GCNS的性能变得不能令人满意。虽然已有的一些工作在整个模型级别上分析了GCNS的成功和失败，但在单个节点级别上分析GCNS仍然没有得到充分的探索。本文从节点度分布的角度对GCNS进行了分析。从经验观察到理论证明，我们证实了GCN偏向于度较大且准确度较高的节点，即使在大多数图中高度较高的节点没有得到充分的表示。在此基础上，我们进一步提出了一种新的自监督学习学位相关GCN(SL-DSGCN)，从模型和数据两个方面缓解了GCNS与学位相关的偏差。首先，我们提出了度相关的GCN层来捕捉不同度节点的异同，减少了由于所有节点共享相同的参数而导致的GCN内部模型方面的偏差。其次，我们设计了一种自监督学习算法，该算法利用贝叶斯神经网络在未标记的节点上生成具有不确定性分数的伪标签。伪标签增加了连接到低度节点的标签邻居的机会，从而从数据的角度减少了GCN的偏差。在SL-DSGCN的随机梯度下降过程中，进一步利用不确定性得分对伪标签权重。在三个基准数据集上的实验表明，SL-DSGCN不仅优于目前最先进的自训练/自监督学习GCN方法，而且显著提高了低度节点的GCN准确率。
Graph Convolutional Networks (GCNs) show promising results for semi-supervised learning tasks on graphs, thus become favorable comparing with other approaches. Despite the remarkable success of GCNs, it is difficult to train GCNs with insufficient supervision. When labeled data are limited, the performance of GCNs becomes unsatisfying for low-degree nodes. While some prior work analyze successes and failures of GCNs on the entire model level, profiling GCNs on individual node level is still underexplored. In this paper, we analyze GCNs in regard to the node degree distribution. From empirical observation to theoretical proof, we confirm that GCNs are biased towards nodes with larger degrees with higher accuracy on them, even if high-degree nodes are underrepresented in most graphs. We further develop a novel Self-Supervised-Learning Degree-Specific GCN (SL-DSGCN) that mitigate the degree-related biases of GCNs from model and data aspects. Firstly, we propose a degree-specific GCN layer that captures both discrepancies and similarities of nodes with different degrees, which reduces the inner model-aspect biases of GCNs caused by sharing the same parameters with all nodes. Secondly, we design a self-supervised-learning algorithm that creates pseudo labels with uncertainty scores on unlabeled nodes with a Bayesian neural network. Pseudo labels increase the chance of connecting to labeled neighbors for low-degree nodes, thus reducing the biases of GCNs from the data perspective. Uncertainty scores are further exploited to weight pseudo labels dynamically in the stochastic gradient descent for SL-DSGCN. Experiments on three benchmark datasets show SL-DSGCN not only outperforms state-of-the-art self-training/self-supervised-learning GCN methods, but also improves GCN accuracy dramatically for low-degree nodes.