Pitfalls in Link Prediction with Graph Neural Networks: Understanding the Impact of Target-link Inclusion & Better Practices
Pitfalls in Link Prediction with Graph Neural Networks: Understanding the Impact of Target-link Inclusion & Better Practices
复制标题
DOI：
10.1145/3616855.3635786
复制
发表时间：
2023-06
期刊：
影响因子：
--
通讯作者：
Jing Zhu;Yuhang Zhou;V. Ioannidis;Sheng Qian;Wei Ai;Xiang Song;Danai Koutra
中科院分区：
文献类型：
--
作者：
Jing Zhu;Yuhang Zhou;V. Ioannidis;Sheng Qian;Wei Ai;Xiang Song;Danai Koutra
虽然图神经网络（GNN）在各种高影响力的应用中非常成功，但我们证明，在链接预测中，在训练和/或测试时将预测的边包含在图中的常见做法对低度节点的性能产生了巨大的影响。我们从理论上和经验上研究了这些做法如何在不同程度上影响节点级性能。具体来说，我们探讨了三个问题出现：（I1）过拟合;（I2）分布移位;（I3）隐式测试泄漏。前两个问题导致测试数据的泛化能力差，而后者导致高估模型的性能，并直接影响GNN的部署。为了系统地解决这些问题，我们引入了一个有效且高效的GNN训练框架SpotTarget，它利用了我们对低度节点的洞察力：（1）在训练时，它排除了一个（训练）要预测的边，如果它与至少一个低度节点关联;以及（2）在测试时，它排除所有要预测的测试边缘（因此，模仿使用GNN的真实的场景，其中测试数据不包括在图中）。SpotTarget帮助研究人员和从业者坚持从图形数据中学习的最佳实践，即使是最广泛使用的框架也经常忽视这些实践。我们在各种真实数据集上的实验表明，SpotTarget使GNN在稀疏图中的准确度提高了15倍，并显着提高了密集图中低度节点的性能。
While Graph Neural Networks (GNNs) are remarkably successful in a variety of high-impact applications, we demonstrate that, in link prediction, the common practices of including the edges being predicted in the graph at training and/or test have outsized impact on the performance of low-degree nodes. We theoretically and empirically investigate how these practices impact node-level performance across different degrees. Specifically, we explore three issues that arise: (I1) overfitting; (I2) distribution shift; and (I3) implicit test leakage. The former two issues lead to poor generalizability to the test data, while the latter leads to overestimation of the model's performance and directly impacts the deployment of GNNs. To address these issues in a systematic way, we introduce an effective and efficient GNN training framework, SpotTarget, which leverages our insight on low-degree nodes: (1) at training time, it excludes a (training) edge to be predicted if it is incident to at least one low-degree node; and (2) at test time, it excludes all test edges to be predicted (thus, mimicking real scenarios of using GNNs, where the test data is not included in the graph). SpotTarget helps researchers and practitioners adhere to best practices for learning from graph data, which are frequently overlooked even by the most widely-used frameworks. Our experiments on various real-world datasets show that SpotTarget makes GNNs up to 15× more accurate in sparse graphs, and significantly improves their performance for low-degree nodes in dense graphs.