Taxonomy of Benchmarks in Graph Representation Learning
Taxonomy of Benchmarks in Graph Representation Learning
复制标题
DOI：
10.48550/arxiv.2206.07729
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
Renming Liu;Semih Cantürk;Frederik Wenkel;Dylan Sandfelder;Devin Kreuzer;A. Little;Sarah McGuire-
中科院分区：
文献类型：
--
作者：
Renming Liu;Semih Cantürk;Frederik Wenkel;Dylan Sandfelder;Devin Kreuzer;A. Little;Sarah McGuire-
图形神经网络(GNN)通过考虑其固有的几何结构将神经网络的成功扩展到图形结构的数据。虽然已经进行了广泛的研究，根据一组图表示学习基准来开发具有优越性能的GNN模型，但目前还不能很好地理解它们探索给定模型的哪些方面。例如，他们在多大程度上测试模型利用图结构与节点特性的能力？在这里，我们开发了一种原则性的方法来根据$\textit{敏感度简档}$对基准数据集进行分类，该$\textit{敏感度简档}$基于由于图形扰动集合而导致的GNN性能变化的程度。我们的数据驱动分析使我们能够更深入地了解GNN利用了哪些基准数据特征。因此，我们的分类可以帮助选择和开发适当的图形基准，并更好地评估未来的GNN方法。最后，我们的方法和在$\exttt{GTaxoGym}$包中的实现可以扩展到多个图预测任务类型和未来的数据集。
Graph Neural Networks (GNNs) extend the success of neural networks to graph-structured data by accounting for their intrinsic geometry. While extensive research has been done on developing GNN models with superior performance according to a collection of graph representation learning benchmarks, it is currently not well understood what aspects of a given model are probed by them. For example, to what extent do they test the ability of a model to leverage graph structure vs. node features? Here, we develop a principled approach to taxonomize benchmarking datasets according to a $\textit{sensitivity profile}$ that is based on how much GNN performance changes due to a collection of graph perturbations. Our data-driven analysis provides a deeper understanding of which benchmarking data characteristics are leveraged by GNNs. Consequently, our taxonomy can aid in selection and development of adequate graph benchmarks, and better informed evaluation of future GNN methods. Finally, our approach and implementation in $\texttt{GTaxoGym}$ package are extendable to multiple graph prediction task types and future datasets.