Design Space for Graph Neural Networks
Design Space for Graph Neural Networks
复制标题
DOI：
--
复制
发表时间：
2020-11
期刊：
影响因子：
--
通讯作者：
Jiaxuan You;Rex Ying;J. Leskovec
中科院分区：
文献类型：
--
作者：
Jiaxuan You;Rex Ying;J. Leskovec
图神经网络（GNNs）的快速发展导致了越来越多的新架构和新应用。然而，目前的研究重点是提出和评估GNN的具体架构设计，而不是研究GNN的更一般的设计空间，该设计空间由不同设计维度的笛卡尔积组成，例如层数或聚合函数的类型。此外，GNN设计通常专用于单个任务，但很少有人努力了解如何快速找到新任务或新数据集的最佳GNN设计。在这里，我们定义并系统地研究了GNN的架构设计空间，该空间由32个不同预测任务的315，000个不同设计组成。我们的方法具有三个关键创新：（1）通用GNN设计空间;（2）具有相似性度量的GNN任务空间，因此对于给定的新任务/数据集，我们可以快速识别/转移性能最佳的架构;（3）高效和有效的设计空间评估方法，可以从大量的模型-任务组合中提取见解。我们的主要成果包括：（1）设计性能良好的GNN的一套全面的指导方针;（2）虽然不同任务的最佳GNN设计差异很大，但GNN任务空间允许在不同任务之间转移最佳设计;（3）使用我们的设计空间发现的模型实现了最先进的性能。总的来说，我们的工作提供了一种原则性和可扩展的方法，从研究特定任务的单个GNN设计过渡到系统地研究GNN设计空间和任务空间。最后，我们发布了GraphGym，这是一个用于探索不同GNN设计和任务的强大平台。GraphGym具有模块化的GNN实现，标准化的GNN评估以及可重复和可扩展的实验管理。
The rapid evolution of Graph Neural Networks (GNNs) has led to a growing number of new architectures as well as novel applications. However, current research focuses on proposing and evaluating specific architectural designs of GNNs, as opposed to studying the more general design space of GNNs that consists of a Cartesian product of different design dimensions, such as the number of layers or the type of the aggregation function. Additionally, GNN designs are often specialized to a single task, yet few efforts have been made to understand how to quickly find the best GNN design for a novel task or a novel dataset. Here we define and systematically study the architectural design space for GNNs which consists of 315,000 different designs over 32 different predictive tasks. Our approach features three key innovations: (1) A general GNN design space; (2) a GNN task space with a similarity metric, so that for a given novel task/dataset, we can quickly identify/transfer the best performing architecture; (3) an efficient and effective design space evaluation method which allows insights to be distilled from a huge number of model-task combinations. Our key results include: (1) A comprehensive set of guidelines for designing well-performing GNNs; (2) while best GNN designs for different tasks vary significantly, the GNN task space allows for transferring the best designs across different tasks; (3) models discovered using our design space achieve state-of-the-art performance. Overall, our work offers a principled and scalable approach to transition from studying individual GNN designs for specific tasks, to systematically studying the GNN design space and the task space. Finally, we release GraphGym, a powerful platform for exploring different GNN designs and tasks. GraphGym features modularized GNN implementation, standardized GNN evaluation, and reproducible and scalable experiment management.