Implicit Graph Neural Networks: A Monotone Operator Viewpoint
Implicit Graph Neural Networks: A Monotone Operator Viewpoint
复制标题
DOI：
--
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
Justin Baker;Qingsong Wang;C. Hauck;Bao Wang
中科院分区：
文献类型：
--
作者：
Justin Baker;Qingsong Wang;C. Hauck;Bao Wang
隐式图神经网络（IGNN）——使用 Pi-card 迭代来求解定点平衡方程以进行表示学习——在学习底层图中的远程依赖关系（LRD）方面表现出了卓越的性能。然而，IGNN 存在几个问题，包括 1）它们的表达能力受到适定性保证的参数化的限制，2）IGNN 在学习 LRD 时不稳定，3）IGNN 在学习 LRD 时变得计算效率低下。在本文中，我们利用单调算子理论为 IGNN 提供了一种新的适定性表征，从而获得比现有参数化更具表现力的参数化。我们还提出了基于凯莱变换的 IGNN 正交参数化来稳定学习 LRD。此外，我们利用安德森加速算子分裂方案有效地求解具有单调或正交参数化的 IGNN 平衡方程的不动点。我们在图和节点级别的各种图学习任务上验证了新模型相对于现有 IGNN 的计算效率和准确性。代码可在 https://github.com/Utah-Math-Data-Science/MIGNN 获取
Implicit graph neural networks (IGNNs) – that solve a fixed-point equilibrium equation using Pi-card iteration for representation learning – have shown remarkable performance in learning long-range dependencies (LRD) in the underlying graphs. However, IGNNs suffer from several issues, including 1) their expressivity is limited by their parameterizations for the well-posedness guarantee, 2) IGNNs are unstable in learning LRD, and 3) IGNNs become computationally inefficient when learning LRD. In this paper, we provide a new well-posedness characterization for IGNNs leveraging monotone operator theory, resulting in a much more expressive parameterization than the existing one. We also pro-pose an orthogonal parameterization for IGNN based on Cayley transform to stabilize learning LRD. Furthermore, we leverage Anderson-accelerated operator splitting schemes to efficiently solve for the fixed point of the equilibrium equation of IGNN with monotone or orthogonal parameterization. We verify the computational efficiency and accuracy of the new models over existing IGNNs on various graph learning tasks at both graph and node levels. Code is available at https://github.com/ Utah-Math-Data-Science/MIGNN