Robust Graph Representation Learning via Predictive Coding
Robust Graph Representation Learning via Predictive Coding
复制标题
DOI：
10.48550/arxiv.2212.04656
复制
发表时间：
2022-12
期刊：
影响因子：
--
通讯作者：
Billy Byiringiro;Tommaso Salvatori;Thomas Lukasiewicz
中科院分区：
文献类型：
--
作者：
Billy Byiringiro;Tommaso Salvatori;Thomas Lukasiewicz
预测编码是一种消息传递框架，最初是为了模拟大脑中的信息处理而开发的，由于一些有趣的特性，现在也成为机器学习的研究主题。其中一个属性是生成模型学习鲁棒表示的自然能力，这要归功于其独特的信用分配规则，该规则允许神经活动在更新突触权重之前收敛到解决方案。图神经网络也是消息传递模型，最近在机器学习的各种类型的任务中表现出了出色的结果，在结构化数据上提供了跨学科的最先进的性能。然而，它们很容易受到难以察觉的对抗性攻击，并且不适合分布外泛化。在这项工作中，我们通过构建与流行的图神经网络架构具有相同结构的模型来解决这个问题，但依赖于预测编码的消息传递规则。通过大量的实验，我们表明所提出的模型（i）在归纳和传导任务的性能方面与标准模型相当，（ii）更好的校准，以及（iii）对多种对抗性攻击具有鲁棒性。
Predictive coding is a message-passing framework initially developed to model information processing in the brain, and now also topic of research in machine learning due to some interesting properties. One of such properties is the natural ability of generative models to learn robust representations thanks to their peculiar credit assignment rule, that allows neural activities to converge to a solution before updating the synaptic weights. Graph neural networks are also message-passing models, which have recently shown outstanding results in diverse types of tasks in machine learning, providing interdisciplinary state-of-the-art performance on structured data. However, they are vulnerable to imperceptible adversarial attacks, and unfit for out-of-distribution generalization. In this work, we address this by building models that have the same structure of popular graph neural network architectures, but rely on the message-passing rule of predictive coding. Through an extensive set of experiments, we show that the proposed models are (i) comparable to standard ones in terms of performance in both inductive and transductive tasks, (ii) better calibrated, and (iii) robust against multiple kinds of adversarial attacks.