Sparsified Subgraph Memory for Continual Graph Representation Learning
Sparsified Subgraph Memory for Continual Graph Representation Learning
复制标题
DOI：
10.1109/icdm54844.2022.00177
复制
发表时间：
2022-11
期刊：
影响因子：
--
通讯作者：
Xikun Zhang;Dongjin Song;D. Tao
中科院分区：
文献类型：
--
作者：
Xikun Zhang;Dongjin Song;D. Tao
内存重放，它存储了一个子集的代表性的历史数据，从以前的任务重放，同时学习新的任务，表现出最先进的性能为各种持续学习应用程序的欧几里德数据。虽然拓扑信息在表征图数据中起着关键作用，但现有的基于存储器重放的图学习技术仅存储用于重放的单个节点，而不考虑它们相关联的边信息。为此，我们提出了一个稀疏化子图存储器（SSM），它将选定的计算图稀疏成固定的大小，然后将它们存储到内存中。通过这种方式，我们可以将计算子图的内存消耗从$\mathcal{O}（d^{L}）$减少到$\mathcal{O}（1）$，并首次使GNN能够利用显式拓扑信息进行内存重放。最后，我们的实证研究表明，SSM在四个不同的公共数据集上的表现优于最先进的方法高达27.8%。与现有的方法，专注于任务增量学习（task-IL）设置，SSM成功地在具有挑战性的类增量学习（class-IL）设置，其中需要一个模型来区分所有学习类没有任务指标，甚至实现了可比的性能联合训练，这是持续学习的性能上限。我们的代码可在https://github.com/QueuQ/SSM上获得。
Memory replay, which stores a subset of representative historical data from previous tasks to replay while learning new tasks, exhibits state-of-the-art performance for various continual learning applications on Euclidean data. While topological information plays a critical role in characterizing graph data, existing memory replay based graph learning techniques only store individual nodes for replay and do not consider their associated edge information. To this end, we propose a sparsified subgraph memory (SSM), which sparsifies the selected computation graphs into fixed size before storing them into the memory. In this way, we can reduce the memory consumption of a computation subgraph from $\mathcal{O}(d^{L})$ to $\mathcal{O}(1)$, and for the first time enable GNNs to utilize the explicit topological information for memory replay. Finally, our empirical studies show that SSM outperforms state-of-the-art approaches by up to 27.8% on four different public datasets. Unlike existing methods which focus on task incremental learning (task-IL) setting, SSM succeeds in the challenging class incremental learning (class-IL) setting in which a model is required to distinguish all learned classes without task indicators, and even achieves comparable performance to joint training which is the performance upper bound for continual learning. Our code is available at https://github.com/QueuQ/SSM.