Hierarchical Prototype Networks for Continual Graph Representation Learning
Hierarchical Prototype Networks for Continual Graph Representation Learning
复制标题
DOI：
10.1109/tpami.2022.3186909
复制
发表时间：
2021-11
影响因子：
23.6
通讯作者：
Xikun Zhang;Dongjin Song;D. Tao
中科院分区：
文献类型：
--
作者：
Xikun Zhang;Dongjin Song;D. Tao
尽管在图形表示学习方面取得了重大进展，但很少有人注意到更实际的持续学习场景，其中新的节点类别(例如，引文网络中的新研究领域，或共同购买网络中的新类型产品)及其相关边不断出现，导致对先前类别的灾难性遗忘。现有的方法要么忽略了丰富的拓扑信息，要么牺牲了塑性来换取稳定性。为此，我们提出了分层原型网络(Hierarchical Prototype Networks，HPN)，它以原型的形式提取不同层次的抽象知识来表示不断扩展的图。具体地说，我们首先利用一组原子特征抽取器(AFES)来编码目标节点的基本属性信息和拓扑结构。接下来，我们开发了HPN来自适应地选择相关的AFE，并用三层原型来表示每个节点。这样，每当给定新的节点类别时，只会激活和细化每个级别上的相关AFE和原型，而其他节点保持不间断，以保持在现有节点上的性能。理论上，我们首先证明了无论遇到多少任务，HPN的内存消耗都是有界的。然后，我们证明了在温和的约束下，学习新任务不会改变与先前数据匹配的原型，从而消除了遗忘问题。在五个数据集上的实验结果支持了理论结果，表明HPN不仅性能优于最先进的基线技术，而且占用的内存也相对较少。代码和数据集可在https://github.com/QueuQ/HPNs.上获得
Despite significant advances in graph representation learning, little attention has been paid to the more practical continual learning scenario in which new categories of nodes (e.g., new research areas in citation networks, or new types of products in co-purchasing networks) and their associated edges are continuously emerging, causing catastrophic forgetting on previous categories. Existing methods either ignore the rich topological information or sacrifice plasticity for stability. To this end, we present Hierarchical Prototype Networks (HPNs) which extract different levels of abstract knowledge in the form of prototypes to represent the continuously expanded graphs. Specifically, we first leverage a set of Atomic Feature Extractors (AFEs) to encode both the elemental attribute information and the topological structure of the target node. Next, we develop HPNs to adaptively select relevant AFEs and represent each node with three levels of prototypes. In this way, whenever a new category of nodes is given, only the relevant AFEs and prototypes at each level will be activated and refined, while others remain uninterrupted to maintain the performance over existing nodes. Theoretically, we first demonstrate that the memory consumption of HPNs is bounded regardless of how many tasks are encountered. Then, we prove that under mild constraints, learning new tasks will not alter the prototypes matched to previous data, thereby eliminating the forgetting problem. The theoretical results are supported by experiments on five datasets, showing that HPNs not only outperform state-of-the-art baseline techniques but also consume relatively less memory. Code and datasets are available at https://github.com/QueuQ/HPNs.