CODA-Prompt: COntinual Decomposed Attention-Based Prompting for Rehearsal-Free Continual Learning
CODA-Prompt: COntinual Decomposed Attention-Based Prompting for Rehearsal-Free Continual Learning
复制标题
DOI：
10.1109/cvpr52729.2023.01146
复制
发表时间：
2022-11
期刊：
影响因子：
--
通讯作者：
James Smith;Leonid Karlinsky;V. Gutta;Paola Cascante-Bonilla;Donghyun Kim-;Assaf Arbelle;Rameswar Panda;R. Feris;Z. Kira
中科院分区：
文献类型：
--
作者：
James Smith;Leonid Karlinsky;V. Gutta;Paola Cascante-Bonilla;Donghyun Kim-;Assaf Arbelle;Rameswar Panda;R. Feris;Z. Kira
计算机视觉模型在从不断变化的训练数据中学习新概念时，会遭受一种被称为灾难性遗忘的现象。针对这种持续学习问题的典型解决方案需要对先前见过的数据进行大量的复习，这增加了内存成本，并且可能违反数据隐私。最近，大规模预训练视觉Transformer模型的出现使得提示方法成为数据复习的一种替代方案。这些方法依靠键 - 查询机制来生成提示，并且在已确立的无复习持续学习设置中被发现对灾难性遗忘具有高度的抵抗力。然而，这些方法的关键机制没有与任务序列进行端到端的训练。我们的实验表明，这导致它们的可塑性降低，从而牺牲了新任务的准确性，并且无法从扩展的参数容量中受益。我们转而提出学习一组提示组件，这些组件与输入条件权重组合以产生输入条件提示，从而产生一种新颖的基于注意力的端到端键 - 查询方案。我们的实验表明，在已确立的基准测试中，我们比当前的最先进方法DualPrompt的平均最终准确率高出多达4.5%。在一个包含类别增量和领域增量任务转移（对应于许多实际设置）的持续学习基准测试中，我们的准确率也比最先进水平高出多达4.4%。我们的代码可在https://github.com/GT - RIPL/CODA - Prompt获取。
Computer vision models suffer from a phenomenon known as catastrophic forgetting when learning novel concepts from continuously shifting training data. Typical solutions for this continual learning problem require extensive rehearsal of previously seen data, which increases memory costs and may violate data privacy. Recently, the emergence of large-scale pre-trained vision transformer models has enabled prompting approaches as an alternative to data-rehearsal. These approaches rely on a key-query mechanism to generate prompts and have been found to be highly resistant to catastrophic forgetting in the well-established rehearsal-free continual learning setting. However, the key mechanism of these methods is not trained end-to-end with the task sequence. Our experiments show that this leads to a reduction in their plasticity, hence sacrificing new task accuracy, and inability to benefit from expanded parameter capacity. We instead propose to learn a set of prompt components which are assembled with input-conditioned weights to produce input-conditioned prompts, resulting in a novel attention-based end-to-end key-query scheme. Our experiments show that we outperform the current SOTA method DualPrompt on established benchmarks by as much as 4.5% in average final accuracy. We also outperform the state of art by as much as 4.4% accuracy on a continual learning benchmark which contains both class-incremental and domain-incremental task shifts, corresponding to many practical settings. Our code is available at https://github.com/GT-RIPL/CODA-Prompt