Don't Memorize; Mimic The Past: Federated Class Incremental Learning Without Episodic Memory-猫眼课题宝

DOI：

10.48550/arxiv.2307.00497

发表时间：

2023-07

期刊：

ArXiv

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Sara Babakniya;Zalan Fabian;Chaoyang He;M. Soltanolkotabi;S. Avestimehr

深度学习模型在新数据上训练时，容易忘记过去学习的信息。这个问题在联邦学习（FL）的背景下变得更加突出，其中数据是分散的，并且每个用户都要进行独立的更改。持续学习（CL）主要在集中式设置中研究这种所谓的灾难性遗忘现象，在这种情况下，学习者可以直接访问完整的训练数据集。然而，应用CL技术FL是不简单的，由于隐私问题和资源的限制。本文提出了一个联邦类增量学习的框架，利用生成模型来合成过去分布的样本，而不是存储过去的数据的一部分。然后，客户可以利用生成模型来减轻本地的灾难性遗忘。生成模型在每个任务结束时使用无数据方法在服务器上进行训练，而无需向客户端请求数据。因此，它降低了数据泄露的风险，而不是在客户的私人数据上进行培训。与现有基线相比，我们证明了CIFAR-100数据集的显著改进。

Deep learning models are prone to forgetting information learned in the past when trained on new data. This problem becomes even more pronounced in the context of federated learning (FL), where data is decentralized and subject to independent changes for each user. Continual Learning (CL) studies this so-called \textit{catastrophic forgetting} phenomenon primarily in centralized settings, where the learner has direct access to the complete training dataset. However, applying CL techniques to FL is not straightforward due to privacy concerns and resource limitations. This paper presents a framework for federated class incremental learning that utilizes a generative model to synthesize samples from past distributions instead of storing part of past data. Then, clients can leverage the generative model to mitigate catastrophic forgetting locally. The generative model is trained on the server using data-free methods at the end of each task without requesting data from clients. Therefore, it reduces the risk of data leakage as opposed to training it on the client's private data. We demonstrate significant improvements for the CIFAR-100 dataset compared to existing baselines.

Don't Memorize; Mimic The Past: Federated Class Incremental Learning Without Episodic Memory

文献摘要

相似文献