Targeted Data Poisoning Attacks Against Continual Learning Neural Networks
Targeted Data Poisoning Attacks Against Continual Learning Neural Networks
复制标题
DOI：
10.1109/ijcnn55064.2022.9892774
复制
发表时间：
2022-07
期刊：
影响因子：
--
通讯作者：
Huayu Li;G. Ditzler
中科院分区：
文献类型：
--
作者：
Huayu Li;G. Ditzler
持续（增量）学习方法是为了解决神经网络中的灾难性遗忘问题而设计的，方法是随着时间的推移对批量或流数据进行训练。在许多实际场景中，生成流数据的环境暴露给不受信任的源。这些不可信的来源可能暴露于被攻击者毒害的数据。攻击者可以操纵和注入恶意样本到训练数据中。因此，不可信的数据源和恶意样本意味着暴露神经网络的漏洞，这些漏洞可能导致需要可靠性能的应用程序产生严重后果。然而，最近关于持续学习的工作只关注对手不可知的场景，而没有考虑数据中毒攻击的可能性。此外，最近的研究表明，在存在后门攻击的情况下，持续学习方法存在漏洞，对操纵数据的限制很宽松。在本文中，我们关注的是一种更普遍和实用的中毒设置，即通过干净标签数据中毒攻击人为地强制灾难性遗忘。我们提出了一种以任务为目标的数据投毒攻击，该攻击迫使神经网络忘记先前学习的知识，而攻击样本保持隐身性。该方法在领域和任务增量学习场景中对三种最先进的持续学习算法进行了基准测试。实验表明，将有毒数据集用于持续任务学习时，目标任务的准确率显著下降。
Continual (incremental) learning approaches are designed to address catastrophic forgetting in neural networks by training on batches or streaming data over time. In many real-world scenarios, the environments that generate streaming data are exposed to untrusted sources. These untrusted sources can be exposed to data poisoned by an adversary. The adversaries can manipulate and inject malicious samples into the training data. Thus, the untrusted data sources and malicious samples are meant to expose the vulnerabilities of neural networks that can lead to serious consequences in applications that require reliable performance. However, recent works on continual learning only focused on adversary agnostic scenarios without considering the possibility of data poisoning attacks. Further, recent work has demonstrated there are vulnerabilities of continual learning approaches in the presence of backdoor attacks with a relaxed constraint on manipulating data. In this paper, we focus on a more general and practical poisoning setting that artificially forces catastrophic forgetting by clean-label data poisoning attacks. We proposed a task targeted data poisoning attack that forces the neural network to forget the previous-learned knowledge, while the attack samples remain stealthy. The approach is benchmarked against three state-of-the-art continual learning algorithms on both domain and task incremental learning scenarios. The experiments demonstrate that the accuracy on targeted tasks significantly drops when the poisoned dataset is used in continual task learning.