CUDA: Convolution-Based Unlearnable Datasets
CUDA: Convolution-Based Unlearnable Datasets
复制标题
DOI：
10.1109/cvpr52729.2023.00376
复制
发表时间：
2023-03
期刊：
影响因子：
--
通讯作者：
Vinu Sankar Sadasivan;M. Soltanolkotabi;S. Feizi
中科院分区：
文献类型：
--
作者：
Vinu Sankar Sadasivan;M. Soltanolkotabi;S. Feizi
现代深度学习模型的大规模训练严重依赖于网络上的公开数据。这种对在线数据的潜在未经授权的使用导致了对数据隐私的担忧。最近的工作旨在通过添加特殊设计的小噪声来解决这个问题，从而为深度学习模型提供不可学习的数据。然而，这些方法容易受到对抗训练（AT）的影响和/或计算量很大。在这项工作中，我们提出了一种新的，无模型的，基于卷积的不可学习数据集（CUDA）生成技术。CUDA是使用受控的类卷积与通过私钥随机生成的过滤器生成的。CUDA鼓励网络学习过滤器和标签之间的关系，而不是对干净数据进行分类的信息特征。我们开发了一些理论分析表明，CUDA可以成功地通过降低最佳贝叶斯分类器的清洁数据性能的高斯混合数据中毒。我们还通过各种数据集（CIFAR-10，CIFAR-100，ImageNet-100和Tiny-ImageNet）和架构（ResNet-18，VGG-16，Wide ResNet-34-10，DenseNet-121，DeIT，EfficientNetV 2-S和MobileNetV 2）实证了CUDA的有效性。我们的实验表明，CUDA对各种数据增强和训练方法（如平滑，不同预算的AT，迁移学习和微调）具有鲁棒性。例如，在ImageNet-100 CUDA上训练ResNet-18，在经验风险最小化（ERM）、L∞AT和L2 AT下，分别仅实现了8.96%、40.08%和20.58%的干净测试准确率。在这里，干净训练数据上的ERM达到了80.66%的干净测试准确率。即使只有一小部分训练数据集被扰动，CUDA也会对ERM表现出不可学习性效应。此外，我们还表明，CUDA是强大的自适应防御专门设计来打破它。
Large-scale training of modern deep learning models heavily relies on publicly available data on the web. This potentially unauthorized usage of online data leads to concerns regarding data privacy. Recent works aim to make unlearnable data for deep learning models by adding small, specially designed noises to tackle this issue. However, these methods are vulnerable to adversarial training (AT) and/or are computationally heavy. In this work, we propose a novel, model-free, Convolution-based Unlearnable DAtaset (CUDA) generation technique. CUDA is generated using controlled class-wise convolutions with filters that are randomly generated via a private key. CUDA encourages the network to learn the relation between filters and labels rather than informative features for classifying the clean data. We develop some theoretical analysis demonstrating that CUDA can successfully poison Gaussian mixture data by reducing the clean data performance of the optimal Bayes classifier. We also empirically demonstrate the effectiveness of CUDA with various datasets (CIFAR-10, CIFAR-100, ImageNet-100, and Tiny-ImageNet), and architectures (ResNet-18, VGG-16, Wide ResNet-34-10, DenseNet-121, DeIT, EfficientNetV2-S, and MobileNetV2). Our experiments show that CUDA is robust to various data augmentations and training approaches such as smoothing, AT with different budgets, transfer learning, and fine-tuning. For instance, training a ResNet-18 on ImageNet-100 CUDA achieves only 8.96%, 40.08%, and 20.58% clean test accuracies with empirical risk minimization (ERM), L∞AT, and L2 AT, respectively. Here, ERM on the clean training data achieves a clean test accuracy of 80.66%. CUDA exhibits unlearnability effect with ERM even when only a fraction of the training dataset is perturbed. Furthermore, we also show that CUDA is robust to adaptive defenses designed specifically to break it.