DP-Mix: Mixup-based Data Augmentation for Differentially Private Learning
DP-Mix: Mixup-based Data Augmentation for Differentially Private Learning
复制标题
DOI：
10.48550/arxiv.2311.01295
复制
发表时间：
2023-11
期刊：
影响因子：
--
通讯作者：
Wenxuan Bao;Francesco Pittaluga;Vijay Kumar;Vincent Bindschaedler
中科院分区：
文献类型：
--
作者：
Wenxuan Bao;Francesco Pittaluga;Vijay Kumar;Vincent Bindschaedler
数据增强技术，如简单的图像变换和组合，在提高计算机视觉模型的泛化能力方面非常有效，特别是在训练数据有限的情况下。然而，这样的技术是从根本上不兼容的差分私人学习方法，由于后者的内置假设，每个训练图像的贡献学习模型是有界的。在本文中，我们研究了为什么天真的应用程序的多样本数据增强技术，如混合，未能实现良好的性能，并提出了两种新的数据增强技术，专门设计的差异私人学习的约束。我们的第一种技术DP-Mix_Self通过对自增强数据执行mixup，在一系列数据集和设置上实现SoTA分类性能。我们的第二种技术DP-Mix_Diff通过将来自预训练扩散模型的合成数据纳入混合过程来进一步提高性能。我们在https://github.com/wenxuan-Bao/DP-Mix上开源代码。
Data augmentation techniques, such as simple image transformations and combinations, are highly effective at improving the generalization of computer vision models, especially when training data is limited. However, such techniques are fundamentally incompatible with differentially private learning approaches, due to the latter's built-in assumption that each training image's contribution to the learned model is bounded. In this paper, we investigate why naive applications of multi-sample data augmentation techniques, such as mixup, fail to achieve good performance and propose two novel data augmentation techniques specifically designed for the constraints of differentially private learning. Our first technique, DP-Mix_Self, achieves SoTA classification performance across a range of datasets and settings by performing mixup on self-augmented data. Our second technique, DP-Mix_Diff, further improves performance by incorporating synthetic data from a pre-trained diffusion model into the mixup process. We open-source the code at https://github.com/wenxuan-Bao/DP-Mix.