Negative Data Augmentation
Negative Data Augmentation
复制标题
DOI：
--
复制
发表时间：
2021-02
期刊：
影响因子：
--
通讯作者：
Abhishek Sinha;Kumar Ayush;Jiaming Song;Burak Uzkent;Hongxia Jin;Stefano Ermon
中科院分区：
文献类型：
--
作者：
Abhishek Sinha;Kumar Ayush;Jiaming Song;Burak Uzkent;Hongxia Jin;Stefano Ermon
数据扩增通常用于根据底层数据分布生成合成样本来扩大数据集。为了实现更大范围的增强，我们探索了负数据增强策略（NDA），故意创建分布外样本。我们表明，这种负分布外样本提供了关于数据分布支持的信息，并且可以用于生成建模和表示学习。我们引入了一个新的GAN训练目标，其中我们使用NDA作为鉴别器的额外合成数据来源。我们证明，在适当的条件下，优化结果目标仍然可以恢复真实的数据分布，但可以直接使生成器偏向于避免缺乏所需结构的样本。根据经验，用我们的方法训练的模型可以改进条件/无条件图像生成以及改进的异常检测能力。此外，我们将相同的负数据增强策略纳入图像和视频的自监督表示学习的对比学习框架中，在下游图像分类、目标检测和动作识别任务上实现了改进的性能。这些结果表明，在一系列无监督学习任务中，对不构成有效数据的先验知识是一种有效的弱监督形式。
Data augmentation is often used to enlarge datasets with synthetic samples generated in accordance with the underlying data distribution. To enable a wider range of augmentations, we explore negative data augmentation strategies (NDA)that intentionally create out-of-distribution samples. We show that such negative out-of-distribution samples provide information on the support of the data distribution, and can be leveraged for generative modeling and representation learning. We introduce a new GAN training objective where we use NDA as an additional source of synthetic data for the discriminator. We prove that under suitable conditions, optimizing the resulting objective still recovers the true data distribution but can directly bias the generator towards avoiding samples that lack the desired structure. Empirically, models trained with our method achieve improved conditional/unconditional image generation along with improved anomaly detection capabilities. Further, we incorporate the same negative data augmentation strategy in a contrastive learning framework for self-supervised representation learning on images and videos, achieving improved performance on downstream image classification, object detection, and action recognition tasks. These results suggest that prior knowledge on what does not constitute valid data is an effective form of weak supervision across a range of unsupervised learning tasks.