Learning Sample-Specific Policies for Sequential Image Augmentation
Learning Sample-Specific Policies for Sequential Image Augmentation
复制标题
DOI：
10.1145/3474085.3475602
复制
发表时间：
2021-10
期刊：
影响因子：
--
通讯作者：
Pu Li;Xiaobai Liu;Xiaohui Xie
中科院分区：
文献类型：
--
作者：
Pu Li;Xiaobai Liu;Xiaohui Xie
本文提出一种策略驱动的序列图像增强方法，用于图像相关任务。我们的方法在训练图像上应用一系列图像变换（例如，平移，旋转），每次一个变换，将前一个时间步长的增强图像作为下一个变换的输入。这种连续的数据增强大大提高了样本多样性，从而提高了测试性能，特别是对于数据饥渴模型（例如，深度神经网络）。然而，由于序列的组合性，在序列的每个时间步寻找每张图像的最优变换具有很高的复杂性。为了解决这一挑战，我们将搜索任务制定为一个顺序决策过程，并引入一个深度策略网络，该网络学习基于图像内容产生转换。我们还开发了一种迭代算法，在强化学习设置中联合训练分类器和策略网络。潜在转换的直接奖励被定义为鼓励为当前分类器生成硬样本的转换。在每次迭代中，我们使用策略网络来增强训练数据集，使用增强的数据训练分类器，并借助分类器训练策略网络。我们将上述方法应用于公共图像分类基准和新收集的图像数据集进行材料识别。与其他增强方法的比较表明，我们的策略驱动方法在使用更少的增强图像的同时实现了相当或改进的分类性能。代码可在https://github.com/Paul-LiPu/rl_autoaug上获得。
This paper presents a policy-driven sequential image augmentation approach for image-related tasks. Our approach applies a sequence of image transformations (e.g., translation, rotation) over a training image, one transformation at a time, with the augmented image from the previous time step treated as the input for the next transformation. This sequential data augmentation substantially improves sample diversity, leading to improved test performance, especially for data-hungry models (e.g., deep neural networks). However, the search for the optimal transformation of each image at each time step of the sequence has high complexity due to its combination nature. To address this challenge, we formulate the search task as a sequential decision process and introduce a deep policy network that learns to produce transformations based on image content. We also develop an iterative algorithm to jointly train a classifier and the policy network in the reinforcement learning setting. The immediate reward of a potential transformation is defined to encourage transformations producing hard samples for the current classifier. At each iteration, we employ the policy network to augment the training dataset, train a classifier with the augmented data, and train the policy net with the aid of the classifier. We apply the above approach to both public image classification benchmarks and a newly collected image dataset for material recognition. Comparisons to alternative augmentation approaches show that our policy-driven approach achieves comparable or improved classification performance while using significantly fewer augmented images. The code is available at https://github.com/Paul-LiPu/rl_autoaug.