Data augmentation via photo-to-sketch translation for sketch-based image retrieval
Data augmentation via photo-to-sketch translation for sketch-based image retrieval
复制标题
通过照片到草图转换来增强数据，以进行基于草图的图像检索
DOI：
10.1117/12.2524230
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
Ohbuchi Ryutarou
中科院分区：
文献类型：
--
作者：
Furuya Takahiko;Ohbuchi Ryutarou
基于草图的图像检索（SBIR）技术通过深度学习从大量草图-照片对中学习草图和照片之间的跨模态距离度量，取得了进展。然而，草图-照片对的数据集很小，因为获取大量这样的对是昂贵的。为了缓解这一问题，通过图像变换（如缩放、翻转、旋转和变形）进行数据增强已被广泛采用。然而，训练集的不足似乎阻碍了深度学习在SBIR中发挥其全部潜力。在本文中，我们提出了一种新的用于SBIR的数据增强方法。一种名为Photo2Sketch （P2S）的深度神经网络将照片转换成视觉上与人类素描相似的线条图。通过将大型图像语料库中的照片输入到P2S中，以低成本人工增强草图-照片对训练数据集。实验评估了在SBIR场景下P2S生成的类草图图像的质量以及所提出的数据增强算法的有效性。特别是将该算法与图像变换增强相结合，检索精度得到显著提高
Sketch-based image retrieval (SBIR) technique has progressed by deep learning to learn cross-modal distance metrics that relate sketches and photos from a large number of sketch-photo pairs. However, datasets of sketch-photo pairs are small, as acquisition of a large number of such pairs is expensive. To alleviate the issue, data augmentation via image transformation such as scaling, flipping, rotation, and deformation has been widely adopted. Still, insufficiency in training set seems to have impeded deep learning from achieving its full potential for SBIR. In this paper, we propose a novel data augmentation approach dedicated for SBIR. A deep neural network called Photo2Sketch (P2S) converts photos into line drawings that are visually similar to those sketched by human. An artificially augmented training dataset of sketch-photo pairs is generated at low cost by feeding photos from a large image corpus into the P2S. Experiments evaluate quality of sketch-like images generated by the P2S as well as efficacy of the proposed data augmentation algorithm under SBIR scenario. In particular, retrieval accuracy is significantly improved when the proposed algorithm is combined with the data augmentation by image transformation