Data augmentation-assisted deep learning of hand-drawn partially colored sketches for visual search.
Data augmentation-assisted deep learning of hand-drawn partially colored sketches for visual search.
复制标题
DOI：
10.1371/journal.pone.0183838
复制
发表时间：
2017
期刊：
影响因子：
3.7
通讯作者：
Baik SW
中科院分区：
文献类型：
--
作者：
Ahmad J;Muhammad K;Baik SW
近年来，图像数据库正以指数级的速度增长，这使得它们的管理、索引和检索变得非常具有挑战性。典型的图像检索系统依赖于样本图像作为查询。但是，在没有示例查询图像的情况下，也使用手绘草图。最近采用的触摸屏输入设备使得快速绘制用于查询图像数据库的对象的阴影草图非常方便。本文提出了一种基于用户使用触摸屏设备绘制的部分彩色草图的视觉信息访问机制。基于草图的图像检索系统面临的一个关键挑战是处理由于缺乏颜色、纹理、阴影和绘图缺陷而导致的草图固有的模糊性。为了解决这些问题，我们提出使用增强数据集对深度卷积神经网络（CNN）进行微调，以从部分着色的手绘草图中提取特征，用于基于草图的图像检索框架中的查询规范。大型增强数据集包含自然图像、边缘地图、手绘草图、去彩色和去纹理图像，这些图像允许CNN以各种形式有效地建模呈现给它的视觉内容。从CNN中提取的深度特征允许使用草图和全彩图像作为查询检索图像。我们还评估了部分着色或阴影在草图中的作用，以提高检索性能。在两个大型数据集上对该方法进行了素描识别和基于素描的图像检索的测试，取得了比现有方法更好的分类和检索性能。
In recent years, image databases are growing at exponential rates, making their management, indexing, and retrieval, very challenging. Typical image retrieval systems rely on sample images as queries. However, in the absence of sample query images, hand-drawn sketches are also used. The recent adoption of touch screen input devices makes it very convenient to quickly draw shaded sketches of objects to be used for querying image databases. This paper presents a mechanism to provide access to visual information based on users’ hand-drawn partially colored sketches using touch screen devices. A key challenge for sketch-based image retrieval systems is to cope with the inherent ambiguity in sketches due to the lack of colors, textures, shading, and drawing imperfections. To cope with these issues, we propose to fine-tune a deep convolutional neural network (CNN) using augmented dataset to extract features from partially colored hand-drawn sketches for query specification in a sketch-based image retrieval framework. The large augmented dataset contains natural images, edge maps, hand-drawn sketches, de-colorized, and de-texturized images which allow CNN to effectively model visual contents presented to it in a variety of forms. The deep features extracted from CNN allow retrieval of images using both sketches and full color images as queries. We also evaluated the role of partial coloring or shading in sketches to improve the retrieval performance. The proposed method is tested on two large datasets for sketch recognition and sketch-based image retrieval and achieved better classification and retrieval performance than many existing methods.
登录
查看更多内容
影响因子：
3.6
作者：
Calisir, Fatih;Bastan, Muhammet;Gudukbay, Ugur
通讯作者：
Gudukbay, Ugur
一种基于三视图草图的新型3D模型检索系统
影响因子：
2
作者：
Cao, Bin;Kang, Yang;Xue, Yu
通讯作者：
Xue, Yu
DOI：
10.1109/tsmca.2004.838464
发表时间：
2005-01-01
影响因子：
--
作者：
Chalechale, A;Naghdy, G;Mertins, A
通讯作者：
Mertins, A
影响因子：
3.6
作者：
Ahmad, Jamil;Sajjad, Muhammad;Baik, Sung Wook
通讯作者：
Baik, Sung Wook
通过基于草图的查询进行风格敏感的 3D 模型检索
影响因子：
2
作者：
Cao, Bin;Kang, Yang;Lv, Zhihan
通讯作者：
Lv, Zhihan