Deep Sketch Hashing: Fast Free-Hand Sketch-Based Image Retrieval
Deep Sketch Hashing: Fast Free-Hand Sketch-Based Image Retrieval
复制标题
DOI：
10.1109/cvpr.2017.247
复制
发表时间：
2017-03
期刊：
影响因子：
--
通讯作者：
Li Liu;Fumin Shen;Yuming Shen;Xianglong Liu;Ling Shao
中科院分区：
文献类型：
--
作者：
Li Liu;Fumin Shen;Yuming Shen;Xianglong Liu;Ling Shao
基于手绘草图的图像检索（SBIR）是一种特定的跨视图检索任务，其中查询是抽象的和模糊的草图，而检索数据库是由自然图像组成的。这一领域的工作主要集中在提取草图和自然图像的代表性和共享特征。然而，这些都不能很好地科普草图和图像之间的几何失真，也是不可行的大规模SBIR由于沉重的连续值的距离计算。在本文中，我们加快SBIR通过引入一种新的二进制编码方法，命名为深度草图哈希（DSH），其中提出了一个半异构的深度架构，并纳入到一个端到端的二进制编码框架。具体来说，三个卷积神经网络被用来编码手绘草图，自然图像，特别是，辅助草图令牌，采用作为桥梁，以减轻草图图像的几何失真。学习的DSH代码可以有效地捕获跨视图的相似性以及不同类别之间的内在语义相关性。据我们所知，DSH是第一个专门为具有端到端深度架构的类别级SBIR设计的哈希工作。建议的DSH在TU-Berlin Extension和Sketchy的两个大规模数据集上进行了全面评估，实验一致表明DSH上级几种最先进的方法的SBIR精度，同时实现了显着减少的检索时间和内存占用。
Free-hand sketch-based image retrieval (SBIR) is a specific cross-view retrieval task, in which queries are abstract and ambiguous sketches while the retrieval database is formed with natural images. Work in this area mainly focuses on extracting representative and shared features for sketches and natural images. However, these can neither cope well with the geometric distortion between sketches and images nor be feasible for large-scale SBIR due to the heavy continuous-valued distance computation. In this paper, we speed up SBIR by introducing a novel binary coding method, named Deep Sketch Hashing (DSH), where a semi-heterogeneous deep architecture is proposed and incorporated into an end-to-end binary coding framework. Specifically, three convolutional neural networks are utilized to encode free-hand sketches, natural images and, especially, the auxiliary sketch-tokens which are adopted as bridges to mitigate the sketch-image geometric distortion. The learned DSH codes can effectively capture the cross-view similarities as well as the intrinsic semantic correlations between different categories. To the best of our knowledge, DSH is the first hashing work specifically designed for category-level SBIR with an end-to-end deep architecture. The proposed DSH is comprehensively evaluated on two large-scale datasets of TU-Berlin Extension and Sketchy, and the experiments consistently show DSHs superior SBIR accuracies over several state-of-the-art methods, while achieving significantly reduced retrieval time and memory footprint.