Triplet Networks Feature Masking for Sketch-Based Image Retrieval
Triplet Networks Feature Masking for Sketch-Based Image Retrieval
复制标题
DOI：
10.1007/978-3-319-59876-5_33
复制
发表时间：
2017-07
期刊：
影响因子：
--
通讯作者：
Omar Seddati;S. Dupont;S. Mahmoudi
中科院分区：
文献类型：
--
作者：
Omar Seddati;S. Dupont;S. Mahmoudi
手绘草图是一种直观的交流工具，适用于各种应用。在本文中，我们提出了一种结合三元组网络和基于草图的图像检索（SBIR）的注意机制的有效方法。这项工作中进行的研究基于使用深度卷积神经网络（ConvNet）提取的特征。为了克服 SBIR 跨域挑战（即从草图查询中搜索照片），我们使用三元组损失来训练 ConvNet 来计算草图和图像的共享嵌入。我们的主要新颖贡献是将这种三元组网络与注意力机制结合起来。我们的方法在具有挑战性的 SBIR 基准测试中优于之前最先进的方法。我们在粗略数据库中实现了 41.66% (at) 的召回率（提高了 4% 以上），在 TU-Berlin SBIR 基准上的 Kendal 分数达到了 42.9 分（接近 5.5 倍的提高），在 Flickr15k（类别级 SBIR 基准）上实现了 31% 的平均精度 (MAP)。
Freehand sketches are an intuitive tool for communication and suitable for various applications. In this paper, we present an effective approach that combines triplet networks and an attention mechanism for sketch-based image retrieval (SBIR). The study conducted in this work is based on features extracted using deep convolutional neural networks (ConvNets). In order to overcome the SBIR cross-domain challenge (i.e. searching for photographs from sketch queries), we use triplet loss to train ConvNets to compute shared embedding for both sketches and images. Our main novel contribution is to combine such triplet networks with an attention mechanism. Our approach outperform previous state-of-the-art on challenging SBIR benchmarks. We achieved a recall of 41.66% (at) for the sketchy database (more than 4% improvement), a Kendal score of 42.9on the TU-Berlin SBIR benchmark (close to 5.5improvement) and a mean average precision (MAP) of 31% on Flickr15k (a category level SBIR benchmark).