Dynamic Match Kernel With Deep Convolutional Features for Image Retrieval
Dynamic Match Kernel With Deep Convolutional Features for Image Retrieval
复制标题
DOI：
10.1109/tip.2018.2845136
复制
发表时间：
2018-06
影响因子：
10.6
通讯作者：
Jufeng Yang;Jie Liang;Hui Shen;K. Wang;Paul L. Rosin;Ming-Hsuan Yang
中科院分区：
文献类型：
--
作者：
Jufeng Yang;Jie Liang;Hui Shen;K. Wang;Paul L. Rosin;Ming-Hsuan Yang
在基于视觉词袋的图像检索方法中，提高图像局部特征的鉴别能力是一个重要的研究方向。虽然检索到的图像通常类似于细节点查询，但从语义角度来看，它们可能会有很大的不同，这可以通过卷积神经网络（CNN）有效地区分开来。此类图像不应被视为相关图像对。为了解决这个问题，我们建议通过基于深度CNN特征之间的成对距离自适应地计算查询和候选图像之间的匹配阈值来构建动态匹配内核。与独立于检索图像的全局外观的典型静态匹配内核相比，动态匹配内核利用语义相似性作为用于确定匹配的约束。因此，我们提出了一个语义约束的检索框架，结合动态匹配内核，它侧重于相关图像之间的匹配补丁和过滤掉不相关的对。此外，我们证明了建议的内核补充最近的方法，如汉明嵌入，多重分配，局部描述符聚合，和基于图的重新排名，而它优于静态的现成的评估指标的各种设置下。我们还建议定量和定性地评估匹配的补丁。在五个基准数据集和大规模干扰项上的实验验证了该方法与现有图像检索方法的优点。
For image retrieval methods based on bag of visual words, much attention has been paid to enhancing the discriminative powers of the local features. Although retrieved images are usually similar to a query in minutiae, they may be significantly different from a semantic perspective, which can be effectively distinguished by convolutional neural networks (CNN). Such images should not be considered as relevant pairs. To tackle this problem, we propose to construct a dynamic match kernel by adaptively calculating the matching thresholds between query and candidate images based on the pairwise distance among deep CNN features. In contrast to the typical static match kernel which is independent to the global appearance of retrieved images, the dynamic one leverages the semantical similarity as a constraint for determining the matches. Accordingly, we propose a semantic-constrained retrieval framework by incorporating the dynamic match kernel, which focuses on matched patches between relevant images and filters out the ones for irrelevant pairs. Furthermore, we demonstrate that the proposed kernel complements recent methods, such as hamming embedding, multiple assignment, local descriptors aggregation, and graph-based re-ranking, while it outperforms the static one under various settings on off-the-shelf evaluation metrics. We also propose to evaluate the matched patches both quantitatively and qualitatively. Extensive experiments on five benchmark data sets and large-scale distractors validate the merits of the proposed method against the state-of-the-art methods for image retrieval.