Efficient and interactive spatial-semantic image retrieval
Efficient and interactive spatial-semantic image retrieval
复制标题
DOI：
10.1007/s11042-018-7148-1
复制
发表时间：
2018-02
影响因子：
3.6
通讯作者：
Ryosuke Furuta;Naoto Inoue;T. Yamasaki
中科院分区：
文献类型：
--
作者：
Ryosuke Furuta;Naoto Inoue;T. Yamasaki
本文提出了一个高效的图像检索系统。当用户希望检索具有语义和空间约束的图像时（例如，一匹马位于图像的中心，一个人骑在马背上），传统的基于文本的检索系统很难准确地检索这样的图像。相比之下，所提出的系统可以考虑语义和空间信息，因为它是基于使用全卷积网络（FCN）的语义分割。所提出的系统可以接受三种类型的图像作为查询：由用户绘制的分割图，自然图像，或两者的组合。查询和数据库中每个图像之间的距离是基于FCN的输出概率图计算的。为了使系统在计算时间和内存使用方面都有效，我们采用了乘积量化（PQ）技术。实验结果表明，PQ与基于模糊控制网的图像检索系统兼容，量化过程导致的信息损失很小。它还表明，我们的方法优于传统的基于文本的搜索系统。
This paper proposes an efficient image retrieval system. When users wish to retrieve images with semantic and spatial constraints (e.g., a horse is located at the center of the image, and a person is riding on the horse), it is difficult for conventional text-based retrieval systems to retrieve such images exactly. In contrast, the proposed system can consider both semantic and spatial information, because it is based on semantic segmentation using fully convolutional networks (FCN). The proposed system can accept three types of images as queries: a segmentation map sketched by the user, a natural image, or a combination of the two. The distance between the query and each image in the database is calculated based on the output probability maps from the FCN. In order to make the system efficient in terms of both the computational time and memory usage, we employ the product quantization (PQ) technique. The experimental results show that the PQ is compatible with the FCN-based image retrieval system, and that the quantization process results in little information loss. It is also shown that our method outperforms a conventional text-based search system.