Effectively Leveraging Attributes for Visual Similarity-猫眼课题宝

DOI：

10.1109/iccv48922.2021.00105

发表时间：

2021-05

期刊：

2021 IEEE/CVF International Conference on Computer Vision (ICCV)

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Samarth Mishra;Zhongping Zhang;Yuan Shen;Ranjitha Kumar;Venkatesh Saligrama;Bryan A. Plummer

测量两个图像之间的相似性通常需要沿沿着不同的轴（例如，颜色、纹理或形状）。注释属性可以提供对度量相似性的重要性的见解。先前的工作倾向于将这些注释视为完整的，导致它们使用预测单个图像上的属性的简单方法，这些属性反过来又用于测量相似性。但是，对数据集来说，完全注释每个可能重要的属性是不切实际的。因此，仅基于这些不完整的注释来表示图像可能会错过关键信息。为了解决这个问题，我们提出了成对属性信息相似性网络（PAN），它将相似性学习分解为从两个图像的联合表示中捕获相似性条件和相关性得分。这使得我们的模型能够识别两个图像包含相同的属性，但可以将其视为不相关的（例如，由于它们之间的细粒度差异），并且在测量两个图像之间的相似性时被忽略。值得注意的是，虽然使用属性注释的现有方法通常无法优于现有技术，但PAN在Polyvore Outfits上的服装物品之间的兼容性预测上获得了4-9%的改进，在使用Caltech-UCSD Birds（CUB）的图像的少数镜头分类上获得了5%的增益，并且在In-Shop Clothes Retrieval上获得了超过1%的Recall@1提升。可在https://github.com/samarth4149/PAN上获得实施

Measuring similarity between two images often requires performing complex reasoning along different axes (e.g., color, texture, or shape). Insights into what might be important for measuring similarity can be provided by annotated attributes. Prior work tends to view these annotations as complete, resulting in them using a simplistic approach of predicting attributes on single images, which are, in turn, used to measure similarity. However, it is impractical for a dataset to fully annotate every attribute that may be important. Thus, only representing images based on these incomplete annotations may miss out on key information. To address this issue, we propose the Pairwise Attribute-informed similarity Network (PAN), which breaks similarity learning into capturing similarity conditions and relevance scores from a joint representation of two images. This enables our model to identify that two images contain the same attribute, but can have it deemed irrelevant (e.g., due to fine-grained differences between them) and ignored for measuring similarity between the two images. Notably, while prior methods of using attribute annotations are often unable to outperform prior art, PAN obtains a 4-9% improvement on compatibility prediction between clothing items on Polyvore Outfits, a 5% gain on few shot classification of images using Caltech-UCSD Birds (CUB), and over 1% boost to Recall@1 on In-Shop Clothes Retrieval. Implementation available at https://github.com/samarth4149/PAN

Effectively Leveraging Attributes for Visual Similarity

文献摘要

相似文献