Object-Part Attention Model for Fine-Grained Image Classification
Object-Part Attention Model for Fine-Grained Image Classification
复制标题
用于细粒度图像分类的对象部分注意力模型
DOI：
10.1109/tip.2017.2774041
复制
发表时间：
2018-03-01
影响因子：
10.6
通讯作者：
Zhao, Junjie
中科院分区：
文献类型：
--
作者：
Peng, Yuxin;He, Xiangteng;Zhao, Junjie
细颗粒的图像分类是识别属于同一基本级别类别的数百个子类别，例如属于鸟类的200个子类别，这是由于在相同的子类别中的较大差异和不同子类别之间的较小差异，这是高度挑战的。现有方法通常首先找到对象或零件，然后区分图像所属的子类别。但是，它们主要有两个局限性：1）依靠对劳动力消耗的物体或部分注释； 2）忽略对象及其部分之间以及这些部分之间的空间关系，这两种关系对于查找区分零件都非常有帮助。因此，本文提出了针对弱监督的细颗粒图像分类的对象零件注意模型（OPAM），主要新颖性是：1）对象零件注意模型集成了两个级别的关注：对象级别的注意力局限于图像的对象，并且部分级别的注意选择对象的歧视部分。两者共同聘请学习多视图和多尺度特征，以增强他们的相互促进； 2）对象零件空间约束模型结合了两个空间约束：对象空间约束确保所选零件高度代表性和部分空间约束可消除冗余，并增强对所选零件的歧视。两者共同使用来利用细微和局部差异来区分子类别。重要的是，在我们提出的方法中均未使用对象和部分注释，从而避免了大量的标签劳动消耗。与四个广泛使用的数据集上的十多种最先进的方法相比，我们的OPAM方法实现了最佳性能。
Fine-grained image classification is to recognize hundreds of subcategories belonging to the same basic-level category, such as 200 subcategories belonging to the bird, which is highly challenging due to large variance in the same subcategory and small variance among different subcategories. Existing methods generally first locate the objects or parts and then discriminate which subcategory the image belongs to. However, they mainly have two limitations: 1) relying on object or part annotations which are heavily labor consuming; and 2) ignoring the spatial relationships between the object and its parts as well as among these parts, both of which are significantly helpful for finding discriminative parts. Therefore, this paper proposes the object-part attention model (OPAM) for weakly supervised fine-grained image classification and the main novelties are: 1) object-part attention model integrates two level attentions: object-level attention localizes objects of images, and part-level attention selects discriminative parts of object. Both are jointly employed to learn multi-view and multi-scale features to enhance their mutual promotion; and 2) Object-part spatial constraint model combines two spatial constraints: object spatial constraint ensures selected parts highly representative and part spatial constraint eliminates redundancy and enhances discrimination of selected parts. Both are jointly employed to exploit the subtle and local differences for distinguishing the subcategories. Importantly, neither object nor part annotations are used in our proposed approach, which avoids the heavy labor consumption of labeling. Compared with more than ten state-of-the-art methods on four widely-used datasets, our OPAM approach achieves the best performance.