Zero-shot Image Recognition Using Relational Matching, Adaptation and Calibration
Zero-shot Image Recognition Using Relational Matching, Adaptation and Calibration
复制标题
DOI：
10.1109/ijcnn.2019.8852315
复制
发表时间：
2019-03
期刊：
影响因子：
--
通讯作者：
Debasmit Das;C. S. G. Lee
中科院分区：
文献类型：
--
作者：
Debasmit Das;C. S. G. Lee
零射击学习（Zero-shot learning， ZSL）用于图像分类的重点是识别没有标记数据可用于训练的新类别。学习通常是在与每个类相关的中级语义描述符的帮助下进行的。这个语义描述符空间通常由可见类别和不可见类别共享。然而，ZSL存在中心性、领域差异和类偏向性。为了解决这些问题，我们提出了一种分三步的零射击学习方法。首先，学习从语义描述符空间到图像特征空间的映射。这种映射学习最小化语义嵌入与相应类的图像特征之间的一对一和成对距离。其次，我们提出了测试时域自适应，使未见类的语义嵌入适应于测试数据。这是通过寻找语义描述符和图像特征之间的对应关系来实现的。第三，我们提出了对所见类的分类分数进行比例校准。这是必要的，因为ZSL模型偏向于可见类，因为未见类不用于训练。最后，为了验证所提出的三步方法，我们在四个基准数据集上进行了实验，其中所提出的方法优于先前的结果。我们还研究和分析了我们提出的ZSL框架的每个组件的性能。
Zero-shot learning (ZSL) for image classification focuses on recognizing novel categories that have no labeled data available for training. The learning is generally carried out with the help of mid-level semantic descriptors associated with each class. This semantic-descriptor space is generally shared by both seen and unseen categories. However, ZSL suffers from hubness, domain discrepancy and biased-ness towards seen classes. To tackle these problems, we propose a three-step approach to zero-shot learning. Firstly, a mapping is learned from the semantic-descriptor space to the image-feature space. This mapping learns to minimize both one-to-one and pairwise distances between semantic embeddings and the image features of the corresponding classes. Secondly, we propose test-time domain adaptation to adapt the semantic embedding of the unseen classes to the test data. This is achieved by finding correspondences between the semantic descriptors and the image features. Thirdly, we propose scaled calibration on the classification scores of the seen classes. This is necessary because the ZSL model is biased towards seen classes as the unseen classes are not used in the training. Finally, to validate the proposed three-step approach, we performed experiments on four benchmark datasets where the proposed method outperformed previous results. We also studied and analyzed the performance of each component of our proposed ZSL framework.