Automatic and quantitative evaluation of attribute discovery methods
Automatic and quantitative evaluation of attribute discovery methods
复制标题
DOI：
10.1109/wacv.2016.7477693
复制
发表时间：
2016-02
期刊：
影响因子：
--
通讯作者：
Liangchen Liu;A. Wiliem;Shaokang Chen;B. Lovell
中科院分区：
文献类型：
--
作者：
Liangchen Liu;A. Wiliem;Shaokang Chen;B. Lovell
已经开发了许多自动属性发现方法来从图像中提取用于各种任务的视觉属性集。然而，尽管在一些图像分类任务中表现良好，但很难评估这些方法是否发现有意义的属性，以及哪一个是最好的图像描述属性。评估这一点的直观方法是手动验证是否存在一致的可识别视觉概念，以区分属性的正面和负面图像。这种人工检查是繁琐的，劳动密集型和昂贵的，并且很难在不同方法之间进行定量比较。在这项工作中，我们解决这个问题，提出了一个属性的意义度量，可以执行自动评估的属性集的意义，以及实现定量比较。我们将我们提出的度量，最近的自动属性发现方法和流行的散列方法的三个属性数据集。还进行了用户研究，以验证该指标的有效性。在我们的评估中，我们收集了一些见解，可能有利于开发自动属性发现方法，以生成有意义的属性。据我们所知，这是第一个定量测量自动发现的属性的语义内容的工作。
Many automatic attribute discovery methods have been developed to extract a set of visual attributes from images for various tasks. However, despite good performance in some image classification tasks, it is difficult to evaluate whether these methods discover meaningful attributes and which one is the best to find the attributes for image descriptions. An intuitive way to evaluate this is to manually verify whether consistent identifiable visual concepts exist to distinguish between positive and negative images of an attribute. This manual checking is tedious, labor intensive and expensive and it is very hard to get quantitative comparisons between different methods. In this work, we tackle this problem by proposing an attribute meaningfulness metric, that can perform automatic evaluation on the meaningfulness of attribute sets as well as achieving quantitative comparisons. We apply our proposed metric to recent automatic attribute discovery methods and popular hashing methods on three attribute datasets. A user study is also conducted to validate the effectiveness of the metric. In our evaluation, we gleaned some insights that could be beneficial in developing automatic attribute discovery methods to generate meaningful attributes. To the best of our knowledge, this is the first work to quantitatively measure the semantic content of automatically discovered attributes.