Systematically evaluating the psychological validity of computer-vision models for vision-based conceptual representations

人类擅长从环境的结构中提取信息，并可以从这些信息中建立一个概念系统，帮助他们导航世界。在过去的二十年里，成功的计算模型已经出现，这些模型模仿了人类的模式学习能力，主要是在语言领域。这是因为大量的语言数据以数字化语料库的形式容易获得，并且因为相对简单的算法（称为分布式语义模型）已经非常成功地从文本中的单词分布中以单词含义及其相似性的形式提取概念信息。然而，人类的概念系统不仅受到语言经验的影响，而且还受到我们对周围世界的感知经验的影响。例如，我们会说比萨饼更像硬币而不是钞票，即使它们的词义是无关的。计算机视觉领域的最新进展现在允许我们从大量图像集合中获得基于视觉的表示，这反过来又允许我们在单个图像和概念原型级别上（通过聚合类别中单个图像的信息）计算这种基于视觉的相似性的定量度量。我们采用这些模型的先前研究已经取得了非常有希望的结果。然而，这一基于视觉的系统仍然缺乏系统的评价，这与基于语言的研究形成鲜明对比。本项目的目的是提供这种系统的评估，以建立这些基于视觉的模型作为一个可靠的和富有成效的资源，心理（和其他）的研究。为此，我们计划调查（a）模型衍生的视觉相似性分数的直观的可扩展性，（B）样本（个人图像）和原型之间的相似性，（c）视觉相似性在时间压力下的在线处理中的作用，以及（d）模型是否可以测量视觉相似性作为一个纯粹的感知现象，没有高层次的概念干扰。所有的调查将以大规模实验的形式进行，以建立一个坚实的经验数据库，并创建高质量，可靠的数据集，可作为未来研究的黄金标准。 此外，该项目旨在创建一个易于使用的网络界面，这样就不需要事先具备技术知识即可访问基于视觉的模型及其衍生措施（此类界面已被证明是基于语言的模型的宝贵资产）。

Humans excel in extracting information from the structure of their environment, and can build a conceptual system from this information that helps them navigate the world. The last two decades have seen the rise of successful computational models that emulate this human capacity of pattern learning, mainly in the domain of language. This is because large amounts of language data are readily available in the form of digitized corpora, and because relatively simple algorithms (referred to as distributional semantic models) have been remarkably successful in extracting conceptual information in the form of word meanings – and their similarities – from word distributions in text. However, the human conceptual system is not only informed by language experience, but is also heavily shaped by our perceptual experience with the world around us. For example, we would say that a pizza is more similar to a coin than to a bank note, even though their word meanings are as unrelated. Recent advancements in the field of computer vision now allow us to derive vision-based representations from large collections of images, which in turn allows us to compute quantitative measures of this vision-based similarity, both on the level of individual images and on a conceptual prototype level (by aggregating information over the individual images in a category). Previous studies in which we employed these models have yielded very promising results. However, a systematic evaluation of this vision-based system is still missing, which stands in stark contrast to the language-based research. The present project is aimed at providing this systematic evaluation, in order to establish these vision-based models as a reliable and fruitful resource for psychological (and other) studies. To this end, we plan to investigate (a) the intuitive plausibility of the model-derived visual similarity scores, (b) the similarity between exemplars (individual images) and prototypes, (c) the role of visual similarity in on-line processing under time pressure, and (d) whether the model can measure visual similarity as a purely perceptual phenomenon, without high-level conceptual interference. All investigations will be conducted in the form of large-scale experiments, to establish a solid empirical database and to create high-quality, reliable datasets that can be used as gold standards in future research. In addition, the project aims at creating an easy-to-use web interface, so that no prior technical knowledge is required to access the vision-based models and the measures derived from them (such interfaces have proven to be valuable assets for the language-based models).