Systematically evaluating the psychological validity of computer-vision models for vision-based conceptual representations

人类擅长于从环境结构中提取信息，并能从这些信息中建立一个概念系统，帮助他们驾驭世界。在过去的二十年里，我们看到了成功的计算模型的兴起，这些模型模仿了人类模式学习的能力，主要是在语言领域。这是因为大量的语言数据可以很容易地以数字化语料库的形式获得，并且因为相对简单的算法（称为分布语义模型）在从文本中的词分布中提取以词义及其相似性形式的概念信息方面已经非常成功。然而，人类的概念系统不仅受到语言经验的影响，而且在很大程度上受到我们对周围世界的感知经验的影响。例如，我们会说披萨更像硬币而不是钞票，尽管它们的意思是不相关的。计算机视觉领域的最新进展现在允许我们从大量图像集合中获得基于视觉的表示，这反过来又允许我们在单个图像和概念原型级别（通过在一个类别中的单个图像上聚合信息）上计算这种基于视觉的相似性的定量度量。我们以前使用这些模型的研究已经产生了非常有希望的结果。然而，对这种基于视觉的系统评估仍然缺乏，这与基于语言的研究形成了鲜明的对比。本项目旨在提供这种系统的评估，以便建立这些基于视觉的模型，作为心理学（和其他）研究的可靠和富有成效的资源。为此，我们计划研究(a)模型衍生的视觉相似性得分的直观合理性，(b)样本（单个图像）与原型之间的相似性，(c)时间压力下视觉相似性在在线处理中的作用，以及(d)模型是否可以将视觉相似性作为一种纯粹的感知现象来衡量，而不受高级概念干扰。所有调查将以大规模实验的形式进行，以建立坚实的经验数据库，并创建高质量，可靠的数据集，可作为未来研究的金标准。此外，该项目旨在创建一个易于使用的web界面，这样就不需要事先的技术知识来访问基于视觉的模型和从中派生的度量（这样的接口已经被证明是基于语言的模型的有价值的资产）。

Humans excel in extracting information from the structure of their environment, and can build a conceptual system from this information that helps them navigate the world. The last two decades have seen the rise of successful computational models that emulate this human capacity of pattern learning, mainly in the domain of language. This is because large amounts of language data are readily available in the form of digitized corpora, and because relatively simple algorithms (referred to as distributional semantic models) have been remarkably successful in extracting conceptual information in the form of word meanings – and their similarities – from word distributions in text. However, the human conceptual system is not only informed by language experience, but is also heavily shaped by our perceptual experience with the world around us. For example, we would say that a pizza is more similar to a coin than to a bank note, even though their word meanings are as unrelated. Recent advancements in the field of computer vision now allow us to derive vision-based representations from large collections of images, which in turn allows us to compute quantitative measures of this vision-based similarity, both on the level of individual images and on a conceptual prototype level (by aggregating information over the individual images in a category). Previous studies in which we employed these models have yielded very promising results. However, a systematic evaluation of this vision-based system is still missing, which stands in stark contrast to the language-based research. The present project is aimed at providing this systematic evaluation, in order to establish these vision-based models as a reliable and fruitful resource for psychological (and other) studies. To this end, we plan to investigate (a) the intuitive plausibility of the model-derived visual similarity scores, (b) the similarity between exemplars (individual images) and prototypes, (c) the role of visual similarity in on-line processing under time pressure, and (d) whether the model can measure visual similarity as a purely perceptual phenomenon, without high-level conceptual interference. All investigations will be conducted in the form of large-scale experiments, to establish a solid empirical database and to create high-quality, reliable datasets that can be used as gold standards in future research. In addition, the project aims at creating an easy-to-use web interface, so that no prior technical knowledge is required to access the vision-based models and the measures derived from them (such interfaces have proven to be valuable assets for the language-based models).