CompCog: A Machine Learning Approach to Human Perceptual Similarity

相似性是人类认知几乎所有方面的基础。感知利用相似性：当看到一个人的脸时，我们（无意识地）计算它与我们认识的人的脸的相似性，以识别我们在看的是谁。分类使用相似性：当判断一个建筑是否由建筑师弗兰克·劳埃德·赖特设计时，我们计算它与已知由赖特设计的建筑的相似性，以便做出我们最好的估计。推理和解决问题使用相似性：当试图解决一个微积分问题时，我们计算它与之前遇到的问题的相似性，以确定一个好的解决策略。然而，人们如何计算两件物品的相似度还不清楚。人们用物品的哪些特征来计算相似性？为了计算相似度，如何比较项目的特征值？这个研究项目将使用人体实验和计算模型来解决这些问题，当物品被观看或抓住时。该项目的一个长期好处是，对人们感知相似性判断的更深入了解将为理解人们如何在其他认知领域计算和使用相似性提供基础。在进行研究时，本科生和研究生将通过参与研究项目的实验和计算方面，在我们的调查中体现的跨学科方法方面得到指导。这个项目的重点是发展一个新的经验和理论基础，以理解人们的相似性概念，特别是在感知相似性领域。认知科学领域非常清楚，理解相似性对于理解人类认知至关重要。尽管如此，这个项目的主要动机是相信，到目前为止，认知科学研究相似性判断的方法过于简单——认知科学家所考虑的有限类别的相似性指标不太可能扩展到大型，现实的设置。这个项目的主要假设是，机器学习领域——尤其是度量学习的研究——可以为认知科学提供丰富的复杂和复杂的模型，这些模型将是在大的、现实的领域中准确表征人们的相似性概念所必需的。机器学习开创了数学上严格的线性和非线性相似性度量的研究。我们认为，认知科学领域利用机器学习最新进展的时机已经成熟。机器学习的度量学习框架在原则性和创新性的新方向上扩展和阐述了认知科学方法。事实上，这个框架为认知科学提供了一个无与伦比的机会，有可能改变这一领域。利用机器学习的经验和理论发现，认知科学家现在可以开始以更复杂和更复杂的方式探索人类的相似性概念——在更现实的领域——比以往任何时候都可能。我们认为这个研究项目是认知科学朝着更复杂地理解人们的相似性概念迈出的第一步。由于该项目不能研究人类认知的所有领域的相似性，因此它专注于感知。未来的工作将需要进一步发展这里提出和评估的模型。如果成功，该计划将建立一个经验和理论基础，随后可以扩展到人类认知的许多其他领域。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Similarity is fundamental to nearly all aspects of human cognition. Perception uses similarity: when viewing a person's face, we (unconsciously) calculate its similarity to the faces of people we know in order to recognize who we are looking at. Categorization uses similarity: when judging whether a building was designed by the architect Frank Lloyd Wright, we calculate its similarity to buildings known to have been designed by Wright in order to make our best estimate. Reasoning and problem solving use similarity: when attempting to solve a calculus problem, we calculate its similarity to previous problems that we have encountered in order to determine a good solution strategy. However, how people calculate the similarity of two items is not yet understood. Which features of items do people use to calculate similarity? And how are the feature values of items compared in order to calculate similarity? This research project will use human experimentation and computational modeling to address these questions when items are viewed or grasped. A long-term benefit of the project is that a greater understanding of people's perceptual similarity judgments will provide a foundation for understanding how people calculate and use similarity in other areas of cognition. While conducting the research, undergraduate and graduate students will be mentored in the cross-disciplinary approach embodied in our investigation through participation in both experimental and computational aspects of the research project. This project focuses on developing a new empirical and theoretical foundation for understanding people's notions of similarity, particularly in the domain of perceptual similarity. The field of cognitive science is well aware that understanding similarity is essential to understanding human cognition. Despite this, the primary motivation for this project is the belief that, to date, cognitive science's approach to the study of similarity judgments is much too simple---the restricted class of similarity metrics considered by cognitive scientists is unlikely to scale to large, realistic settings. The primary hypothesis of this project is that the field of machine learning---especially the study of metric learning---can supply cognitive science with a rich array of complex and sophisticated models, models that will be necessary to accurately characterize people's similarity notions in large, realistic domains. Machine learning has pioneered the study of mathematically rigorous linear and nonlinear similarity metrics. We believe that the time is ripe for the field of cognitive science to make use of machine learning's recent advances. Machine learning's metric learning framework extends and elaborates the cognitive science approach in principled and innovative new directions. Indeed, this framework presents an unparalleled opportunity for cognitive science with the potential for transforming this field. Using the empirical and theoretical findings from machine learning, cognitive scientists can now begin to explore human notions of similarity in more complex and sophisticated ways---and in more realistic domains---than has ever been possible. We regard the research project as an early step for cognitive science towards a more sophisticated understanding of people's notions of similarity. Because the project cannot study similarity in all domains of human cognition, it concentrates on perception. Future work will need to develop further the models proposed and evaluated here. If successful, the program will establish an empirical and theoretical foundation that can subsequently be extended to many other areas of human cognition.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.