Modeling rich inter-image relationships in big visual collections

在大型视觉集合中建模丰富的图像间关系

• 批准号: 1514512
• 负责人: Alexei Efros
• 金额: $ 22.59万
• 依托单位: University of California-Berkeley
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-09-01 至 2017-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1514512&HistoricalAwards=false
• 关键词: Modeling; rich; inter; image; relationships

The Directorate of Social, Behavioral and Economic Sciences offers postdoctoral research fellowships to provide opportunities for recent doctoral graduates to obtain additional training, to gain research experience under the sponsorship of established scientists, and to broaden their scientific horizons beyond their undergraduate and graduate training. Postdoctoral fellowships are further designed to assist new scientists to direct their research efforts across traditional disciplinary lines and to avail themselves of unique research resources, sites, and facilities, including at foreign locations. This postdoctoral fellowship supports a rising scientist in the interdisciplinary area overlapping computer vision and psychology, with a research project that investigates the web of relationships within visual data in both humans and machines. To a human observer, no photograph is an island: it is connected to the rest of the visual world by a web of similarities, associations, and other relationships. For example, two photos of Paris share a certain similarity; images of boats are associated with images of water; a photo of a tadpole and a photo of a frog show the same organism at two stages of life. In each of these cases, a human can readily reason about the link between two images. Not only can people identify that a relationship exists, but can also identify the nature of this relationship. These relationships shed light on how the human brain organizes visual information, and also give insight into how to build intelligent systems that automatically make visual connections. The latter will bring the field closer to producing an intelligent visual web, able to organize visual information in the same way as the current Internet is able to organize text. Computer vision scientists and psychologists have both studied relationships between visual data, but from different directions. In computer vision, the focus has been on models of natural image similarity. These models handle complex stimuli but are usually limited to one simple kind of relationship, namely similarity in appearance. Psychologists have studied a richer set of relationships - association, causation, analogy, antonymy, transformation, etc. - but their models usually only apply to simple, artificial stimuli. This project unites the best of both fields by modeling subtle visual relationships between complex, natural images. The objective is to model both which images humans consider to be related and how are those images related. An additional objective is to study how certain relationships, such as visual associations, can arise in an unsupervised manner from natural visual experience. This will help explain how humans might learn about the relationships in the first place. Better models of inter-image relationships will have deep implications across cognitive psychology. In particular, similarity and association play fundamental roles in theories of human learning and memory. A sense of similarity underlies our ability to learn from one visual experience and then apply our knowledge in a future, similar setting. The associations made from the experience additionally impact human memory of it. The present project also has applications toward computer vision systems. Reverse image search has recently become a popular tool. However, current systems are only able to retrieve look-alike images. If the computer is instead able to retrieve images linked by more diverse kinds of relationships, many possibilities open up. For example, one could imagine a system that lets users navigate through artistic styles, or that recommends shoes that match a pair of pants. If successful, this project could pave the way toward a world-wide web of visual connections that parallels the current web of hypertext connections.

社会、行为和经济科学局提供博士后研究奖学金，为新近毕业的博士毕业生提供获得额外培训的机会，在知名科学家的赞助下获得研究经验，并在本科生和研究生培训之外拓宽他们的科学视野。博士后奖学金的进一步设计是为了帮助新科学家引导他们的研究努力跨越传统的学科界限，并利用独特的研究资源、地点和设施，包括在国外。这一博士后奖学金支持一位在计算机视觉和心理学交叉学科领域崭露头角的科学家，他的研究项目调查了人类和机器视觉数据中的关系网络。对于人类观察者来说，没有一张照片是一座孤岛：它通过相似之处、联想和其他关系的网络与视觉世界的其他部分相连。例如，巴黎的两张照片有一定的相似性；船只的图像与水的图像联系在一起；一张蝌蚪的照片和一张青蛙的照片显示了同一生物在生命的两个阶段。在每一种情况下，人类都可以很容易地推理出两个图像之间的联系。人们不仅可以识别一段关系的存在，还可以识别这种关系的性质。这些关系揭示了人脑如何组织视觉信息，也让我们深入了解如何建立自动建立视觉联系的智能系统。后者将使该领域更接近于生产智能可视网络，能够像目前的互联网能够组织文本一样组织视觉信息。计算机视觉科学家和心理学家都从不同的角度研究了视觉数据之间的关系。在计算机视觉中，重点一直放在自然图像相似性模型上。这些模型处理复杂的刺激，但通常仅限于一种简单的关系，即外观相似。心理学家已经研究了一组更丰富的关系--联想、因果关系、类比、反义词、转化等等--但他们的模型通常只适用于简单的、人为的刺激。该项目通过对复杂的自然图像之间微妙的视觉关系进行建模，将这两个领域的最佳效果结合在一起。目标是对人类认为哪些图像相关以及这些图像是如何相关的进行建模。另一个目标是研究某些关系，如视觉联想，如何在自然的视觉体验中以无人监督的方式产生。这将有助于解释人类最初是如何了解这些关系的。更好的图像间关系模型将对认知心理学产生深远的影响。特别是，相似性和联想在人类学习和记忆理论中发挥着基础性的作用。相似感是我们从一次视觉体验中学习，然后将我们的知识应用到未来类似环境中的能力。从这一经历中产生的联想还会影响人类对它的记忆。本项目也适用于计算机视觉系统。最近，反向图像搜索已经成为一种流行的工具。然而，目前的系统只能检索外观相似的图像。相反，如果计算机能够检索由更多种类的关系链接的图像，就会出现许多可能性。例如，你可以想象一个让用户浏览艺术风格的系统，或者推荐一双与裤子相匹配的鞋子。如果成功，这个项目可能会为建立一个与当前的超文本连接网络相媲美的全球视觉连接网络铺平道路。