Towards a Compositional Generative Model of Human Vision

迈向人类视觉的组合生成模型

• 批准号: 10018020
• 负责人: DANIEL J KERSTEN
• 金额: $ 33.29万
• 依托单位: UNIVERSITY OF MINNESOTA
• 依托单位国家: 美国
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-09-30 至 2023-07-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10018020
• 关键词: Affect; Area; Articulation; Behavioral; Body Image; Body part; Brain; Brain imaging; Complex; Computer Vision Systems; Confusion; Cues; Data; Development; Disease; Elbow; Feedback; Functional Magnetic Resonance Imaging; Future; Goals; Human; Human body; Hybrids; Image; Impairment; Individual; Knowledge; Link; Measures; Modeling; Perception; Performance; Predictive Value; Process; Psychophysics; Published Comment; Research; Structure; System; Testing; Training; Vision; Visual; Visual system structure; Work; Wrist; base; convolutional neural network; crowdsourcing; human model; imaging modality; improved; object recognition; relating to nervous system; spatial relationship; success; theories; vision science

Understanding object recognition has long been a central problem in vision science, because of its applied utility and computational difficulty. Progress has been slow, because of an inability to process complex natural images, where the largest challenges arise. Recently, advances in Deep Convolutional Neural Networks (DCNNs) spurred unprecedented success in natural image recognition. The general goal of this proposal is to leverage this success to test computational theories of human object recognition in natural images. However, DCNNs still markedly underperform humans when challenged with high levels of ambiguity, occlusion, and articulation. We hypothesize that humans' superior performance arises from the use of knowledge about how images and objects are structured. Preliminary evidence for this claim comes from the success of hybrid models, that combine DCNNS for identifying features and parts in images, with explicit knowledge of object and image structure. These computations occur within a hierarchy, which includes both top-down and bottom- up processing. The specific goal of the work proposed here is to strongly test whether these computational strategies, structured, hierarchical representations and bidirectional processing, are used to recognize objects in natural images. Human bodies are composed of hierarchically organized configurable parts, making them an ideal test domain. We examine the complete recognition process, from parts, to pairs of parts, to whole bodies, each in its own aim. Each aim also tests important sub-hypotheses about when and how the computational strategies are used. Aim 1 examines recognition of individual body parts, testing whether it is dependent on parsing images into more basic features and relationships, for example edges and materials. Aim 2 examines pairs of parts, testing the importance of knowledge of body connectedness relationships. Aim 3 examines perception of entire bodies, testing whether knowledge of global body structure guides bidirectional processing. In each aim, we first develop nested computer vision models that either do or do not make use of structural knowledge, to test whether it aids recognition. We then test whether human performance can be accounted for by the availability of that structural knowledge. We next measure neural activity with functional MRI to identify where and how it is used in cortex. Finally, we integrate these results to produce even stronger tests, using the nested models to predict human performance and confusion matrices as well as fMRI activity levels and confusion matrices. Altogether, this work will strongly test key theoretical accounts of object recognition in the most important domain, perception of natural images. The work, based on extensive preliminary data, measures and models the entire body recognition system. The models developed and tested here should surpass the state-of-the-art, and be useful for many real-world recognition tasks. The proposal will also lay the groundwork for future studies of recognition impaired by disease.

理解物体识别长期以来一直是视觉科学的一个中心问题，因为它的应用 实用性和计算难度。由于无法处理复杂的自然现象，进展缓慢 图像，其中出现最大的挑战。最近，深度卷积神经网络的进展 （DCNN）在自然图像识别领域取得了前所未有的成功。该提案的总体目标是 利用这一成功来测试自然图像中人体物体识别的计算理论。然而， 当面临高水平的模糊性、遮挡和模糊性挑战时，DCNN 的表现仍然明显不如人类。 关节。我们假设人类的卓越表现源于对如何运用知识的运用 图像和物体都是结构化的。这一说法的初步证据来自于混合动力车的成功 模型，结合 DCNNS 来识别图像中的特征和部分，并具有对象的明确知识 和图像结构。这些计算发生在一个层次结构中，其中包括自上而下和自下而上的计算。 上处理。这里提出的工作的具体目标是强烈测试这些计算是否 策略、结构化、分层表示和双向处理用于识别对象 在自然图像中。人体由分层组织的可配置部分组成，使它们成为 理想的测试域。我们检查完整的识别过程，从零件到零件对，再到整个身体， 每个人都有自己的目标。每个目标还测试关于计算何时以及如何进行的重要子假设。 使用策略。目标 1 检查对各个身体部位的识别，测试它是否依赖于 将图像解析为更基本的特征和关系，例如边缘和材质。目标 2 检查 成对的零件，测试身体关联关系知识的重要性。目标 3 检查 整个身体的感知，测试整体身体结构的知识是否指导双向处理。 在每个目标中，我们首先开发嵌套的计算机视觉模型，这些模型要么使用或不使用结构 知识，测试它是否有助于识别。然后我们测试是否可以解释人类的表现 通过结构知识的可用性。接下来我们用功能性 MRI 测量神经活动来识别 它在皮质中的使用地点和方式。最后，我们使用以下方法整合这些结果以产生更强大的测试 嵌套模型来预测人类表现和混淆矩阵以及功能磁共振成像活动水平和 混淆矩阵。总而言之，这项工作将有力地测试物体识别的关键理论解释。 最重要的领域，自然图像的感知。这项工作基于大量的初步数据， 测量和建模整个身体识别系统。这里开发和测试的模型应该 超越最先进的技术，并且对许多现实世界的识别任务很有用。该提案还将规定 为未来研究因疾病而受损的识别奠定基础。