Human and machine perception of 2D and 3D shape from contour

人类和机器从轮廓感知 2D 和 3D 形状

• 批准号: RGPIN-2022-04533
• 负责人: Elder, James
• 金额: $ 4.01万
• 依托单位: York University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=764005
• 关键词: Human; machine; perception; 2D; 3D

We associate vision with the eye, but in fact it is a network of visual areas in the brain that allows us to make sense of the visual images formed by the eye. This network performs the computations required to solve a remarkable diversity of visual tasks - rapidly organizing the image into the distinct surfaces forming the 3D scene, recognizing and manipulating objects, and navigating safely and efficiently. The goal of the proposed research is to understand these computational processes, and to develop better machine vision systems for artificial intelligence applications based on this understanding. We focus here specifically on human and machine perception of 2D and 3D shape and scene layout from contour. Image contours arise from sharp changes in luminance, colour and texture corresponding to important features of the scene, including the boundaries of objects and changes in material and lighting. These contours form strong cues to object/surface shape and the human visual system is known to be exquisitely sensitive to these cues. Yet much remains unknown about the neural computations that underlie this sensitivity, and these cues are under-utilized by computer vision systems. Our research aims to elucidate these neural computations and deliver useful shape-from-contour computer vision algorithms based on our findings. This is a deeply interdisciplinary endeavour, involving 1) systems-level psychophysical studies of human perception, 2) modeling of neural computations in the brain, 3) mathematical and statistical modeling relating the physics of our visual environment to image observations and 4) computational theory, models and computer vision algorithms for making useful inferences from image data. Deep neural network (DNN) models play a central role in current computational vision research. Performance has reached human levels on some tasks and DNN models now serve as our most accurate predictors for physiological and behavioural response to object stimuli. However, significant deviations between DNN models and biological perception have been noted and their high dimensionality limits their contribution to scientific understanding and their trustability in applications. Our research will address these limitations through changes in training and architecture and through integration with more explainable computational approaches that rely on information theory and optimal estimation theory. In the short term, results of this research will lead to a better scientific understanding of the human brain and more trustable object processing systems for performance-critical applications such as autonomous driving and robot navigation. In the long term, this research program will strengthen Canada's position as a world-leader in computational neuroscience and bio-inspired AI research.

我们将视觉与眼睛联系在一起，但实际上它是大脑中视觉区域的网络，使我们能够理解眼睛形成的视觉图像。该网络执行解决各种视觉任务所需的计算-快速将图像组织成形成3D场景的不同表面，识别和操纵物体，以及安全有效地导航。拟议研究的目标是了解这些计算过程，并在此基础上为人工智能应用开发更好的机器视觉系统。 我们在这里特别关注人类和机器对2D和3D形状的感知以及轮廓的场景布局。图像轮廓是由亮度、颜色和纹理的急剧变化引起的，这些变化对应于场景的重要特征，包括物体的边界以及材料和照明的变化。这些轮廓形成对物体/表面形状的强烈提示，并且已知人类视觉系统对这些提示非常敏感。然而，关于这种敏感性背后的神经计算仍有很多未知之处，并且这些线索未被计算机视觉系统充分利用。 我们的研究旨在阐明这些神经计算，并根据我们的发现提供有用的轮廓形状计算机视觉算法。这是一项深入的跨学科努力，涉及1）人类感知的系统级心理物理学研究，2）大脑中神经计算的建模，3）将我们的视觉环境物理学与图像观察相关联的数学和统计建模，以及4）计算理论，模型和计算机视觉算法，用于从图像数据中进行有用的推断。 深度神经网络（DNN）模型在当前的计算视觉研究中发挥着核心作用。在某些任务上，性能已经达到了人类的水平，DNN模型现在是我们对物体刺激的生理和行为反应的最准确的预测器。然而，DNN模型和生物感知之间的显着偏差已经被注意到，它们的高维性限制了它们对科学理解的贡献及其在应用中的可信度。我们的研究将通过改变训练和架构，并通过与依赖于信息论和最优估计理论的更可解释的计算方法相结合来解决这些限制。在短期内，这项研究的结果将有助于更好地科学理解人类大脑，并为自动驾驶和机器人导航等性能关键型应用提供更可靠的对象处理系统。从长远来看，这项研究计划将加强加拿大在计算神经科学和生物启发人工智能研究方面的世界领先地位。