Human and machine perception of 2D and 3D shape from contour

我们将视觉与眼睛联系在一起，但事实上，大脑中的视觉区域网络让我们能够理解眼睛形成的视觉图像。该网络执行解决各种视觉任务所需的计算--快速将图像组织成形成3D场景的不同表面、识别和操纵对象以及安全高效地导航。拟议研究的目标是了解这些计算过程，并在此基础上开发更好的人工智能应用机器视觉系统。我们在这里特别关注人类和机器从轮廓中感知2D和3D形状和场景布局。图像轮廓产生于与场景的重要特征相对应的亮度、颜色和纹理的急剧变化，包括对象的边界以及材质和照明的变化。这些轮廓形成了物体/表面形状的强烈线索，众所周知，人类的视觉系统对这些线索非常敏感。然而，这种敏感性背后的神经计算仍有许多未知之处，计算机视觉系统也没有充分利用这些线索。我们的研究旨在阐明这些神经计算，并基于我们的发现提供有用的从轮廓形状计算机视觉算法。这是一项深入的跨学科研究，涉及1)人类感知的系统级心理物理研究，2)大脑神经计算的建模，3)将我们视觉环境的物理与图像观察联系起来的数学和统计建模，4)从图像数据中做出有用推断的计算理论、模型和计算机视觉算法。深度神经网络(DNN)模型是当前计算视觉研究的核心。在某些任务上的表现已经达到了人类的水平，DNN模型现在可以作为我们对对象刺激的生理和行为反应的最准确的预测指标。然而，DNN模型与生物感知之间存在着显著的偏差，它们的高维性限制了它们对科学理解的贡献以及它们在应用中的可信度。我们的研究将通过改变培训和架构，并通过与依赖于信息论和最优估计理论的更可解释的计算方法相结合，来解决这些限制。短期内，这项研究的结果将导致对人脑的更好的科学理解，以及用于自动驾驶和机器人导航等性能关键型应用的更可靠的对象处理系统。从长远来看，这一研究计划将巩固加拿大在计算神经科学和生物启发的人工智能研究方面的世界领先地位。

We associate vision with the eye, but in fact it is a network of visual areas in the brain that allows us to make sense of the visual images formed by the eye. This network performs the computations required to solve a remarkable diversity of visual tasks - rapidly organizing the image into the distinct surfaces forming the 3D scene, recognizing and manipulating objects, and navigating safely and efficiently. The goal of the proposed research is to understand these computational processes, and to develop better machine vision systems for artificial intelligence applications based on this understanding. We focus here specifically on human and machine perception of 2D and 3D shape and scene layout from contour. Image contours arise from sharp changes in luminance, colour and texture corresponding to important features of the scene, including the boundaries of objects and changes in material and lighting. These contours form strong cues to object/surface shape and the human visual system is known to be exquisitely sensitive to these cues. Yet much remains unknown about the neural computations that underlie this sensitivity, and these cues are under-utilized by computer vision systems. Our research aims to elucidate these neural computations and deliver useful shape-from-contour computer vision algorithms based on our findings. This is a deeply interdisciplinary endeavour, involving 1) systems-level psychophysical studies of human perception, 2) modeling of neural computations in the brain, 3) mathematical and statistical modeling relating the physics of our visual environment to image observations and 4) computational theory, models and computer vision algorithms for making useful inferences from image data. Deep neural network (DNN) models play a central role in current computational vision research. Performance has reached human levels on some tasks and DNN models now serve as our most accurate predictors for physiological and behavioural response to object stimuli. However, significant deviations between DNN models and biological perception have been noted and their high dimensionality limits their contribution to scientific understanding and their trustability in applications. Our research will address these limitations through changes in training and architecture and through integration with more explainable computational approaches that rely on information theory and optimal estimation theory. In the short term, results of this research will lead to a better scientific understanding of the human brain and more trustable object processing systems for performance-critical applications such as autonomous driving and robot navigation. In the long term, this research program will strengthen Canada's position as a world-leader in computational neuroscience and bio-inspired AI research.