New computational models of human visual perception of surface colour, 3D shape, and lighting

人类视觉感知表面颜色、3D 形状和照明的新计算模型

• 批准号: RGPIN-2022-04583
• 负责人: Murray, Richard
• 金额: $ 2.04万
• 依托单位: York University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=766062
• 关键词: New; computational; models; human; visual

What could be easier than seeing? We see without trying, and usually without thinking about it. Seeing seems easy because our visual cortex is a powerful computing device with a lifetime of experience. One reason why vision is a challenging computational task, though, is that all images are highly ambiguous: any given image could conceivably be seen as depicting a wide range of shapes, colours, and lighting conditions. In order to perceive things correctly, our visual system must overcome this ambiguity. The long-term goal of my research program is to develop computational models that solve this problem in the same way people do, and see what people see in complex, realistic scenes. The research proposed here will approach this goal in two ways. First, using powerful new methods developed for machine learning and computer vision, I will train artificial neural networks to perceive surface colour, 3D shape, and lighting conditions in complex scenes. To do this I will use rendering software to generate training data that includes images of many diverse scenes, along with representations that show the true colour and shape of objects in those scenes, as well as the true lighting conditions. The networks will be trained to view just the images, and deduce the shape, colour, and lighting in the scenes as well as possible. Previous research suggests that simply because the networks are trained on naturalistic data, they will have some of the same characteristics as human vision: they will find the same visual tasks easy or hard, and they will see many of the same illusions. I will test this prediction, and as it will probably not turn out to be completely true, I will also revise the networks as necessary so that they see shape, colour, and lighting as similarly to human vision as possible. The second approach I will take is to run perceptual experiments with human participants that investigate what fundamental visual features make up our visual world. For example, we obviously perceive colour and 3D shape, and just as obviously we do not perceive the polarization of light. I will focus these experiments on the claim that 'brightness' is a fundamental perceptual dimension, defined as the point-by-point intensity of images (technically, 'perceived luminance'). I will systematically vary surface colour and lighting of test patches in real and computer-generated scenes, and measure how these variations affect judgements of surface colour and brightness. These measurements will help to establish whether 'brightness' is a feature that we actually perceive, separate from surface colour and lighting conditions. This research will help us to understand normal human vision, both in real life and in the simulated virtual environments that are becoming increasingly important for many applications. It will also provide information that will be useful for developing computer vision systems that see what people see in complex, realistic scenes.

还有什么比看更容易的呢？我们不用去尝试就能看到，而且通常不用去思考，看东西似乎很容易，因为我们的视觉皮层是一个强大的计算设备，有着一生的经验。然而，视觉之所以是一项具有挑战性的计算任务，原因之一是所有图像都是高度模糊的：任何给定的图像都可以被视为描绘了各种形状，颜色和照明条件。为了正确地感知事物，我们的视觉系统必须克服这种模糊性。我的研究项目的长期目标是开发计算模型，以与人们相同的方式解决这个问题，并看到人们在复杂，现实的场景中看到的东西。这里提出的研究将通过两种方式实现这一目标。首先，使用为机器学习和计算机视觉开发的强大的新方法，我将训练人工神经网络来感知复杂场景中的表面颜色，3D形状和照明条件。为此，我将使用渲染软件来生成训练数据，其中包括许多不同场景的图像，沿着显示这些场景中对象的真实颜色和形状以及真实照明条件的表示。这些网络将被训练成只查看图像，并尽可能地推断出场景中的形状、颜色和照明。之前的研究表明，仅仅因为网络是在自然主义数据上训练的，它们将具有一些与人类视觉相同的特征：它们会发现相同的视觉任务容易或困难，并且它们会看到许多相同的错觉。我将测试这个预测，因为它可能不会完全正确，我还将根据需要修改网络，以便它们看到的形状，颜色和照明尽可能与人类视觉相似。第二种方法是对人类参与者进行感知实验，研究构成我们视觉世界的基本视觉特征。例如，我们明显地感知到颜色和3D形状，同样明显地，我们没有感知到光的偏振。我将把这些实验的重点放在“亮度”是一个基本的感知维度上，定义为图像的逐点强度（技术上，“感知亮度”）。我将系统地改变表面颜色和照明的测试补丁在真实的和计算机生成的场景，并测量这些变化如何影响判断的表面颜色和亮度。这些测量将有助于确定“亮度”是否是我们实际感知的特征，与表面颜色和照明条件分开。这项研究将有助于我们了解正常的人类视觉，无论是在真实的生活中，在模拟的虚拟环境中，这是越来越重要的许多应用。它还将提供对开发计算机视觉系统有用的信息，这些系统可以看到人们在复杂，逼真的场景中看到的东西。