New computational models of human visual perception of surface colour, 3D shape, and lighting

点击翻译按钮获取中文摘要

What could be easier than seeing? We see without trying, and usually without thinking about it. Seeing seems easy because our visual cortex is a powerful computing device with a lifetime of experience. One reason why vision is a challenging computational task, though, is that all images are highly ambiguous: any given image could conceivably be seen as depicting a wide range of shapes, colours, and lighting conditions. In order to perceive things correctly, our visual system must overcome this ambiguity. The long-term goal of my research program is to develop computational models that solve this problem in the same way people do, and see what people see in complex, realistic scenes. The research proposed here will approach this goal in two ways. First, using powerful new methods developed for machine learning and computer vision, I will train artificial neural networks to perceive surface colour, 3D shape, and lighting conditions in complex scenes. To do this I will use rendering software to generate training data that includes images of many diverse scenes, along with representations that show the true colour and shape of objects in those scenes, as well as the true lighting conditions. The networks will be trained to view just the images, and deduce the shape, colour, and lighting in the scenes as well as possible. Previous research suggests that simply because the networks are trained on naturalistic data, they will have some of the same characteristics as human vision: they will find the same visual tasks easy or hard, and they will see many of the same illusions. I will test this prediction, and as it will probably not turn out to be completely true, I will also revise the networks as necessary so that they see shape, colour, and lighting as similarly to human vision as possible. The second approach I will take is to run perceptual experiments with human participants that investigate what fundamental visual features make up our visual world. For example, we obviously perceive colour and 3D shape, and just as obviously we do not perceive the polarization of light. I will focus these experiments on the claim that 'brightness' is a fundamental perceptual dimension, defined as the point-by-point intensity of images (technically, 'perceived luminance'). I will systematically vary surface colour and lighting of test patches in real and computer-generated scenes, and measure how these variations affect judgements of surface colour and brightness. These measurements will help to establish whether 'brightness' is a feature that we actually perceive, separate from surface colour and lighting conditions. This research will help us to understand normal human vision, both in real life and in the simulated virtual environments that are becoming increasingly important for many applications. It will also provide information that will be useful for developing computer vision systems that see what people see in complex, realistic scenes.