Mid-level visual processing studied by generative image models and computational modeling

点击翻译按钮获取中文摘要

Vision scientists have developed a detailed understanding of the first steps of visual processing. For many visual tasks involving detection and discrimination of simple stimuli, we can make accurate predictions about visual performance and how the corresponding processing steps are implemented in the brain. However, visual perception involves more than detection and discrimination and we encounter many images that are far beyond the simple stimuli used in typical studies of early visual processing. For example, we are able to decide which parts of an image belong to the same object, which objects are at the front of others, and we can tell apart different objects based on their shape or their surface texture. Importantly, this ability is not restricted to simple images, but it works in our everyday environment. And it works so well, that we are often not even aware of it.***There are two reasons why it is difficult to extend our understanding of detection and discrimination to more natural tasks and viewing conditions. For one, it is difficult to manipulate images with naturalistic complexity in ways that gradually change these, yet precise characterization of visual processing requires exactly these gradual manipulations of images. Secondly, visual processing beyond detection and discrimination is really more complex; it often requires information from different parts of an image to be combined or it relies on feedback loops in which the interpretation of an image unfolds over a short period of time.***Recent developments in the field of artificial intelligence (AI) have a potential to resolve both of these points. Modern AI algorithms can generate images that look like photographs and importantly, they can manipulate these images in exactly the gradual way that is required for precise characterization of visual processing. Furthermore, these algorithms are extract and combine information from different image parts and refine this information in a hierarchy of processing steps. In the proposed research program, I will build on these developments to study how humans assign objects to either foreground or background, how they understand which parts of an image belong to the same object and how object shape and surface texture are integrated to form a coherent perception of the world.***This work will close gap between our understanding of early visual processing and high-level visual perception. As such, it will be important to researchers working on visual perception in humans and non-human primates. By rigorously comparing cutting edge AI algorithms to human performance, this work will further be relevant for researchers in AI, in particular in the domain of computer vision. Finally, this work will also inform our understanding, diagnosis and treatment of clinical conditions affecting visual cortex, and will help guide the development of more intelligent, human-like machine vision systems.**