Mid-level visual processing studied by generative image models and computational modeling

视觉科学家已经对视觉处理的第一步有了详细的了解。对于许多涉及检测和区分简单刺激的视觉任务，我们可以准确预测视觉表现以及相应的处理步骤如何在大脑中实现。然而，视觉感知涉及的不仅仅是检测和辨别，我们遇到的许多图像远远超出了早期视觉处理典型研究中使用的简单刺激。例如，我们能够确定图像的哪些部分属于同一个物体，哪些物体在其他物体的前面，我们可以根据它们的形状或表面纹理区分不同的物体。重要的是，这种能力并不局限于简单的图像，而是在我们的日常环境中发挥作用。它的效果很好，我们甚至经常没有意识到它。我们很难将对探测和辨别的理解扩展到更自然的任务和观察条件，原因有二。首先，很难以逐渐改变这些的方式来操纵具有自然主义复杂性的图像，然而视觉处理的精确表征恰恰需要这些渐进的图像操纵。其次，视觉处理超出了检测和辨别的范围，实际上更加复杂;它通常需要将来自图像不同部分的信息组合在一起，或者依赖于反馈回路，在反馈回路中，图像的解释在短时间内展开。人工智能（AI）领域的最新发展有可能解决这两个问题。现代人工智能算法可以生成看起来像照片的图像，重要的是，它们可以以精确表征视觉处理所需的渐进方式操纵这些图像。此外，这些算法是从不同图像部分提取和联合收割机信息，并在处理步骤的层次结构中细化该信息。在拟议的研究计划中，我将在这些发展的基础上研究人类如何将物体分配给前景或背景，他们如何理解图像的哪些部分属于同一个物体，以及物体形状和表面纹理如何整合以形成对世界的连贯感知。这项工作将缩小我们对早期视觉处理和高级视觉感知的理解之间的差距。因此，这对研究人类和非人类灵长类动物视觉感知的研究人员来说非常重要。通过严格比较前沿AI算法与人类表现，这项工作将进一步与AI研究人员相关，特别是在计算机视觉领域。最后，这项工作还将为我们理解、诊断和治疗影响视觉皮层的临床疾病提供信息，并将有助于指导更智能、更人性化的机器视觉系统的开发。

Vision scientists have developed a detailed understanding of the first steps of visual processing. For many visual tasks involving detection and discrimination of simple stimuli, we can make accurate predictions about visual performance and how the corresponding processing steps are implemented in the brain. However, visual perception involves more than detection and discrimination and we encounter many images that are far beyond the simple stimuli used in typical studies of early visual processing. For example, we are able to decide which parts of an image belong to the same object, which objects are at the front of others, and we can tell apart different objects based on their shape or their surface texture. Importantly, this ability is not restricted to simple images, but it works in our everyday environment. And it works so well, that we are often not even aware of it.***There are two reasons why it is difficult to extend our understanding of detection and discrimination to more natural tasks and viewing conditions. For one, it is difficult to manipulate images with naturalistic complexity in ways that gradually change these, yet precise characterization of visual processing requires exactly these gradual manipulations of images. Secondly, visual processing beyond detection and discrimination is really more complex; it often requires information from different parts of an image to be combined or it relies on feedback loops in which the interpretation of an image unfolds over a short period of time.***Recent developments in the field of artificial intelligence (AI) have a potential to resolve both of these points. Modern AI algorithms can generate images that look like photographs and importantly, they can manipulate these images in exactly the gradual way that is required for precise characterization of visual processing. Furthermore, these algorithms are extract and combine information from different image parts and refine this information in a hierarchy of processing steps. In the proposed research program, I will build on these developments to study how humans assign objects to either foreground or background, how they understand which parts of an image belong to the same object and how object shape and surface texture are integrated to form a coherent perception of the world.***This work will close gap between our understanding of early visual processing and high-level visual perception. As such, it will be important to researchers working on visual perception in humans and non-human primates. By rigorously comparing cutting edge AI algorithms to human performance, this work will further be relevant for researchers in AI, in particular in the domain of computer vision. Finally, this work will also inform our understanding, diagnosis and treatment of clinical conditions affecting visual cortex, and will help guide the development of more intelligent, human-like machine vision systems.**