Computational and neural mechanisms of perceptual grouping

人类大脑如何将视觉信息分类成有意义的单位？我们如何利用人类视觉的洞察力来促进机器视觉呢？我和我的受训者将寻求通过三个方面的方法来回答这些问题：1.开发用于真实世界图像中感知分组的计算算法，并测试其对人类视觉的效用；2.测量它们在人脑中的实现；3.为计算机视觉的神经网络添加感知分组机制，以提高其性能和健壮性。 在早期提取简单的视觉元素之后，所谓的中级人类视觉将这些元素分组为形状，形成高级视觉任务的基础：识别对象、人脸和场景。在被广泛理解的早期视觉和由于功能磁共振成像而被大量研究的高级视觉之间，中级视觉的计算和神经机制(主要包括知觉分组)在我们对现实世界环境的视觉感知的理解中存在着明显的差距。通过拟议的研究，我和我的受训人员试图填补这一空白，使我们能够更全面地掌握人脑执行的级联计算，以从视网膜中光感受器的激活中获得意义。我们将开发在真实世界图像中感知分组的计算算法，测试它们对人类视觉的效用，并探索它们在人脑中的实现。这项工作将改变我们对人类视觉的理解，将感知分组从一组看似随机的定性规则领域转移到具体的计算和神经机制领域。 人工智能系统已经开始渗透到我们的日常生活中，想想自动驾驶汽车，识别自己内容的冰箱，或者能够全自动为照片拍摄最佳画面的手机相机。这些系统需要处理杂乱的、真实的环境。然而，尽管基于卷积神经网络(CNN)的计算机视觉算法具有令人印象深刻的识别性能，但它们被对抗性图像以及像轮廓和玻璃雕像这样简单的刺激所愚弄。我们认为这是由于缺乏中层视觉，导致CNN学习纹理统计而不是将边缘元素组织成形状。我们将使用人类视觉研究的洞察力来实现CNN的感知分组规则。将感知分组应用于计算机视觉，将赋予人工智能代理更健壮的视觉系统，以理解现实生活中的情况，它们在其中得到了越来越多的应用。这种对感知分组的机械式理解可能会带来计算机视觉的进步，这将使人工智能系统能够更好地与它们与我们共存的现实世界的混乱互动。

How does the human brain sort visual information into meaningful units? And how can we use the insights from human vision to boost machine vision? My trainees and I will seek to answer these questions with a three-pronged approach: 1. develop computational algorithms for perceptual grouping in real-world images and test their utility for human vision; 2. measure their implementation in the human brain; 3. add perceptual grouping mechanisms to neural networks for computer vision to improve their performance and robustness. Following extraction of simple visual elements early on, so-called mid-level human vision groups the elements into shapes that form the basis for high-level vision tasks: the recognition of objects, faces, and scenes. Between well-understood early vision and, thanks to fMRI, heavily researched high-level vision, the computational and neural mechanisms of mid-level vision, largely comprising perceptual grouping, are a glaring gap in our understanding of visual perception of real-world environments. With the proposed research my trainees and I seek to fill this gap to allow us to more fully grasp the cascade of computations performed by the human brain to derive meaning from the activations of photoreceptors in the retina. We will develop computational algorithms for perceptual grouping in real-world images, test their utility for human vision, and explore their implementation in the human brain. The work will transform our understanding of human vision by moving perceptual grouping from the realm of a seemingly random set of qualitative rules into the realm of concrete computational and neural mechanisms. Artificially intelligent systems have started to permeate our daily lives think of self-driving cars, refrigerators that recognize their own content, or cell phone cameras that are able to frame the best shot for a photo fully automatically. These systems need to deal with a cluttered, real-world environment. However, despite their impressive recognition performance, computer vision algorithms based on convolutional neural networks (CNNs) are fooled by adversarial images as well as stimuli as simple as silhouettes and glass figurines. We argue that this is due to a lack of mid-level vision, leading to CNNs learning texture statistics rather than organizing edge elements into shapes. We will use the insights from human vision research to implement perceptual grouping rules for CNNs. Applied to computer vision, perceptual grouping will endow artificially intelligent agents with more robust vision systems for comprehending the real-life situations, in which they are increasingly deployed. Such a mechanistic understanding of perceptual grouping is likely to give rise to advances in computer vision that will allow AI systems to better interact with the messiness of the real world that they cohabit with us.