Computational and neural mechanisms of perceptual grouping

人脑如何将视觉信息分类为有意义的单元？我们如何利用人类视觉的洞察力来提升机器视觉？我和我的学员将寻求通过三管齐下的方法来回答这些问题： 1. 开发用于现实世界图像中感知分组的计算算法，并测试它们对人类视觉的实用性； 2. 测量它们在人脑中的实施情况； 3.为计算机视觉的神经网络添加感知分组机制，以提高其性能和鲁棒性。 在早期提取简单的视觉元素之后，所谓的中级人类视觉将这些元素分组为构成高级视觉任务基础的形状：物体、面部和场景的识别。在充分理解的早期视觉和通过功能磁共振成像进行大量研究的高级视觉之间，中级视觉的计算和神经机制（主要包括感知分组）是我们对现实世界环境视觉感知的理解中的一个明显差距。通过拟议的研究，我和我的学员试图填补这一空白，使我们能够更全面地掌握人脑执行的级联计算，从视网膜感光细胞的激活中获取意义。我们将开发用于现实世界图像中感知分组的计算算法，测试它们对人类视觉的实用性，并探索它们在人脑中的实现。这项工作将通过将感知分组从看似随机的一组定性规则领域转移到具体的计算和神经机制领域来改变我们对人类视觉的理解。 人工智能系统已经开始渗透到我们的日常生活中，比如自动驾驶汽车、能够识别自己内容的冰箱，或者能够全自动拍摄最佳照片的手机摄像头。这些系统需要处理混乱的现实环境。然而，尽管基于卷积神经网络 (CNN) 的计算机视觉算法具有令人印象深刻的识别性能，但它还是会被对抗性图像以及像剪影和玻璃雕像这样简单的刺激所愚弄。我们认为这是由于缺乏中级视觉，导致 CNN 学习纹理统计而不是将边缘元素组织成形状。我们将利用人类视觉研究的见解来实现 CNN 的感知分组规则。应用于计算机视觉，感知分组将赋予人工智能代理更强大的视觉系统，以理解它们越来越多地部署的现实生活情况。这种对感知分组的机械理解可能会促进计算机视觉的进步，从而使人工智能系统能够更好地与它们与我们共存的现实世界的混乱进行交互。

How does the human brain sort visual information into meaningful units? And how can we use the insights from human vision to boost machine vision? My trainees and I will seek to answer these questions with a three-pronged approach: 1. develop computational algorithms for perceptual grouping in real-world images and test their utility for human vision; 2. measure their implementation in the human brain; 3. add perceptual grouping mechanisms to neural networks for computer vision to improve their performance and robustness. Following extraction of simple visual elements early on, so-called mid-level human vision groups the elements into shapes that form the basis for high-level vision tasks: the recognition of objects, faces, and scenes. Between well-understood early vision and, thanks to fMRI, heavily researched high-level vision, the computational and neural mechanisms of mid-level vision, largely comprising perceptual grouping, are a glaring gap in our understanding of visual perception of real-world environments. With the proposed research my trainees and I seek to fill this gap to allow us to more fully grasp the cascade of computations performed by the human brain to derive meaning from the activations of photoreceptors in the retina. We will develop computational algorithms for perceptual grouping in real-world images, test their utility for human vision, and explore their implementation in the human brain. The work will transform our understanding of human vision by moving perceptual grouping from the realm of a seemingly random set of qualitative rules into the realm of concrete computational and neural mechanisms. Artificially intelligent systems have started to permeate our daily lives think of self-driving cars, refrigerators that recognize their own content, or cell phone cameras that are able to frame the best shot for a photo fully automatically. These systems need to deal with a cluttered, real-world environment. However, despite their impressive recognition performance, computer vision algorithms based on convolutional neural networks (CNNs) are fooled by adversarial images as well as stimuli as simple as silhouettes and glass figurines. We argue that this is due to a lack of mid-level vision, leading to CNNs learning texture statistics rather than organizing edge elements into shapes. We will use the insights from human vision research to implement perceptual grouping rules for CNNs. Applied to computer vision, perceptual grouping will endow artificially intelligent agents with more robust vision systems for comprehending the real-life situations, in which they are increasingly deployed. Such a mechanistic understanding of perceptual grouping is likely to give rise to advances in computer vision that will allow AI systems to better interact with the messiness of the real world that they cohabit with us.