Computational and neural mechanisms of perceptual grouping

点击翻译按钮获取中文摘要

How does the human brain sort visual information into meaningful units? And how can we use the insights from human vision to boost machine vision? My trainees and I will seek to answer these questions with a three-pronged approach: 1. develop computational algorithms for perceptual grouping in real-world images and test their utility for human vision; 2. measure their implementation in the human brain; 3. add perceptual grouping mechanisms to neural networks for computer vision to improve their performance and robustness. Following extraction of simple visual elements early on, so-called mid-level human vision groups the elements into shapes that form the basis for high-level vision tasks: the recognition of objects, faces, and scenes. Between well-understood early vision and, thanks to fMRI, heavily researched high-level vision, the computational and neural mechanisms of mid-level vision, largely comprising perceptual grouping, are a glaring gap in our understanding of visual perception of real-world environments. With the proposed research my trainees and I seek to fill this gap to allow us to more fully grasp the cascade of computations performed by the human brain to derive meaning from the activations of photoreceptors in the retina. We will develop computational algorithms for perceptual grouping in real-world images, test their utility for human vision, and explore their implementation in the human brain. The work will transform our understanding of human vision by moving perceptual grouping from the realm of a seemingly random set of qualitative rules into the realm of concrete computational and neural mechanisms. Artificially intelligent systems have started to permeate our daily lives - think of self-driving cars, refrigerators that recognize their own content, or cell phone cameras that are able to frame the best shot for a photo fully automatically. These systems need to deal with a cluttered, real-world environment. However, despite their impressive recognition performance, computer vision algorithms based on convolutional neural networks (CNNs) are fooled by adversarial images as well as stimuli as simple as silhouettes and glass figurines. We argue that this is due to a lack of mid-level vision, leading to CNNs learning texture statistics rather than organizing edge elements into shapes. We will use the insights from human vision research to implement perceptual grouping rules for CNNs. Applied to computer vision, perceptual grouping will endow artificially intelligent agents with more robust vision systems for comprehending the real-life situations, in which they are increasingly deployed. Such a mechanistic understanding of perceptual grouping is likely to give rise to advances in computer vision that will allow AI systems to better interact with the messiness of the real world that they cohabit with us.