Measuring uncertainty in human visual segmentation.
Measuring uncertainty in human visual segmentation.
复制标题
DOI：
10.1371/journal.pcbi.1011483
复制
发表时间：
2023-09
影响因子：
4.3
通讯作者：
中科院分区：
文献类型：
--
作者：
将视觉刺激分割成不同的特征和视觉对象组是视觉功能的核心。经典的心理物理方法帮助揭示了人类感知分割的许多规则，机器学习的最新进展产生了成功的算法。然而，人类分割的计算逻辑仍然不清楚，部分原因是我们缺乏良好控制的范例来衡量感知分割图和定量比较模型。在这里，我们提出了一种新的综合方法：给定一幅图像，我们测量多个基于像素的相同和不同的判断，并执行基于模型的底层分割地图的重建。重建对几种实验操作是稳健的，并捕捉到了个体参与者的可变性。实验结果表明，该方法对自然图像和复合纹理的人体分割是有效的。我们表明，图像的不确定性影响测量的人类变异性，并影响参与者如何权衡不同的视觉特征。由于可以插入任何假定的分割算法来执行重建，我们的范例为感知理论提供了定量测试，并为分割算法提供了新的基准。视觉分割是将视野分解成有意义的部分的过程。分割是视觉知觉和神经科学中大量文献的焦点，因为它是视觉系统的核心功能，涉及整个视觉皮质的自下而上和自上而下的整合。同样，分割是计算机视觉系统的一项基本任务，因为它是无数实际应用所必需的。然而，缺乏与分割相关的不确定性的严格经验测量是这两个领域的主要障碍，因为主观不确定性是视觉感知的核心特征，也因为现有数据库不允许校准确实计算不确定性的分割算法。这份手稿中提出的工作建议克服这些限制。具体地说，我们的贡献有三个：(I)我们引入了第一个实验方法来测量任意图像的感知分割。(2)我们捕捉个体水平的可变性，并将其与感知的不确定性联系起来，这是理解人类感知所必需的。(Iii)我们提供计算工具，使任何分割算法适合数据，这将为计算机视觉算法提供新的基准，并测试感知分割的计算理论。
Segmenting visual stimuli into distinct groups of features and visual objects is central to visual function. Classical psychophysical methods have helped uncover many rules of human perceptual segmentation, and recent progress in machine learning has produced successful algorithms. Yet, the computational logic of human segmentation remains unclear, partially because we lack well-controlled paradigms to measure perceptual segmentation maps and compare models quantitatively. Here we propose a new, integrated approach: given an image, we measure multiple pixel-based same–different judgments and perform model–based reconstruction of the underlying segmentation map. The reconstruction is robust to several experimental manipulations and captures the variability of individual participants. We demonstrate the validity of the approach on human segmentation of natural images and composite textures. We show that image uncertainty affects measured human variability, and it influences how participants weigh different visual features. Because any putative segmentation algorithm can be inserted to perform the reconstruction, our paradigm affords quantitative tests of theories of perception as well as new benchmarks for segmentation algorithms. Visual segmentation is the process of decomposing the visual field into meaningful parts. Segmentation is the focus of a vast literature in visual perception and neuroscience, because it is a core function of the visual system that involves bottom/up and top/down integration across the whole visual cortex. Similarly, segmentation is an essential task of computer vision systems, because it is required for countless practical applications. However, the lack of rigorous empirical measures of segmentation-related uncertainty represents a major roadblock for both fields, because subjective uncertainty is a central feature of visual perception, and also because existing databases do not allow to calibrate segmentation algorithms that do compute uncertainty. The work presented in this manuscript proposes to overcome these limitations. Specifically, our contributions are threefold: (i) We introduce the first experimental method to measure perceptual segmentation on arbitrary images. (ii) We capture individual-level variability and relate it to perceptual uncertainty, which is necessary to understand human perception. (iii) We offer computational tools to fit any segmentation algorithm to the data, which will enable new benchmarks for computer vision algorithms, and testing computational theories of perceptual segmentation.
登录
查看更多内容
影响因子：
25
作者：
Kar K;Kubilius J;Schmidt K;Issa EB;DiCarlo JJ
通讯作者：
DiCarlo JJ
影响因子：
16.2
作者：
Li, Wu;Piëch, Valentin;Gilbert, Charles D.
通讯作者：
Gilbert, Charles D.
影响因子：
4.3
作者：
Doerig, Adrien;Schmittwilken, Lynn;Herzog, Michael H.
通讯作者：
Herzog, Michael H.
影响因子：
1.8
作者：
LANDY, MS;BERGEN, JR
通讯作者：
BERGEN, JR
影响因子：
1.9
作者：
Vancleef K;Putzeys T;Gheorghiu E;Sassi M;Machilsen B;Wagemans J
通讯作者：
Wagemans J