Perceptual integration of luminance, texture and color cues for visual boundary segmentation

项目摘要 视觉系统执行的最基本的计算之一是分割图像 分成不同表面的区域这反过来又需要确定边界 分离图像区域，这一过程称为边界分割。计算分析 已经揭示了在区域边界处可以获得许多视觉线索， 包括亮度、纹理和颜色的差异。众所周知，这些线索联合收割机 例如边缘定位和方向辨别。然而，目前尚不清楚这些 对各种线索进行加权和组合以用于边界分割。 通过与蒙特利尔麦吉尔大学的加拿大同事合作，我们 开发了一种新的机器学习框架，用于描述人类在 边界分割任务使用自然的微观模式刺激。我们的方法利用了 滤波-校正-滤波（FRF）模型常用于纹理边界的表征 细分我们的方法的主要创新是我们直接将FRF模型拟合为 成千上万的心理物理刺激反应观察，以估计其主要定义 参数我们最近应用这种方法来研究对比的空间策略 边界分割和比较对比度调制如何 跨方向通道集成。在这个补助金中，我们建议同时应用经典的 心理物理学技术和我们新颖的机器学习方法来理解 用于联合收割机组合亮度、纹理和颜色线索以进行分割的计算。 在目标1中，我们专注于对亮度定义的边界进行建模分割， 比较每个表面具有均匀亮度的情况，产生尖锐边缘 （亮度阶跃），到两个表面具有不同比例的更自然的情况 在边界的任一侧上的暗和亮的微图案，没有尖锐边缘（亮度 纹理）。我们将应用我们的机器学习方法来测试不同的假设， 神经机制可能参与分割这两种不同的亮度 边界在目标2中，我们询问观察者如何整合一阶（亮度）和二阶 （纹理）线索的边界分割，如果有不同的线索组合 亮度步长和亮度纹理的策略。我们还将比较体现 线索组合的潜在神经机制的竞争假设。在目标3中， 将目标1和2中的分析扩展到简单的亮度差异之外，以包括 彩色的。最后，目标4是促进本科生研究的教学目标。

Project Summary One of the most essential computations performed by the visual system is segmenting images into regions corresponding to distinct surfaces. This in turn requires identifying the boundaries separating image regions, a process known as boundary segmentation. Computational analyses of natural images have revealed that many visual cues are available at region boundaries, including differences in luminance, texture, and color. It is known that these cues combine for tasks like edge localization and orientation discrimination. However, it remains unclear how these various cues are weighted and combined for boundary segmentation. In collaborative work with Canadian colleagues at McGill University in Montreal, we have developed a novel machine learning framework for characterizing human performance on boundary segmentation tasks using naturalistic micro-pattern stimuli. Our method makes use of the Filter-Rectify-Filter (FRF) model often applied to characterizing texture boundary segmentation. The major innovation of our approach is that we fit the FRF model directly to thousands of psychophysical stimulus-response observations to estimate its major defining parameters. We have recently applied this approach to investigating spatial strategies for contrast boundary segmentation and comparing competing hypotheses of how contrast modulation is integrated across orientation channels. In this grant, we propose to apply both classical psychophysical techniques and our novel machine learning methodology to understanding the computations employed to combine luminance, texture and color cues for segmentation. In Aim 1, we focus on modeling segmentation of luminance-defined boundaries, comparing the case where each surface has uniform luminance, giving rise to a sharp edge (luminance step), to the more naturalistic case where the two surfaces have differing proportions of dark and light micro-patterns on either side of the boundary with no sharp edge (luminance texture). We will apply our machine learning methodology to test the hypothesis that different neural mechanisms may be involved in segmenting these two different kinds of luminance boundaries. In Aim 2, we ask how observers integrate first-order (luminance) and second-order (texture) cues for boundary segmentation, and if there are differences in cue combination strategies for luminance steps and luminance textures. We will also compare models embodying competing hypotheses of the underlying neural mechanisms of cue combination. In Aim 3, we extend the analyses in Aims 1 and 2 beyond simple luminance differences to include differences in color. Finally, Aim 4 is a pedagogical aim of promoting undergraduate research.