Perceptual integration of luminance, texture and color cues for visual boundary segmentation

项目摘要 视觉系统执行的最基本的计算之一是分割图像 分成对应于不同表面的区域。这反过来又需要确定边界 分离图像区域，这一过程称为边界分割。计算分析 已经揭示了许多视觉线索在区域边界处可用， 包括亮度、纹理和颜色的差异。众所周知，这些线索结合在一起形成了 边缘定位和方向辨别等任务。然而，目前还不清楚这些因素是如何 对各种线索进行加权和组合，以进行边界分割。 在与蒙特利尔麦吉尔大学的加拿大同事合作中，我们有 开发了一种新的机器学习框架，用于表征人类在 使用自然微模式刺激的边界分割任务。我们的方法利用了 滤光片-校正-滤光片(FRF)模型常用于纹理边界的表征 分段。我们方法的主要创新是我们将FRF模型直接匹配到 数以千计的心理物理刺激-反应观察以估计其主要定义 参数。我们最近用这种方法来研究对比的空间策略。 边界分割和比较关于对比度调制的相互竞争的假设 集成了多个定向渠道。在这项拨款中，我们建议将经典的 心理物理技术和我们新的机器学习方法来理解 用于组合亮度、纹理和颜色线索以进行分割的计算。 在目标1中，我们专注于对亮度定义边界的分割进行建模， 比较每个表面具有均匀亮度的情况，从而产生尖锐的边缘 (亮度步长)，到两个表面具有不同比例的更自然的情况 边界两侧没有锐边(亮度)的暗和亮微图案 纹理)。我们将应用我们的机器学习方法来测试不同的假设 神经机制可能涉及到分割这两种不同的亮度 边界。在目标2中，我们询问观测者如何整合一阶(亮度)和二阶 (纹理)用于边界分割的线索，以及线索组合是否存在差异 亮度步长和亮度纹理的策略。我们还将比较体现在 线索组合的潜在神经机制的相互竞争的假说。在目标3中，我们 将目标1和目标2中的分析从简单的亮度差异扩展到包括差异 在颜色上。最后，目标4是促进本科生研究的教学目标。

Project Summary One of the most essential computations performed by the visual system is segmenting images into regions corresponding to distinct surfaces. This in turn requires identifying the boundaries separating image regions, a process known as boundary segmentation. Computational analyses of natural images have revealed that many visual cues are available at region boundaries, including differences in luminance, texture, and color. It is known that these cues combine for tasks like edge localization and orientation discrimination. However, it remains unclear how these various cues are weighted and combined for boundary segmentation. In collaborative work with Canadian colleagues at McGill University in Montreal, we have developed a novel machine learning framework for characterizing human performance on boundary segmentation tasks using naturalistic micro-pattern stimuli. Our method makes use of the Filter-Rectify-Filter (FRF) model often applied to characterizing texture boundary segmentation. The major innovation of our approach is that we fit the FRF model directly to thousands of psychophysical stimulus-response observations to estimate its major defining parameters. We have recently applied this approach to investigating spatial strategies for contrast boundary segmentation and comparing competing hypotheses of how contrast modulation is integrated across orientation channels. In this grant, we propose to apply both classical psychophysical techniques and our novel machine learning methodology to understanding the computations employed to combine luminance, texture and color cues for segmentation. In Aim 1, we focus on modeling segmentation of luminance-defined boundaries, comparing the case where each surface has uniform luminance, giving rise to a sharp edge (luminance step), to the more naturalistic case where the two surfaces have differing proportions of dark and light micro-patterns on either side of the boundary with no sharp edge (luminance texture). We will apply our machine learning methodology to test the hypothesis that different neural mechanisms may be involved in segmenting these two different kinds of luminance boundaries. In Aim 2, we ask how observers integrate first-order (luminance) and second-order (texture) cues for boundary segmentation, and if there are differences in cue combination strategies for luminance steps and luminance textures. We will also compare models embodying competing hypotheses of the underlying neural mechanisms of cue combination. In Aim 3, we extend the analyses in Aims 1 and 2 beyond simple luminance differences to include differences in color. Finally, Aim 4 is a pedagogical aim of promoting undergraduate research.