Perceptual integration of luminance, texture and color cues for visual boundary segmentation

项目总结

Project Summary One of the most essential computations performed by the visual system is segmenting images into regions corresponding to distinct surfaces. This in turn requires identifying the boundaries separating image regions, a process known as boundary segmentation. Computational analyses of natural images have revealed that many visual cues are available at region boundaries, including differences in luminance, texture, and color. It is known that these cues combine for tasks like edge localization and orientation discrimination. However, it remains unclear how these various cues are weighted and combined for boundary segmentation. In collaborative work with Canadian colleagues at McGill University in Montreal, we have developed a novel machine learning framework for characterizing human performance on boundary segmentation tasks using naturalistic micro-pattern stimuli. Our method makes use of the Filter-Rectify-Filter (FRF) model often applied to characterizing texture boundary segmentation. The major innovation of our approach is that we fit the FRF model directly to thousands of psychophysical stimulus-response observations to estimate its major defining parameters. We have recently applied this approach to investigating spatial strategies for contrast boundary segmentation and comparing competing hypotheses of how contrast modulation is integrated across orientation channels. In this grant, we propose to apply both classical psychophysical techniques and our novel machine learning methodology to understanding the computations employed to combine luminance, texture and color cues for segmentation. In Aim 1, we focus on modeling segmentation of luminance-defined boundaries, comparing the case where each surface has uniform luminance, giving rise to a sharp edge (luminance step), to the more naturalistic case where the two surfaces have differing proportions of dark and light micro-patterns on either side of the boundary with no sharp edge (luminance texture). We will apply our machine learning methodology to test the hypothesis that different neural mechanisms may be involved in segmenting these two different kinds of luminance boundaries. In Aim 2, we ask how observers integrate first-order (luminance) and second-order (texture) cues for boundary segmentation, and if there are differences in cue combination strategies for luminance steps and luminance textures. We will also compare models embodying competing hypotheses of the underlying neural mechanisms of cue combination. In Aim 3, we extend the analyses in Aims 1 and 2 beyond simple luminance differences to include differences in color. Finally, Aim 4 is a pedagogical aim of promoting undergraduate research.