CRCNS: Joint coding of shape and texture in the primate brian

项目说明 协作PIS和顾问 美国 安妮莎·帕苏帕西，系美国西雅图华盛顿大学生物结构系 公司名称：Wyeth Bair，Department.美国西雅图华盛顿大学生物结构系 日本 PL：伊萨木·本吉，系。日本东京大学生命科学系 顾问：小松秀彦，玉川大学，日本 具体目标 我们的视觉系统赋予我们一套不同的能力：识别和操纵 物体，避免航行中的障碍和危险，评估食物质量，阅读文本， 解读面部表情等。这依赖于神经元对信息的处理 灵长类视觉系统腹侧通路的形态和材料结构(Ungerleider& Mishkin，1982；Felleman&Van Essen，1991)。过去几十年的研究表明 制作了视觉信息在V1中如何处理的详细模型，V1是 。这种途径(Hubel&Wiseel，1959,1968；Movshon等人，1978a，b；Albrecht等人，1980)，但是 在V1之外，我们对视觉处理和表示的理解是有限的。这是 尤其是在我们理解形式和形式的视觉表现如何 纹理对物体的感知和识别有共同的作用。这项提案的总体目标是 两重--开发一种实验驱动的图像可计算模型，以显示其自然主义程度 视觉刺激在V4区进行处理，这是沿腹侧视觉的一个重要中间阶段 路径(目标1)，并发现这种表征如何有助于感知(目标2)。 过去的研究表明，V4神经元对这种形式都很敏感(Desimone和Schein， 1987年；Kobatake和Tanaka，1994；Gallant等人，1993；Pasuthy和Connor，2001；Nandy et 等人，2013)和视觉刺激的表面纹理(Arcizet等人，2008；Goda等人，2014； Okazawa等人，2015)。但是，由于专业知识有限，实验时间有限， 科学家倾向于只关注形状或质地的编码，而不是它们的关节 编码。例如，在这次合作的美国部分的实验室中，我们直到 现在专注于通过使用2D形状进行神经生理学研究来处理形状 使用统一的表面属性来研究对象边界是如何编码的(Oleskiw et 等人，2014年；波波夫金娜等人，2016年)。我们已经通过比较表示法对数据进行了建模 与AlexNet中单位的V4神经元的相同(Pospisil等人，2015)，一个显著的卷积 神经网络(CNN)被训练成识别物体(Krizevsky等人，2012年)。同时， 这次合作的日本代表团研究了表面纹理的编码和 没有相关形式编码的人类感知中的光泽(Motoyoshi等人，2007；Sharan等人 Al.，2008；Motoyoshi，2010；Motoyoshi&Matoba，2012)。在这里，我们建议将我们的 各自在研究形式和纹理编码方面的专长如何 具有形式和表面提示的自然主义刺激在区域V4中被编码，以及这些 表征支持人类的视觉感知。我们的具体目标是： 目的：1.为神经元对形状和形状的反应建立统一的图像可计算模型 区域V4中的纹理 可以解释V4对具有均匀亮度/色度特征的2D形状的响应 通过强调边界特征的对象识别的层次-最大(HMAX)模型 (Cadieu等人，2007年)。这种反应也可以用人工深水中的单位来解释。 卷积网络，其中边界要素未明确强调(所有要素 是从最初的随机权重学习的)。另一方面，V4对纹理补丁的响应 可以用基于高阶图像统计的模型很好地解释(Okazawa等人，2015年)。 使用帕苏帕西实验室的形状数据和小松实验室的纹理数据(日语 ，我们将询问V4神经元对形状和纹理的反应是否可以 第21页

PROJECT DESCRIPTION Collaborating Pis and Consultant United States Pl: Anitha Pasupathy, Dept. of Biological Structure, University of Washington, Seattle, USA Co-Pl: Wyeth Bair, Dept. of Biological Structure, University of Washington, Seattle, USA Japan Pl: lsamu Motoyoshi, Dept. of Life Sciences, The University of Tokyo, Japan Consultant: Hidehiko Komatsu, Tamagawa University, Japan Specific Aims Our visual system endows us with a diverse set of abilities: to recognize and manipulate objects, avoid obstacles and danger during navigation, evaluate the quality of food, read text, interpret facial expressions, etc. This relies on the neuronal processing of information about form and material texture along the ventral pathway of the primate visual system (Ungerleider & Mishkin, 1982; Felleman & Van Essen, 1991). Studies over the past several decades have produced detailed models of how visual information is processed in V1, the earliest stage along . this pathway (Hubel & Wiesel, 1959, 1968; Movshon et al., 1978a, b; Albrecht et al., 1980), but beyond V1 our understanding of visual processing and representation is limited. This is particularly true with regard to our understanding of how visual representations of form and texture jointly contribute to object perception and recognition. The broad goal of this proposal is two-fold-to develop an experimentally-driven image-computable model for how naturalistic visual stimuli are processed in area V4, an important intermediate stage along the ventral visual pathway (Aim 1) and to discover how such a representation contributes to perception (Aim 2). Past studies have shown that V4 neurons are sensitive to both the form (Desimone and Schein, 1987; Kobatake and Tanaka, 1994; Gallant et al., 1993; Pasupathy and Connor, 2001; Nandy et al., 2013) and the surface texture of visual stimuli (Arcizet et al., 2008; Goda et al., 2014; Okazawa et al., 2015). But, because expertise is narrow and experimental time limited, scientists tend to focus exclusively on the encoding of form or texture and not on their joint coding. For example, in the laboratories of the USA portion of this collaboration, we have until now focused on form processing by carrying out neurophysiological studies using 2D shapes with uniform surface properties to investigate how object boundaries are encoded (Oleskiw et al., 2014; Popovkina et al., 2016). We have modeled our data by comparing the representation of V4 neurons to that of the units in AlexNet (Pospisil et al., 2015), a prominent convolutional neural net (CNN) trained to recognize objects (Krizhevsky et al., 2012). At the same time, the Japanese contingent of this collaboration has investigated the encoding of surface texture and gloss in human perception without associated form encoding (Motoyoshi et al., 2007; Sharan et al., 2008; Motoyoshi, 2010; Motoyoshi & Matoba, 2012). Here we propose to bring our respective expertise in studying form and texture encoding to bear on the question of how naturalistic stimuli with both form and surface cues are encoded in area V4 and how these representations support human visual perception. Our specific aims are: Aim1. To build a unified image-computable model for neuronal responses to shapes and textures in area V4 V4 responses to 2D shapes with uniform luminance/chromatic characteristics can be explained by a hierarchical-Max (HMax) model for object recognition that emphasizes boundary features (Cadieu et al., 2007). Such responses can also be explained by units in artificial deep convolutional networks, in which boundary features are not explicitly emphasized (all features are learned from initially random weights). On the other hand, V4 responses to texture patches can be well explained by a higher-order image-statistics-based model (Okazawa et al., 2015). Using shape data from the Pasupathy lab and texture data from the Komatsu lab (Japanese consultant), we will ask whether responses of V4 neurons to shapes and textures can be Page 21