Texture Interpolation for Probing Visual Perception
Texture Interpolation for Probing Visual Perception
复制标题
DOI：
--
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
Jonathan Vacher;Aida Davila;A. Kohn;Ruben Coen-Cagli
中科院分区：
文献类型：
--
作者：
Jonathan Vacher;Aida Davila;A. Kohn;Ruben Coen-Cagli
纹理合成模型是理解视觉处理的重要工具。特别是，基于神经相关特征的统计方法有助于理解视觉感知和神经编码的各个方面。基于深度学习的新方法进一步提高了合成纹理的质量。然而，目前还不清楚为什么深度纹理合成的表现如此之好，而且这种新框架在探索视觉感知方面的应用也很少。在这里，我们证明了纹理的深卷积神经网络(CNN)激活的分布可以用椭圆分布来很好地描述，因此，遵循最优传输理论，约束它们的均值和协方差就足以产生新的纹理样本。然后，我们提出了自然测地线(即两点之间的最短路径)在任意纹理之间进行插补的最优传输度量。与其他基于CNN的方法相比，我们的插值法似乎更接近纹理感知的几何形状，并且我们的数学框架更适合于研究其统计性质。我们应用我们的方法，测量与人类观察者的插补参数相关的感知尺度，以及猕猴视觉皮质不同区域的神经敏感度。
Texture synthesis models are important tools for understanding visual processing. In particular, statistical approaches based on neurally relevant features have been instrumental in understanding aspects of visual perception and of neural coding. New deep learning-based approaches further improve the quality of synthetic textures. Yet, it is still unclear why deep texture synthesis performs so well, and applications of this new framework to probe visual perception are scarce. Here, we show that distributions of deep convolutional neural network (CNN) activations of a texture are well described by elliptical distributions and therefore, following optimal transport theory, constraining their mean and covariance is sufficient to generate new texture samples. Then, we propose the natural geodesics (i.e. the shortest path between two points) arising with the optimal transport metric to interpolate between arbitrary textures. Compared to other CNN-based approaches, our interpolation method appears to match more closely the geometry of texture perception, and our mathematical framework is better suited to study its statistical nature. We apply our method by measuring the perceptual scale associated to the interpolation parameter in human observers, and the neural sensitivity of different areas of visual cortex in macaque monkeys.