Learning diagnostic latent representations for human material perception: common mechanisms and individual variability

摘要 视觉辨别和识别材料（如判断杯子是塑料还是玻璃） 对于日常任务至关重要，例如在不同表面上行走，使用工具和选择食物;然而， 物质感知仍然知之甚少。主要的挑战是，给定的材料可以采取巨大的 各种外观取决于3D形状，照明和对象类，人类必须解开这些 来达到感知的恒定性。先前的研究揭示了有用的图像线索，并发现3D几何 以复杂的方式与物质感知相互作用。然而，发现的图像线索并不普遍 跨越材质和场景。拟议的工作将联合收割机无监督生成模型与人类 心理物理学，以确定一个代表，可以解开物理特性和发现诊断图像 没有标记的图像数据的特征。具体目标1是识别预测人类的潜在表示 材料辨别，使用计算机渲染图像训练的无监督深度神经网络。的 具体目标2是表征高级语义材料感知，高级识别的效果， 以及在属性评定和再认任务上的个体差异。去发现一个真实世界的表象 材料，PI和团队将训练一个无监督的基于风格的生成对抗网络（StyleGAN） 真实世界的照片。初步结果表明，StyleGAN可以生成逼真多样的图像 的材料。总的来说，这些研究将探讨语义层面的材料感知过程如何与 从无监督模型中学习到的自然环境的统计结构。拟议的工作将 还揭示了高层次视觉和中级表示之间的任务依赖性相互作用，并提供 寻求材料感知的神经相关性的指导。本提案中提出的方法，如 由于发现具有有限人类标记数据的感知维度并表征个体可变性， 对其他认知研究有影响。该地区的建议提供了一个独特的多学科培训 有机会让美国大学的不同本科生参与心理物理学的研究， 机器学习和图像处理。PI和学生们还将研究一种新的招募方法 用“同龄人招募”的方式来招募代表性不足的人类实验对象。“最后，这项建议的预期结果将有 对长期争论的影响，即感知表征的预先确定程度 通过进化或经验学习。

Abstract Visually discriminating and identifying materials (such as judging whether a cup is made of plastic or glass) is crucial for everyday tasks, such as walking on different surfaces, using tools, and selecting food; and yet material perception remains poorly understood. The main challenge is that a given material can take an enormous variety of appearances depending on the 3D shape, lighting, and object class, and humans must untangle these to achieve perceptual constancy. Previous research revealed useful image cues and found that 3D geometry interacts with the material perception in intricate ways. The discovered image cues, however, do not generalize across materials and scenes. The proposed work will combine unsupervised generative models with human psychophysics to identify a representation that can disentangle physical properties and discover diagnostic image features without labeled image data. The specific Aim 1 is to identify a latent representation that predicts human material discrimination, using unsupervised deep neural networks trained with computer rendered images. The specific Aim 2 is to characterize high-level semantic material perception, the effects of high-level recognition as well as individual differences on attribute rating and recognition tasks. To discover a representation of real-world materials, the PI and the team will train a unsupervised style-based Generative Adversarial Network (StyleGAN) on real-world photographs. The preliminary results show that StyleGAN can generate realistic and diverse images of materials. Collectively, these studies will explore how the semantic-level material perception process relates to the statistical structure of the natural environment learned from unsupervised models. The proposed work will also uncover the task-dependent interplay between high-level vision and mid-level representations, and provide guidance for seeking neural correlates of material perception. The methods developed in this proposal, such as discovering perceptual dimensions with limited human labeled data and characterizing individual variability, have impact for other research in cognition. The AREA proposal provides a unique multidisciplinary training opportunity to engage diverse undergraduate students at American University in the research of psychophysics, machine learning, and image processing. The PI and students will also investigate a novel method of recruiting under-represented human subjects using "peer-recruiting." Finally, the expected findings of this proposal will have implications for the long-standing debate about the degree to which perceptual representations are predetermined by evolution or learned via experience.