Learning diagnostic latent representations for human material perception: common mechanisms and individual variability

摘要 在视觉上辨别和识别材料(例如判断杯子是塑料还是玻璃) 对于日常任务至关重要，例如在不同的表面上行走、使用工具和选择食物；然而 人们对物质的认知仍然知之甚少。主要的挑战是，一种给定的材料可能需要巨大的 根据3D形状、照明和对象类别的不同外观，人类必须理清这些 以达到知觉上的恒定。之前的研究揭示了有用的图像线索，并发现3D几何学 以错综复杂的方式与物质知觉相互作用。然而，发现的图像线索并不能一概而论 跨材质和场景。拟议的工作将把无监督的产生式模型与人类结合起来 心理物理学，以确定可以解开物理属性并发现诊断图像的表示 没有标记图像数据的要素。该特定fic目标1是识别预测人类的潜在表征 材料识别，使用与计算机渲染图像一起训练的无监督深度神经网络。这个 SPECIfic目标2表征高级语义材料感知，高级识别的效果如下 以及在属性评级和识别任务上的个体差异。发现真实世界的表现形式 材料，PI和团队将训练一个无监督的基于风格的生成性对抗网络(StyleGAN) 在真实世界的照片上。初步结果表明，StyleGan能够生成逼真多样的图像 材料。总而言之，这些研究将探索语义层面的材料知觉过程是如何联系的 从非监督模型中学习到的自然环境的统计结构。拟议的工作将 还揭示了高级视觉和中级表示之间依赖任务的相互作用，并提供 寻找物质知觉的神经关联的指导。本提案中提出的方法，如 由于利用有限的人类标记数据发现感知维度并表征个体的可变性， 对认知领域的其他研究也有影响。该领域的建议提供了独特的多学科培训 有机会让美国大学的不同本科生参与心理物理学的研究， 机器学习和图像处理。PI和学生们还将研究一种新的招聘方法 使用“同行招募”的代表不足的人类受试者。最后，此提案的预期fi结点将具有 对长期以来关于知觉表征被预先确定的程度的争论的影响 通过进化或通过经验学习。

Abstract Visually discriminating and identifying materials (such as judging whether a cup is made of plastic or glass) is crucial for everyday tasks, such as walking on different surfaces, using tools, and selecting food; and yet material perception remains poorly understood. The main challenge is that a given material can take an enormous variety of appearances depending on the 3D shape, lighting, and object class, and humans must untangle these to achieve perceptual constancy. Previous research revealed useful image cues and found that 3D geometry interacts with the material perception in intricate ways. The discovered image cues, however, do not generalize across materials and scenes. The proposed work will combine unsupervised generative models with human psychophysics to identify a representation that can disentangle physical properties and discover diagnostic image features without labeled image data. The specific Aim 1 is to identify a latent representation that predicts human material discrimination, using unsupervised deep neural networks trained with computer rendered images. The specific Aim 2 is to characterize high-level semantic material perception, the effects of high-level recognition as well as individual differences on attribute rating and recognition tasks. To discover a representation of real-world materials, the PI and the team will train a unsupervised style-based Generative Adversarial Network (StyleGAN) on real-world photographs. The preliminary results show that StyleGAN can generate realistic and diverse images of materials. Collectively, these studies will explore how the semantic-level material perception process relates to the statistical structure of the natural environment learned from unsupervised models. The proposed work will also uncover the task-dependent interplay between high-level vision and mid-level representations, and provide guidance for seeking neural correlates of material perception. The methods developed in this proposal, such as discovering perceptual dimensions with limited human labeled data and characterizing individual variability, have impact for other research in cognition. The AREA proposal provides a unique multidisciplinary training opportunity to engage diverse undergraduate students at American University in the research of psychophysics, machine learning, and image processing. The PI and students will also investigate a novel method of recruiting under-represented human subjects using "peer-recruiting." Finally, the expected findings of this proposal will have implications for the long-standing debate about the degree to which perceptual representations are predetermined by evolution or learned via experience.