Neural and computational basis of dynamic material perception

当我们观察物体时，我们不仅确定了它们的身份和位置，还确定了它们是由什么组成的。材料特性的视觉推断，如柔软度或脆性，对于预测和与我们的环境互动至关重要。然而，目前还不清楚大脑是如何实现这一非凡壮举的。我将汇集两个领先实验室(Tenenbaum博士和Kanwisher博士)的专业知识，他们在计算建模、脑成像方面的能力互补，以及我自己在材料感知方面的经验，以从根本上提高我们对动态显示中我们如何感知和推理材料的理解。这项为期两年的研究计划分为四个不同的工作包(WP)。我将首先使用计算机模拟和摄影来创建描述自然材料相互作用的视频的基准数据集，例如变形物体或倒液体(WP1)。场景的一个子集将用于研究动态材料的感知表征的行为实验(WP2)和fMRI任务，以揭示潜在的神经表征(WP3)。为了了解动态材料的视觉推理所需的计算，我们将开发一组在我们的基准数据集(WP4)上训练的竞争计算模型。我们将使用表象相似性分析来测试动态材料的感知、神经和计算表象之间的一致性。重要的是，每个WP还将独立地推进我们的知识，因为相同的数据将用于互补分析，例如发现动态场景中材料属性的神经编码维度。拟议的研究直接建立在我在材料感知、心理物理学、物理模拟和计算机图形学方面的专业知识基础上，并给了我在麻省理工学院两个世界著名的实验室获得先进建模技术、机器学习和神经成像方面的新技能的独特机会。

When we look at objects, we determine not only their identity and location, but what they are made of. Visual inference of material properties like softness or fragility is crucial to predicting and interacting with our environment. Yet, it is unclear how the brain achieves this remarkable feat. I will bring together expertise from two leading labs (Dr. Tenenbaum, and Dr. Kanwisher) with complementary competencies in computational modeling, brain imaging and my own experience in material perception to fundamentally advance our understanding of how we perceive and reason about materials in dynamic displays. The two-year research program is structured into four distinct work packages (WP). I will first use computer simulations and photography to create a benchmark dataset of videos depicting naturalistic material interactions, e.g. deforming objects, or pouring liquids (WP1). A subset of scenes will be used in a behavioral experiment investigating the perceptual representation of dynamic materials (WP 2) and in an fMRI task to uncover the underlying neural representations (WP3). To understand the computations entailed in visual inference about dynamic materials, we will develop a set of competing computational models trained on our benchmark dataset (WP4). We will test the correspondence between the perceptual, neural, and computational representations of dynamic materials using representational similarity analysis. Importantly, each WP will also advance our knowledge independently, because the same data will be used in complementary analyses, e.g. discovering dimensions of neural coding of material properties in dynamic scenes. The proposed research builds directly on my expertise in material perception, psychophysics, physics simulations, and computer graphics, and gives me the unique opportunity to acquire new skills in advanced modelling techniques, machine learning and neuroimaging in two world-renowned laboratories at MIT.