Neural and computational basis of dynamic material perception

当我们看物体时，我们不仅确定它们的身份和位置，还确定它们的构成。对柔软或脆弱等材料特性的视觉推断对于预测和与我们的环境互动至关重要。然而，目前还不清楚大脑是如何实现这一非凡壮举的。我将汇集两个领先实验室（Tenenbaum博士和Kanwisher博士）的专业知识，在计算建模、脑成像和我自己在材料感知方面的经验方面具有互补的能力，从根本上推进我们对动态显示中材料的感知和推理的理解。为期两年的研究计划分为四个不同的工作包（WP）。我将首先使用计算机模拟和摄影来创建一个描述自然材料相互作用的视频基准数据集，例如变形物体或倒液体（WP1）。场景子集将用于研究动态材料的感知表征的行为实验（WP 2）和fMRI任务以揭示潜在的神经表征（WP3）。为了理解动态材料的视觉推理所需的计算，我们将开发一组基于基准数据集（WP4）训练的竞争性计算模型。我们将使用表征相似性分析测试动态材料的感知、神经和计算表征之间的对应关系。重要的是，每个WP也将独立地推进我们的知识，因为相同的数据将用于互补分析，例如发现动态场景中材料属性的神经编码维度。拟议的研究直接建立在我在物质感知、心理物理学、物理模拟和计算机图形学方面的专业知识基础上，并为我提供了在麻省理工学院两个世界知名实验室获得先进建模技术、机器学习和神经成像新技能的独特机会。

When we look at objects, we determine not only their identity and location, but what they are made of. Visual inference of material properties like softness or fragility is crucial to predicting and interacting with our environment. Yet, it is unclear how the brain achieves this remarkable feat. I will bring together expertise from two leading labs (Dr. Tenenbaum, and Dr. Kanwisher) with complementary competencies in computational modeling, brain imaging and my own experience in material perception to fundamentally advance our understanding of how we perceive and reason about materials in dynamic displays. The two-year research program is structured into four distinct work packages (WP). I will first use computer simulations and photography to create a benchmark dataset of videos depicting naturalistic material interactions, e.g. deforming objects, or pouring liquids (WP1). A subset of scenes will be used in a behavioral experiment investigating the perceptual representation of dynamic materials (WP 2) and in an fMRI task to uncover the underlying neural representations (WP3). To understand the computations entailed in visual inference about dynamic materials, we will develop a set of competing computational models trained on our benchmark dataset (WP4). We will test the correspondence between the perceptual, neural, and computational representations of dynamic materials using representational similarity analysis. Importantly, each WP will also advance our knowledge independently, because the same data will be used in complementary analyses, e.g. discovering dimensions of neural coding of material properties in dynamic scenes. The proposed research builds directly on my expertise in material perception, psychophysics, physics simulations, and computer graphics, and gives me the unique opportunity to acquire new skills in advanced modelling techniques, machine learning and neuroimaging in two world-renowned laboratories at MIT.