CAREER: Computational and Neural Basis of Social Perception

为了恰当地与他人互动，人们需要识别身份、面部表情和动作。这种认知背后的大脑机制是什么？该项目提出，面部特征和表情是由共同的大脑区域处理的，而不同的区域专门处理静态和动态信息。这可能是一个更普遍的现象的特例：“互补性”，即不同的任务，如识别身份和识别表情，由相同的大脑区域执行，因为解决其中一个任务有助于解决另一个任务。互补性可能是大脑组织的一般原则。这项研究旨在通过测量神经活动和构建人工神经网络，为社会感知和更广泛地了解人脑的大规模组织提供新的线索。这旨在提高对社会认知及其损害的理解，例如自闭症谱系障碍。该项目将为学生设计课程和研究机会，让他们在神经科学和人工智能(AI)的交叉点接受培训，为他们利用新的AI工具学习神经科学问题做好准备，并在神经科学最先进研究的启发下建立新的AI。该项目还将通过面对面和在线活动相结合的方式，接触到更广泛的社区。根据一个经典理论，面孔身份和面部表情是由不同的神经通路处理的。面部的身份由颞叶腹侧(枕面部面积：OFA，梭形面部面积：FFA)识别，表情由外侧颞叶(后上颞沟：PSTS)识别。相比之下，最近的研究确定了pSTS中的身份信息，以及OFA和FFA中的表达信息。该项目假设：1)面孔身份和表情等正交属性的表征在识别过程中协同工作，并在相同的大脑区域自发出现；2)这种现象不仅出现在面孔身份和表情的情况下，而且发生在身体身份和动作的情况下，构成了更广泛的社会知觉组织原则；3)腹侧区域不是通过它们编码的内容来区分，而是通过它们处理的属性的类型来区分。该项目将通过让参与者在接受功能磁共振检查时观看受控的、自然的表情和动作视频，并使用深层神经网络的特征用编码模型对他们的反应进行建模，来测试这些假设。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

In order to interact appropriately with others, people need to recognize identities, facial expressions, and actions. What are the brain mechanisms underlying this recognition? This project proposes that face identity and expressions are processed by common brain regions, and that distinct regions are specialized to process static and dynamic information. This could be a particular case of a more general phenomenon: “complementarity”, in which different tasks like recognizing identity and recognizing expressions are performed by the same brain regions, because solving one of the tasks helps to solve the other. Complementarity might be a general principle of organization of the brain. This research aims to shed new light into social perception and more broadly into the large-scale organization of the human brain, through measurement of neural activity and building artificial neural networks. This aims to improve understanding of social cognition and its impairments, such as autism spectrum disorders. The project will design courses and research opportunities for students to receive training at the intersection between neuroscience and artificial intelligence (AI), preparing them to study neuroscience questions with new AI tools, and to build new AI inspired by state-of-the-art research in neuroscience. The project will also reach out to the broader community, using a combination of in-person and online events. According to a classical theory, face identity and facial expressions are processed by separate neural pathways. The identity of a face is recognized by the ventral temporal lobe (occipital face area: OFA, fusiform face area: FFA), while expressions are recognized by the lateral temporal lobe (posterior superior temporal sulcus: pSTS). By contrast, recent research identified identity information in pSTS, and expression information in OFA and FFA. This project hypothesizes that: 1) representations of orthogonal properties like face identity and expressions operate synergistically during recognition, and arise spontaneously within the same brain regions; 2) this phenomenon occurs not only in the case of face identity and expressions, but also in the case of body identity and actions, constituting a broader principle of organization of social perception; 3) ventral and lateral regions are not distinguished by the content they encode, but by the type of properties they process. The project will test these hypotheses by asking participants to watch controlled and naturalistic videos of expressions and actions while undergoing fMRI, and by modeling their responses with encoding models using features from deep neural networks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.