NCS-FO:Collaborative Research:Decoding and Reconstructing the Neural Basis of Real World Social Perception

社会和情感感知是决定我们在日常生活中如何与他人互动的关键输入。因此，拥有社会和情感感知的神经生物学基础模型对于理解人类行为的神经基础至关重要。我们对社会和情感感知的神经基础的绝大多数理解来自于在人工实验室环境中进行的研究，这些研究无法捕捉到现实世界社会互动的丰富性、复杂性和显著性。该项目旨在填补这一知识空白。为了实现这一目标，研究人员将记录接受神经外科治疗的癫痫患者的脑电活动。为了确定导致癫痫发作的大脑区域，这些患者在大脑的不同部位植入电极，然后他们在医院呆1-2周，在此期间他们与医生、护士、朋友和家人来访者等进行互动。该奖项将支持研究使用来自他们大脑的记录来了解这些患者如何感知和理解这些互动过程中的行为、情绪和交流。这些研究的结果有可能通过阐明在现实生活中有意义的互动中这些过程的神经基础来改变我们对社会和情感感知的理解。缺乏自然的、真实世界的社会和情感感知的神经基础模型是理解这些过程的关键障碍，并最终阻碍了对社会和情感感知的衰弱性神经和精神疾病（如自闭症、创伤后应激障碍等）的治疗。此外，通过教育、指导和教学，该奖项将为新的研究人员提供一条途径，使他们能够利用由人类大脑直接记录提供的罕见而宝贵的基础神经科学研究机会。本研究得到了EHR核心研究计划的支持，为STEM学习和学习环境的基础研究提供资金，扩大STEM参与和STEM劳动力发展。利用非自然刺激建立的社会视觉感知模型通常假设神经元具有不变的反应敏感性，并被组织成自下而上的层次结构。虽然最近的一些模型承认反馈的作用，但它们仍然过于简单，核心系统的数量相对有限，并且经常忽视社会背景和动态先验知识的作用。这些模型不太可能完全推广到自然社会视觉，在自然社会视觉中，系统可以快速主动地调整其响应，以优化对丰富而复杂的自然视觉输入的处理。PI及其同事将把在长时间自然视觉行为中捕获的颅内脑电图（iEEG）记录与尖端的计算机视觉、机器学习和统计分析相结合，以了解自然的、现实世界的视觉感知的神经基础。他们研究项目的目标是开发第一个完全经过生态学验证的社会感知模型。研究人员将利用iEEG的最新进展，结合尖端的注视跟踪技术、视频分析工具、大数据统计和机器学习工具，了解在现实世界的社会视觉中发生的快速、复杂的神经信息处理。该项目将涉及解码神经活动的时空模式，并重建他们在这些不同层次上每时每刻看到的人的表达特征。这个项目的多学科性质为学生和博士后在计算方法、统计学和神经科学方面的训练提供了一个很好的环境。鉴于神经科学中高级计算和统计方法的快速发展，这种多学科培训对现代神经科学家至关重要。加强对社会认知机制的理解对教学和学习具有重要意义。例如，更多地了解人们如何形成彼此的印象，可以提高教师在教育环境中识别和回应学生和其他利益相关者的能力。该项目由理解神经和认知系统的综合策略（NSF-NCS）资助，这是一个由计算机与信息科学与工程（CISE）、教育与人力资源（EHR）、工程（ENG）和社会、行为和经济科学（SBE）联合支持的多学科项目。

Social and affective perception is the critical input that governs how we interact with others during everyday life. Consequently, having a model of the neurobiological basis of social and affective perception is critical for understanding the neural basis of human behavior. The overwhelming majority of our understanding of the neural basis of social and affective perception comes from studies done in artificial lab settings, which cannot capture the richness, complexity, and salience of real-world social interactions. This project aims to fill this gap in knowledge. To accomplish this goal, the researchers will record electrical brain activity from patients undergoing neurosurgical treatment for epilepsy. To determine the region of the brain responsible for their seizures, these patients are implanted with electrodes in various parts of their brain and then they spend 1-2 weeks in the hospital during which they interact with doctors, nurses, friend and family visitors, etc. This award will support research into using the recordings from their brains to understand how these patients perceive and understand the actions, emotions, and communication during these interactions on a moment-to-moment basis. The results of these studies have the potential to transform our understanding of social and affective perception by illuminating the neural basis of these processes during real life, meaningful interactions. The lack of models of the neural basis of natural, real world social and affective perception is a critical impediment to understanding these processes and ultimately a developing treatments for debilitating neurological and psychiatric disorders of social and affective perception, such as autism, post traumatic stress disorder, etc. In addition, through education, mentoring, and teaching, this award will provide an avenue for new researchers to take advantage of the rare and valuable opportunity for basic neuroscientific research provided by direct recordings from the human brain. This research is supported by the EHR Core Research Program, providing funding for fundamental research in STEM learning and learning environments, broadening participation in STEM, and STEM workforce development. Models of social visual perception developed using unnatural stimuli often assume that neurons have unchanging response sensitivity and are organized into bottom-up hierarchies. While some recent models acknowledge the role of feedback, they remain simplistic with a relatively limited number of core systems and often neglect of the role of social context and dynamic prior knowledge. These models are unlikely to fully generalize to natural social vision where the system can rapidly and actively adapt its response to optimize processing of rich and complex natural visual input. The PI and colleagues will combine intracranial EEG (iEEG) recordings captured during long stretches of natural visual behavior with cutting-edge computer vision, machine learning, and statistical analyses to understand the neural basis of natural, real-world visual perception. The goal of their program of research is to develop the first fully ecologically validated models of social perception. The researchers will use recent advances in iEEG in combination with cutting-edge gaze tracking technology, video analysis tools, and big data statistical and machine learning tools to understand the rapid, complex neural information processing that occurs during real-world social vision. The project will involve decoding the spatiotemporal patterns of neural activity and reconstruct the expressive features of people they see at these different levels on a moment-to-moment basis. The multidisciplinary nature of this project provides an excellent environment for students and postdocs to be trained in computational methods, statistics, and neuroscience. Given the rapid advance of high-level computational and statistical methods in neuroscience, this multidisciplinary training is critical for modern neuroscientists. Enhanced understanding of the mechanisms involved in social cognition has implications for teaching and learning. For example, knowing more about how people form impressions of one another can inform teachers' abilities to recognize and respond to students and other stakeholders in educational settings.This project is funded by Integrative Strategies for Understanding Neural and Cognitive Systems (NSF-NCS), a multidisciplinary program jointly supported by the Directorates for Computer and Information Science and Engineering (CISE), Education and Human Resources (EHR), Engineering (ENG), and Social, Behavioral, and Economic Sciences (SBE).