NCS-FO:Collaborative Research:Decoding and Reconstructing the Neural Basis of Real World Social Perception

社会和情感感知是决定我们在日常生活中如何与他人互动的关键输入。因此，拥有一个社会和情感感知的神经生物学基础的模型对于理解人类行为的神经基础至关重要。我们对社会和情感感知的神经基础的绝大多数理解来自于在人工实验室环境中进行的研究，这些研究无法捕捉真实世界社会互动的丰富性，复杂性和显著性。该项目旨在填补这一知识空白。为了实现这一目标，研究人员将记录接受神经外科治疗癫痫患者的脑电活动。为了确定负责癫痫发作的大脑区域，这些患者在大脑的各个部位植入电极，然后他们在医院度过1-2周，在此期间他们与医生，护士，朋友和家人访客等互动。以及在这些互动过程中的交流。这些研究的结果有可能通过阐明真实的生活中有意义的互动中这些过程的神经基础来改变我们对社会和情感感知的理解。缺乏自然的、真实的世界的社会和情感感知的神经基础的模型是理解这些过程的关键障碍，并且最终是开发用于社会和情感感知的衰弱性神经和精神障碍（诸如自闭症、创伤后应激障碍等）的治疗的关键障碍。这个奖项将为新的研究人员提供一个途径，让他们利用人脑直接记录所提供的难得和宝贵的基础神经科学研究机会。这项研究得到了EHR核心研究计划的支持，为STEM学习和学习环境的基础研究提供资金，扩大STEM的参与，以及STEM劳动力的发展。使用非自然刺激开发的社会视觉感知模型通常假设神经元具有不变的反应灵敏度，并被组织成自下而上的层次结构。虽然最近的一些模型承认反馈的作用，但它们仍然过于简单，核心系统的数量相对有限，而且往往忽视社会背景和动态先验知识的作用。这些模型不太可能完全推广到自然的社会视觉，在自然的社会视觉中，系统可以快速主动地调整其响应，以优化对丰富而复杂的自然视觉输入的处理。PI及其同事将联合收割机结合在长时间自然视觉行为中捕获的颅内EEG（iEEG）记录与尖端计算机视觉，机器学习和统计分析，以了解自然，真实世界视觉感知的神经基础。他们的研究计划的目标是开发第一个完全生态验证的社会感知模型。研究人员将利用iEEG的最新进展，结合尖端的凝视跟踪技术、视频分析工具以及大数据统计和机器学习工具，来了解在现实社会视觉中发生的快速、复杂的神经信息处理。该项目将涉及解码神经活动的时空模式，并在每时每刻的基础上重建他们在这些不同层次上看到的人的表达特征。该项目的多学科性质为学生和博士后提供了一个良好的环境，在计算方法，统计和神经科学方面进行培训。鉴于神经科学中高级计算和统计方法的快速发展，这种多学科培训对现代神经科学家至关重要。加强对社会认知机制的理解对教与学具有重要意义。例如，了解更多关于人们如何形成对彼此的印象可以告知教师在教育环境中识别和回应学生和其他利益相关者的能力。该项目由理解神经和认知系统的综合策略（NSF-NCS）资助，这是一个由计算机和信息科学与工程（CISE），教育和人力资源（EHR），工程（ENG）和社会，行为和经济科学（SBE）。

Social and affective perception is the critical input that governs how we interact with others during everyday life. Consequently, having a model of the neurobiological basis of social and affective perception is critical for understanding the neural basis of human behavior. The overwhelming majority of our understanding of the neural basis of social and affective perception comes from studies done in artificial lab settings, which cannot capture the richness, complexity, and salience of real-world social interactions. This project aims to fill this gap in knowledge. To accomplish this goal, the researchers will record electrical brain activity from patients undergoing neurosurgical treatment for epilepsy. To determine the region of the brain responsible for their seizures, these patients are implanted with electrodes in various parts of their brain and then they spend 1-2 weeks in the hospital during which they interact with doctors, nurses, friend and family visitors, etc. This award will support research into using the recordings from their brains to understand how these patients perceive and understand the actions, emotions, and communication during these interactions on a moment-to-moment basis. The results of these studies have the potential to transform our understanding of social and affective perception by illuminating the neural basis of these processes during real life, meaningful interactions. The lack of models of the neural basis of natural, real world social and affective perception is a critical impediment to understanding these processes and ultimately a developing treatments for debilitating neurological and psychiatric disorders of social and affective perception, such as autism, post traumatic stress disorder, etc. In addition, through education, mentoring, and teaching, this award will provide an avenue for new researchers to take advantage of the rare and valuable opportunity for basic neuroscientific research provided by direct recordings from the human brain. This research is supported by the EHR Core Research Program, providing funding for fundamental research in STEM learning and learning environments, broadening participation in STEM, and STEM workforce development. Models of social visual perception developed using unnatural stimuli often assume that neurons have unchanging response sensitivity and are organized into bottom-up hierarchies. While some recent models acknowledge the role of feedback, they remain simplistic with a relatively limited number of core systems and often neglect of the role of social context and dynamic prior knowledge. These models are unlikely to fully generalize to natural social vision where the system can rapidly and actively adapt its response to optimize processing of rich and complex natural visual input. The PI and colleagues will combine intracranial EEG (iEEG) recordings captured during long stretches of natural visual behavior with cutting-edge computer vision, machine learning, and statistical analyses to understand the neural basis of natural, real-world visual perception. The goal of their program of research is to develop the first fully ecologically validated models of social perception. The researchers will use recent advances in iEEG in combination with cutting-edge gaze tracking technology, video analysis tools, and big data statistical and machine learning tools to understand the rapid, complex neural information processing that occurs during real-world social vision. The project will involve decoding the spatiotemporal patterns of neural activity and reconstruct the expressive features of people they see at these different levels on a moment-to-moment basis. The multidisciplinary nature of this project provides an excellent environment for students and postdocs to be trained in computational methods, statistics, and neuroscience. Given the rapid advance of high-level computational and statistical methods in neuroscience, this multidisciplinary training is critical for modern neuroscientists. Enhanced understanding of the mechanisms involved in social cognition has implications for teaching and learning. For example, knowing more about how people form impressions of one another can inform teachers' abilities to recognize and respond to students and other stakeholders in educational settings.This project is funded by Integrative Strategies for Understanding Neural and Cognitive Systems (NSF-NCS), a multidisciplinary program jointly supported by the Directorates for Computer and Information Science and Engineering (CISE), Education and Human Resources (EHR), Engineering (ENG), and Social, Behavioral, and Economic Sciences (SBE).