Active Social Vision: How the Brain Processes Visual Information During Natural Social Perception

摘要/摘要 社会视觉神经科学的主导范式一直是呈现简化的刺激，通常 由孤立的静态图像组成，没有上下文，被动地呈现给需要保持凝视的受试者 定格。虽然使用这种范式已经学到了很多，但它不能捕捉到社会愿景(SV)的所有方面 行得通。具体地说，主动观察者结合了运动信息收集行为(眼睛和头部运动) 在复杂、自然的多感官社会环境中以最佳方式采样感知信息。事实上，在那里 在SV的神经生物学研究中，很少有地方的主动视觉比这更重要。现实世界中的SV是一个 典型的主动感知过程，其中我们使用我们的经验，以及我们所有的感知和解释 在复杂和动态的环境中从他人那里收集社交和情感信息的能力。简单 PUT，SV，正如通常研究的那样，不足以接近于积极地与朋友、家人或你的 医生。计算机视觉、机器学习和计算分析的最新进展现在提供了 攻击社会神经科学的一个关键目标：社会和情感系统如何引导活跃的信息 聚集在自然的社会环境中。我们提出了一种新的自然方法来研究SV。 该项目将利用一种强大的技术来测量人脑的活动：直接记录 从植入外科癫痫患者大脑的电极中提取。外科癫痫患者花费1-2 在医院呆上几周，同时监测他们的大脑活动，为他们提供了记录的独特机会 在与朋友、家人、医生、护士、实验者等自然互动期间的多尺度神经活动 此外，患者将玩一个社交游戏，以允许使用半控制和 可重复的任务。此外，参与者将参与更标准的实验室SV范例，以评估如何 现实世界条件下的结果与传统实验的结果进行了比较。神经记录 将与视频和音频监控以及眼睛跟踪同时获得。最先进的计算机 视力分析将提供对社交线索(例如，眼睛凝视和面部表情)的持续评估 病人与之互动的人。计算分析和机械神经测量 将决定患者所感知的内容、神经编码和 该代码的神经生物学实现。这种方法将使我们能够跨越以下三个关键级别 了解SV信息处理系统所需的分析。我们假设现实世界中的SV 是一个扩展的迭代过程，它结合了主动的、动态的运动/注意采样策略和之前的 和上下文信息，以主动计划信息收集，这既支持也限制了流 通过系统获取信息。我们的结果将对基本的社会问题产生根本的影响 神经科学和我们对这些过程如何在SV障碍中改变的理解，如情绪 焦虑症和精神分裂症。

Summary/Abstract The dominant paradigm in social visual neuroscience has been to present simplified stimuli, often consisting of static images in isolation presented without context, passively, to subjects required to maintain gaze fixation. While much has been learned using this paradigm, it cannot capture all aspects of how social vision (SV) works. Specifically, active observers combine motoric information gathering behaviors (eye and head movement) to optimally sample perceptual information in a complex, natural multisensory social environment. Indeed, there are few places where active vision is more critical than in the study of the neurobiology of SV. Real world SV is a prototypic active sensing process where we use our experience, along with all of our sensing and interpretive capacities to gather social and affective information from other people in a complex and dynamic setting. Simply put, SV, as typically studied, does not adequately approximate actively interacting with friends, family, or your doctor. Recent advances in computer vision, machine learning, and computational analysis now provide means to attack a critical goal of social neuroscience: how the social and affective system guides active information gathering in natural social contexts. We propose a novel natural approach to studying SV. This project will leverage a powerful technique to measure activity in the human brain: direct recording from electrodes implanted in the brains of surgical epilepsy patients. Surgical epilepsy patients who spend 1-2 weeks in the hospital while having their brain activity monitored afford the unique opportunity to record multiscale neural activity during natural interactions with friends, family, doctors, nurses, experimenters, etc. In addition, patients will play a social game to allow for the study of real world SV using a semi-controlled and repeatable task. Furthermore, participants will engage in more standard laboratory SV paradigms to assess how results from real world conditions compare to the results from traditional experiments. The neural recordings will be acquired simultaneously with video and audio monitoring, and eye tracking. State-of-the-art computer vision analysis will provide a continuous assessment of social cues (e.g., eye gaze and facial expression) from people with whom the patients are interacting. Computational analyses and mechanistic neural measurements will determine the correspondence between what is perceived by the patient, the neural coding, and the neurobiological implementation of that code. This approach will allow us to bridge across three critical levels of analysis required for understanding the SV information processing systems. We hypothesize that real world SV is an extended, iterative process that combines active, dynamic motor/attentional sampling strategies with prior and contextual information to actively plan information gathering, which both enables and constrains the flow of information through the system. Our results will have fundamental implications both for basic social neuroscience and for our understanding of how these processes may be altered in disorders of SV, such as mood and anxiety disorders and schizophrenia.