NCS-FO: Active Listening and Attention in 3D Natural Scenes

当人类和其他动物四处移动时，他们的身体和环境中的物体之间的距离和方向不断变化。 判断物体的位置，重新调整身体动作以绕过物体，需要大脑中不断更新的三维空间地图。 生成这个地图，并在运动过程中保持更新，需要视觉或听觉线索，注意力和行为输出之间的动态交互。 对空间感知如何在大脑中产生的理解来自数十年来在限制条件下使用视觉或听觉刺激的研究。 关于自然景物如何在自由移动的动物身上表现的动态，人们知之甚少。 该项目将通过研究自由飞行的蝙蝠如何利用回声定位在环境中导航来弥合这一差距。 具体来说，一个由工程师和神经科学家组成的团队将研究蝙蝠大脑如何处理与飞行导航相关的信息。该项目小组将为公立学校、本科生和研究生以及博士后研究人员提供工程和科学方面的教育和培训。这项研究还将有助于丰富的材料库，包括视频和网站，这将提供给教育工作者和科学家在私营和公共部门的工作。该项目利用创新的工程工具，尖端的神经科学方法和神经行为学建模，追求3D场景表示，从事自然任务的自由移动动物的注意力和行动选择。回声定位蝙蝠是该项目的研究对象，它积极地产生声音信号，用于代表自然场景，因此提供了直接获得指导行为的感官信息的途径。该项目的具体目标是测试以下假设：1）自然场景表征通过感觉处理，适应性运动行为和注意力反馈之间的相互作用进行操作，2）当动物适应其行为以关注选定的目标时，对神经元集合的感觉流的时空反应变得尖锐，以及3）神经反应的时空锐化使得自然环境中的图形-背景分离成为可能。该项目集成了1）基于蝙蝠声纳发射器和接收器特性的重建，结合环境的3D声学模型，新颖的声学测量和计算分析来表示声纳场景，2）定量分析回声定位蝙蝠在复杂环境中的适应性回声定位和飞行行为，3）当自由飞行的蝙蝠关注其周围的目标、障碍物和其他声学信号时，来自其中脑的多通道神经遥测记录，以及4）听觉系统架构、注意力和工作记忆机制的计算建模。 总的来说，本研究将加深对自然场景表征的行为调制的理解。该项目由理解神经和认知系统的综合策略（NSF-NCS）资助，这是一个由计算机和信息科学与工程（CISE），教育和人力资源（EHR），工程（ENG）以及社会，行为和经济科学（SBE）董事会共同支持的多学科计划。

As humans and other animals move around, the distance and direction between their bodies and objects in their environment are constantly changing. Judging the position of objects, and readjusting body movements to steer around the objects, requires a constantly updated map of three-dimensional space in the brain. Generating this map, and keeping it updated during movement, requires dynamic interaction between visual or auditory cues, attention, and behavioral output. An understanding of how spatial perception is generated in the brain comes from decades of research using visual or auditory stimuli under restricted conditions. Far less is known about the dynamics of how natural scenes are represented in freely moving animals. This project will bridge this gap by studying how freely flying bats navigate through their environment using echolocation. Specifically, a team of engineers and neuroscientists will investigate how the bat brain processes information associated with flight navigation. The project team will provide education and training in engineering and science to public school, undergraduate and graduate students, and to postdoctoral researchers. This research will also contribute to a rich library of materials, including videos and a website, which will be available to educators and scientists working in both the private and public sectors.This project leverages innovative engineering tools, cutting-edge neuroscience methods and neuroethological modeling to pursue a multidisciplinary investigation of dynamic feedback between 3D scene representation, attention and action-selection in freely moving animals engaged in natural tasks. The echolocating bat, the subject of the project's research, actively produces the acoustic signals that it uses to represent natural scenes and therefore provides direct access to the sensory information that guides behavior. The specific goals of the project are to test the hypotheses that 1) natural scene representation operates through the interplay between sensory processing, adaptive motor behaviors, and attentional feedback, 2) spatio-temporal responses to sensory streams across ensembles of neurons sharpen when an animal adapts its behavior to attend to selected targets, and 3) spatio-temporal sharpening of neural responses enables figure-ground segregation in the natural environment. The project integrates 1) novel acoustic measurements and computational analyses to represent the sonar scene based on reconstructions of the bat's sonar transmitter and receiver characteristics, combined with a 3D acoustic model of the environment, 2) quantitative analysis of the echolocating bat's adaptive echolocation and flight behaviors as it negotiates complex environments, 3) multichannel neural telemetry recordings from the midbrain of the free-flying bat as it attends to targets, obstacles and other acoustic signals in its surroundings, and 4) computational modeling of auditory system architecture, attention and working memory mechanisms. Collectively, this research will deepen the understanding of behavioral modulation of natural scene representation. This project is funded by Integrative Strategies for Understanding Neural and Cognitive Systems (NSF-NCS), a multidisciplinary program jointly supported by the Directorates for Computer and Information Science and Engineering (CISE), Education and Human Resources (EHR), Engineering (ENG), and Social, Behavioral, and Economic Sciences (SBE).