NCS-FO: Active Listening and Attention in 3D Natural Scenes

当人类和其他动物四处走动时，他们身体和环境中物体之间的距离和方向不断变化。判断物体的位置，调整身体动作以绕过物体，需要大脑中不断更新的三维空间地图。生成这个地图，并在移动过程中保持它的更新，需要视觉或听觉提示、注意力和行为输出之间的动态交互。对空间知觉是如何在大脑中产生的理解来自于数十年来在受限条件下使用视觉或听觉刺激的研究。关于自然场景如何在自由移动的动物身上表现出来的动态，我们知道的要少得多。这个项目将通过研究自由飞行的蝙蝠如何使用回声定位在它们的环境中导航来弥补这一差距。具体地说，一个由工程师和神经学家组成的团队将研究蝙蝠大脑是如何处理与飞行导航相关的信息的。该项目团队将为公立学校、本科生和研究生以及博士后研究人员提供工程和科学方面的教育和培训。这项研究还将为丰富的材料库做出贡献，包括视频和网站，供私营和公共部门的教育工作者和科学家使用。该项目利用创新的工程工具、尖端的神经科学方法和神经行为学建模，对参与自然任务的自由移动动物的3D场景表示、注意力和动作选择之间的动态反馈进行多学科调查。回声定位蝙蝠是该项目研究的对象，它主动产生声音信号，用来代表自然场景，因此提供了直接访问指导行为的感觉信息的途径。该项目的具体目标是测试以下假设：1)自然场景表征通过感觉处理、自适应运动行为和注意反馈之间的相互作用来运作；2)当动物调整其行为以关注选定的目标时，通过神经元集合对感觉流的时空反应变得尖锐；以及3)神经反应的时空锐化使自然环境中的图形-背景分离。该项目包括1)新的声学测量和计算分析，以重建蝙蝠的声纳发射器和接收器特性，并结合环境的3D声学模型来表示声纳场景；2)定量分析回声定位蝙蝠在穿越复杂环境时的自适应回声定位和飞行行为；3)自由飞行蝙蝠中脑的多通道神经遥测记录，因为它关注周围的目标、障碍物和其他声音信号；以及4)听觉系统架构、注意力和工作记忆机制的计算建模。综上所述，本研究将深化对自然场景表征行为调控的理解。该项目由理解神经和认知系统的综合策略(NSF-NCS)资助，NSF-NCS是一个多学科项目，由计算机和信息科学与工程(CEISE)、教育和人力资源(EHR)、工程(ENG)以及社会、行为和经济科学(SBE)的主管部门联合支持。

As humans and other animals move around, the distance and direction between their bodies and objects in their environment are constantly changing. Judging the position of objects, and readjusting body movements to steer around the objects, requires a constantly updated map of three-dimensional space in the brain. Generating this map, and keeping it updated during movement, requires dynamic interaction between visual or auditory cues, attention, and behavioral output. An understanding of how spatial perception is generated in the brain comes from decades of research using visual or auditory stimuli under restricted conditions. Far less is known about the dynamics of how natural scenes are represented in freely moving animals. This project will bridge this gap by studying how freely flying bats navigate through their environment using echolocation. Specifically, a team of engineers and neuroscientists will investigate how the bat brain processes information associated with flight navigation. The project team will provide education and training in engineering and science to public school, undergraduate and graduate students, and to postdoctoral researchers. This research will also contribute to a rich library of materials, including videos and a website, which will be available to educators and scientists working in both the private and public sectors.This project leverages innovative engineering tools, cutting-edge neuroscience methods and neuroethological modeling to pursue a multidisciplinary investigation of dynamic feedback between 3D scene representation, attention and action-selection in freely moving animals engaged in natural tasks. The echolocating bat, the subject of the project's research, actively produces the acoustic signals that it uses to represent natural scenes and therefore provides direct access to the sensory information that guides behavior. The specific goals of the project are to test the hypotheses that 1) natural scene representation operates through the interplay between sensory processing, adaptive motor behaviors, and attentional feedback, 2) spatio-temporal responses to sensory streams across ensembles of neurons sharpen when an animal adapts its behavior to attend to selected targets, and 3) spatio-temporal sharpening of neural responses enables figure-ground segregation in the natural environment. The project integrates 1) novel acoustic measurements and computational analyses to represent the sonar scene based on reconstructions of the bat's sonar transmitter and receiver characteristics, combined with a 3D acoustic model of the environment, 2) quantitative analysis of the echolocating bat's adaptive echolocation and flight behaviors as it negotiates complex environments, 3) multichannel neural telemetry recordings from the midbrain of the free-flying bat as it attends to targets, obstacles and other acoustic signals in its surroundings, and 4) computational modeling of auditory system architecture, attention and working memory mechanisms. Collectively, this research will deepen the understanding of behavioral modulation of natural scene representation. This project is funded by Integrative Strategies for Understanding Neural and Cognitive Systems (NSF-NCS), a multidisciplinary program jointly supported by the Directorates for Computer and Information Science and Engineering (CISE), Education and Human Resources (EHR), Engineering (ENG), and Social, Behavioral, and Economic Sciences (SBE).