US-French Collaboration: Auditory computations for interpreting and producing communication signals.

人类语言和动物交流需要从声音中提取意义，并处理自己的声音来指导这些发声的产生。这些过程需要非平凡的计算，这对语言学家和工程师来说是一个挑战，但我们的大脑可以毫不费力地完成。为了了解解码通信信号的行为意义和声音姿态的神经计算，本研究将研究鸣禽的听觉皮层如何处理其自身物种的完整声乐曲目。 泰尼森实验室获得了一个独特的数据库，其中包含了成年和幼年斑胸草雀以及雄性和雌性斑胸草雀发出的所有声音。 该数据库包含完整的曲目，其中包含许多人每种发声类型的多个样本。因为每个交流声音的行为背景都被仔细记录下来，所以这些声音被分为意义类别。因此，这个数据库将能够详细研究听觉系统如何从发声中提取意义，同时控制发声类型内以及个体之间的产生变化。该项目的方法包括使用先进的神经生理学记录技术获得对这些交流声音的神经反应，然后通过找到最能预测这些反应的统计模型来研究神经计算。多电极阵列将被用来记录在初级和次级听觉区的大量单神经元的同时神经活动。然后，这些神经元的反应将使用统计模型进行拟合，这些模型包含越来越多的抽象层次：从基本的声音特征到声音手势和语义标签。 将从一个简化的鸟类发声器官的物理模型中获得在发声姿态方面的表示。 这种分析不仅会指出参与语义处理的大脑区域，还会指出导致这些更高层次表征的分层计算的性质。 该研究还将通过直接评估高级听觉区域中基于运动的声音表征的作用来研究感知和产生之间的联系。 通过结合行为学，神经生理学和计算研究的声音通信在鸣禽，该项目将建立一个适当的动物模型系统，以阐明听觉皮层如何提取和分类的声音特征，以连接声音的意义。鉴于脊椎动物听觉系统在解剖学和生理学上的相似性以及所有语音通信中共有的共同信号处理问题，这项研究也可以为神经生理学对语音感知神经机制的理解做出重大贡献。该奖项由NSF国际科学与工程主任办公室共同资助。 一个配套项目正在由法国国家研究机构（ANR）资助。

Human speech and animal communication require both the extraction of meaning from sound and the processing of one's own voice to guide the production of these vocalizations. These processes require non-trivial computations that have challenged linguists and engineers but that are performed effortlessly by our brains. To understand what are the neural computations performed to decode the behavioral meaning and vocal gestures of communication signals, this study will examine how the auditory cortex of a songbird processes the complete vocal repertoire of its own species. The Theunissen Lab acquired a unique database of all the vocalizations emitted by adult and juvenile, and both male and female zebra finches. This database contains the complete repertoire with multiple exemplars of each vocalization type for many individuals. Because the behavioral context of each communication sound was carefully recorded, these sounds are classified in meaning categories. This database will thus enable the detailed investigation of how the auditory system extract meaning from vocalizations, while controlling for variability of production within vocalization type as well as between individuals.The approach of this project consists in obtaining neural responses to these communication sounds using advanced neurophysiological recording techniques, and then investigating the neural computations by finding the statistics models that best predict these responses. Multi-electrode arrays will be used to record the simultaneous neural activity of large sets of single neurons in the primary and secondary auditory areas. The response of these neurons will then be fitted using statistical models that incorporate increasing levels of abstraction: from elementary sound features, to vocal gestures and semantic labels. The representation in terms of vocal gestures will be obtained from a reduced physical model of the avian vocal organ. This analysis will not only point out the brain regions that are involved in semantic processing but also the nature of the hierarchical computations that lead to these higher-level representations. The research will also investigate the link between perception and production by directly assessing the role of a motor-based representation of sounds in high-level auditory areas. By combining ethological, neurophysiological and computational studies of acoustic communication in a songbird, the project will establish an appropriate animal model system to elucidate how the auditory cortex extracts and categorizes sound features in order to link sound to meaning. Given the similarities in the anatomy and physiology of the auditory system across vertebrates and the common signal processing problems shared in all vocal communications, this study can also contribute significantly to the neurophysiological understanding of neural mechanisms underlying speech perception. This award is being co-funded by NSF's Office of the Director, International Science and Engineering. A companion project is being funded by the French National Research Agency (ANR).