CAREER: Integrating information across levels of processing during real-time spoken language comprehension

语言理解是成功交流的核心。然而，为了理解口语，人们必须科普嘈杂的环境，说话者的声音和语言背景的变化，以及单词发音方式的差异。因此，人们依赖于上下文，例如句子中的其他单词，来解释他们所听到的内容。该项目将通过研究大脑对语音的反应和开发在上下文中识别口语单词的计算机模型来研究人们如何在不同的上下文中理解口语。其目的是了解人类听众在嘈杂的世界中成功沟通的方式。这项研究还可以让人们深入了解，通过让计算机系统以类似于人类的方式处理语言，如何使旨在识别语音的计算机系统变得更智能。这项研究将与该项目的教育部分相结合，该项目将为学生提供对STEM劳动力至关重要的先进计算和神经科学技术的培训。该项目通过使用神经科学技术在实验中研究语言理解来解决这些问题，这些技术揭示了大脑在听到声音的前几百毫秒内如何感知语音。通过研究这些早期的大脑反应，研究人员将能够确定听众使用什么信息来区分口语，并确定这些大脑反应如何受到听众对他们将听到的单词的期望的影响。研究人员还将创建神经网络模型，这些模型使用机器学习技术来识别语音，并以模仿儿童学习语言的方式进行训练。该模型的行为将与来自人类听众的数据进行比较，以确定该模型是否准确地捕捉到大脑在不同背景下理解口语的方式。从这项研究中开发的技术也将用于课堂和实验室设置，以培训本科生和研究生使用这些approaches.This奖项反映了NSF的法定使命，并已被认为是值得通过评估使用基金会的智力价值和更广泛的影响审查标准的支持。

Language comprehension is central to successful communication. However, to understand spoken language, people must cope with noisy environments, variability across speakers' voices and language backgrounds, and differences in the way words are pronounced. As a result, people rely on context, such as the other words in a sentence, to interpret the meaning of what they hear. This project will investigate how people understand spoken language in different contexts by studying brain responses to speech and developing computer models that recognize spoken words in context. The aim is to understand the ways that human listeners successfully communicate in a noisy world. The research may also provide insights into how computer systems designed to recognize speech can be made smarter by having them process language in ways similar to humans. This research will be integrated with the educational component of the project, which will provide students with training in advanced computational and neuroscience techniques that are vital to the STEM workforce. The project addresses these issues by studying language comprehension in experiments using neuroscience techniques that reveal how speech is perceived by the brain in the first few hundred milliseconds of hearing a sound. By studying these early brain responses, the investigators will be able to identify what information the listener uses to distinguish spoken words and determine how these brain responses are affected by the listeners' expectation of which words they will hear. The investigators will also create neural network models that use techniques from machine learning to recognize speech and are trained in a way that mimics how children learn language. The behavior of the model will be compared with the data from human listeners to determine whether the model accurately captures the way the brain understands spoken language in different contexts. The techniques developed from this research will also be used in classroom and laboratory settings to train undergraduate and graduate students in the use of these approaches.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.