CAREER: Integrating information across levels of processing during real-time spoken language comprehension

语言理解是成功沟通的关键。然而，为了理解口语，人们必须应对嘈杂的环境，说话者的声音和语言背景的变化，以及单词发音方式的差异。因此，人们依赖上下文，比如句子中的其他单词，来解释他们听到的意思。该项目将通过研究大脑对语音的反应和开发识别上下文语音的计算机模型来研究人们如何在不同的环境中理解口语。其目的是了解人类听众在嘈杂的世界中成功沟通的方式。这项研究还可能提供一些见解，让用于识别语音的计算机系统以类似于人类的方式处理语言，从而使它们变得更智能。这项研究将与该项目的教育部分相结合，为学生提供对STEM劳动力至关重要的先进计算和神经科学技术的培训。该项目通过使用神经科学技术在实验中研究语言理解来解决这些问题，这些技术揭示了大脑在听到声音的前几百毫秒是如何感知语言的。通过研究这些早期的大脑反应，研究人员将能够确定听者使用哪些信息来区分口语单词，并确定听者对将要听到的单词的期望如何影响这些大脑反应。研究人员还将创建神经网络模型，利用机器学习技术来识别语音，并以模仿儿童学习语言的方式进行训练。该模型的行为将与人类听众的数据进行比较，以确定该模型是否准确地捕捉到了大脑在不同环境下理解口语的方式。本研究开发的技术也将用于课堂和实验室环境，以训练本科生和研究生使用这些方法。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Language comprehension is central to successful communication. However, to understand spoken language, people must cope with noisy environments, variability across speakers' voices and language backgrounds, and differences in the way words are pronounced. As a result, people rely on context, such as the other words in a sentence, to interpret the meaning of what they hear. This project will investigate how people understand spoken language in different contexts by studying brain responses to speech and developing computer models that recognize spoken words in context. The aim is to understand the ways that human listeners successfully communicate in a noisy world. The research may also provide insights into how computer systems designed to recognize speech can be made smarter by having them process language in ways similar to humans. This research will be integrated with the educational component of the project, which will provide students with training in advanced computational and neuroscience techniques that are vital to the STEM workforce. The project addresses these issues by studying language comprehension in experiments using neuroscience techniques that reveal how speech is perceived by the brain in the first few hundred milliseconds of hearing a sound. By studying these early brain responses, the investigators will be able to identify what information the listener uses to distinguish spoken words and determine how these brain responses are affected by the listeners' expectation of which words they will hear. The investigators will also create neural network models that use techniques from machine learning to recognize speech and are trained in a way that mimics how children learn language. The behavior of the model will be compared with the data from human listeners to determine whether the model accurately captures the way the brain understands spoken language in different contexts. The techniques developed from this research will also be used in classroom and laboratory settings to train undergraduate and graduate students in the use of these approaches.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.