Computational approaches to human spoken word recognition

这个项目解决了认知科学面临的一个重大挑战——人类如何理解语言。人们识别单词甚至比最好的计算机语音识别系统要容易得多，尽管我们听到的实际声音（辅音和元音）根据上下文（什么声音出现在前面或后面）、说话的人以及环境（安静的房间还是拥挤的机场）有很大的不同。目前大多数语音识别模型都不能处理真实语音中的巨大变异性，因为它们没有对实际语音信号进行操作。此外，他们不学习，所以他们不能模拟人们如何习得语言。这个项目通过比较当前的语音识别模型和人类的能力来解决这些挑战，目的是理解人类语音处理是如何如此强大和灵活的。此外，简化的“深度学习”网络将被开发和评估为人类语音识别模型。深度学习网络类似于认知模型，因为它们学习数据的抽象表示，而不是特定于任务的规则或算法。这些网络已被用于创建精确的商业语音识别系统。通过将它们与人类的表现进行比较，研究人员可能会对人类语音识别如此强大的原因提供新的见解。该项目的结果将具有技术意义（更好地了解人类的灵活性可能有助于改进计算机语音识别）和健康意义（更好地了解人类语音识别将有助于开发更好的语言障碍干预措施）。该项目还将支持培养一名博士后研究员和一名博士生，他们都将培养可用于学术界或工业界研究和发展的技能。这个项目的重点是开发一个名为“DeepListener”的“浅层深层网络”模型，该模型将与人类听众的行为进行比较。网络的毫秒级行为（例如，单词之间会暂时混淆）与人类表现的密切匹配表明，人类的语音处理可能来自与模型中相似的原理。在初步工作中，DeepListener学会了识别2000个真实单词（10个说话者产生的200个单词）中的93%。DeepListener将通过与认知理论的标准神经网络模型和人类表现的详细比较来评估。DeepListener与人类表现和竞争模型的相似和不同之处将有助于推进人类语音识别的科学理论。该项目将遵循开放科学的新兴标准：实验将预先注册，数据和计算机代码将免费公开提供。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project addresses one of the grand challenges facing cognitive science -- how humans understand speech. People recognize words far more easily than even the best computer speech recognition systems, even though the actual sounds we hear as consonants and vowels vary greatly depending on context (what sounds come before or after), who is talking, and the setting (a quiet room versus a crowded airport). Most current models of speech recognition cannot handle the huge variability in real speech because they do not operate on the actual speech signal. Also, they do not learn, so they cannot model how people acquire language. This project addresses these challenges by comparing current models of speech recognition to each other and to human capabilities, with the goal of understanding how human speech processing is so robust and flexible. In addition, simplified "deep learning" networks will be developed and evaluated as models of human speech recognition. Deep learning networks are similar to cognitive models in that they learn abstract representations of the data, not task-specific rules or algorithms. These networks have been used to create accurate commercial speech recognition systems. By comparing them to human performance, the investigators may provide new insights into why human speech recognition is so robust. The results of this project will have technical implications (better understanding of human flexibility may aid in improving computer speech recognition) and health implications (better understanding of human speech recognition will aid in developing better interventions for language disorders). The project will also support the training of a postdoctoral researcher and a PhD student, both of whom will develop skills that can be used to contribute to research and development in academia or industry. This project focuses on the development of a "shallow deep network" model called "DeepListener" that will be compared with the behavior of human listeners. A close match in the millisecond-level behavior of the network (for example, in which words are temporarily confusable with each other) and human performance suggests that human speech processing may emerge from similar principles as those in the model. In preliminary work, DeepListener learned to recognize 93% of 2000 real words (200 words produced by 10 talkers). DeepListener will be evaluated by detailed comparison to standard neural network models of cognitive theories and to human performance. The ways in which DeepListener is similar and dissimilar to human performance and competing models will help to advance scientific theories of human speech recognition. This project will follow emerging standards for open science: experiments will be pre-registered and data and computer code will be made freely and publicly available.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.