Building the next generation of computational psycholinguistic models of speech perception

人类是如何毫不费力地感知和理解语言的？我们的大脑给我们一种错觉，认为这个过程很简单，但即使是最先进的人工智能系统也经常会犯奇怪的错误，当我们听到一种不熟悉的语言时，我们甚至难以判断自己听到了什么，这些都清楚地表明，这个过程实际上一点也不容易。令人难以置信的是，婴儿的感知能力在他们会说话之前就已经专注于他们在家里听到的语言：早在6个月大的时候，实验也表明他们认识并理解了几十个或几百个常见单词的意思。语言感知的认知科学极大地促进了我们对人类语言感知如何工作的理解，以及在较小程度上对婴儿语言感知能力如何发展的理解。然而，我们的理解还远远没有先进到足以构建“类人”的计算机系统来学习和感知语音，而且我们目前的语音技术依赖于令人难以置信的大量数据，在许多方面的行为与人类非常不同。我们试图利用机器学习和语音技术的最新进展来推进我们对(1)学习的理解：什么样的系统可以完成婴儿大脑的工作，并自主学习将他们听到的语音信号解码成单个的辅音和元音（目前我们还不知道）？这些系统最终会像人类听众一样产生同样的误解错误吗？(2)听觉处理的早期阶段：许多新的语音处理系统乍一看，比前几代语音技术表现得更像人类听觉系统，但需要对人类听众进行进一步的实验来评估这一点，并了解如果这是真的，对我们理解人类听觉处理的影响；(3)语音是如何在我们的记忆中被大脑编码的。这些问题的答案会影响我们对人类如何解码语言的理解，我们如何在幼年时学会解码，以及我们如何构建不那么脆弱的人工智能系统，使其能够在世界上更多的语言中运行。

How do human beings perceive and understand speech so effortlessly? Our brains give us the illusion that the process is simple, but the often strange errors made by even the most advanced artificial intelligence systems, and the baffling difficulty we have even taking stock of what we hear when we hear an unfamiliar language are clear clues that the process is in fact not easy at all. Incredibly, infants' perception becomes specialized in the language(s) they hear at home well before they can speak: as early as six months, a time when experiments also demonstrate that they recognize and understand the meaning of dozens or hundreds of common words. The cognitive science of speech perception has greatly advanced our understanding of how human speech perception works, and, to a lesser extent, of how the ability develops in infants. Nevertheless, our understanding is still very far from being advanced enough to build "human-like" computer systems that learn and perceive speech, and our current speech technology, tuned on implausibly large quantities of data, behave in many ways very differently from human beings. We seek to take advantage of recent advances in machine learning and speech technology to advance our understanding of (1) learning: what kind of systems can do the work of the infant brain and autonomously learn to decode the speech signals they hear into individual consonant and vowel sounds (currently we know of none)? do these systems end up making the same kinds of misperception errors as human listeners? (2) the early stages of auditory processing: many new speech processing systems appear, at first glance, to behave much more like the human auditory system than previous generations of speech technology, but further experiments with human listeners are needed to assess this, and to understand the implications for our understanding of human auditory processing if it is true; and, (3), how speech sounds are encoded by the brain in our memory for words. The answers to these questions have consequences for our understanding of how humans decode speech, how we learn to do this at an early age, and how we can build artificial intelligence systems that are less fragile, and that are capable of operating in far more of the world's languages.