Building the next generation of computational psycholinguistic models of speech perception

人类是如何毫不费力地感知和理解语言的呢？我们的大脑给我们一种错觉，认为这个过程很简单，但即使是最先进的人工智能系统也经常犯下奇怪的错误，甚至当我们听到一种不熟悉的语言时，我们都很难评估自己听到的东西，这些都清楚地表明，这个过程实际上并不容易。令人难以置信的是，婴儿的感知能力在他们会说话之前就已经对他们在家里听到的语言(S)变得专门了：早在六个月前，实验也证明了他们认识并理解了数十或数百个常见单词的含义。言语知觉的认知科学极大地促进了我们对人类言语知觉如何工作的理解，在较小程度上也促进了我们对婴儿言语知觉能力发展的理解。然而，我们的理解仍然远远不够先进，不足以建立学习和感知语音的“类人”计算机系统，而我们目前的语音技术，基于令人难以置信的大量数据，在许多方面与人类的行为截然不同。我们试图利用机器学习和语音技术的最新进展来推进我们对(1)学习的理解：什么样的系统可以完成婴儿大脑的工作，并自主地学习将他们听到的语音信号解码为单独的辅音和元音(目前我们还不知道)？这些系统最终会犯与人类听者相同的误解错误吗？(2)听觉处理的早期阶段：乍一看，许多新的语音处理系统的行为似乎比前几代语音技术更像人类的听觉系统，但需要对人类听者进行进一步的实验，以评估这一点，并了解如果这是真的，对我们理解人类听觉处理的影响；以及，(3)大脑如何在我们的单词记忆中对语音进行编码。这些问题的答案将影响我们对人类如何解码语音的理解，我们如何在幼年学习这一点，以及我们如何构建不那么脆弱、能够在世界上更多语言中运行的人工智能系统。

How do human beings perceive and understand speech so effortlessly? Our brains give us the illusion that the process is simple, but the often strange errors made by even the most advanced artificial intelligence systems, and the baffling difficulty we have even taking stock of what we hear when we hear an unfamiliar language are clear clues that the process is in fact not easy at all. Incredibly, infants' perception becomes specialized in the language(s) they hear at home well before they can speak: as early as six months, a time when experiments also demonstrate that they recognize and understand the meaning of dozens or hundreds of common words. The cognitive science of speech perception has greatly advanced our understanding of how human speech perception works, and, to a lesser extent, of how the ability develops in infants. Nevertheless, our understanding is still very far from being advanced enough to build "human-like" computer systems that learn and perceive speech, and our current speech technology, tuned on implausibly large quantities of data, behave in many ways very differently from human beings. We seek to take advantage of recent advances in machine learning and speech technology to advance our understanding of (1) learning: what kind of systems can do the work of the infant brain and autonomously learn to decode the speech signals they hear into individual consonant and vowel sounds (currently we know of none)? do these systems end up making the same kinds of misperception errors as human listeners? (2) the early stages of auditory processing: many new speech processing systems appear, at first glance, to behave much more like the human auditory system than previous generations of speech technology, but further experiments with human listeners are needed to assess this, and to understand the implications for our understanding of human auditory processing if it is true; and, (3), how speech sounds are encoded by the brain in our memory for words. The answers to these questions have consequences for our understanding of how humans decode speech, how we learn to do this at an early age, and how we can build artificial intelligence systems that are less fragile, and that are capable of operating in far more of the world's languages.