Spreading the Word: The Theory of Distributed Representations in Speech and Natural Language Processing

由于最近神经网络在语音识别和机器翻译等领域的应用取得了进展，该基金将资助神经网络在语言技术中广泛应用的研究。****尽管人们对神经网络的兴奋之处大多来自于它们与人类认知机制的对应，但让工程师和计算机科学家兴奋的是它们作为非常紧凑的序列分类器的实用性——这些工具可以准确地标记长序列的数据，比如语音或文本单词，只需要给出训练的例子。****由一个在语音识别领域率先使用神经网络的研究实验室承担，这项资助的项目不仅考虑将语音转录成文本，而且考虑对所说或所写内容的理解。这种对意义的强调将使新一代的语言技术应用成为可能，例如，它可以将大型报纸文章总结为最相关的10%或20%。这样做的一个关键挑战是确定单个单词的表示如何组合成短语和整个句子的表示。应用数学的一个分支研究这种组合，称为数学语言学，它将在这项研究中发挥至关重要的作用。我们将学习神经网络在模拟这些组合中的应用。

Motivated by recent progress in the use of neural networks in areas such as speech recognition and machine translation, this grant will fund research into the application of neural networks to a wide variety of other tasks in language technology.****Although much of the popular excitement over neural networks arises from their alleged correspondence to human cognitive mechanisms, what excites engineers and computer scientists about them is their utility as very compact sequence classifiers - tools that can accurately label long sequences of data, such as speech sounds or words of text, given only examples to train on.****Undertaken by one of the research labs that pioneered the use of neural networks in the area of speech recognition, the projects funded by this grant will consider not merely the transcription of speech into text, but the understanding of what was said or written. This emphasis on meaning will enable a new generation of language technology applications that can, for example, summarize large newspaper articles into their most relevant 10 or 20 percent. A key challenge to doing this is determining how individual representations of words combine to form representations of phrases and whole sentences. There is a branch of applied mathematics that studies this sort of combination, called mathematical linguistics, which will play a crucial role in this research. We will study the use of neural networks in mimicking those combinations.