Spreading the Word: The Theory of Distributed Representations in Speech and Natural Language Processing

点击翻译按钮获取中文摘要

Motivated by recent progress in the use of neural networks in areas such as speech recognition and machine translation, this grant will fund research into the application of neural networks to a wide variety of other tasks in language technology. Although much of the popular excitement over neural networks arises from their alleged correspondence to human cognitive mechanisms, what excites engineers and computer scientists about them is their utility as very compact sequence classifiers - tools that can accurately label long sequences of data, such as speech sounds or words of text, given only examples to train on. Undertaken by one of the research labs that pioneered the use of neural networks in the area of speech recognition, the projects funded by this grant will consider not merely the transcription of speech into text, but the understanding of what was said or written. This emphasis on meaning will enable a new generation of language technology applications that can, for example, summarize large newspaper articles into their most relevant 10 or 20 percent. A key challenge to doing this is determining how individual representations of words combine to form representations of phrases and whole sentences. There is a branch of applied mathematics that studies this sort of combination, called mathematical linguistics, which will play a crucial role in this research. We will study the use of neural networks in mimicking those combinations.