RI: Small: Developing Large Scale Distributed Syntactic, Semantic and Lexical Language Models for Machine Translation and Speech Recognition

该项目旨在建立大规模的分布式语法、语义和词法语言模型，这些模型由超级计算机上具有web级数据的语料库训练，以大幅提高机器翻译和语音识别系统的性能。它在有向马尔可夫随机场范式下进行，将主题和语法整合在一起，形成自然语言的复杂分布，并使用分层Pitman-Yor过程来模拟自然语言的长尾特性。通过利用这种特殊的结构，复杂的统计估计和推理算法被分解并在分布式环境中执行。将语言模型应用到机器翻译系统的单通解码器中，将点阵评分解码器应用到语音识别系统中。此外，可能会发现一个长期存在的开放性问题的原则性解决方案，即平滑由于Kneser-Ney意义上的潜在变量而导致的分数计数。该项目符合美国国家科学基金会关于21世纪科学与工程网络基础设施框架（CIF21）的长期战略愿景。该项目集成了各种已知的语言模型，并提供了一种方法来克服语言模型的现有组合方法的局限性，并部署算法上有趣的方法，这些方法可扩展到Web上可用的数据集。该项目为信息技术领域的跨学科教育提供了一个环境，将语言和语音处理、机器学习、数据密集型科学和工程等领域连接起来，使学生在多个层面上受益。

This project aims to build large scale distributed syntactic, semantic, and lexical language models that are trained by corpora with up to Web-scale data on a supercomputer in order to substantially improve the performance of machine translation and speech recognition systems. It is conducted under the directed Markov random field paradigm to integrate both topics and syntax to form complex distributions for natural language, and uses hierarchical Pitman-Yor processes to model long-tail properties of natural language. By exploiting this particular structure, the complex statistical estimation and inference algorithms are decomposed and performed in a distributed environment. The language models are put into one-pass decoders of machine translation systems, and the lattice rescoring decoder into a speech recognition system. In addition, a principled solution to a long-standing open problem, smoothing fractional counts due to latent variables in Kneser-Ney's sense, might be found. This project fits into the NSF's strategic long term vision of a Cyber-infrastructure Framework for 21st Century Science and Engineering (CIF21). The project integrates various kinds of known language models and provides a way to overcome the limitations of existing combination methods for language models and to deploy algorithmically interesting methodologies that are scalable to data sets available on the Web. The project provides an environment for interdisciplinary education in information technology that bridges areas of language and speech processing, machine learning, and data-intensive science and engineering to benefit students at several levels.