RI: Small: Developing Large Scale Distributed Syntactic, Semantic and Lexical Language Models for Machine Translation and Speech Recognition

该项目旨在构建大规模分布式句法，语义和词汇语言模型，这些模型由语料库在超级计算机上进行训练，数据可达Web规模，以大幅提高机器翻译和语音识别系统的性能。 它是在有向马尔可夫随机场范式下进行的，以整合主题和语法来形成自然语言的复杂分布，并使用分层Pitman-Yor过程来建模自然语言的长尾属性。 通过利用这种特殊的结构，复杂的统计估计和推理算法被分解，并在分布式环境中执行。 语言模型被放入机器翻译系统的单程解码器中，而格重评分解码器被放入语音识别系统中。 此外，一个原则性的解决方案，一个长期存在的开放问题，平滑分数计数由于潜在变量在Kneser-Ney的sense，可能会found. This项目符合NSF的战略长期愿景的网络基础设施框架的21世纪世纪科学和工程（CIF 21）。 该项目集成了各种已知的语言模型，并提供了一种方法来克服现有的语言模型组合方法的局限性，并部署算法上有趣的方法，可扩展到Web上的数据集。 该项目为信息技术的跨学科教育提供了一个环境，该环境将语言和语音处理，机器学习以及数据密集型科学和工程领域联系起来，使多个层次的学生受益。

This project aims to build large scale distributed syntactic, semantic, and lexical language models that are trained by corpora with up to Web-scale data on a supercomputer in order to substantially improve the performance of machine translation and speech recognition systems. It is conducted under the directed Markov random field paradigm to integrate both topics and syntax to form complex distributions for natural language, and uses hierarchical Pitman-Yor processes to model long-tail properties of natural language. By exploiting this particular structure, the complex statistical estimation and inference algorithms are decomposed and performed in a distributed environment. The language models are put into one-pass decoders of machine translation systems, and the lattice rescoring decoder into a speech recognition system. In addition, a principled solution to a long-standing open problem, smoothing fractional counts due to latent variables in Kneser-Ney's sense, might be found. This project fits into the NSF's strategic long term vision of a Cyber-infrastructure Framework for 21st Century Science and Engineering (CIF21). The project integrates various kinds of known language models and provides a way to overcome the limitations of existing combination methods for language models and to deploy algorithmically interesting methodologies that are scalable to data sets available on the Web. The project provides an environment for interdisciplinary education in information technology that bridges areas of language and speech processing, machine learning, and data-intensive science and engineering to benefit students at several levels.