SGER: Scaling up unsupervised grammar induction

SGER：扩大无监督语法归纳

• 批准号: 0836431
• 负责人: Noah Smith
• 金额: --
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2008
• 资助国家: 美国
• 起止时间: 2008-07-01 至 2009-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0836431&HistoricalAwards=false
• 关键词: SGER; Scaling; up; unsupervised; grammar

This SGER project seeks to determine the scalability of computationally intensive, iterative statistical learning algorithms on a MapReduce architecture. Such algorithms underlie much research in natural language processing, yet their scalability to even moderately large training datasets (text corpora) has been under-explored. On the surface, scaling to more data appears to be a good fit for the MapReduce paradigm, and this exploratory project aims to identify whether such algorithms benefit from more data and more complex data than used in prior work. A special emphasis is given to unsupervised learning algorithms, such as the Expectation-Maximization algorithm, which have been widely studied on small problems and rarely studied on large ones. The technique is applicable to many other methods, as well.At the same time, the project seeks to explore how to leverage supercomputers and MapReduce to make these learning algorithms faster, permitting a faster research cycle. Concretely, the "E step" (or itsanalogue) is the most computationally demanding part of an iteration, but the standard assumption that the training data are independently and identically distributed permits parallelization. To the extent that this parallelization is affected by network and input-output overhead, each iteration of training may be made faster, perhaps reducing training time from days or weeks to hours. This project explores this tradeoff and others like it.This work leverages a resource donated by Yahoo for use by the PI's research group: a 4,000-node supercomputer running Hadoop (an open-source implementation of MapReduce).

这个SGER项目旨在确定MapReduce架构上计算密集型迭代统计学习算法的可扩展性。 这些算法是自然语言处理中许多研究的基础，但它们对中等规模的训练数据集（文本语料库）的可扩展性尚未得到充分探索。 从表面上看，扩展到更多的数据似乎很适合MapReduce范式，这个探索性的项目旨在确定这些算法是否受益于比以前工作中使用的更多数据和更复杂的数据。 特别强调的是无监督学习算法，如期望最大化算法，这已经被广泛研究的小问题，很少研究大的。 该技术也适用于许多其他方法。同时，该项目旨在探索如何利用超级计算机和MapReduce来使这些学习算法更快，从而加快研究周期。 具体地说，“E步骤”（或其sanitary）是迭代中计算要求最高的部分，但训练数据独立且同分布的标准假设允许并行化。 在这种并行化受到网络和输入输出开销影响的程度上，可以使训练的每次迭代更快，可能将训练时间从几天或几周减少到几小时。 这个项目探索了这种权衡和其他类似的权衡。这项工作利用了雅虎捐赠的资源，供PI的研究小组使用：一台运行Hadoop（MapReduce的开源实现）的4,000个节点的超级计算机。