INCA: An Integrated Cluster Computing Architecture for Machine Translation

INCA：用于机器翻译的集成集群计算架构

• 批准号: 0844507
• 负责人: Stephan Vogel
• 金额: $ 44.91万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2009
• 资助国家: 美国
• 起止时间: 2009-02-15 至 2012-01-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0844507&HistoricalAwards=false
• 关键词: INCA; Integrated; Cluster; Computing; Architecture

Progress in the field of machine translation (MT) has come to depend heavily on open-source toolkits, which make it easier for new research groups to tackle the problem at lower cost, broadening participation. Unfortunately, toolkits have not kept up with modern computing infrastructure (e.g., the MapReduce framework) required for modern "big data" approaches to MT, the "primitives" in most toolkits are hardly extensible to new models since they focus on pipeline components rather than algorithmic concepts, and experiment management has been all but ignored.This project is developing the Integrated Cluster Computing Architecture (INCA) for translation to overcome these challenges, by implementing an extensible, open-source toolkit that can leverage MapReduce clusters and flexibly implement many types of MT systems. MT is not a perfect fit for MapReduce (it has massive memory footprints and requires iterative algorithms); new algorithms are being developed to take advantage of the framework without being limited by it. Experiment management, evaluation, and advice about "best practices" are also part of the toolkit, to make it as widely accessible as possible.This project is expected to have broad impact in MT research through the open-source toolkit to be made available to the research community. A course project suitable for undergraduates will be developed and shared openly using the toolkit. Technical solutions to problems in large-scale, parallelized MT will be applicable in areas of data-intensive natural language processing and machine learning, and elements of the toolkit are expected to be useful in such research efforts as well.

机器翻译（MT）领域的进步很大程度上依赖于开源工具包，这使得新的研究小组更容易以更低的成本解决问题，扩大参与。不幸的是，工具包并没有跟上现代MT“大数据”方法所需的现代计算基础设施（例如，MapReduce框架），大多数工具包中的“原语”几乎无法扩展到新模型，因为它们专注于管道组件而不是算法概念，并且实验管理几乎被忽略了。该项目正在开发用于翻译的集成集群计算架构（INCA），通过实现一个可扩展的开源工具包来克服这些挑战，该工具包可以利用MapReduce集群并灵活地实现多种类型的机器翻译系统。机器翻译并不完全适合MapReduce（它占用大量内存，需要迭代算法）；新的算法正在开发中，以利用该框架而不受其限制。实验管理、评估和关于“最佳实践”的建议也是工具包的一部分，以使其尽可能广泛地可用。该项目预计将通过向研究社区提供的开源工具包对机器翻译研究产生广泛的影响。将开发一个适合本科生的课程项目，并使用该工具包公开分享。大规模并行机器翻译问题的技术解决方案将适用于数据密集型自然语言处理和机器学习领域，并且该工具包的元素也有望在此类研究工作中发挥作用。