RI-Small: Exploiting Comparable Corpora for Machine Translation (CC4MT)

RI-Small：利用可比语料库进行机器翻译 (CC4MT)

• 批准号: 0916866
• 负责人: Stephan Vogel
• 金额: $ 10万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2009
• 资助国家: 美国
• 起止时间: 2009-09-15 至 2012-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0916866&HistoricalAwards=false
• 关键词: RI; Small; Exploiting; Comparable; Corpora

Parallel corpora, i.e. texts that are translations of each other, are an important resource for many natural language processing tasks, and especially for building data-driven machine translation systems. Unfortunately, for the majority of languages, parallel corpora are virtually non-existent. To be able to develop machine translation systems for those languages, we need to be able to learn from non-parallel corpora. Comparable corpora ? i.e. documents covering at least partially the same content ? are available in far larger quantities and can be easily collected on the Web. Examples include news published in many languages by Voice of America or BBC, and the multi-lingual Wikipedia.To make best use of comparable corpora it is not sufficient to extract sentence pairs, which are sufficiently parallel, thereby building a parallel corpus and then using proven training procedures. Rather, new techniques are required to find sub-sentential translation equivalences in non-parallel sentences. To extract phrase pairs from comparable corpora requires a cascaded approach:- find comparable documents using, for example, cross-lingual information retrieval techniques;- detect promising sentence pairs, i.e. those, which may contain translational equivalences; - apply robust phrase alignment techniques to detect phrase translation pairs within non-parallel sentence pairs;The main focus of the project lies on this third step: developing novel alignment algorithms, which do not rely on aligning all words within the sentences, as traditional word alignment algorithms do, but can separate parallel from non-parallel regions.The long term benefit of this work will be that machine translation technology can be applied to those languages, for which so far no translation systems are available, due to the lack of the language resources required by current technology. This will enable communication across language barriers, esp. in critical situations like medical assistance or disaster relieve.

平行语料库，即相互翻译的文本，是许多自然语言处理任务的重要资源，特别是对于构建数据驱动的机器翻译系统。 不幸的是，对于大多数语言来说，平行语料库几乎不存在。 为了能够为这些语言开发机器翻译系统，我们需要能够从非平行语料库中学习。 可比语料库？即至少部分涵盖相同内容的文件？数量要多得多，可以很容易地在网上收集。 例子包括美国之音或英国广播公司以多种语言发布的新闻，以及多语言的维基百科。为了最好地利用可比语料库，提取足够平行的句子对是不够的，从而建立一个平行语料库，然后使用经过验证的训练过程。 相反，需要新的技术来找到非平行句子中的子句子翻译对等项。 从可比语料库中提取短语对需要一种级联方法：-使用例如跨语言信息检索技术找到可比文档;-检测有希望的句子对，即可能包含翻译等价物的句子对; -应用鲁棒的短语对齐技术来检测非平行句子对中的短语翻译对;该项目的主要重点在于第三步：开发新的对齐算法，它不像传统的单词对齐算法那样依赖于对齐句子中的所有单词，而是可以将平行区域与非平行区域分开。这项工作的长期好处是机器翻译技术可以应用于这些语言，由于缺乏当前技术所需的语言资源，迄今为止还没有翻译系统。 这将使沟通跨越语言障碍，特别是在紧急情况下，如医疗援助或救灾。