RI-Small: Exploiting Comparable Corpora for Machine Translation (CC4MT)

平行语料库，即相互翻译的文本，是许多自然语言处理任务的重要资源，特别是用于构建数据驱动的机器翻译系统。不幸的是，对于大多数语言来说，平行语料库实际上是不存在的。为了能够为这些语言开发机器翻译系统，我们需要能够从非平行语料库中学习。可比语料库？即文件至少部分内容相同？的数量要大得多，而且很容易在网上收集到。例如，美国之音或英国广播公司用多种语言发布的新闻，以及多语种的维基百科。为了充分利用可比较的语料库，仅仅提取足够平行的句子对是不够的，因此建立一个平行语料库，然后使用经过验证的训练程序。相反，在非平行句中寻找子句翻译对等需要新的技术。从可比语料库中提取短语对需要一种级联方法：-使用例如跨语言信息检索技术查找可比文档；-检测有希望的句子对，即那些可能包含翻译等价的句子；-应用强大的短语对齐技术来检测非平行句子对中的短语翻译对；该项目的主要重点在于第三步：开发新的对齐算法，它不像传统的单词对齐算法那样依赖于对齐句子中的所有单词，而是可以将并行区域与非并行区域分开。这项工作的长期好处将是机器翻译技术可以应用于这些语言，由于缺乏当前技术所需的语言资源，迄今为止还没有翻译系统可用。这将使沟通跨越语言障碍，特别是在紧急情况下，如医疗援助或救灾。

Parallel corpora, i.e. texts that are translations of each other, are an important resource for many natural language processing tasks, and especially for building data-driven machine translation systems. Unfortunately, for the majority of languages, parallel corpora are virtually non-existent. To be able to develop machine translation systems for those languages, we need to be able to learn from non-parallel corpora. Comparable corpora ? i.e. documents covering at least partially the same content ? are available in far larger quantities and can be easily collected on the Web. Examples include news published in many languages by Voice of America or BBC, and the multi-lingual Wikipedia.To make best use of comparable corpora it is not sufficient to extract sentence pairs, which are sufficiently parallel, thereby building a parallel corpus and then using proven training procedures. Rather, new techniques are required to find sub-sentential translation equivalences in non-parallel sentences. To extract phrase pairs from comparable corpora requires a cascaded approach:- find comparable documents using, for example, cross-lingual information retrieval techniques;- detect promising sentence pairs, i.e. those, which may contain translational equivalences; - apply robust phrase alignment techniques to detect phrase translation pairs within non-parallel sentence pairs;The main focus of the project lies on this third step: developing novel alignment algorithms, which do not rely on aligning all words within the sentences, as traditional word alignment algorithms do, but can separate parallel from non-parallel regions.The long term benefit of this work will be that machine translation technology can be applied to those languages, for which so far no translation systems are available, due to the lack of the language resources required by current technology. This will enable communication across language barriers, esp. in critical situations like medical assistance or disaster relieve.