Cross-language Learning-to-Rank for Patent Retrieval, Phase 2: Weakly Supervised Learning of Cross-lingual Systems

机器翻译或跨语言信息检索等跨语言技术的学习关键依赖于人工监督，其形式是域内并行文档集合或每对语言的相关性排名。在策划数据之外的任何句子或文档都被认为不是翻译或不相关（强监管）。收集和校对这些信息是一项非常繁重和昂贵的任务，即使在资源丰富的语言中，也只能在非常狭窄的领域中实现。与此同时，数据并行性从根本上控制着整体性能：首先，由于它在学习过程的早期就存在，因此它的错误和特性会沿着学习管道传播。第二，强监督建模与自然语言的灵活性和障碍跨语言应用程序的域和任务自适应冲突。正如我们在当前的DFG项目中所示，强监督对于跨语言检索应用程序并不是严格必要的。该项目第一阶段最重要的成果之一是开发了一种方法，通过直接从相关性指标（如专利引用或维基百科页面中的超链接）监督较弱但不严格平行的数据中学习跨语言排名，从而为跨语言信息检索带来非凡的改进。在该项目的第二阶段，我们打算通过将已成功用于跨语言检索的学习排名的技术应用于大量非并行数据的机器翻译的区分训练，并在此过程中，进一步改进我们的跨语言检索方法。我们提出的技术的关键成分将是从弱监督数据的学习与通过使用细粒度稀疏特征来最好地部署弱监督信号的技术相结合，并尝试从正面和负面的例子中学习。我们通过应用于医学领域的翻译和跨语言检索来激励我们的研究，其中大量的准并行训练数据可以在互联网上获得，研究出版物和专利数据。此外，最近一项医疗预防基准测试的公开数据可供评估。

Cross-lingual technologies such as machine translation or cross-lingual information retrieval crucially rely for their learning on human-sourced supervision in the form of in-domain sentence-parallel document collections or relevance rankings for each pair of languages. Any sentence or document outside of the curated data is understood as not a translation or as irrelevant (strong supervision). Collecting and proof-reading such information is a hugely onerous and expensive task and is achieved only for very narrow domains even in well-resourced languages. At the same time, data parallelism governs the overall performance in a fundamental way: First, as it is present early in the learning process, its errors and idiosyncrasies propagate down the learning pipeline. Second, modeling with strong supervision conflicts with the flexibility of natural language and handicaps domain- and task-adaptation of cross-lingual applications.As we have shown in the current DFG project, strong supervision is not strictly necessary for the application of cross-lingual retrieval. One of the most important outcomes of the first phase of the project is the development of methods that yield extraordinary improvements for cross-lingual information retrieval by learning cross-lingual rankings directly from data that are weakly supervised by relevance indicators such as citations in patents or hyperlinks in Wikipedia pages, but are not strictly parallel. In the proposed second phase of the project we intend to turn the idea on its head by applying the techniques that have been successful for learning-to-rank for cross-lingual retrieval to discriminative training of machine translation on massive non-parallel data, and in the process, further improve our methods for cross-lingual retrieval. The key ingredients of our proposed techniques will be the combination of learning from weakly supervised data with techniques that best deploy the weak supervision signals by using fine-grained sparse features and attempt at learning from positive and negative examples.We motivate our research by an application to translation and cross-lingual retrieval in the medical domain where massive amounts of quasi-parallel training data are available on the Internet, in research publications, and patent data. Furthermore, public data from a recent benchmark testing on medical translationare available for evaluation.