Cross-language Learning-to-Rank for Patent Retrieval, Phase 2: Weakly Supervised Learning of Cross-lingual Systems

跨语言技术，如机器翻译或跨语言信息检索，主要依赖于以领域内句子平行文档集合或每对语言的相关性排名的形式进行的人力监督。管理数据之外的任何句子或文档都被理解为不是翻译或不相关（强监督）。收集和校对这些信息是一项非常繁重和昂贵的任务，即使在资源丰富的语言中，也只能在非常狭窄的领域中实现。与此同时，数据并行性以一种基本的方式控制着整体性能：首先，由于它出现在学习过程的早期，它的错误和特性会沿着学习管道传播。其次，具有强监督的建模与自然语言的灵活性相冲突，阻碍了跨语言应用的领域和任务适应。正如我们在当前的DFG项目中所显示的那样，跨语言检索的应用并不严格需要强有力的监督。该项目的第一阶段最重要的成果之一是开发方法，通过直接从相关性指标（如专利引文或维基百科页面中的超链接）弱监督的数据中学习跨语言排名，从而显著提高跨语言信息检索的效率，但这些数据并非严格平行。在项目的第二阶段中，我们打算将已经成功的跨语言检索的学习排序技术应用于大量非并行数据上的机器翻译判别训练，并在此过程中进一步改进我们的跨语言检索方法。我们提出的技术的关键成分是将弱监督数据的学习与通过使用细粒度稀疏特征最好地部署弱监督信号的技术相结合，并尝试从正例和负例中学习。我们通过应用于医学领域的翻译和跨语言检索来激励我们的研究，在这个领域中，大量的准并行训练数据可以在互联网、研究出版物和专利数据中获得。此外，最近医学翻译基准测试的公开数据可用于评估。

Cross-lingual technologies such as machine translation or cross-lingual information retrieval crucially rely for their learning on human-sourced supervision in the form of in-domain sentence-parallel document collections or relevance rankings for each pair of languages. Any sentence or document outside of the curated data is understood as not a translation or as irrelevant (strong supervision). Collecting and proof-reading such information is a hugely onerous and expensive task and is achieved only for very narrow domains even in well-resourced languages. At the same time, data parallelism governs the overall performance in a fundamental way: First, as it is present early in the learning process, its errors and idiosyncrasies propagate down the learning pipeline. Second, modeling with strong supervision conflicts with the flexibility of natural language and handicaps domain- and task-adaptation of cross-lingual applications.As we have shown in the current DFG project, strong supervision is not strictly necessary for the application of cross-lingual retrieval. One of the most important outcomes of the first phase of the project is the development of methods that yield extraordinary improvements for cross-lingual information retrieval by learning cross-lingual rankings directly from data that are weakly supervised by relevance indicators such as citations in patents or hyperlinks in Wikipedia pages, but are not strictly parallel. In the proposed second phase of the project we intend to turn the idea on its head by applying the techniques that have been successful for learning-to-rank for cross-lingual retrieval to discriminative training of machine translation on massive non-parallel data, and in the process, further improve our methods for cross-lingual retrieval. The key ingredients of our proposed techniques will be the combination of learning from weakly supervised data with techniques that best deploy the weak supervision signals by using fine-grained sparse features and attempt at learning from positive and negative examples.We motivate our research by an application to translation and cross-lingual retrieval in the medical domain where massive amounts of quasi-parallel training data are available on the Internet, in research publications, and patent data. Furthermore, public data from a recent benchmark testing on medical translationare available for evaluation.