Inducing Crosslingual Distributed Representations of Words
Inducing Crosslingual Distributed Representations of Words
复制标题
DOI：
--
复制
发表时间：
2012-12
期刊：
影响因子：
--
通讯作者：
A. Klementiev;Ivan Titov;Binod Bhattarai
中科院分区：
文献类型：
--
作者：
A. Klementiev;Ivan Titov;Binod Bhattarai
单词的分布式表示已被证明在许多自然语言处理任务中非常有用。它们的吸引力在于，它们可以帮助缓解监督学习中常见的数据稀疏问题。用于归纳这些表示的方法仅需要未标记的语言数据，这对于许多自然语言来说是丰富的。在这项工作中，我们共同诱导一对语言的分布式表示。我们把它作为一个多任务的学习问题，每个任务对应于一个单词，任务相关性来自双语并行数据的共现统计。这些表示可以用于许多跨语言学习任务，其中学习者可以在一种语言中存在的注释上进行训练，并应用于另一种语言中的测试数据。我们表明，我们的表示是信息丰富的，通过使用它们进行跨语言文档分类，在这些表示上训练的分类器在应用于新语言时大大优于强基线（例如机器翻译）。
Distributed representations of words have proven extremely useful in numerous natural language processing tasks. Their appeal is that they can help alleviate data sparsity problems common to supervised learning. Methods for inducing these representations require only unlabeled language data, which are plentiful for many natural languages. In this work, we induce distributed representations for a pair of languages jointly. We treat it as a multitask learning problem where each task corresponds to a single word, and task relatedness is derived from co-occurrence statistics in bilingual parallel data. These representations can be used for a number of crosslingual learning tasks, where a learner can be trained on annotations present in one language and applied to test data in another. We show that our representations are informative by using them for crosslingual document classification, where classifiers trained on these representations substantially outperform strong baselines (e.g. machine translation) when applied to a new language.