Grouping Words with Semantic Diversity
Grouping Words with Semantic Diversity
复制标题
DOI：
10.18653/v1/2021.naacl-main.257
复制
发表时间：
2021-06
期刊：
影响因子：
--
通讯作者：
Karine Chubarian;A. Khan;Anastasios Sidiropoulos;Jia Xu
中科院分区：
文献类型：
--
作者：
Karine Chubarian;A. Khan;Anastasios Sidiropoulos;Jia Xu
基于深度学习的NLP系统可能对看不见的标记敏感，并且难以使用高维输入进行学习，这严重阻碍了学习泛化。我们介绍了一种方法，通过分组输入单词的语义多样性的基础上，以简化输入语言表示低歧义。由于语义上不同的词存在于不同的语境中，我们可以用它们的组来替换词，并且仍然可以根据它们的语境来区分词义。我们设计了几个算法，计算不同的分组的基础上随机抽样，几何距离，熵最大化，我们证明了正式的保证，基于熵的算法。实验结果表明，我们的方法推广了NLP模型，并在POS标记和LM任务上表现出更高的准确性，在中等规模的机器翻译任务上也有显着改进，最高可达+6.5 BLEU点。我们的源代码可以在https://github.com/abdulrafae/dg上找到。
Deep Learning-based NLP systems can be sensitive to unseen tokens and hard to learn with high-dimensional inputs, which critically hinder learning generalization. We introduce an approach by grouping input words based on their semantic diversity to simplify input language representation with low ambiguity. Since the semantically diverse words reside in different contexts, we are able to substitute words with their groups and still distinguish word meanings relying on their contexts. We design several algorithms that compute diverse groupings based on random sampling, geometric distances, and entropy maximization, and we prove formal guarantees for the entropy-based algorithms. Experimental results show that our methods generalize NLP models and demonstrate enhanced accuracy on POS tagging and LM tasks and significant improvements on medium-scale machine translation tasks, up to +6.5 BLEU points. Our source code is available at https://github.com/abdulrafae/dg.