Compositionality and Generalization In Emergent Languages
Compositionality and Generalization In Emergent Languages
复制标题
新兴语言中的组合性和泛化
DOI：
--
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Marco Baroni
中科院分区：
文献类型：
--
作者：
Rahma Chaabouni;E. Kharitonov;Diane Bouchacourt;Emmanuel Dupoux;Marco Baroni
自然语言允许我们通过根据系统规则组合表示其部分的表达式来引用新的组合概念，这种特性称为组合性。在本文中，我们研究了深度多智能体模拟中出现的语言是否具有类似的参考新原始组合的能力，以及它是否通过类似于人类语言组合性的策略来完成这一壮举。在表征学习的解纠缠启发下，我们采用了新的方法来测量新兴语言的组合性，我们建立了三个主要结果：首先，给定足够大的输入空间，新兴语言将自然地发展出参考新组合概念的能力。第二，新兴语言的组合程度与其概括能力之间没有相关性。第三，虽然组合性不是泛化所必需的，但它在语言传播方面提供了优势：一种语言的组合性越强，新学习者就越容易掌握它，即使后者在架构上与原始智能体不同。我们的结论是，组合性并不是由简单的泛化压力产生的，但如果一种新兴语言偶然出现，它将更有可能生存和发展。
Natural language allows us to refer to novel composite concepts by combining expressions denoting their parts according to systematic rules, a property known as compositionality. In this paper, we study whether the language emerging in deep multi-agent simulations possesses a similar ability to refer to novel primitive combinations, and whether it accomplishes this feat by strategies akin to human-language compositionality. Equipped with new ways to measure compositionality in emergent languages inspired by disentanglement in representation learning, we establish three main results: First, given sufficiently large input spaces, the emergent language will naturally develop the ability to refer to novel composite concepts. Second, there is no correlation between the degree of compositionality of an emergent language and its ability to generalize. Third, while compositionality is not necessary for generalization, it provides an advantage in terms of language transmission: The more compositional a language is, the more easily it will be picked up by new learners, even when the latter differ in architecture from the original agents. We conclude that compositionality does not arise from simple generalization pressure, but if an emergent language does chance upon it, it will be more likely to survive and thrive.