Encoding Spatial Relations from Natural Language
Encoding Spatial Relations from Natural Language
复制标题
从自然语言编码空间关系
DOI：
--
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
Karl Moritz Hermann
中科院分区：
文献类型：
--
作者：
Tiago Ramalho;Tomás Kociský;F. Besse;S. Eslami;Gábor Melis;Fabio Viola;Phil Blunsom;Karl Moritz Hermann
自然语言处理已经通过分布式方法在学习单词语义方面取得了重大进展，但是通过这些方法学习的表示无法捕获隐含在真实的世界中的某些类型的信息。特别是，空间关系的编码方式是不符合人类的空间推理和缺乏不变性的观点的变化。我们提出了一个系统，能够捕捉语义的空间关系，如后面，离开，等从自然语言。我们的主要贡献是一个新的多模态目标的基础上产生的图像的场景从他们的文本描述，和一个新的数据集上，以训练it.We证明，内部表示是强大的意义保持变换的描述（释义不变性），而观点不变性是一个新兴的属性系统。
Natural language processing has made significant inroads into learning the semantics of words through distributional approaches, however representations learnt via these methods fail to capture certain kinds of information implicit in the real world. In particular, spatial relations are encoded in a way that is inconsistent with human spatial reasoning and lacking invariance to viewpoint changes. We present a system capable of capturing the semantics of spatial relations such as behind, left of, etc from natural language. Our key contributions are a novel multi-modal objective based on generating images of scenes from their textual descriptions, and a new dataset on which to train it. We demonstrate that internal representations are robust to meaning preserving transformations of descriptions (paraphrase invariance), while viewpoint invariance is an emergent property of the system.