Neural Variational Learning for Grounded Language Acquisition
Neural Variational Learning for Grounded Language Acquisition
复制标题
DOI：
10.1109/ro-man50785.2021.9515374
复制
发表时间：
2021-07
期刊：
影响因子：
--
通讯作者：
Nisha Pillai;Cynthia Matuszek;Francis Ferraro
中科院分区：
文献类型：
--
作者：
Nisha Pillai;Cynthia Matuszek;Francis Ferraro
我们提出了一个学习系统，其中语言是基于视觉感知没有特定的预定义类别的条款。我们提出了一个统一的生成方法来获取共享的语义/视觉嵌入，使语言学习的范围广泛的现实世界的对象。我们通过预测对象的语义并比较神经和非神经输入的性能来评估这种学习的有效性。我们发现，这种生成方法在低资源设置下，在没有预先指定视觉类别的情况下，在语言基础方面表现出很好的效果。我们的实验表明，这种方法是通用的多语言，高度多样化的数据集。
We propose a learning system in which language is grounded in visual percepts without specific pre-defined categories of terms. We present a unified generative method to acquire a shared semantic/visual embedding that enables the learning of language about a wide range of real-world objects. We evaluate the efficacy of this learning by predicting the semantics of objects and comparing the performance with neural and non-neural inputs. We show that this generative approach exhibits promising results in language grounding without pre-specifying visual categories under low resource settings. Our experiments demonstrate that this approach is generalizable to multilingual, highly varied datasets.