Grounding Language Attributes to Objects using Bayesian Eigenobjects
Grounding Language Attributes to Objects using Bayesian Eigenobjects
复制标题
使用贝叶斯特征对象将语言属性基础化为对象
DOI：
10.1109/iros40897.2019.8968603
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
G. Konidaris
中科院分区：
文献类型：
--
作者：
Vanya Cohen;B. Burchfiel;Thao Nguyen;N. Gopalan;Stefanie Tellex;G. Konidaris
我们开发了一个基于简单的物理描述的系统，以在同一类中消除歧义对象实例。该系统将自然语言短语和包含分段对象的深度图像作为输入，并预测观察到的对象与短语描述的对象的相似性。我们的系统旨在仅从少量的人类标记的语言数据中学习，并将其推广到语言通知的深度图像训练集中未表示的观点。通过将3D形状表示与语言表示解耦，该方法能够使用少量语言的深度数据和较大的未标记的3D对象网格的较大的语料语言将基础语言转化为新颖对象，即使从异常观点中部分观察到这些对象。我们的系统能够根据自然语言描述通过深度图像观察到新颖对象之间的歧义。我们的方法还可以使观点转移；尽管在训练集中没有这样的深度图像，但我们的系统对从额叶视角捕获的一小部分深度图像进行了培训，从后视图中成功预测了对象属性。最后，我们在百特机器人上演示了我们的方法，使其能够基于人类提供的自然语言描述来选择特定的对象。
We develop a system to disambiguate object instances within the same class based on simple physical descriptions. The system takes as input a natural language phrase and a depth image containing a segmented object and predicts how similar the observed object is to the object described by the phrase. Our system is designed to learn from only a small amount of human-labeled language data and generalize to viewpoints not represented in the language-annotated depth image training set. By decoupling 3D shape representation from language representation, this method is able to ground language to novel objects using a small amount of language-annotated depth-data and a larger corpus of unlabeled 3D object meshes, even when these objects are partially observed from unusual viewpoints. Our system is able to disambiguate between novel objects, observed via depth images, based on natural language descriptions. Our method also enables viewpoint transfer; trained on human-annotated data on a small set of depth images captured from frontal viewpoints, our system successfully predicted object attributes from rear views despite having no such depth images in its training set. Finally, we demonstrate our approach on a Baxter robot, enabling it to pick specific objects based on human-provided natural language descriptions.