TOUCHDOWN: Natural Language Navigation and Spatial Reasoning in Visual Street Environments
TOUCHDOWN: Natural Language Navigation and Spatial Reasoning in Visual Street Environments
复制标题
DOI：
10.1109/cvpr.2019.01282
复制
发表时间：
2018-11
期刊：
影响因子：
--
通讯作者：
Howard Chen;Alane Suhr;Dipendra Kumar Misra;Noah Snavely;Yoav Artzi
中科院分区：
文献类型：
--
作者：
Howard Chen;Alane Suhr;Dipendra Kumar Misra;Noah Snavely;Yoav Artzi
我们通过导航和空间推理任务研究了共同推理语言和愿景的问题。我们介绍了达阵任务和数据集，在该任务和数据集中，代理必须首先遵循街道视图环境中的导航说明到目标位置，然后猜测其自然语言中观察到的环境中的位置以找到隐藏的对象。数据包含9326个英语说明的示例，并配对了演示。我们进行定性语言分析，并表明数据显示了空间推理的丰富使用。经验分析表明，该数据对现有方法提出了公开挑战。
We study the problem of jointly reasoning about language and vision through a navigation and spatial reasoning task. We introduce the Touchdown task and dataset, where an agent must first follow navigation instructions in a Street View environment to a goal position, and then guess a location in its observed environment described in natural language to find a hidden object. The data contains 9326 examples of English instructions and spatial descriptions paired with demonstrations. We perform qualitative linguistic analysis, and show that the data displays a rich use of spatial reasoning. Empirical analysis shows the data presents an open challenge to existing methods.