Talk to the Vehicle: Language Conditioned Autonomous Navigation of Self Driving Cars
Talk to the Vehicle: Language Conditioned Autonomous Navigation of Self Driving Cars
复制标题
与车辆对话：自动驾驶汽车的语言条件自主导航
DOI：
10.1109/iros40897.2019.8967929
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
K. Krishna
中科院分区：
文献类型：
--
作者：
Sriram Narayanan;T. Maniar;Jayaganesh Kalyanasundaram;Vineet Gandhi;Brojeshwar Bhowmick;K. Krishna
我们提出了一种新的管道，混合编码从自然语言和3D语义地图从视觉图像生成本地轨迹，由一个低级别的控制器执行。该管道排除了通过本地航路点生成器神经网络对先前注册地图的需要。航路点生成器网络（WGN）将语义和自然语言编码（NLE）映射到本地航路点。然后，本地规划器生成从车辆（在这种情况下是室外汽车）的自我位置到这些本地生成的路点的轨迹，而低级控制器忠实地执行这些计划。在CARLA模拟器环境中以及从真实世界的KITTI数据集构建的本地语义地图上验证了管道的有效性。在这两个环境（模拟和现实世界）中，我们显示的WGN生成航路点的能力，准确地映射不同的序列长度和复杂程度的NLE。我们比较基线的方法，并显示显着的性能增益。最后，我们展示了我们的电动汽车上的真实的实现，验证了管道在不受控制的户外环境中的实际和有形的实现。循环执行所提出的管道，涉及重复调用的网络是至关重要的任何这样的基于语言的导航框架。这一努力成功地实现了这一点，从而绕过了在遍历期间对先前度量图或度量级定位策略的需要。
We propose a novel pipeline that blends encodings from natural language and 3D semantic maps obtained from visual imagery to generate local trajectories that are executed by a low-level controller. The pipeline precludes the need for a prior registered map through a local waypoint generator neural network. The waypoint generator network (WGN) maps semantics and natural language encodings (NLE) to local waypoints. A local planner then generates a trajectory from the ego location of the vehicle (an outdoor car in this case) to these locally generated waypoints while a low-level controller executes these plans faithfully. The efficacy of the pipeline is verified in the CARLA simulator environment as well as on local semantic maps built from real-world KITTI dataset. In both these environments (simulated and real-world) we show the ability of the WGN to generate waypoints accurately by mapping NLE of varying sequence lengths and levels of complexity. We compare with baseline approaches and show significant performance gain over them. And finally, we show real implementations on our electric car verifying that the pipeline lends itself to practical and tangible realizations in uncontrolled outdoor settings. In loop execution of the proposed pipeline that involves repetitive invocations of the network is critical for any such language-based navigation framework. This effort successfully accomplishes this thereby bypassing the need for prior metric maps or strategies for metric level localization during traversal.
DOI：
10.1109/cvpr.2019.01282
发表时间：
2018-11
期刊：
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
影响因子：
--
作者：
Howard Chen;Alane Suhr;Dipendra Kumar Misra;Noah Snavely;Yoav Artzi
通讯作者：
Howard Chen;Alane Suhr;Dipendra Kumar Misra;Noah Snavely;Yoav Artzi