Vision-Only Robot Navigation in a Neural Radiance World
Vision-Only Robot Navigation in a Neural Radiance World
复制标题
DOI：
10.1109/lra.2022.3150497
复制
发表时间：
2022-04-01
影响因子：
5.2
通讯作者：
Schwager, Mac
中科院分区：
文献类型：
--
作者：
Adamkiewicz, Michal;Chen, Timothy;Schwager, Mac
神经辐射场 (NeRF) 最近已成为表示自然、复杂 3D 场景的强大范例。神经辐射场 (NeRF) 表示神经网络中的连续体积密度和 RGB 值，并通过光线追踪从看不见的相机视点生成逼真的图像。我们提出了一种算法，用于在 NeRF 表示的 3D 环境中导航机器人，仅使用机载 RGB 相机进行定位。我们假设场景的 NeRF 已经离线预训练，机器人的目标是导航穿过 NeRF 中的未占用空间以达到目标姿势。我们引入了一种轨迹优化算法，该算法基于差分平坦度的离散时间版本，可以避免与 NeRF 中的高密度区域发生碰撞，该算法适合约束机器人的完整姿态和控制输入。我们还引入了一种基于优化的滤波方法，在仅给定机载 RGB 相机的情况下估计 NeRF 中机器人的 6DoF 位姿和速度。我们将轨迹规划器与位姿过滤器结合在一个在线重新规划循环中，以提供基于视觉的机器人导航管道。我们展示了四旋翼机器人仅使用 RGB 相机在丛林健身房环境、教堂内部和巨石阵中导航的模拟结果。我们还演示了一个全向地面机器人在教堂中导航，要求它重新定向以适应狭窄的间隙。
Neural Radiance Fields (NeRFs) have recently emerged as a powerful paradigm for the representation of natural, complex 3D scenes. Neural Radiance Fields (NeRFs) represent continuous volumetric density and RGB values in a neural network, and generate photo-realistic images from unseen camera viewpoints through ray tracing. We propose an algorithm for navigating a robot through a 3D environment represented as a NeRF using only an onboard RGB camera for localization. We assume the NeRF for the scene has been pre-trained offline, and the robot's objective is to navigate through unoccupied space in the NeRF to reach a goal pose. We introduce a trajectory optimization algorithm that avoids collisions with high-density regions in the NeRF based on a discrete time version of differential flatness that is amenable to constraining the robot's full pose and control inputs. We also introduce an optimization based filtering method to estimate 6DoF pose and velocities for the robot in the NeRF given only an onboard RGB camera. We combine the trajectory planner with the pose filter in an online replanning loop to give a vision-based robot navigation pipeline. We present simulation results with a quadrotor robot navigating through a jungle gym environment, the inside of a church, and Stonehenge using only an RGB camera. We also demonstrate an omnidirectional ground robot navigating through the church, requiring it to reorient to fit through a narrow gap.