Self-Supervised Visual Motor Skills via Neural Radiance Fields
Self-Supervised Visual Motor Skills via Neural Radiance Fields
复制标题
DOI：
10.1109/iros55552.2023.10341682
复制
发表时间：
2023-10
期刊：
影响因子：
--
通讯作者：
Paul Gesel;Noushad Sojib;M. Begum
中科院分区：
文献类型：
--
作者：
Paul Gesel;Noushad Sojib;M. Begum
在本文中，我们提出了一种新的视觉模仿学习的网络架构，利用神经辐射场（NeRFs）和关键点对应的自我监督视觉运动策略学习。建议的网络架构采用了动态系统输出层的政策学习。将动态系统的稳定性和目标自适应特性与基于关键点的对应关系的鲁棒性相结合，产生了一种对显著的杂乱、遮挡、照明条件变化和目标配置中的空间变化不变的策略。在多个操作任务上的实验表明，当使用少量训练样本时，我们的方法在分布内和分布外的情况下都优于可比的视觉运动策略学习方法。
In this paper, we propose a novel network architecture for visual imitation learning that exploits neural radiance fields (NeRFs) and key-point correspondence for self-supervised visual motor policy learning. The proposed network architecture incorporates a dynamic system output layer for policy learning. Combining the stability and goal adaption properties of dynamic systems with the robustness of keypoint-based correspondence yields a policy that is invariant to significant clutter, occlusions, lighting conditions changes, and spatial variations in goal configurations. Experiments on multiple manipulation tasks show that our method outperforms comparable visual motor policy learning methods on both in-distribution and out-of-distribution scenarios when using a small number of training samples.