ACID: Action-Conditional Implicit Visual Dynamics for Deformable Object Manipulation
ACID: Action-Conditional Implicit Visual Dynamics for Deformable Object Manipulation
复制标题
DOI：
10.1177/02783649231191222
复制
发表时间：
2022-03
期刊：
影响因子：
--
通讯作者：
Bokui Shen;Zhenyu Jiang;C. Choy;L. Guibas;S. Savarese;Anima Anandkumar;Yuke Zhu
中科院分区：
文献类型：
--
作者：
Bokui Shen;Zhenyu Jiang;C. Choy;L. Guibas;S. Savarese;Anima Anandkumar;Yuke Zhu
在现实世界中操纵体积可变形对象，如毛绒玩具和披萨面团，由于无限的形状变化、非刚性运动和部分可观察性，带来了巨大的挑战。本文介绍了ACID，一种基于结构化隐式神经表示的体积可变形物体的动作条件视觉动力学模型。ACID集成了两种新技术：动作条件动力学的隐式表示和基于测地线的对比学习。为了从部分RGB-D观测中表示可变形的动力学，我们学习了占用的隐式表示和基于流的向前动力学。为了准确识别非刚性大变形下的状态变化，我们通过一种新的基于测地线的对比损耗来学习对应的嵌入场。为了评估我们的方法，我们开发了一个模拟框架，用于在真实场景中操纵复杂的可变形形状，并开发了一个基准，该基准包含超过17,000个动作轨迹，包括6种毛绒玩具和78种变体。与现有方法相比，我们的模型在几何、对应和动力学预测方面取得了最好的性能。ACID动力学模型被成功地应用于目标条件可变形操纵任务，使得任务成功率比最强基线提高了30%。此外，我们将仿真训练的ACID模型直接应用于真实世界的对象，并成功地将它们操作为目标配置。Https://b0ku1.github.io/acid/
Manipulating volumetric deformable objects in the real world, like plush toys and pizza dough, brings substantial challenges due to infinite shape variations, non-rigid motions, and partial observability. We introduce ACID, an action-conditional visual dynamics model for volumetric deformable objects based on structured implicit neural representations. ACID integrates two new techniques: implicit representations for action-conditional dynamics and geodesics-based contrastive learning. To represent deformable dynamics from partial RGB-D observations, we learn implicit representations of occupancy and flow-based forward dynamics. To accurately identify state change under large non-rigid deformations, we learn a correspondence embedding field through a novel geodesics-based contrastive loss. To evaluate our approach, we develop a simulation framework for manipulating complex deformable shapes in realistic scenes and a benchmark containing over 17,000 action trajectories with six types of plush toys and 78 variants. Our model achieves the best performance in geometry, correspondence, and dynamics predictions over existing approaches. The ACID dynamics models are successfully employed for goal-conditioned deformable manipulation tasks, resulting in a 30% increase in task success rate over the strongest baseline. Furthermore, we apply the simulation-trained ACID model directly to real-world objects and show success in manipulating them into target configurations. https://b0ku1.github.io/acid/