Differentiable Physics Simulation of Dynamics-Augmented Neural Objects
Differentiable Physics Simulation of Dynamics-Augmented Neural Objects
复制标题
DOI：
10.1109/lra.2023.3257707
复制
发表时间：
2022-10
影响因子：
5.2
通讯作者：
Simon Le Cleac'h;Hong Yu;Michelle Guo;Taylor A. Howell;Ruohan Gao;Jiajun Wu;Zachary Manchester;M. Schwager
中科院分区：
文献类型：
--
作者：
Simon Le Cleac'h;Hong Yu;Michelle Guo;Taylor A. Howell;Ruohan Gao;Jiajun Wu;Zachary Manchester;M. Schwager
我们提出了一个可微的流程，用于模拟将物体几何形状表示为以深度网络为参数的连续密度场的物体运动。这包括神经辐射场（NeRFs）以及其他相关模型。从密度场中，我们估计物体的动力学属性，包括其质量、质心和惯性矩阵。然后，我们引入了一种基于密度场的可微接触模型，用于计算碰撞产生的法向力和摩擦力。这使得机器人能够从运动物体的静止图像和视频中自主构建视觉和动力学上都准确的物体模型。由此产生的动力学增强神经物体（DANOs）通过现有的可微模拟引擎Dojo进行模拟，与其他标准模拟物体（如球体、平面以及以URDF格式指定的机器人）相互作用。机器人可以利用这种模拟来优化神经物体的抓取和操作轨迹，或者通过基于梯度的真实到模拟的迁移来改进神经物体模型。我们展示了从一块肥皂在桌子上滑动的真实视频中学习肥皂摩擦系数的流程。我们还通过从合成数据中与熊猫机器人手臂的交互学习斯坦福兔子的摩擦系数和质量，并且我们在模拟中为熊猫手臂优化轨迹以将兔子推到目标位置。
We present a differentiable pipeline for simulating the motion of objects that represent their geometry as a continuous density field parameterized as a deep network. This includes Neural Radiance Fields (NeRFs), and other related models. From the density field, we estimate the dynamical properties of the object, including its mass, center of mass, and inertia matrix. We then introduce a differentiable contact model based on the density field for computing normal and friction forces resulting from collisions. This allows a robot to autonomously build object models that are visually and dynamically accurate from still images and videos of objects in motion. The resulting Dynamics-Augmented Neural Objects (DANOs) are simulated with an existing differentiable simulation engine, Dojo, interacting with other standard simulation objects, such as spheres, planes, and robots specified as URDFs. A robot can use this simulation to optimize grasps and manipulation trajectories of neural objects, or to improve the neural object models through gradient-based real-to-simulation transfer. We demonstrate the pipeline to learn the coefficient of friction of a bar of soap from a real video of the soap sliding on a table. We also learn the coefficient of friction and mass of a Stanford bunny through interactions with a Panda robot arm from synthetic data, and we optimize trajectories in simulation for the Panda arm to push the bunny to a goal location.