Ditto: Building Digital Twins of Articulated Objects from Interaction
Ditto: Building Digital Twins of Articulated Objects from Interaction
复制标题
DOI：
10.1109/cvpr52688.2022.00553
复制
发表时间：
2022-02
期刊：
影响因子：
--
通讯作者：
Zhenyu Jiang;Cheng-Chun Hsu;Yuke Zhu
中科院分区：
文献类型：
--
作者：
Zhenyu Jiang;Cheng-Chun Hsu;Yuke Zhu
将物理对象数字化到虚拟世界中有可能开启嵌入式AI和混合现实的新研究和应用。这项工作的重点是重新创建现实世界的铰接对象的交互式数字双胞胎，可以直接导入到虚拟环境中。我们引入Ditto来学习通过交互感知的关节模型估计和关节对象的3D几何重建。给定交互之前和之后对铰接对象的一对视觉观察，Ditto重建零件级几何形状并估计对象的铰接模型。我们采用隐式神经表示关节几何和关节建模。我们的实验表明，Ditto以一种类别不可知的方式有效地构建了连接对象的数字孪生。我们还将Ditto应用于现实世界的对象，并在物理模拟中部署重新创建的数字孪生模型。代码和其他结果可在https://ut-austin-rpl.github.io/Ditto/上获得
Digitizing physical objects into the virtual world has the potential to unlock new research and applications in embodied AI and mixed reality. This work focuses on recreating interactive digital twins of real-world articulated objects, which can be directly imported into virtual environments. We introduce Ditto to learn articulation model estimation and 3D geometry reconstruction of an articulated object through interactive perception. Given a pair of visual observations of an articulated object before and after interaction, Ditto reconstructs part-level geometry and estimates the articulation model of the object. We employ implicit neural representations for joint geometry and articulation modeling. Our experiments show that Ditto effectively builds digital twins of articulated objects in a category-agnostic way. We also apply Ditto to real-world objects and deploy the recreated digital twins in physical simulation. Code and additional results are available at https://ut-austin-rpl.github.io/Ditto/