Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objects
Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objects
复制标题
DOI：
10.1109/cvpr52688.2022.00366
复制
发表时间：
2021-12
期刊：
影响因子：
--
通讯作者：
Atsuhiro Noguchi;Umar Iqbal-;Jonathan Tremblay;T. Harada;Orazio Gallo
中科院分区：
文献类型：
--
作者：
Atsuhiro Noguchi;Umar Iqbal-;Jonathan Tremblay;T. Harada;Orazio Gallo
在控制对象姿态的同时渲染关节对象对于虚拟现实或电影动画等应用至关重要。然而，操纵对象的姿势需要了解其底层结构，即其关节以及它们如何相互作用。不幸的是，假设结构是已知的，如现有的方法所做的那样，排除了在新的对象类别上工作的能力。我们建议通过观察它们从多个视图移动来学习以前看不见的铰接对象的外观和结构，没有关节注释监督或有关结构的信息。我们观察到，相对于彼此静止的3D点应该属于同一部分，并且相对于彼此移动的相邻部分必须通过关节连接。为了利用这一洞察力，我们将3D中的对象部分建模为椭球体，这使我们能够识别关节。我们结合联合收割机这一明确的表示与隐式的近似介绍补偿。我们表明，我们的方法适用于不同的结构，从四足动物，单臂机器人，人类。代码可以在https://github.com/NVlabs/watch-it-move上找到，使用动画的手稿版本可以在https://arxiv.org/abs/2112.11347上找到
Rendering articulated objects while controlling their poses is critical to applications such as virtual reality or animation for movies. Manipulating the pose of an object, however, requires the understanding of its underlying structure, that is, its joints and how they interact with each other. Unfortunately, assuming the structure to be known, as existing methods do, precludes the ability to work on new object categories. We propose to learn both the appearance and the structure of previously unseen articulated objects by ob-serving them move from multiple views, with no joints annotation supervision, or information about the structure. We observe that 3D points that are static relative to one another should belong to the same part, and that adjacent parts that move relative to each other must be connected by a joint. To leverage this insight, we model the object parts in 3D as ellipsoids, which allows us to identify joints. We combine this explicit representation with an implicit one that compensates for the approximation introduced. We show that our method works for different structures, from quadrupeds, to single-arm robots, to humans. The code is available at https://github.com/NVlabs/watch-it-move and a version of this manuscript that uses animations is at https://arxiv.org/abs/2112.11347