Articulated Objects in Free-form Hand Interaction
Articulated Objects in Free-form Hand Interaction
复制标题
自由形式手部交互中的铰接对象
DOI：
10.48550/arxiv.2204.13662
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Otmar Hilliges
中科院分区：
文献类型：
--
作者：
Zicong Fan;Omid Taheri;Dimitrios Tzionas;Muhammed Kocabas;Manuel Kaufmann;Michael J. Black;Otmar Hilliges
.我们用手来与物体互动和操纵物体。有关节的物体特别有趣，因为它们通常需要人手的全部灵巧来操纵它们。为了理解、建模和合成这样的交互，需要从彩色图像以3D重建手和铰接对象的自动且鲁棒的方法。用于从图像估计3D手和对象姿态的现有方法集中于刚性对象。部分原因是，这种方法依赖于训练数据，并且不存在铰接对象操纵的数据集。因此，我们介绍了ARCTIC -第一个数据集的自由形式的相互作用的手和铰接对象。北极有1。200万张图像与精确的3D网格配对，适用于双手和随着时间推移而移动和变形的物体。该数据集还提供手-物体接触信息。为了显示我们的数据集的价值，我们在ARCTIC上执行了两个新的任务：（1）交互中的双手和关节对象的3D重建;（2）估计密集的手-对象相对距离，我们称之为交互场估计。对于第一个任务，我们提出了ArcticNet，这是一种基线方法，用于从RGB图像联合重建双手和关节对象的任务。对于交互场估计，我们预测从每个手顶点到物体表面的相对距离，反之亦然。我们介绍InterField，这是第一种从单个RGB图像估计这种距离的方法。我们为这两个任务提供了定性和定量的实验，并对数据进行了详细的分析。代码
. We use our hands to interact with and to manipulate objects. Articu-lated objects are especially interesting since they often require the full dexterity of human hands to manipulate them. To understand, model, and synthesize such interactions, automatic and robust methods that reconstruct hands and articulated objects in 3D from a color image are needed. Existing methods for estimating 3D hand and object pose from images focus on rigid objects. In part, because such methods rely on training data and no dataset of articulated object manipulation exists. Consequently, we introduce ARCTIC – the first dataset of free-form interactions of hands and articulated objects. ARCTIC has 1 . 2 M images paired with accurate 3D meshes for both hands and for objects that move and deform over time. The dataset also provides hand-object contact information. To show the value of our dataset, we perform two novel tasks on ARCTIC: (1) 3D reconstruction of two hands and an articulated object in interaction; (2) an estimation of dense hand-object relative distances, which we call interaction field estimation . For the first task, we present ArcticNet, a baseline method for the task of jointly reconstructing two hands and an articulated object from an RGB image. For interaction field estimation, we predict the relative distances from each hand vertex to the object surface, and vice versa. We introduce InterField, the first method that estimates such distances from a single RGB image. We provide qualitative and quantitative experiments for both tasks, and provide detailed analysis on the data. Code
DOI：
--
发表时间：
2020-10
期刊：
ArXiv
影响因子：
--
作者：
Supreeth Narasimhaswamy;Trung Nguyen;Minh Hoai
通讯作者：
Supreeth Narasimhaswamy;Trung Nguyen;Minh Hoai