FLEX: Full-Body Grasping Without Full-Body Grasps
FLEX: Full-Body Grasping Without Full-Body Grasps
复制标题
DOI：
10.1109/cvpr52729.2023.02029
复制
发表时间：
2022-11
期刊：
影响因子：
--
通讯作者：
Purva Tendulkar;D'idac Sur'is;Carl Vondrick
中科院分区：
文献类型：
--
作者：
Purva Tendulkar;D'idac Sur'is;Carl Vondrick
合成与场景逼真交互的3D人类化身是AR/VR，视频游戏和机器人技术中应用的重要问题。为了实现这一目标，我们解决的任务是生成一个虚拟的人-手和全身抓日常物品。现有方法通过收集人类与对象交互的3D数据集并在此数据上进行训练来解决这个问题。然而，1）这些方法不能推广到不同的对象位置和方向或场景中家具的存在，以及2）它们生成的全身姿势的多样性非常有限。在这项工作中，我们解决了所有上述挑战，在日常场景中生成逼真的，多样化的全身抓握，而不需要任何3D全身抓握数据。我们的关键见解是利用全身姿势和手抓先验的存在，使用3D几何约束来组合它们以获得全身抓地力。我们的经验验证，这些约束条件可以产生各种可行的人类抓地力是上级的基线定量和定性。请参阅我们的网页了解更多详情：flex.cs.columbia.edu。
Synthesizing 3D human avatars interacting realistically with a scene is an important problem with applications in AR/VR, video games, and robotics. Towards this goal, we address the task of generating a virtual human – hands and full body – grasping everyday objects. Existing methods approach this problem by collecting a 3D dataset of humans interacting with objects and training on this data. However, 1) these methods do not generalize to different object positions and orientations or to the presence of furniture in the scene, and 2) the diversity of their generated full-body poses is very limited. In this work, we address all the above challenges to generate realistic, diverse full-body grasps in everyday scenes without requiring any 3D full-body grasping data. Our key insight is to leverage the existence of both full-body pose and hand-grasping priors, composing them using 3D geometrical constraints to obtain full-body grasps. We empirically validate that these constraints can generate a variety of feasible human grasps that are superior to baselines both quantitatively and qualitatively. See our webpage for more details: flex.cs.columbia.edu.