Full-Body Motion from a Single Head-Mounted Device: Generating SMPL Poses from Partial Observations
Full-Body Motion from a Single Head-Mounted Device: Generating SMPL Poses from Partial Observations
复制标题
DOI：
10.1109/iccv48922.2021.01148
复制
发表时间：
2021-10
期刊：
影响因子：
--
通讯作者：
Andrea Dittadi;Sebastian Dziadzio;D. Cosker;Bengt Lundell;Tom Cashman;J. Shotton
中科院分区：
文献类型：
--
作者：
Andrea Dittadi;Sebastian Dziadzio;D. Cosker;Bengt Lundell;Tom Cashman;J. Shotton
头戴式和可穿戴设备的可用性和成熟度不断提高，为远程通信和协作带来了机会。然而，这些设备提供的信号流（例如头部姿势、手部姿势和注视方向）并不代表整个人。因此，主要的开放问题之一是如何利用这些信号来建立用户的忠实表示。在本文中，我们提出了一种基于变分自动编码器的方法，根据头部和手部姿势的噪声流生成人体骨骼的关节姿势。我们的方法依赖于一种新颖且理论上有充分依据的姿势可能性模型。我们在公开可用的数据集上证明，即使信号非常贫乏，我们的方法也是有效的，并研究如何使姿势预测更加准确和真实。
The increased availability and maturity of head-mounted and wearable devices opens up opportunities for remote communication and collaboration. However, the signal streams provided by these devices (e.g., head pose, hand pose, and gaze direction) do not represent a whole person. One of the main open problems is therefore how to leverage these signals to build faithful representations of the user. In this paper, we propose a method based on variational autoencoders to generate articulated poses of a human skeleton based on noisy streams of head and hand pose. Our approach relies on a model of pose likelihood that is novel and theoretically well-grounded. We demonstrate on publicly available datasets that our method is effective even from very impoverished signals and investigate how pose prediction can be made more accurate and realistic.