Learning to Reconstruct People in Clothing From a Single RGB Camera
Learning to Reconstruct People in Clothing From a Single RGB Camera
复制标题
DOI：
10.1109/cvpr.2019.00127
复制
发表时间：
2019-03
期刊：
影响因子：
--
通讯作者：
Thiemo Alldieck;M. Magnor;Bharat Lal Bhatnagar;C. Theobalt;Gerard Pons-Moll
中科院分区：
文献类型：
--
作者：
Thiemo Alldieck;M. Magnor;Bharat Lal Bhatnagar;C. Theobalt;Gerard Pons-Moll
我们提出了 Octopus，一种基于学习的模型，可以从单眼视频的几帧（1-8）中推断人的个性化 3D 形状，其中人在移动，重建精度为 4 到 5mm，同时比以前的方法快几个数量级。我们的章鱼模型根据语义分割图像重建 3D 形状，包括 SMPL 参数以及衣服和头发，只需 10 秒或更短的时间。该模型基于两个关键的设计选择实现了快速、准确的预测。首先，通过预测规范 T 姿势空间中的形状，网络学习将人的图像编码为姿势不变的潜在代码，其中信息被融合。其次，基于前馈预测速度快但并不总是与输入图像对齐的观察，我们使用自下而上和自上而下的流（每个视图一个）进行预测，从而允许信息在两个方向上流动。学习仅依赖于合成 3D 数据。一旦学习完毕，Octopus 就可以采用可变数量的帧作为输入，并且甚至能够从单个图像中重建形状，精度为 5 毫米。 3 个不同数据集的结果证明了我们方法的有效性和准确性。
We present Octopus, a learning-based model to infer the personalized 3D shape of people from a few frames (1-8) of a monocular video in which the person is moving with a reconstruction accuracy of 4 to 5mm, while being orders of magnitude faster than previous methods. From semantic segmentation images, our Octopus model reconstructs a 3D shape, including the parameters of SMPL plus clothing and hair in 10 seconds or less. The model achieves fast and accurate predictions based on two key design choices. First, by predicting shape in a canonical T-pose space, the network learns to encode the images of the person into pose-invariant latent codes, where the information is fused. Second, based on the observation that feed-forward predictions are fast but do not always align with the input images, we predict using both, bottom-up and top-down streams (one per view) allowing information to flow in both directions. Learning relies only on synthetic 3D data. Once learned, Octopus can take a variable number of frames as input, and is able to reconstruct shapes even from a single image with an accuracy of 5mm. Results on 3 different datasets demonstrate the efficacy and accuracy of our approach.