SimPose: Effectively Learning DensePose and Surface Normals of People from Simulated Data
SimPose: Effectively Learning DensePose and Surface Normals of People from Simulated Data
复制标题
DOI：
10.1007/978-3-030-58526-6_14
复制
发表时间：
2020-07
期刊：
影响因子：
--
通讯作者：
Tyler Lixuan Zhu;Per Karlsson;C. Bregler
中科院分区：
文献类型：
--
作者：
Tyler Lixuan Zhu;Per Karlsson;C. Bregler
随着通用领域适应方法的激增，我们报告了一种简单而有效的技术，用于学习清晰的人的困难的每像素 2.5D 和 3D 回归表示。我们为 2.5D DensePose 估计任务和 3D 人体表面法线估计任务获得了强大的模拟到真实域泛化能力。在多人 DensePose MSCOCO 基准上，我们的方法优于在密集标记的真实图像上进行训练的最先进方法。这是一个重要的结果，因为在真实图像上获取人体流形的固有紫外坐标非常耗时并且容易产生标签噪声。此外，我们在 MSCOCO 数据集上展示了模型的 3D 表面法线预测，该数据集缺乏任何真实的 3D 表面法线标签。我们方法的关键是通过从域样本的混合中精心挑选的训练批次、深度批量归一化残差网络和修改后的多任务学习目标来减轻“域间协变量偏移”。我们的方法是对现有域适应技术的补充，并且可以应用于其他密集的每像素姿态估计问题。
With a proliferation of generic domain-adaptation approaches, we report a simple yet effective technique for learning difficult per-pixel 2.5D and 3D regression representations of articulated people. We obtained strong sim-to-real domain generalization for the 2.5D DensePose estimation task and the 3D human surface normal estimation task. On the multi-person DensePose MSCOCO benchmark, our approach outperforms the state-of-the-art methods which are trained on real images that are densely labelled. This is an important result since obtaining human manifold’s intrinsicuvcoordinates on real images is time consuming and prone to labeling noise. Additionally, we present our model’s 3D surface normal predictions on the MSCOCO dataset that lacks any real 3D surface normal labels. The key to our approach is to mitigate the “Inter-domain Covariate Shift” with a carefully selected training batch from a mixture of domain samples, a deep batch-normalized residual network, and a modified multi-task learning objective. Our approach is complementary to existing domain-adaptation techniques and can be applied to other dense per-pixel pose estimation problems.