LASR: Learning Articulated Shape Reconstruction from a Monocular Video
LASR: Learning Articulated Shape Reconstruction from a Monocular Video
复制标题
LASR：从单目视频学习铰接形状重建
DOI：
--
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Ce Liu
中科院分区：
文献类型：
--
作者：
Gengshan Yang;Deqing Sun;V. Jampani;Daniel Vlasic;Forrester Cole;Huiwen Chang;Deva Ramanan;W. Freeman;Ce Liu
从视频或图像集合中进行刚性结构的三维重建已经取得了显着的进展。然而，由于其约束不足的性质，从RGB输入重建非刚性结构仍然具有挑战性。虽然基于模板的方法，如参数化形状模型，在建模已知对象类别的“封闭世界”方面取得了巨大的成功，但它们不能很好地处理新颖对象类别或离群形状的“开放世界”。在这项工作中，我们引入了一种无模板的方法来从单个视频中学习3D形状。它采用合成分析策略，向前渲染对象轮廓，光流和像素值以与视频观察结果进行比较，从而生成梯度以调整相机，形状和运动参数。在不使用特定类别的形状模板的情况下，我们的方法忠实地从人类，动物和未知类别的对象的视频中重建非刚性3D结构。我们的代码可以在lasr-google.github.io上找到。
Remarkable progress has been made in 3D reconstruction of rigid structures from a video or a collection of images. However, it is still challenging to reconstruct nonrigid structures from RGB inputs, due to its under-constrained nature. While template-based approaches, such as parametric shape models, have achieved great success in modeling the "closed world" of known object categories, they cannot well handle the "open-world" of novel object categories or outlier shapes. In this work, we introduce a template-free approach to learn 3D shapes from a single video. It adopts an analysis-by-synthesis strategy that forward-renders object silhouette, optical flow, and pixel values to compare with video observations, which generates gradients to adjust the camera, shape and motion parameters. Without using a category-specific shape template, our method faithfully reconstructs nonrigid 3D structures from videos of human, animals, and objects of unknown classes. Our code is available at lasr-google.github.io.