Real-time Non-rigid Reconstruction using an RGB-D Camera
Real-time Non-rigid Reconstruction using an RGB-D Camera
复制标题
DOI：
10.1145/2601097.2601165
复制
发表时间：
2014-07-01
影响因子：
6.2
通讯作者：
Stamminger, Marc
中科院分区：
文献类型：
--
作者：
Zollhoefer, Michael;Niessner, Matthias;Stamminger, Marc
本文提出了一种软硬件结合的无标记点实时重建任意形状非刚性变形物体的方法。我们的系统使用一个独立的立体摄像机单元，由现成的组件和消费者图形硬件构建，以30 Hz的频率生成时空相干的3D模型。一种新的立体匹配算法估计实时RGB-D数据。我们首先扫描一个光滑的模板模型的主题，因为他们刚性移动。这种几何表面先验避免了强场景假设，例如运动人体骨架或参数形状模型。接下来，一种新的GPU流水线使用扩展的非线性尽可能刚性（ARAP）框架执行实时RGB-D数据到平滑模板的非刚性配准。使用线性变形模型将高频细节融合到最终网格上。该系统比最先进的方法快一个数量级，同时匹配许多离线算法的质量和鲁棒性。我们展示了各种场景的精确实时重建，包括：用户头部、手部和上身的大变形;皮肤和衣服的精细皱纹和褶皱;以及用户在玩具等柔性物体上进行的非刚性交互。我们演示了如何获得的模型可以用于许多交互式场景，包括重新纹理，在线性能捕获和预览，以及实时形状和运动重新定位。
We present a combined hardware and software solution for marker-less reconstruction of non-rigidly deforming physical objects with arbitrary shape in real-time. Our system uses a single self-contained stereo camera unit built from off-the-shelf components and consumer graphics hardware to generate spatio-temporally coherent 3D models at 30 Hz. A new stereo matching algorithm estimates real-time RGB-D data. We start by scanning a smooth template model of the subject as they move rigidly. This geometric surface prior avoids strong scene assumptions, such as a kinematic human skeleton or a parametric shape model. Next, a novel GPU pipeline performs non-rigid registration of live RGB-D data to the smooth template using an extended non-linear as-rigid-as-possible (ARAP) framework. High-frequency details are fused onto the final mesh using a linear deformation model. The system is an order of magnitude faster than state-of-the-art methods, while matching the quality and robustness of many offline algorithms. We show precise real-time reconstructions of diverse scenes, including: large deformations of users' heads, hands, and upper bodies; fine-scale wrinkles and folds of skin and clothing; and non-rigid interactions performed by users on flexible objects such as toys. We demonstrate how acquired models can be used for many interactive scenarios, including re-texturing, online performance capture and preview, and real-time shape and motion re-targeting.