Detailed Surface Geometry and Albedo Recovery from RGB-D Video under Natural Illumination
Detailed Surface Geometry and Albedo Recovery from RGB-D Video under Natural Illumination
复制标题
DOI：
10.1109/tpami.2019.2955459
复制
发表时间：
2017-02
影响因子：
23.6
通讯作者：
X. Zuo;Sen Wang;Jiangbin Zheng;Zhigeng Pan;Ruigang Yang
中科院分区：
文献类型：
--
作者：
X. Zuo;Sen Wang;Jiangbin Zheng;Zhigeng Pan;Ruigang Yang
本文提出了一种从RGB-D视频序列增强深度图的新方法。其基本思想是利用颜色序列中的光度信息来解决阴影问题带来的形状固有的模糊性。我们没有对表面反照率或受控物体运动和照明做任何假设，而是使用随机物体运动引入的照明变化。我们在自然光照下有效地计算运动物体的光度立体。关键的技术挑战之一是在整个图像集上建立对应关系。因此，我们开发了一种光照不敏感的鲁棒像素匹配技术，该技术在光照变化情况下优于光流方法。引入了一种自适应参考系选择方法，增强了系统对不完全朗伯反射的鲁棒性。此外，我们还提出了一个期望最大化框架，可以同时恢复地表法向和反照率，而不需要任何正则化项。我们在合成数据集和真实数据集上验证了我们的方法，显示了它在表面细节恢复和内在分解方面的优越性能。
This article presents a novel approach for depth map enhancement from an RGB-D video sequence. The basic idea is to exploit the photometric information in the color sequence to resolve the inherent ambiguity of shape from shading problem. Instead of making any assumption about surface albedo or controlled object motion and lighting, we use the lighting variations introduced by casual object movement. We are effectively calculating photometric stereo from a moving object under natural illuminations. One of the key technical challenges is to establish correspondences over the entire image set. We, therefore, develop a lighting insensitive robust pixel matching technique that out-performs optical flow method in presence of lighting variations. An adaptive reference frame selection procedure is introduced to get more robust to imperfect lambertian reflections. In addition, we present an expectation-maximization framework to recover the surface normal and albedo simultaneously, without any regularization term. We have validated our method on both synthetic and real datasets to show its superior performance on both surface details recovery and intrinsic decomposition.