Deep Reflectance Scanning: Recovering Spatially‐varying Material Appearance from a Flash‐lit Video Sequence
Deep Reflectance Scanning: Recovering Spatially‐varying Material Appearance from a Flash‐lit Video Sequence
复制标题
DOI：
10.1111/cgf.14387
复制
发表时间：
2021-08
影响因子：
2.5
通讯作者：
Wenjie Ye;Yue Dong;P. Peers;B. Guo
中科院分区：
文献类型：
--
作者：
Wenjie Ye;Yue Dong;P. Peers;B. Guo
在本文中，我们提出了一种新的方法来恢复高分辨率的空间变化的各向同性表面反射率的平面样本从闪光灯的特写镜头的视频序列捕获与定期的手持移动的手机。我们不需要仔细校准摄像机和照明参数，而是使用深度神经网络计算每像素流图来对齐输入视频帧。对于每个视频帧，我们还提取反射率参数，并直接使用每像素流扭曲神经反射率特征，然后将扭曲的特征集中起来。我们的方法便于非专业用户使用商品硬件方便地手持采集空间变化的表面反射率。此外，我们的方法能够从仅在捕获的视频帧的子集中可见的表面点聚合反射特征，从而能够创建超过原生相机分辨率的高分辨率反射图。我们在各种合成和真实的空间变化材料上演示和验证了我们的方法。
In this paper we present a novel method for recovering high‐resolution spatially‐varying isotropic surface reflectance of a planar exemplar from a flash‐lit close‐up video sequence captured with a regular hand‐held mobile phone. We do not require careful calibration of the camera and lighting parameters, but instead compute a per‐pixel flow map using a deep neural network to align the input video frames. For each video frame, we also extract the reflectance parameters, and warp the neural reflectance features directly using the per‐pixel flow, and subsequently pool the warped features. Our method facilitates convenient hand‐held acquisition of spatially‐varying surface reflectance with commodity hardware by non‐expert users. Furthermore, our method enables aggregation of reflectance features from surface points visible in only a subset of the captured video frames, enabling the creation of high‐resolution reflectance maps that exceed the native camera resolution. We demonstrate and validate our method on a variety of synthetic and real‐world spatially‐varying materials.