TöRF: Time-of-Flight Radiance Fields for Dynamic Scene View Synthesis
TöRF: Time-of-Flight Radiance Fields for Dynamic Scene View Synthesis
复制标题
DOI：
--
复制
发表时间：
2021-09
期刊：
影响因子：
--
通讯作者：
Benjamin Attal;Eliot Laidlaw;Aaron Gokaslan;Changil Kim;Christian Richardt;J. Tompkin;Matthew O'Toole
中科院分区：
文献类型：
--
作者：
Benjamin Attal;Eliot Laidlaw;Aaron Gokaslan;Changil Kim;Christian Richardt;J. Tompkin;Matthew O'Toole
神经网络可以表示并准确地重建静态3D场景（例如，NeRF）的亮度场。一些作品将这些扩展到用单目视频捕捉的动态场景，具有很好的表现。然而，已知单目设置是一个约束不足的问题，因此方法依赖于数据驱动的先验来重建动态内容。我们用飞行时间（ToF）相机的测量值代替了这些先验，并引入了基于连续波ToF相机图像形成模型的神经表示。我们不再使用处理过的深度图，而是对原始ToF传感器测量数据进行建模，以提高重建质量，避免低反射区域、多路径干扰和传感器有限的明确深度范围等问题。我们表明，这种方法提高了动态场景重建对错误校准和大运动的鲁棒性，并讨论了集成RGB+ToF传感器的优点和局限性，这些传感器现在可以在现代智能手机上使用。
Neural networks can represent and accurately reconstruct radiance fields for static 3D scenes (e.g., NeRF). Several works extend these to dynamic scenes captured with monocular video, with promising performance. However, the monocular setting is known to be an under-constrained problem, and so methods rely on data-driven priors for reconstructing dynamic content. We replace these priors with measurements from a time-of-flight (ToF) camera, and introduce a neural representation based on an image formation model for continuous-wave ToF cameras. Instead of working with processed depth maps, we model the raw ToF sensor measurements to improve reconstruction quality and avoid issues with low reflectance regions, multi-path interference, and a sensor's limited unambiguous depth range. We show that this approach improves robustness of dynamic scene reconstruction to erroneous calibration and large motions, and discuss the benefits and limitations of integrating RGB+ToF sensors that are now available on modern smartphones.