3D Scene Inference from Transient Histograms
3D Scene Inference from Transient Histograms
复制标题
DOI：
10.48550/arxiv.2211.05094
复制
发表时间：
2022-11
期刊：
影响因子：
--
通讯作者：
Sacha Jungerman;A. Ingle;Yin Li;Mohit Gupta
中科院分区：
文献类型：
--
作者：
Sacha Jungerman;A. Ingle;Yin Li;Mohit Gupta
以皮秒到纳秒的时间尺度捕捉光线的时间分辨率图像传感器一度仅限于小众应用，但现在正迅速成为消费设备的主流。我们提出了低成本和低功耗的成像方式，从最小的时间分辨率图像传感器捕获场景信息，只需一个像素。其关键思想是用脉冲光源泛洪照亮大型场景块(或整个场景)，并通过在整个照明区域进行积分来测量时间分辨的反射光。一维测量的时间波形称为\emph{瞬变}，它对所有可见场景点处的距离和反照率进行编码，因此是场景3D几何体的聚合代理。我们探讨了暂态波形本身在恢复场景信息方面的可行性和局限性，以及与传统RGB相机结合时的可行性和局限性。我们表明，平面估计可以从单个瞬变进行，并且只使用更多的几个瞬变就可以恢复整个场景的深度图。我们还展示了两个概念验证硬件原型，它们展示了我们的方法在紧凑型、移动和预算有限的应用中的可行性。
Time-resolved image sensors that capture light at pico-to-nanosecond timescales were once limited to niche applications but are now rapidly becoming mainstream in consumer devices. We propose low-cost and low-power imaging modalities that capture scene information from minimal time-resolved image sensors with as few as one pixel. The key idea is to flood illuminate large scene patches (or the entire scene) with a pulsed light source and measure the time-resolved reflected light by integrating over the entire illuminated area. The one-dimensional measured temporal waveform, called \emph{transient}, encodes both distances and albedoes at all visible scene points and as such is an aggregate proxy for the scene's 3D geometry. We explore the viability and limitations of the transient waveforms by themselves for recovering scene information, and also when combined with traditional RGB cameras. We show that plane estimation can be performed from a single transient and that using only a few more it is possible to recover a depth map of the whole scene. We also show two proof-of-concept hardware prototypes that demonstrate the feasibility of our approach for compact, mobile, and budget-limited applications.