Learned Compressive Representations for Single-Photon 3D Imaging
Learned Compressive Representations for Single-Photon 3D Imaging
复制标题
DOI：
10.1109/iccv51070.2023.00987
复制
发表时间：
2023-10
期刊：
影响因子：
--
通讯作者：
Felipe Gutierrez-Barragan;Fangzhou Mu;Andrei Ardelean;A. Ingle;C. Bruschini;E. Charbon;Yin Li;Mohit Gupta;A. Velten
中科院分区：
文献类型：
--
作者：
Felipe Gutierrez-Barragan;Fangzhou Mu;Andrei Ardelean;A. Ingle;C. Bruschini;E. Charbon;Yin Li;Mohit Gupta;A. Velten
单光子3D相机可以每秒记录数十亿光子的到达时间，精度为皮秒。总结光子数据流的一种常见方法是构建每像素时间戳直方图，从而产生沿时间轴沿着编码距离的3D直方图张量。随着直方图张量的时空分辨率增加，像素内存储器要求和输出数据速率可能很快变得不切实际。为了克服这一限制，我们提出了一个家庭的线性压缩表示的直方图张量，可以有效地计算，在一个在线的方式，作为一个矩阵运算。我们设计了实用的轻量级压缩表示，适合于像素内实现，并考虑每个时间戳的时空信息。此外，我们将我们提出的框架实现为神经网络的第一层，这使得压缩表示和下游SPAD数据处理模型的联合端到端优化成为可能。我们发现，一个精心设计的压缩表示可以减少传感器内存和数据速率高达2个数量级，而不会显着降低3D成像质量。最后，我们分析了功耗的影响，通过片上实现。
Single-photon 3D cameras can record the time-of-arrival of billions of photons per second with picosecond accuracy. One common approach to summarize the photon data stream is to build a per-pixel timestamp histogram, resulting in a 3D histogram tensor that encodes distances along the time axis. As the spatio-temporal resolution of the histogram tensor increases, the in-pixel memory requirements and output data rates can quickly become impractical. To overcome this limitation, we propose a family of linear compressive representations of histogram tensors that can be computed efficiently, in an online fashion, as a matrix operation. We design practical lightweight compressive representations that are amenable to an in-pixel implementation and consider the spatio-temporal information of each timestamp. Furthermore, we implement our proposed framework as the first layer of a neural network, which enables the joint end-to-end optimization of the compressive representations and a downstream SPAD data processing model. We find that a well-designed compressive representation can reduce in-sensor memory and data rates up to 2 orders of magnitude without significantly reducing 3D imaging quality. Finally, we analyze the power consumption implications through an on-chip implementation.