MantissaCam: Learning Snapshot High-dynamic-range Imaging with Perceptually-based In-pixel Irradiance Encoding
MantissaCam: Learning Snapshot High-dynamic-range Imaging with Perceptually-based In-pixel Irradiance Encoding
复制标题
DOI：
10.1109/iccp54855.2022.9887659
复制
发表时间：
2021-12
期刊：
影响因子：
--
通讯作者：
Haley M. So;Julien N. P. Martel;P. Dudek;Gordon Wetzstein
中科院分区：
文献类型：
--
作者：
Haley M. So;Julien N. P. Martel;P. Dudek;Gordon Wetzstein
对高动态范围（HDR）场景进行成像的能力在许多计算机视觉应用中至关重要。然而，传统传感器的动态范围从根本上受到其阱容量的限制，导致亮场景部分饱和。为了克服这一限制，新兴的传感器提供像素内处理能力来编码入射辐照度。其中最有前途的编码方案是模包裹，这导致计算摄影问题，其中HDR场景由辐照度解包裹算法从包裹的低动态范围（LDR）传感器图像计算。在这里，我们设计了一种基于神经网络的算法，其性能优于以前的辐照度展开方法，并且我们设计了一种感知启发的“尾数”或对数模编码方案，该方案可以更有效地将HDR场景包装到LDR传感器中。结合我们的重建框架，MantissaCam在模块型快照HDR成像方法中实现了最先进的结果。我们证明了我们的方法在模拟中的有效性，并显示了我们的算法的好处与可编程传感器实现的原型捕获的模图像。
The ability to image high-dynamic-range (HDR) scenes is crucial in many computer vision applications. The dynamic range of conventional sensors, however, is fundamentally limited by their well capacity, resulting in saturation of bright scene parts. To overcome this limitation, emerging sensors offer in-pixel processing capabilities to encode the incident irradiance. Among the most promising encoding schemes is modulo wrapping, which results in a computational photography problem where the HDR scene is computed by an irradiance unwrapping algorithm from the wrapped low-dynamic-range (LDR) sensor image. Here, we design a neural network-based algorithm that outperforms previous irradiance unwrapping methods and we design a perceptually inspired “mantissa,” or log-modulo, encoding scheme that more efficiently wraps an HDR scene into an LDR sensor. Combined with our reconstruction framework, MantissaCam achieves state-of-the-art results among modulo-type snapshot HDR imaging approaches. We demonstrate the efficacy of our method in simulation and show benefits of our algorithm on modulo images captured with a prototype implemented with a programmable sensor.