CAREER: Quanta Computational Imaging with Single-Photon Cameras

单光子雪崩二极管（SPAD）是一种新兴的传感器技术，能够检测单个入射光子并以皮秒精度捕获它们的到达时间。由于其高灵敏度和时间分辨率，SPAD正在推动成像革命，实现迄今为止被认为是不可能的极端应用：每秒万亿帧的成像，非视线成像和纳米时间尺度的显微成像。尽管有这些功能，但SPAD被认为是仅在超暗环境中使用的专用设备，并且仅限于有限的利基应用。该项目开发技术，以扩大SPAD作为具有广泛应用的通用相机的范围。所开发的技术不仅将促进SPAD在生命科学、天文学和医学等领域的广泛采用，在这些领域，尽可能少的光对成功至关重要，而且还将实现新的、高要求的应用。3D相机将能够在远距离实现比当前最先进技术更高的深度分辨率，使车辆（空中，陆地和水下）能够在具有挑战性的天气条件和崎岖地形中自主导航。这项研究的结果将通过科学会议和期刊传播。一些材料将被整合到一本关于主动3D成像技术的教科书中。这项研究为一类新的成像和计算技术奠定了数学和物理基础，这类技术将把SPAD转变为能够在各种条件下（从黑暗到明亮的阳光）工作的“通用”相机，以便在整个成像条件下恢复高质量图像和三维场景信息。该项目开发（a）编码和异步单光子成像，这是两种新颖的主动单光子成像技术，可最大限度地减少非线性失真，并可在高通量环境中可靠地工作;（B）单光子计算成像技术，用于在被动、不受控制的照明下捕获场景强度（例如，sunlight）;以及（c）基于尖峰神经网络从单光子传感器数据中提取高级场景信息的新型机器学习算法，从而在动态环境中，在低功耗设备上实现快速，节能的场景理解。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Single-photon avalanche diodes (SPADs) are an emerging sensor technology capable of detecting individual incident photons and capturing their time-of-arrival with picosecond precision. Due to their high sensitivity and time resolution, SPADs are driving an imaging revolution, enabling extreme applications that were hitherto considered impossible: imaging at trillion frames-per-second, non-line-of-sight imaging, and microscopic imaging at nano time-scales. Despite these capabilities, SPADs are considered specialized devices used only in ultra-dark environments and restricted to a limited set of niche applications. This project develops technologies to expand the scope of SPADs as general-purpose cameras with a broad range of applications. The developed technologies will not only spur wide-spread adoption of SPADs in fields like life science, astronomy, and medicine, where operating with the smallest amount of light possible is critical to success, but also enable new, highly demanding applications. 3D cameras will be able to achieve substantially higher depth resolution than current state-of-the-art at long distances, enabling vehicles (aerial, terrestrial, and underwater) to navigate autonomously in challenging weather conditions and on rugged terrains. The results from this research will be disseminated through scientific conferences and journals. Some materials will be integrated into a textbook on active 3D imaging techniques. This research develops mathematical and physical foundations for a new class of imaging and computational techniques which will transform SPADs into `all-purpose' cameras capable of operating in diverse conditions (dark to bright sunlight), for recovering high-quality images and 3D scene information over the entire gamut of imaging conditions. This project develops (a) coded and asynchronous single-photon imaging, two novel families of active single-photon imaging techniques that minimize non-linear distortions and can reliably operate in high-flux environments; (b) single-photon computational imaging techniques for capturing scene intensity under passive, uncontrolled lighting (e.g., sunlight); and (c) novel machine learning algorithms for extracting high-level scene information from single-photon sensor data, based on spiking neural networks, enabling rapid, power-efficient scene understanding in dynamic environments, on low-power devices.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.