Short-Exposure Imaging through Atmospheric Turbulence using Single Photon Image Sensors

地面远程被动成像系统是民用和军事应用中最关键的传感方式之一，但由于大气湍流，它们经常受到失真的影响。在大气湍流的存在下，捕捉到的图像会变得不稳定、模糊，最终会丢失物体的细节。如果没有任何缓解措施，图像质量的下降将严重降低我们做出明智决策、识别威胁对象、在未知环境中导航和执行科学测量的能力。人们已经研究了几十年的湍流缓解方法，但大多数基于光学的方法需要相干光源，这并不总是可行的，而后处理方法在很大程度上仅限于静止场景。当感兴趣的物体移动或摄像机不静止时，图像的捕获和恢复都是非常具有挑战性的任务。该项目的目标是开发一种新的计算成像框架，使用单光子图像传感器，通过捕获和恢复因湍流而扭曲的运动物体。通过实现这一目标，该项目将填补当前成像技术的关键空白。这反过来将为监视、导航、国防和遥感提供新的成像能力，所有这些都对经济和国土安全至关重要。拟议的项目将开发获取极短曝光图像的方法，以及恢复动态场景的新算法。这涉及到一种新型的图像传感器，它提供了最先进的光子分辨率，即使在极短的曝光下也可以在足够的信噪比下运行。将开展四项研究工作：(1)通过分析光子统计数据及其与大气的相互作用，得出穿过湍流的单光子成像的理论界限；(2)开发动态采样机制，以获取非均匀信号，并开发最佳曝光控制，以实现高动态范围成像；(3)开发人工智能增强的图像重建算法；(4)制定评价框架，以量化图像重建算法的性能，包括建立快速湍流模拟器和模式识别评价计划。在教育方面，该项目将为9-12年级的学生创建在线STEM课外项目，重点是计算机视觉和机器学习。在线内容交付的重点将在面对面会议困难的情况下实现远程教学，并允许跨地域教学。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Ground-based long-range passive imaging systems are one of the most critical sensing modalities in civilian and military applications, but they often suffer from distortions due to a turbulent atmosphere. In the presence of atmospheric turbulence, the captured images will become unstable, blurred, and ultimately the details of the objects will be lost. Without any mitigation, the degraded image quality will severely reduce our ability to make informed decisions, identify threat objects, navigate in unknown environments, and perform scientific measurements. Turbulence mitigation methods have been studied for decades, but most optics-based approaches require coherent light sources which are not always feasible, whereas post-processing methods are largely limited to stationary scenes. When an object of interest moves or when the camera is not stationary, both the capturing and restoration of the images remain very challenging tasks. The goal of this project is to develop a new computational imaging framework using the single-photon image sensors, by capturing and recovering moving objects distorted by turbulence. By accomplishing this goal, the project will fill the critical gap in the current imaging technology. This, in turn, will empower new imaging abilities for surveillance, navigation, defense, and remote sensing, all of which are critical to the economy and homeland security.The proposed project will develop methods to acquire very short exposure images, and new algorithms to recover the dynamic scenes. This involves a new type of image sensors that offer state-of-the-art photon resolving capability, which can operate with sufficient signal-to-noise at even extremely short exposures. Four research thrusts will be pursued: (i) to derive the theoretical limits of single-photon imaging through turbulence, by analyzing the photon statistics and its interaction with the atmosphere; (ii) to develop dynamic sampling mechanisms to acquire signals non-uniformly, and to develop optimal exposure controls to enable high-dynamic range imaging; (iii) to develop AI-enhanced image reconstruction algorithms; (iv) to develop an evaluation framework to quantify the performance of image reconstruction algorithms, including building fast turbulence simulators and pattern recognition evaluation schemes. On the education front, the project will create online STEM afterschool programs for 9-12th grade students, focusing on computer vision and machine learning. The emphasis of online content delivery will enable remote teaching when face-to-face meetings are difficult, and allow teaching beyond geographical boundaries.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.