Short-Exposure Imaging through Atmospheric Turbulence using Single Photon Image Sensors

地基远程被动成像系统是民用和军事应用中最关键的传感模式之一，但它们经常受到由于湍流大气而引起的失真。在大气湍流的存在下，捕获的图像会变得不稳定，模糊，最终会丢失物体的细节。 如果不采取任何缓解措施，图像质量下降将严重降低我们做出明智决策、识别威胁物体、在未知环境中导航以及进行科学测量的能力。湍流缓解方法已经研究了几十年，但大多数基于光学的方法需要相干光源，这并不总是可行的，而后处理方法在很大程度上限于静止场景。当感兴趣的对象移动时，或者当相机不是静止的时，图像的捕获和恢复仍然是非常具有挑战性的任务。该项目的目标是开发一种新的计算成像框架，使用单光子图像传感器，通过捕获和恢复被湍流扭曲的运动物体。通过实现这一目标，该项目将填补当前成像技术的关键空白。这反过来将为监视、导航、国防和遥感提供新的成像能力，所有这些都对经济和国土安全至关重要。拟议的项目将开发获取极短曝光图像的方法，以及恢复动态场景的新算法。这涉及到一种新型的图像传感器，它提供了最先进的光子分辨能力，即使在极短的曝光时间内也能以足够的信噪比工作。将开展四项研究：（一）通过分析光子统计及其与大气的相互作用，推导出通过湍流进行单光子成像的理论极限;（二）开发动态采样机制，以非均匀地获取信号，并开发最佳曝光控制，以实现高动态范围成像;（三）开发人工智能增强的图像重建算法;（iv）发展一个评估架构，以量化图像重建算法的表现，包括建立快速湍流模拟器和模式识别评估计划。在教育方面，该项目将为9- 12年级的学生创建在线STEM课后课程，重点是计算机视觉和机器学习。在线内容交付的重点将使远程教学时，面对面的会议是困难的，并允许教学超越地理界限。这个奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

Ground-based long-range passive imaging systems are one of the most critical sensing modalities in civilian and military applications, but they often suffer from distortions due to a turbulent atmosphere. In the presence of atmospheric turbulence, the captured images will become unstable, blurred, and ultimately the details of the objects will be lost. Without any mitigation, the degraded image quality will severely reduce our ability to make informed decisions, identify threat objects, navigate in unknown environments, and perform scientific measurements. Turbulence mitigation methods have been studied for decades, but most optics-based approaches require coherent light sources which are not always feasible, whereas post-processing methods are largely limited to stationary scenes. When an object of interest moves or when the camera is not stationary, both the capturing and restoration of the images remain very challenging tasks. The goal of this project is to develop a new computational imaging framework using the single-photon image sensors, by capturing and recovering moving objects distorted by turbulence. By accomplishing this goal, the project will fill the critical gap in the current imaging technology. This, in turn, will empower new imaging abilities for surveillance, navigation, defense, and remote sensing, all of which are critical to the economy and homeland security.The proposed project will develop methods to acquire very short exposure images, and new algorithms to recover the dynamic scenes. This involves a new type of image sensors that offer state-of-the-art photon resolving capability, which can operate with sufficient signal-to-noise at even extremely short exposures. Four research thrusts will be pursued: (i) to derive the theoretical limits of single-photon imaging through turbulence, by analyzing the photon statistics and its interaction with the atmosphere; (ii) to develop dynamic sampling mechanisms to acquire signals non-uniformly, and to develop optimal exposure controls to enable high-dynamic range imaging; (iii) to develop AI-enhanced image reconstruction algorithms; (iv) to develop an evaluation framework to quantify the performance of image reconstruction algorithms, including building fast turbulence simulators and pattern recognition evaluation schemes. On the education front, the project will create online STEM afterschool programs for 9-12th grade students, focusing on computer vision and machine learning. The emphasis of online content delivery will enable remote teaching when face-to-face meetings are difficult, and allow teaching beyond geographical boundaries.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.