CAREER: Seeing Through Atmospheric Turbulence: Image Restoration and Understanding using Deep Convolutional Neural Networks

大气湍流会造成大气折射率在空间和时间上的随机波动，从而显著降低远程成像系统获取的图像质量。折射率的变化导致捕获的图像在几何上扭曲和模糊。这些失真对后续计算机视觉算法(例如目标检测和识别)的性能产生不利影响。因此，对大气湍流引起的图像视觉退化进行补偿是非常重要的。基于自适应光学的技术可以用来补偿图像中的湍流效应。然而，它们需要庞大、复杂和昂贵的硬件。另一方面，基于图像处理的方法是廉价和有效的。该方法的思想是将湍流退化图像恢复问题归结为一个非线性回归问题，其中最优参数从综合生成的数据中学习。作为函数逼近器，我们建议使用深度卷积神经网络。这一职业项目的目标是开发以数据为基础、以学习为基础的方法，恢复和理解因大气湍流而退化的图像。该项目将有助于开设关于深度学习和形象恢复的新的本科生/研究生课程。此外，该项目将影响许多多样性推广活动，包括确保妇女、少数群体和弱势群体广泛参与的具体推广活动。我们的研究将为恢复和理解因大气湍流而退化的图像/视频提供一个全面的框架。我们将通过开发新的端到端可训练的深卷积神经网络和相应的损失函数，在监督、半监督和非监督图像恢复技术领域进行重大创新。此外，还将开发基于域转移学习的方法，使对象检测和分割等计算机视觉算法适用于湍流退化的图像。该项目将开发的算法将显著提高远程可见光和红外成像系统收集的图像和视频的质量，并将导致对因大气湍流而退化的图像的理解。建议的方法将在遥感、远程监视、光通信、天文学、道路交通监测、水下成像和自主导航的许多应用中有用。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Atmospheric turbulence can significantly degrade the quality of images acquired by long-range imaging systems by causing spatially and temporally random fluctuations in the index of refraction of the atmosphere. Variations in the refractive index causes the captured images to be geometrically distorted and blurry. These distortions adversely affect the performance of subsequent computer vision algorithms such as object detection and recognition. Hence, it is important to compensate for the visual degradation in images caused by atmospheric turbulence. Adaptive optics-based techniques can be used to compensate for turbulence effects in images. However, they require large, complex and expensive hardware. On the other hand, image processing-based approaches are cheap and effective. The idea of the proposed approach is to pose the turbulence degraded image restoration problem as a nonlinear regression problem, where the optimal parameters are learned from synthetically generated data. As a function approximator, we propose to use deep convolutional neural networks. The goal of this CAREER project is to develop data-driven learning-based approaches for restoration and understanding of images degraded by atmospheric turbulence. This project will help create new undergraduate/graduate courses on Deep Learning and Image Restoration. Further, this project will impact many diversity outreach activities, including specific outreach to ensure broad participation of women, minorities and disadvantaged groups.Our research will provide a comprehensive framework for restoring and understanding images/videos degraded by atmospheric turbulence. We will significantly innovate in the areas of supervised, semi-supervised and unsupervised image restoration techniques by developing novel end-to-end trainable deep convolutional neural networks and corresponding loss functions. Furthermore, domain transfer learning-based methods for adapting computer vision algorithms such as object detection and segmentation to turbulence-degraded images will be developed. Algorithms that will be developed in this project will significantly enhance the quality of images and videos collected by long-range visible and infrared imagining systems, and will result in improved understanding of images degraded by atmospheric turbulence. The proposed methods will be useful in many applications of remote sensing, long-range surveillance, optical communications, astronomy, road traffic monitoring, underwater imaging, and autonomous navigation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.