CCSS: Uncertainty-Aware Computational Imaging in the Wild: a Bayesian Deep Learning Approach in the Latent Space

从显微镜和断层扫描到生物识别和监控，计算成像（CI）极大地扩展了人类在各个科学和工程领域的视觉能力。与需要昂贵光学器件的基于硬件的解决方案相比，由深度学习驱动的数据驱动或计算方法推动了支持AI的计算成像系统的发展。贝叶斯深度学习的新兴范式赋予下一代CI系统更大的灵活性来处理各种不确定性-无论它们是否与图像采集的对抗条件有关（例如，移动平台或恶劣天气）或人为的无意错误（例如，哈勃太空望远镜发射升空后出现故障的主镜）。开发具有不确定性意识的CI系统对科学探索（例如，在诸如微观和天文的极端尺度下成像）和我们的日常生活（例如，该项目旨在采用贝叶斯深度学习（BDL）方法来建模真实世界成像场景中的不确定性。该项目带来的一个重要的新见解是将基于流的不确定性核估计（用于似然建模）与潜在空间中基于内存的不确定性图像生成（用于先验建模）统一起来。在完全贝叶斯框架下，但使用重新参数化来简化建模过程，所提出的BDL极大地促进了在不确定性不可避免的现实场景中的退化学习和图像重建。该研究由三个任务组成：1）基于流的潜在空间非均匀核估计; 2）用于图像重建的内存增强深度生成模型; 3）超分辨率显微镜等深层组织成像的实际应用。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响力评审标准进行评估，被认为值得支持。

From microscopy and tomography to biometrics and surveillance, computational imaging (CI) has greatly expanded mankind’s vision capability in various scientific and engineering fields. In contrast to hardware-based solutions that require expensive optics, data-driven or computational approaches powered by deep learning have fueled the development of AI-enabled computational imaging systems. The emerging paradigm of Bayesian deep learning endows the next-generation CI systems more flexibility to handle various uncertainties in the wild – no matter whether they are related to the adversarial conditions of image acquisition (e.g., moving platform or bad weather) or unintentional mistakes made by humans (e.g., the mal-functioned main mirror of Hubble Space Telescope after the launch into the space). Developing uncertainty-aware CI systems have a wide range of impact on both scientific exploration (e.g., imaging at extreme scales such as microscopic and astronomical) and our daily lives (e.g., smartphone applications).This project aims at taking a Bayesian deep learning (BDL) approach to modeling uncertainty in real-world imaging scenarios. An important new insight brought about by this project is to unify the flow-based uncertainty kernel estimation (for likelihood modeling) with memory-based uncertainty image generation in latent space (for prior modeling). Under a fully Bayesian framework but using reparameterization to simplify the modeling process, the proposed BDL greatly facilitates both degradation learning and image reconstruction in the realistic scenario when uncertainty is inevitable. The proposed research consists of three tasks: 1) Flow-based Nonuniform Kernel Estimation in the Latent Space; 2) Memory-enhanced Deep Generative Models for Image Reconstruction; and 3) Real-world Applications in Deep Tissue Imaging such as Superresolution Microscopy.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.