CCSS: Uncertainty-Aware Computational Imaging in the Wild: a Bayesian Deep Learning Approach in the Latent Space

从显微镜和层析成像到生物特征识别和监视，计算成像(CI)极大地扩展了人类在各个科学和工程领域的视觉能力。与需要昂贵光学设备的基于硬件的解决方案相比，深度学习支持的数据驱动或计算方法推动了支持人工智能的计算成像系统的发展。贝叶斯深度学习的新兴范式赋予下一代CI系统更大的灵活性，以处理野外的各种不确定性-无论这些不确定性是与图像获取的不利条件(例如，移动的平台或恶劣天气)有关，还是与人类犯下的无意错误(例如，哈勃太空望远镜在发射进入太空后出现故障)有关。开发不确定性感知的CI系统对科学探索(例如，极端尺度的成像，如显微镜和天文)和我们的日常生活(例如，智能手机应用程序)都有广泛的影响。该项目旨在采用贝叶斯深度学习(BDL)的方法来建模真实世界成像场景中的不确定性。该项目带来的一个重要的新见解是将基于流的不确定性核估计(用于似然建模)和基于记忆的潜在空间不确定性图像生成(用于先验建模)统一起来。在完全贝叶斯框架下，通过重新参数化简化建模过程，在不确定性不可避免的真实场景中，BDL极大地方便了退化学习和图像重建。建议的研究包括三个任务：1)潜在空间中基于流的非均匀核估计；2)记忆增强的深度生成图像重建模型；3)在深度组织成像中的真实应用，如超分辨率显微镜。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

From microscopy and tomography to biometrics and surveillance, computational imaging (CI) has greatly expanded mankind’s vision capability in various scientific and engineering fields. In contrast to hardware-based solutions that require expensive optics, data-driven or computational approaches powered by deep learning have fueled the development of AI-enabled computational imaging systems. The emerging paradigm of Bayesian deep learning endows the next-generation CI systems more flexibility to handle various uncertainties in the wild – no matter whether they are related to the adversarial conditions of image acquisition (e.g., moving platform or bad weather) or unintentional mistakes made by humans (e.g., the mal-functioned main mirror of Hubble Space Telescope after the launch into the space). Developing uncertainty-aware CI systems have a wide range of impact on both scientific exploration (e.g., imaging at extreme scales such as microscopic and astronomical) and our daily lives (e.g., smartphone applications).This project aims at taking a Bayesian deep learning (BDL) approach to modeling uncertainty in real-world imaging scenarios. An important new insight brought about by this project is to unify the flow-based uncertainty kernel estimation (for likelihood modeling) with memory-based uncertainty image generation in latent space (for prior modeling). Under a fully Bayesian framework but using reparameterization to simplify the modeling process, the proposed BDL greatly facilitates both degradation learning and image reconstruction in the realistic scenario when uncertainty is inevitable. The proposed research consists of three tasks: 1) Flow-based Nonuniform Kernel Estimation in the Latent Space; 2) Memory-enhanced Deep Generative Models for Image Reconstruction; and 3) Real-world Applications in Deep Tissue Imaging such as Superresolution Microscopy.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.