CAREER: Co-Optimized Sensing and Reconstruction for Next-Generation Computational Cameras

成像技术在推动科学发展方面发挥着至关重要的作用。然而，随着科学不断突破知识的界限，传统的成像传感器正在达到它们的极限。例如，传统的望远镜不能大到足以分辨黑洞的阴影；传统的显微镜无法看到透明的细胞；传统的相机由于散射而无法用来研究云的核心。只有通过用计算算法取代光学的计算相机的出现，才有可能打破这些基本限制；这种新的范式转变使成像过程成为可能，而这些过程以前对于传统的光学成像是不可行的。计算机相机的全部潜力还远远没有实现；到目前为止，它们主要是通过人类的聪明才智来识别和开发的。因此，计算成像管道往往明显得不到优化，毫无疑问，许多这样的“相机”甚至还没有被识别出来。开发下一代计算相机需要从根本上改变在成像管道设计中依赖人类直觉和过于简化的模型。该项目旨在开发基于现代学习的方法来联合优化计算摄像机流水线中的传感器和算法设计，以自动发现新的成像策略。该项目的目标是开发一个数据驱动的、可推广的学习框架，以解决计算成像流水线中的传感器设计和重建算法的联合优化。可概括的协同设计框架将被开发成容易地结合领域知识和尊重物理约束。与领域专家合作，研究人员将研究这一框架在从天文成像到地震成像的一系列问题上的应用。研究人员将在四个方面进行基础工作：1)单次概率联合设计以结合重建方法优化传感器设计，2)在线序贯概率联合设计以特定目标的先前测量为条件优化下一次传感器测量，3)与随机演变目标的联合设计，以及4)与不匹配正向模型的联合设计。研究人员将利用机器学习、信号处理、优化、应用数学和控制方面的新兴计算技术和机械来有效地共同优化计算成像管道。这项研究将改变识别和开发新型成像管道的方式，并将导致新方法的开发，这些方法将影响一系列重要的成像问题，包括天文、医学、地震和显微成像。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Imaging technology plays a critical role in advancing science. However, as science continues to push the boundaries of knowledge, traditional imaging sensors are reaching their limits. For example, traditional telescopes cannot be constructed large enough to resolve a black hole’s shadow; traditional microscopes are not able to see transparent cells, and traditional cameras cannot be used to study the inner core of a cloud due to scattering. Breaking these fundamental limits has only been possible through the emergence of computational cameras, which replace optics with computational algorithms; this new paradigm shift has enabled image formation processes that were previously infeasible for conventional optical imaging. The full potential of computational cameras is far from being realized; thus far they have primarily been identified and developed though human ingenuity. Consequently, computational imaging pipelines are often significantly under-optimized, and there is no doubt that many such “cameras” have yet to even be identified. Developing the next generation of computational cameras requires a fundamental shift away from relying on human intuition and overly simplified models in the design of imaging pipelines. This project aims to develop modern learning-based approaches to jointly optimize sensor and algorithm designs in computational camera pipelines in order to automatically discover new imaging strategies.The objective of this project is to develop a data-driven, generalizable learning framework that solves for a jointly optimized sensor design and reconstruction algorithm for computational imaging pipelines. The generalizable co-design framework will be developed to easily incorporate domain knowledge and respect physical constraints. In collaboration with domain-experts, the investigator will study the application of this framework to problems ranging from astronomical imaging to seismic imaging. The investigator will pursue fundamental work in four areas: 1) single-shot probabilistic co-design to optimize sensor design jointly with reconstruction methods, 2) online sequential probabilistic co-design for optimizing the next sensor measurement conditioned on previous measurements for a particular target, 3) co-design with a stochastically evolving target, and 4) co-design with a mismatched forward model. The investigator will make use of emerging computational techniques and machinery in machine learning, signal processing, optimization, applied math, and controls to efficiently co-optimize the computational imaging pipeline. This research will transform the way novel imaging pipelines are identified and developed, and will result in the development of new methods that will impact a wide array of important imaging problems, including astronomical, medical, seismic, and microscopic imaging.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.