CAREER: Generative Physical Modeling for Computational Imaging Systems

职业：计算成像系统的生成物理建模

• 批准号: 2239687
• 负责人: Jonathan Tamir
• 金额: $ 57.67万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-01 至 2028-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2239687&HistoricalAwards=false
• 关键词: CAREER; Generative; Physical; Modeling; Computational

Imaging devices, from microscopes to medical-imaging scanners, have transformed science and diagnostic medicine by providing safe and noninvasive techniques for observing the environment and seeing inside the body. However, imaging-system design choices are often based on idealized operating conditions, resulting in highly promising "benchtop demonstrations" that quickly degrade when deployed outside of a controlled laboratory environment. This project aims to develop a framework for robust computational-imaging system design, where the data acquisition and data processing are jointly designed in tandem to address the mismatch between the idealized performance of physical systems and their real-world behavior. The research aims to enable reliable imaging in dynamically evolving clinical and scientific-research settings, for example by reducing acquisition times, imaging moving objects, and compensating for system imperfections. The project also involves the dissemination of results through tutorials, webinars, and reproducible code, as well as community-outreach initiatives based on hands-on interactive demos of computational-imaging systems with the goal of broadening participation in engineering and science. The objective of this project is to develop a machine-learning framework for robust computational-imaging system design with principled methods and theoretical performance guarantees. Central to the approach is the separation between modeling the physical system, which is governed by established imaging physics, and modeling the statistical prior knowledge, which is learned from imaging data. The project involves four research thrusts, each intended to tackle a specific problem foundational to computational-imaging system robustness and reliability: 1) adapting to dynamically changing operating conditions; 2) accounting for motion during image acquisition; 3) learning directly from noisy, sub-sampled measurements; and 4) improving resiliency to system imperfections. The project is founded on a fruitful synthesis between imaging physics, signal processing, optimization, and machine learning. Image acquisition and recovery will be formalized using newly developed deep generative physical models instead of poorly understood and non-generalizable black-box deep-learning methods. In addition to impacting applications including microscopic, medical, and automotive imaging, the work will also inform the design of non-imaging systems, such as in wireless communications, where similar challenges exist surrounding the deployment of deep-learning-based algorithms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

从显微镜到医学成像扫描仪的成像设备，通过提供安全和非侵入性的技术来观察环境和观察身体内部，改变了科学和诊断医学。然而，成像系统的设计选择往往是基于理想化的操作条件，导致非常有前途的“台式演示”，当部署在受控的实验室环境之外时会迅速退化。该项目旨在为强大的计算成像系统设计开发一个框架，其中数据采集和数据处理联合设计，以解决物理系统的理想性能与其现实世界行为之间的不匹配。该研究旨在通过减少采集时间、对移动物体进行成像以及补偿系统缺陷等方式，在动态发展的临床和科学研究环境中实现可靠的成像。该项目还涉及通过教程，网络研讨会和可复制代码传播结果，以及基于计算成像系统的动手互动演示的社区外展计划，旨在扩大工程和科学的参与。该项目的目标是开发一个机器学习框架，用于具有原则性方法和理论性能保证的鲁棒计算成像系统设计。该方法的核心是物理系统建模（由已建立的成像物理学控制）和统计先验知识建模（从成像数据中学习）之间的分离。该项目涉及四个研究方向，每个方向都旨在解决计算成像系统鲁棒性和可靠性的基础问题：1）适应动态变化的操作条件; 2）在图像采集过程中考虑运动; 3）直接从噪声，子采样测量中学习; 4）提高系统缺陷的弹性。该项目建立在成像物理学、信号处理、优化和机器学习之间富有成效的综合基础上。图像采集和恢复将使用新开发的深度生成物理模型来形式化，而不是理解不深和不可推广的黑盒深度学习方法。除了影响包括显微镜、医疗和汽车成像在内的应用外，这项工作还将为无线通信等非成像系统的设计提供信息，这些系统在部署基于深度学习的算法方面存在类似的挑战。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。