CAREER: Mixed-State Computational Imaging and Light Transport Probing

计算成像是一个结合光学、电子学和计算处理来捕捉新形式视觉信息的领域。任何计算成像技术的理论基础都是它的光传播模型。视觉和图形学研究一般采用光传输的几何光学近似，光沿着直线传播，直到经历散射事件或被吸收。在小尺度上，光的波状特性变得更加明显，并且需要以额外的计算开销为代价将光建模为波。实际上，成像系统既不是完全非相干的（如几何光学），也不是完全相干的（如波动光学）。光更适合建模为一种状态混合：相干波的统计集合。该项目开发了一个全面的理论和相应的成像技术，以利用状态混合进行各种成像应用，包括开发新的计算显微镜、3D传感器和光学振动传感器。此外，该项目结合了非连贯和连贯环境中使用的技术，借鉴了多个不同研究团体开发的成像研究。此外，该项目将开发一个低成本、开源的投影仪相机平台，以轻松实现一系列重要的计算成像技术。该平台将用于教育各个层次的学生（研究生、本科生和K-12）。该项目解决了三个独立但紧密耦合的研究重点，集中在(1)混合状态传感，(2)混合状态照明和(3)混合状态场景分析。第一个推力探索在传感器上观察到的状态混合。许多成像技术依赖于入射光是相干的（如相干衍射成像），而不考虑入射光具有多个波长或偏振态。该推力开发了灵活的成像系统和重建算法，可以从多个相干场的非相干混合中独特地恢复复杂场。第二部分研究了一种基于编程混合状态照明的传输探测新形式。传输探测为相机提供了增强或减弱特定光路的能力。这一推动力开发了一种完全不同的探测方法，包括选择性干涉光路对，这在3D传感和直接光路和全局光路的分离中得到了应用。第三个推力使用混合状态场景分析技术来推断场景动态，包括光学放大和测量物体的难以察觉的振动。这项研究的一个重点是在更高的空间分辨率下，以更高效的方式，在多达六个运动维度上捕捉振动测量，提供更多更好的视觉场景信息。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Computational imaging is a field that combines optics, electronics, and computational processing to capture new forms of visual information. The theoretical underpinning of any computational imaging technique is its model for light propagation. Vision and graphics research generally adopts a geometric optics approximation of light transport, where light travels along straight lines until it undergoes a scattering event or gets absorbed. At small scales, the wave-like properties of light become far more noticeable and require modeling light as a wave at the cost of additional computational overhead. In practice, imaging systems are neither perfectly incoherent (as in geometric optics) nor completely coherent (as in wave optics). Light is more aptly modeled as a state mixture: a statistical ensemble of coherent waves. This project develops a comprehensive theory and corresponding imaging techniques to leverage state mixtures for a wide variety of imaging applications, including the development of new computational microscopes, 3D sensors, and optical vibration sensors. Moreover, this project connects techniques used in both the incoherent and coherent settings, drawing upon imaging research developed across multiple different research communities. In addition, the project will develop a low-cost, open-source projector-camera platform to easily implement a collection of important computational imaging techniques. This platform will be used to educate students at all levels (graduate, undergraduate, and K-12).This project tackles three independent, but tightly coupled, research thrusts focused on (1) mixed-state sensing, (2) mixed-state illumination, and (3) mixed-state scene analysis. The first thrust explores state mixtures observed at the sensor. Many imaging techniques depend on the incident light being coherent (such as in coherent diffraction imaging), and do not account for the incident light having multiple wavelengths or polarization states. This thrust develops flexible imaging systems and reconstruction algorithms that can uniquely recover complex fields from incoherent mixtures of multiple coherent fields. The second thrust investigates a new form of transport probing based on programming mixed-state illumination. Transport probing provides a camera the ability to enhance or attenuate specific light paths. This thrust develops a fundamentally different approach to probing that involves selectively interfering pairs of light paths, which has applications in 3D sensing and the separation of direct and global light paths. The third thrust uses mixed-state scene analysis techniques to infer scene dynamics, which includes optically amplifying and measuring the imperceptible vibrations of objects. A focus of this thrust is to capture vibration measurements at higher spatial resolutions, in a more light-efficient fashion, and across up to six dimensions of motion, providing more and better information about visual scenes.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.