CAREER: Mixed-State Computational Imaging and Light Transport Probing

计算成像是一个结合了光学、电子学和计算处理来捕捉新形式的视觉信息的领域。任何计算成像技术的理论基础都是它的光传播模型。视觉和图形研究通常采用光传输的几何光学近似，即光沿着直线传播，直到它经历散射事件或被吸收。在较小的比例下，光的波状属性变得更加明显，需要以额外的计算开销为代价将光建模为波。在实践中，成像系统既不是完全非相干的(如几何光学)，也不是完全相干的(如波动光学)。光更贴切地被建模为一种状态混合体：一组相干波的统计集合。该项目开发了一套全面的理论和相应的成像技术，以利用状态混合进行广泛的成像应用，包括开发新的计算显微镜、3D传感器和光学振动传感器。此外，该项目结合了在不连贯和连贯的环境中使用的技术，借鉴了跨多个不同研究社区开发的成像研究。此外，该项目将开发一个低成本、开源的投影仪-相机平台，以轻松实施一系列重要的计算成像技术。这个平台将用于教育所有级别的学生(研究生、本科生和K-12)。该项目解决三个独立但紧密耦合的研究主题，重点是(1)混合状态传感，(2)混合状态照明，和(3)混合状态场景分析。第一个推力探测在传感器上观察到的状态混合物。许多成像技术依赖于入射光是相干的(例如在相干衍射成像中)，并且不考虑具有多个波长或偏振态的入射光。这一推力开发了灵活的成像系统和重建算法，可以唯一地从多个相干场的非相干混合中恢复复杂的场。第二个推力研究了一种基于编程混合状态照明的新形式的传输探测。传输探测为相机提供了增强或衰减特定光路的能力。这种推力开发了一种根本不同的探测方法，包括选择性地干扰光路对，这在3D传感和直接光路和全局光路的分离中有应用。第三个推力使用混合状态场景分析技术来推断场景动力学，其中包括光学放大和测量物体的不可感知的振动。这一努力的重点是以更高的空间分辨率，以更省光的方式捕捉振动测量，并跨越多达六个运动维度，提供更多和更好的视觉场景信息。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Computational imaging is a field that combines optics, electronics, and computational processing to capture new forms of visual information. The theoretical underpinning of any computational imaging technique is its model for light propagation. Vision and graphics research generally adopts a geometric optics approximation of light transport, where light travels along straight lines until it undergoes a scattering event or gets absorbed. At small scales, the wave-like properties of light become far more noticeable and require modeling light as a wave at the cost of additional computational overhead. In practice, imaging systems are neither perfectly incoherent (as in geometric optics) nor completely coherent (as in wave optics). Light is more aptly modeled as a state mixture: a statistical ensemble of coherent waves. This project develops a comprehensive theory and corresponding imaging techniques to leverage state mixtures for a wide variety of imaging applications, including the development of new computational microscopes, 3D sensors, and optical vibration sensors. Moreover, this project connects techniques used in both the incoherent and coherent settings, drawing upon imaging research developed across multiple different research communities. In addition, the project will develop a low-cost, open-source projector-camera platform to easily implement a collection of important computational imaging techniques. This platform will be used to educate students at all levels (graduate, undergraduate, and K-12).This project tackles three independent, but tightly coupled, research thrusts focused on (1) mixed-state sensing, (2) mixed-state illumination, and (3) mixed-state scene analysis. The first thrust explores state mixtures observed at the sensor. Many imaging techniques depend on the incident light being coherent (such as in coherent diffraction imaging), and do not account for the incident light having multiple wavelengths or polarization states. This thrust develops flexible imaging systems and reconstruction algorithms that can uniquely recover complex fields from incoherent mixtures of multiple coherent fields. The second thrust investigates a new form of transport probing based on programming mixed-state illumination. Transport probing provides a camera the ability to enhance or attenuate specific light paths. This thrust develops a fundamentally different approach to probing that involves selectively interfering pairs of light paths, which has applications in 3D sensing and the separation of direct and global light paths. The third thrust uses mixed-state scene analysis techniques to infer scene dynamics, which includes optically amplifying and measuring the imperceptible vibrations of objects. A focus of this thrust is to capture vibration measurements at higher spatial resolutions, in a more light-efficient fashion, and across up to six dimensions of motion, providing more and better information about visual scenes.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.