RI: Small: New Optical Frontiers for 4D Computational Imaging Systems

成像是一项核心技术，它为关于世界的智能传感和推理提供了支持。该项目通过支持具有从根本上优越的性能的“4D镜头”设计的广阔的新空间的开发，推动了成像的前沿。研究团队开发了记录和处理有关流经透镜的光的更丰富数据的技术，从而实现了对光学图像质量的计算增强。研究团队还重新设计了镜头本身，降低了对光学的要求，并在亮度、大小、重量、变焦和成本等方面取得了前所未有的性能。该项目扩展了各种成像应用的前沿，包括3D成像、摄影、科学成像、投影、光刻以及用于虚拟和增强现实、显微镜、安全和防御的耳机。这项研究开发了用于设计和优化所述4D透镜的理论、算法和工具。促成技术转变的是这样一种假设，即新的透镜可以用于新兴的计算成像系统类别，这些系统使用完整的4D光场(所有光线的集合)，而不仅仅是2D图像。这个项目从理论和算法上解决了几个智力挑战。首先，研究建立了长期存在的透镜设计问题的新公式，不仅考虑了2D光学图像质量，而且考虑了端到端4D成像和处理后的计算图像质量。新配方显著增加了评估和优化镜头设计的复杂性。其次，探索和绘制了这种方法开辟的镜片设计空间的新前沿，以了解其成长特征和终极极限。新的设计方法有可能从根本上颠覆成像能力，因为它使镜头技术随着像素和处理技术的不断改进而不断改进，像素和处理技术呈指数级增长。

Imaging is a core technology that feeds into intelligent sensing and reasoning about the world. This project advances the frontiers of imaging by supporting development of a broad, new space of "4D lens" designs with fundamentally superior performance. The research team develops technologies to record and process richer data about light flowing through the lens, enabling computational enhancement over the optical image quality. The research team also redesigns the lens itself, lowering the optical requirements and pushing to unprecedented performance in figures of merit such as brightness, size, weight, zoom and cost. This project extends the frontier of what is possible in diverse imaging applications, including 3D imaging, photography, scientific imaging, projection, lithography, and headsets for virtual and augmented reality, microscopy, security and defense. This research develops the theory, algorithms and tools for designing and optimizing the described 4D lenses. The enabling technical shift is the assumption that the new lenses can be used in the emerging class of computational imaging systems that work with the full 4D light field (set of all rays), rather than simply a 2D image. This project addresses several intellectual challenges theoretically and algorithmically. First, the research establishes a new formulation of the long-standing lens design problem, considering not just 2D optical image quality, but rather the computed image quality after end-to-end 4D imaging and processing. The new formulation significantly increases the complexity of evaluating and optimizing lens designs. Secondly, the research explores and maps the new frontier of lens design space that this approach opens up, to understand its growth characteristics and ultimate limits. The new design approach has potential for fundamental disruption of imaging capability, because it enables lens technology to improve continuously with technology for pixels and processing, which grow exponentially.