RI: Small: New Optical Frontiers for 4D Computational Imaging Systems

成像技术是智能感知和推理世界的核心技术。该项目通过支持开发具有根本优越性能的“4D镜头”设计的广阔新空间，推动了成像领域的发展。该研究团队开发了一种技术，可以记录和处理关于穿过透镜的光的更丰富的数据，从而提高光学图像质量的计算能力。研究小组还重新设计了镜头本身，降低了光学要求，并在亮度、尺寸、重量、变焦和成本等方面达到了前所未有的性能。该项目扩展了各种成像应用的前沿，包括3D成像、摄影、科学成像、投影、光刻、虚拟和增强现实耳机、显微镜、安全和国防。本研究发展了设计和优化所描述的四维透镜的理论、算法和工具。实现技术转变的假设是，新透镜可以用于新兴的计算成像系统，这些系统可以处理全4D光场（所有光线的集合），而不仅仅是2D图像。该项目在理论上和算法上解决了几个智力挑战。首先，该研究建立了一个长期存在的透镜设计问题的新公式，不仅考虑二维光学图像质量，而且考虑端到端四维成像和处理后的计算图像质量。新配方显著增加了评估和优化透镜设计的复杂性。其次，探索和绘制这种方法开辟的镜头设计空间的新前沿，了解其生长特征和最终极限。新的设计方法有可能从根本上破坏成像能力，因为它使镜头技术随着像素和处理技术的不断改进而呈指数级增长。

Imaging is a core technology that feeds into intelligent sensing and reasoning about the world. This project advances the frontiers of imaging by supporting development of a broad, new space of "4D lens" designs with fundamentally superior performance. The research team develops technologies to record and process richer data about light flowing through the lens, enabling computational enhancement over the optical image quality. The research team also redesigns the lens itself, lowering the optical requirements and pushing to unprecedented performance in figures of merit such as brightness, size, weight, zoom and cost. This project extends the frontier of what is possible in diverse imaging applications, including 3D imaging, photography, scientific imaging, projection, lithography, and headsets for virtual and augmented reality, microscopy, security and defense. This research develops the theory, algorithms and tools for designing and optimizing the described 4D lenses. The enabling technical shift is the assumption that the new lenses can be used in the emerging class of computational imaging systems that work with the full 4D light field (set of all rays), rather than simply a 2D image. This project addresses several intellectual challenges theoretically and algorithmically. First, the research establishes a new formulation of the long-standing lens design problem, considering not just 2D optical image quality, but rather the computed image quality after end-to-end 4D imaging and processing. The new formulation significantly increases the complexity of evaluating and optimizing lens designs. Secondly, the research explores and maps the new frontier of lens design space that this approach opens up, to understand its growth characteristics and ultimate limits. The new design approach has potential for fundamental disruption of imaging capability, because it enables lens technology to improve continuously with technology for pixels and processing, which grow exponentially.