CRII: RI: General Surface Reconstruction via Polarized Computational Imaging

对真实的世界进行完整而忠实的数字重建对许多应用至关重要，如虚拟和增强现实、全息渲染、自动驾驶、数字文化遗产保护、虚拟建筑和室内设计。物理世界由不同材料制成的物体组成，这些材料赋予它们不同的外观。扫描具有高度视觉多样性的场景是具有挑战性的。所有现有的3D扫描系统都限于它们可以工作的特定类型的表面。例如，有些只适用于哑光表面，而有些则是专为高反射的。因此，需要设计一种通用的解决方案，用于数字化具有不同外观的真实世界表面。该项目将产生一个综合解决方案，用于重建一般表面，无论其外观如何。该成果将大大简化3D扫描程序，并允许公众广泛使用这项技术。此外，该研究将被纳入路易斯安那州立大学的计算机视觉和计算成像新课程的课程中。该项目将开发鲁棒、快速和精确的基于偏振的成像系统，用于重建从漫射到纯粹镜面反射的一般表面。本研究的动机是光的偏振态嵌入光传输的基本信息。为了实现这一目标，本项目有两个研究方向：1）制定一个包含偏振的光传输框架和2）开发一个实用的系统，用于重建一般表面。第一个推力将描述光与各种反射特性的表面相互作用后偏振的转变。该框架为系统开发提供了理论基础。第二个目标是利用偏振传感器和光学器件的最新进展，设计新的可编程偏振编码光源和相机。新的基于物理的三维表面重建算法将通过偏振分析开发。该项目的成果有望突破现有3D扫描技术在普遍适用性方面的主要局限性。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Complete and faithful digital reconstruction of the real world is critical to many applications, such as virtual and augmented reality, holographic rendering, autonomous driving, digital cultural heritage preservation, virtual architecture and interior design. The physical world consists of objects made with different materials that give them diverse appearances. Scanning a scene with high visual diversity is challenging. All existing 3D scanning systems are limited to specific types of surfaces that they can work for. For example, some only works for the matte surfaces while some are designed for highly reflective ones. Therefore, there is need to devise a generic solution for digitalizing the real-world surfaces with diverse appearances. This project will result in an integrated solution for reconstructing general surfaces, regardless of their appearances. The outcome will significantly simplify the 3D scanning procedure and allow wide use of this technology by the general public. In addition, this research will be integrated into the curricula of new courses on computer vision and computational imaging at Louisiana State University.This project will develop robust, fast, and accurate polarization-based imaging system for reconstructing general surfaces, ranging from diffuse to purely specular. This research is motivated by the fact that the polarization state of light embeds essential information on light transport. To achieve the goal, this project has two research thrusts: 1) formulate a polarization-inclusive light transport framework and 2) develop a practical system for reconstructing general surfaces. The first thrust will characterize the transformation of polarization after light interacts with surfaces of various reflectance properties. This framework provides the theoretical foundation for system development. The second thrust will design new programmable polarization-coded light sources and cameras, by leveraging recent advances in polarization sensors and optics. New physics-based 3D surface reconstruction algorithms will be developed through polarization analysis. Result of this project is expected to breakthrough major limitations of existing 3D scanning technology on general applicability.applicability.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.