CAREER: Towards Polarimetric Visual Understanding

偏振光在我们周围很常见。由于大多数视觉推理算法都利用图像的颜色和亮度，因此偏振在计算机视觉中的使用尚未充分发挥其潜力。这个CAREER项目将研究如何利用表面反射偏振光的方式来帮助更有效地识别物体。该项目将解决以下两个问题：在与各种类型的表面相互作用后，光的偏振如何变化？偏振光可以告诉我们什么样的物体，它已经相互作用？解决这些问题可以通过加强机器视觉系统的几何和语义场景理解能力来显著改善机器视觉系统，其中形状被描述和命名。该项目将为自主导航和智能制造带来新的基于偏振的视觉系统。该项目还将为通过散射介质（包括水、雾、霾、云和人体组织）进行成像和传感的挑战性问题提供新的解决方案。该项目将研究偏振光传输，并通过综合分析解决偏振视觉理解问题。研究团队将推导出偏振表面反射率和体积光传输的理论模型，设计用于场景理解的视觉推理算法，并开发用于真实世界数据采集的计算成像系统。具体而言，该项目将首先研究局部表面相互作用下的偏振态转移。将采用基于物理的分层材料模型来表征直接镜面反射之外的复杂表面散射效应。然后，该项目将研究偏振光通过空间体积的传输。一个自下而上的方法将采取发展一个基于体素的光传输模型，以减轻不适定的传输矩阵分解问题。该项目还将研究混浊介质的挑战性场景，这可能导致通过散射介质成像的新技术。利用偏振光传输模型，视觉推理算法将被设计和开发用于理解场景属性，如深度，形状，反射率和语义组成。该项目将开发用于获取真实世界数据的新型计算成像系统，以验证和分析我们提出的模型和算法。本项目将收集各种表面的高精度偏振反射率，并与研究界共享。该项目的其他更广泛的影响包括将研究成果整合到现有的和新的计算机视觉课程中。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Polarized light is very common in our surroundings. As most visual inference algorithms leverage the color and brightness of images, the use of polarization in computer vision has not yet reached its full potential. This CAREER project will study how the way surfaces reflect polarized light can be used to help recognize objects more effectively. The project will address the following two questions: How does the polarization of light change after interacting with various types of surfaces? What can polarized light tell us about the kinds of objects that it has interacted with? Addressing these questions can lead to significant improvements in machine vision systems by strengthening their capability to do geometric and semantic scene understanding in which shapes are described and named. This project will result in new polarization-based vision systems for autonomous navigation and smart manufacturing. It will also produce novel solutions to the challenging problems of imaging and sensing through scattering media, including water, fog, haze, clouds, and bodily tissues.This project will investigate polarimetric light transport and tackle the problem of polarimetric visual understanding through analysis by synthesis. The research team will derive theoretical models for polarimetric surface reflectance and volumetric light transport, design visual inference algorithms for scene understanding, and develop computational imaging systems for real-world data acquisition. Specifically, the project will first study the transfer of polarization state upon local surface interactions. Physics-based layered material models will be adopted for characterizing complex surface scattering effects beyond direct mirror reflection. The project will then study the transport of polarized light through a spatial volume. A bottom-up approach will be taken to develop a voxel-based light transport model to mitigate the ill-posed transport matrix decomposition problem. The project will also study the challenging scenarios of turbid media, which could result in new techniques for imaging through scattering media. Leveraging the polarimetric light transport models, visual inference algorithms will be designed and developed for understanding scene properties, such as depth, shape, reflectance, and semantic composition. This project will develop novel computational imaging systems for acquiring real-world data to validate and analyze our proposed models and algorithms. A high precision polarimetric reflectance of various surfaces will be collected in this project and shared with the research community. Other broader impacts of this project include integrating the research results into existing and new course curricula on computer vision.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.