CAREER: Towards Polarimetric Visual Understanding

偏振光在我们的环境中非常常见。由于大多数视觉推理算法利用图像的颜色和亮度，极化在计算机视觉中的使用尚未达到其全部潜力。这个职业项目将研究如何利用表面反射偏振光的方式来帮助更有效地识别物体。该项目将解决以下两个问题：光的偏振在与各种类型的表面相互作用后如何变化？关于它与之相互作用的物体的种类，偏振光能告诉我们什么？解决这些问题可以通过增强机器视觉系统对几何和语义场景的理解能力来显著改进机器视觉系统，在场景中描述和命名形状。该项目将为自主导航和智能制造带来新的基于偏振的视觉系统。它还将产生新的解决方案，以解决通过散射介质成像和传感的挑战性问题，包括水、雾、雾霾、云和身体组织。本项目将研究偏振光传输，并通过综合分析解决偏振视觉理解问题。研究小组将推导出偏振表面反射率和体积光传输的理论模型，设计用于场景理解的视觉推理算法，并开发用于真实世界数据采集的计算成像系统。具体地说，该项目将首先研究局部表面相互作用时偏振态的转移。基于物理的层状材料模型将被用来描述超出直接镜面反射的复杂表面散射效应。然后，该项目将研究偏振光在空间体积中的传输。采用自下而上的方法建立了一个基于体素的光传输模型，以缓解传输矩阵的病态分解问题。该项目还将研究混浊介质的挑战性场景，这可能导致通过散射介质成像的新技术。利用偏振光传输模型，将设计和开发视觉推理算法来理解场景属性，如深度、形状、反射率和语义组成。该项目将开发用于获取真实世界数据的新型计算成像系统，以验证和分析我们提出的模型和算法。这个项目将收集各种表面的高精度偏振反射比，并与研究社区共享。该项目的其他更广泛的影响包括将研究成果整合到现有的和新的计算机视觉课程中。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Polarized light is very common in our surroundings. As most visual inference algorithms leverage the color and brightness of images, the use of polarization in computer vision has not yet reached its full potential. This CAREER project will study how the way surfaces reflect polarized light can be used to help recognize objects more effectively. The project will address the following two questions: How does the polarization of light change after interacting with various types of surfaces? What can polarized light tell us about the kinds of objects that it has interacted with? Addressing these questions can lead to significant improvements in machine vision systems by strengthening their capability to do geometric and semantic scene understanding in which shapes are described and named. This project will result in new polarization-based vision systems for autonomous navigation and smart manufacturing. It will also produce novel solutions to the challenging problems of imaging and sensing through scattering media, including water, fog, haze, clouds, and bodily tissues.This project will investigate polarimetric light transport and tackle the problem of polarimetric visual understanding through analysis by synthesis. The research team will derive theoretical models for polarimetric surface reflectance and volumetric light transport, design visual inference algorithms for scene understanding, and develop computational imaging systems for real-world data acquisition. Specifically, the project will first study the transfer of polarization state upon local surface interactions. Physics-based layered material models will be adopted for characterizing complex surface scattering effects beyond direct mirror reflection. The project will then study the transport of polarized light through a spatial volume. A bottom-up approach will be taken to develop a voxel-based light transport model to mitigate the ill-posed transport matrix decomposition problem. The project will also study the challenging scenarios of turbid media, which could result in new techniques for imaging through scattering media. Leveraging the polarimetric light transport models, visual inference algorithms will be designed and developed for understanding scene properties, such as depth, shape, reflectance, and semantic composition. This project will develop novel computational imaging systems for acquiring real-world data to validate and analyze our proposed models and algorithms. A high precision polarimetric reflectance of various surfaces will be collected in this project and shared with the research community. Other broader impacts of this project include integrating the research results into existing and new course curricula on computer vision.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.