Data-Driven Modeling of Shape, Reflection, and Interreflection

该项目的目标是开发强大的算法来重建现实世界场景的3D形状和反射特性。该方法的显著特点是对全局光传输进行建模，考虑了未知场景中光与表面的真实反射和全局互反射。模拟真实的光传输是计算机视觉中一个重要的开放问题，因为日常材料以复杂的方式反射光（例如，木材，头发，天鹅绒），并且因为几何复杂场景的外观受到互反射的强烈影响。这个项目试图在一个非常普遍的环境中建立反射模型，在这个环境中，场景的形状和反射属性是未知的，也是不受约束的。互反射的建模将重点放在漫射散射上，并充分考虑光在场景中的全局传播。为了实现这一目标，引入了一种新的计算框架，该框架使用可以直接从照片中捕获的光传输数据驱动模型，并恢复场景结构，而无需复杂的模拟或优化底层物理过程。这种数据驱动模型的一个关键优势是，它们可以在非常一般的条件下健壮地工作。提出的工作开辟了3D形状传感技术的新途径，这是一个广泛应用于机器人，可视化，制图，航空成像和虚拟现实的问题。在真实物体上捕获材料模型的能力将提高计算机图形学的真实感，并影响娱乐、可视化和视觉媒体交流等应用。该项目将涉及本科生和研究生的研究项目，其结果将是一套成果和工具，这些成果和工具将被广泛传播，并纳入华盛顿大学的研究项目和教育倡议。

The goal of this project is to develop robust algorithms for reconstructing the 3D shape and reflectance properties of real world scenes. The distinguishing feature of the proposed approach is the modeling of global light transport, taking into account both realistic reflection and global interreflection of light with surfaces in the unknown scene. Modeling realistic light transport is a open problem with major importance in computer vision, due to the fact that everyday materials reflect light in complex ways (e.g., wood, hair, velvet), and because the appearance of geometrically complex scenes is strongly affected by interreflection. This project seeks to model reflection in a very general setting, where the shape and the reflectance properties of the scene are both unknown and unconstrained. Modeling of interreflection will focus on diffuse scattering, and fully account for global light propagation through the scene. Toward this objective, a new computational framework is introduced that uses data-driven models of light transport that can be captured directly from photographs, and recovers scene structure without the need for complex simulations or optimizations of the underlying physical process. A key advantage of such data-driven models is that they work robustly in very general conditions.The proposed work opens up new avenues in 3D shape sensing technology, a problem with widespread applications in robotics, visualization, mapping, aerial imaging, and virtual reality. The ability to capture material models on real objects will improve realism in computer graphics, and impacts applications such as entertainment, visualization, and the communication of visual media. The project will involve undergraduate and graduate research projects and the outcome will be a set of results and tools that will be broadly disseminated and incorporated into research projects and educational initiatives at the University of Washington.