RI: Small: Collaborative Research: Dynamic Light Transport Acquisition and Applications to Computational Illumination

点击翻译按钮获取中文摘要

Cameras that view a dynamic scene typically capture interactions of moving objects with light. Computer vision algorithms can use these measurements to infer properties of these objects, such as depth, motion and appearance. However, there is a subtler, richer visual back-story that occurs as an object moves in a scene, and usually these effects are ignored in traditional algorithms, sometimes causing errors. This project studies all the interactions of light with dynamic scenes, which we term as dynamic light transport, and the goal is to understand and recover effects such as multiple reflections and scattering as objects move in a scene. The innovations of the project include new computational cameras and projectors to capture light transport for dynamic scenes, and to explore new physics-based and data-driven algorithms to exploit this information for improved computer vision and graphics applications. The project further seeks to include broadening access to computing education and research through curriculum material and capstone experiences which emphasize the intersection of light transport and digital media as well as outreach to middle and high school students in summer programs to discover imaging and optics applications.This research focuses on designing new light transport acquisition frameworks to capture dynamic scenes, characterization of dynamic light transport properties including sparsity and low-rank, and algorithms to exploit this information for computer vision applications. In particular, the project focuses on three main objectives. The first is design of an MEMs-based optical scanner coupled with high frame rate cameras to capture the full set of light transport paths at extremely fast timescales. The second contribution is new algorithms for adaptive light transport sampling using both physics-based and data-driven priors for light transport interpolation via generalized light transport flow. Finally, the project will provide applications of dynamic light transport for 3D scanning of deformable, moving, and specular objects. These innovations are evaluated in an integrated testbed via the optical scanner and the collection of a dataset of dynamic light transport for real-world scenes.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.