Transport-Aware Imaging for Computer Vision

点击翻译按钮获取中文摘要

The main goal of my research program for the next five years will be to study, design, and use a new class of computational cameras whose key property is that they are "transport-aware." Unlike conventional cameras which record all incident light, transport-aware cameras can be programmed to block some of that light, based on the actual 3D paths it followed through a scene. Transport-aware cameras use a programmable light source for illumination and a programmable sensor mask for imaging, and were pioneered by my research group three years ago. Live video from a transport-aware camera can offer a very unconventional view of our everyday world in which refraction and scattering can be selectively blocked or enhanced, visual structures too subtle to notice with the naked eye can become apparent, and object surfaces can be reconstructed in 3D under challenging conditions well beyond the state of the art. Our initial work on harnessing their power brings together computer vision, computer graphics, numerical optimization and geometric optics to demonstrate never-seen-before capabilities with potential applications in 3D shape acquisition; robust time-of-flight imaging; visual material analysis; scene understanding; robotic navigation; industrial inspection; and medical/scientific imaging among others. Our program seeks to unlock the full potential of these cameras by (1) generalizing the concept of transport-aware imaging and advancing its mathematical underpinnings; (2) designing and building novel transport-aware camera systems; and (3) exploiting these newfound sources of visual input to solve challenging 3D and 2D visual inference tasks.