Transport-Aware Imaging for Computer Vision

计算机视觉的传输感知成像

• 批准号: RGPIN-2016-06124
• 负责人: Kutulakos, Kiriakos
• 金额: $ 6.85万
• 依托单位: University of Toronto
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2019
• 资助国家: 加拿大
• 起止时间: 2019-01-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=692514
• 关键词: Transport; Aware; Imaging; Computer; Vision

The main goal of my research program for the next five years will be to study, design, and use a new class of computational cameras whose key property is that they are "transport-aware." Unlike conventional cameras which record all incident light, transport-aware cameras can be programmed to block some of that light, based on the actual 3D paths it followed through a scene. Transport-aware cameras use a programmable light source for illumination and a programmable sensor mask for imaging, and were pioneered by my research group three years ago. ******Live video from a transport-aware camera can offer a very unconventional view of our everyday world in which refraction and scattering can be selectively blocked or enhanced, visual structures too subtle to notice with the naked eye can become apparent, and object surfaces can be reconstructed in 3D under challenging conditions well beyond the state of the art. Our initial work on harnessing their power brings together computer vision, computer graphics, numerical optimization and geometric optics to demonstrate never-seen-before capabilities with potential applications in 3D shape acquisition; robust time-of-flight imaging; visual material analysis; scene understanding; robotic navigation; industrial inspection; and medical/scientific imaging among others.******Our program seeks to unlock the full potential of these cameras by (1) generalizing the concept of transport-aware imaging and advancing its mathematical underpinnings; (2) designing and building novel transport-aware camera systems; and (3) exploiting these newfound sources of visual input to solve challenging 3D and 2D visual inference tasks.

我未来五年研究计划的主要目标将是研究、设计和使用一类新的计算相机，其关键特性是它们是“交通感知”的。“与记录所有入射光的传统相机不同，运输感知相机可以根据它在场景中的实际3D路径进行编程，以阻挡部分光线。运输感知相机使用可编程光源进行照明，并使用可编程传感器掩模进行成像，三年前由我的研究小组率先开发。** 来自交通感知摄像头的实时视频可以提供我们日常世界的非常非常规的视图，其中折射和散射可以选择性地被阻挡或增强，肉眼无法注意到的视觉结构可以变得明显，和物体表面可以在挑战性的条件下进行3D重建，远远超出了最先进的水平。我们最初的工作是利用它们的力量，视觉、计算机图形学、数值优化和几何光学，以展示前所未有的能力，并在3D形状采集、强大的飞行时间成像、视觉材料分析、场景理解、机器人导航、工业检测和医疗/科学成像等方面具有潜在应用。我们的计划旨在通过以下方式释放这些相机的全部潜力：（1）推广运输感知成像的概念并推进其数学基础;（2）设计和构建新型运输感知相机系统;（3）利用这些新发现的视觉输入来源来解决具有挑战性的3D和2D视觉推理任务。