RI: Small: Collaborative Research: Dynamic Light Transport Acquisition and Applications to Computational Illumination

RI：小型：合作研究：动态光传输采集及其在计算照明中的应用

• 批准号: 1909729
• 负责人: Sanjeev Koppal
• 金额: $ 25万
• 依托单位: University of Florida
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-08-15 至 2023-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1909729&HistoricalAwards=false
• 关键词: RI; Small; Collaborative; Research; Dynamic

Cameras that view a dynamic scene typically capture interactions of moving objects with light. Computer vision algorithms can use these measurements to infer properties of these objects, such as depth, motion and appearance. However, there is a subtler, richer visual back-story that occurs as an object moves in a scene, and usually these effects are ignored in traditional algorithms, sometimes causing errors. This project studies all the interactions of light with dynamic scenes, which we term as dynamic light transport, and the goal is to understand and recover effects such as multiple reflections and scattering as objects move in a scene. The innovations of the project include new computational cameras and projectors to capture light transport for dynamic scenes, and to explore new physics-based and data-driven algorithms to exploit this information for improved computer vision and graphics applications. The project further seeks to include broadening access to computing education and research through curriculum material and capstone experiences which emphasize the intersection of light transport and digital media as well as outreach to middle and high school students in summer programs to discover imaging and optics applications.This research focuses on designing new light transport acquisition frameworks to capture dynamic scenes, characterization of dynamic light transport properties including sparsity and low-rank, and algorithms to exploit this information for computer vision applications. In particular, the project focuses on three main objectives. The first is design of an MEMs-based optical scanner coupled with high frame rate cameras to capture the full set of light transport paths at extremely fast timescales. The second contribution is new algorithms for adaptive light transport sampling using both physics-based and data-driven priors for light transport interpolation via generalized light transport flow. Finally, the project will provide applications of dynamic light transport for 3D scanning of deformable, moving, and specular objects. These innovations are evaluated in an integrated testbed via the optical scanner and the collection of a dataset of dynamic light transport for real-world scenes.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

查看动态场景的相机通常捕捉移动对象与灯光的交互。计算机视觉算法可以使用这些测量来推断这些对象的属性，例如深度，运动和外观。然而，当物体在场景中移动时，会出现一个更微妙、更丰富的视觉背景故事，而传统算法通常会忽略这些效果，有时会导致错误。该项目研究光与动态场景的所有相互作用，我们称之为动态光传输，目标是理解和恢复物体在场景中移动时的多次反射和散射等效果。该项目的创新包括新的计算相机和投影仪，以捕捉动态场景的光传输，并探索新的基于物理和数据驱动的算法，以利用这些信息来改进计算机视觉和图形应用。该项目进一步寻求通过强调光传输和数字媒体交叉的课程材料和顶级体验来扩大计算机教育和研究的范围，并在暑期项目中向初中和高中学生提供探索成像和光学应用的机会。这项研究的重点是设计新的光传输采集框架来捕捉动态场景，包括稀疏性和低秩的动态光传输特性的表征，以及将该信息用于计算机视觉应用的算法。具体而言，该项目侧重于三个主要目标。 第一个是基于MEMS的光学扫描仪的设计，该扫描仪与高帧速率相机耦合，以极快的时间尺度捕获光传输路径的全部集合。 第二个贡献是自适应光传输采样的新算法，使用基于物理和数据驱动的先验，通过广义光传输流进行光传输插值。 最后，该项目将提供动态光传输的应用程序，用于可变形，移动和镜面物体的3D扫描。这些创新是通过光学扫描仪和真实场景动态光传输数据集在集成测试台中进行评估的。该奖项反映了NSF的法定使命，并通过使用基金会的知识产权进行评估而被认为值得支持优点和更广泛的影响审查标准。

项目成果

The Security-Utility Trade-off for Iris Authentication and Eye Animation for Social Virtual Avatars

• DOI: 10.1109/tvcg.2020.2973052
• 发表时间: 2020-02
• 期刊: IEEE Transactions on Visualization and Computer Graphics
• 影响因子: 5.2
• 作者: Brendan David-John;S. Jörg;S. Koppal;Eakta Jain

FoveaCam: A MEMS Mirror-Enabled Foveating Camera

• DOI: 10.1109/iccp48838.2020.9105183
• 发表时间: 2020-04
• 期刊: 2020 IEEE International Conference on Computational Photography (ICCP)
• 作者: Brevin Tilmon;Eakta Jain;S. Ferrari;S. Koppal

Design and Calibration of a Fast Flying-Dot Projector for Dynamic Light Transport Acquisition

用于动态光传输采集的快速飞点投影仪的设计和校准

• DOI: 10.1109/tci.2020.2964246
• 发表时间: 2020
• 期刊: IEEE Transactions on Computational Imaging
• 影响因子: 5.4
• 作者: Henderson, Kristofer;Liu, Xiaomeng;Folden, Justin;Tilmon, Brevin;Jayasuriya, Suren;Koppal, Sanjeev
• 通讯作者: Koppal, Sanjeev