权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

CAREER: Plenoptic Signal Processing --- A Framework for Sampling, Detection, and Estimation using Plenoptic Functions

职业：全光信号处理——使用全光功能进行采样、检测和估计的框架

基本信息

批准号：
1652569
负责人：
Aswin Sankaranarayanan
金额：
$ 53.2万
依托单位：
Carnegie-Mellon University
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2017
资助国家：
美国
起止时间：
2017-02-15 至 2024-01-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1652569&HistoricalAwards=false
关键词：
CAREER Plenoptic Signal Processing Framework

项目摘要

The interactions of light with objects in a scene are often complex. An image --- which only captures 2D spatial variations --- is poorly equipped to unravel these interactions and infer properties of a scene including its shape, reflectance, and its composition. This is especially true for scenes that have sharp reflections, refractions, and volumetric scattering. This research models interactions of light with scenes using light rays and their transformations. The central hypothesis underlying the research is the idea that problems of shape, reflectance and material composition estimation are often simpler and well-posed when they are studied using light rays and their transformations. A wide-range of real-world objects and scenes stand to benefit from progress made in this research; this includes scenes with complex configurations that lead to inter-reflections, objects with shine, specularities, and spatially-varying reflectances, as well as objects that are transparent, or translucent. A diverse set of applications including machine vision, microscopy, and consumer photography stand to benefit from this research. The education and outreach components of this project disseminates image processing research in the broader Pittsburgh area via camera building workshops and lab demos for middle/high-school students, and professional development courses for physics teachers.The focus of the research is to develop novel acquisition and processing methods for scene understanding by studying characterizations of light that go beyond images. In particular, the research analyzes the properties of two signals: the plenoptic function, which captures spatial, temporal, angular, and spectral variations of light, and the plenoptic light transport, which captures how light propagates through a scene. The central hypothesis of the research is that the plenoptic function and light transport provide a rich encoding of how light interacts with a scene; hence, unlike image-based inference, plenoptic inference can be fundamentally well-conditioned even for scenes that interact with light in a complex manner. To this end, the research develops novel low-dimensional models for plenoptic functions that are based on physical laws governing interaction of light with a scene. The research also builds novel computational cameras that acquire light propagates in a scene by decomposing into light paths of varying complexity, and subsequently estimating the 3D shape, reflectance, and material composition.

灯光与场景中对象的交互通常很复杂。一幅图像-只捕捉2D空间变化-不足以解开这些相互作用，并推断场景的属性，包括其形状、反射率和组成。对于具有尖锐反射、折射和体积散射的场景尤其如此。这项研究使用光线及其变换来模拟光线与场景的相互作用。这项研究背后的中心假设是，当使用光线及其变换来研究形状、反射率和材料成分估计时，它们的问题往往更简单、更合适。广泛的真实世界对象和场景将从这项研究中取得的进展中受益；这包括具有导致相互反射的复杂配置的场景、具有光泽、镜面反射率和空间变化的反射率的对象，以及透明或半透明的对象。包括机器视觉、显微镜和消费者摄影在内的各种应用都将从这项研究中受益。该项目的教育和推广部分通过为初中生和高中生举办的相机制造研讨会和实验室演示，以及为物理教师提供的专业发展课程，在更广泛的匹兹堡地区传播图像处理研究。研究的重点是通过研究超出图像的光的特征，开发用于场景理解的新的获取和处理方法。特别是，这项研究分析了两种信号的特性：全光函数，捕获光的空间、时间、角度和光谱变化，以及全光传输，捕获光如何通过场景传播。这项研究的中心假设是，全光功能和光传输为光如何与场景相互作用提供了丰富的编码；因此，与基于图像的推理不同，全光推理即使对于以复杂方式与光相互作用的场景，也可以从根本上良好地进行。为此，该研究开发了全光函数的新型低维模型，该模型基于控制光与场景相互作用的物理定律。这项研究还建立了新型的计算相机，通过将光分解成不同复杂程度的光路，然后估计3D形状、反射率和材料组成，来获取光在场景中的传播。

项目成果

期刊论文数量（28）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Illuminant Spectra-Based Source Separation Using Flash Photography

DOI：
10.1109/cvpr.2018.00650
发表时间：
2017-04
期刊：
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
影响因子：
0
作者：
Zhuo Hui;Kalyan Sunkavalli;Sunil Hadap;Aswin C. Sankaranarayanan
通讯作者：
Zhuo Hui;Kalyan Sunkavalli;Sunil Hadap;Aswin C. Sankaranarayanan

Wide-Baseline Light Fields using Ellipsoidal Mirrors

使用椭圆面镜的宽基线光场

DOI：
10.1109/tpami.2022.3202513
发表时间：
2022
期刊：
IEEE Transactions on Pattern Analysis and Machine Intelligence
影响因子：
23.6
作者：
De Zeeuw, Michael;Sankaranarayanan, Aswin C.
通讯作者：
Sankaranarayanan, Aswin C.

3PointTM: Faster Measurement of High-Dimensional Transmission Matrices

3PointTM：更快地测量高维传输矩阵

DOI：
10.1007/978-3-030-58598-3_19
发表时间：
2020
期刊：
European Conference on Computer Vision
影响因子：
0
作者：
Chen, Yujun;Sharma, Manoj Kumar;Sabharwal, Ashutosh;Veeraraghavan, Ashok;Sankaranarayanan, Aswin C.
通讯作者：
Sankaranarayanan, Aswin C.

Fine-Grain Prediction of Strawberry Freshness using Subsurface Scattering