CIF: Small: Computational Tools for Visual Inference of Complex Materials

成像过程的核心是光与场景中物体的相互作用。在过去的几百年里，在解决由具有朗伯反射率的简单材料构成的一大类物体的推理问题（如检测、分类或估计）方面取得了显着的进展。这样的物体散射光，使得视亮度对于观察者的视角是不变的。不幸的是，真实的世界物体由相当复杂的材料制成，这些材料不能根据这种各向同性反射率来表征。虽然人类能够毫不费力地推理复杂的材料，但今天的图像分析和处理算法却失败得很惨。原因是复杂的非朗伯材料只能用高维函数来表征，而高维函数的理解相对较少，建模也更差。这项研究正在开发新的方法来建模，捕获和处理复杂材料的丰富反射模式。 关键的工具是物体的全光传输函数，它描述了由于材料的性质而导致的入射光到照射光的变换。一般而言，全光传输函数是14维的;因此，用于复杂材料的感测、分析和处理系统的基本复杂性是高维全光函数与大多数常规传感器（相机）获取最多2D或3D图像投影的能力之间的维度差距。 正在开发的工具和技术包括稀疏和流形模型（以弥合维度差距），几何特征（以减轻环境照明和其他滋扰参数的存在），以及新的传感器设计（以最有效地从自然场景中获取全光信息）。

Central to the imaging process is the interaction of light with the objects in the scene. Remarkable progress has been made over the past several hundred years on solving inference problems (such as detection, classification, or estimation) for a large class of objects constructed from simple materials with Lambertian reflectance. Such objects scatter light such that the apparent brightness is invariant to the observer's view angle. Unfortunately, real world objects are made of considerably more complex materials that cannot be characterized in terms of such an isotropic reflectance. While humans are able to effortlessly reason about complex materials, today's image analysis and processing algorithms fail miserably. The reason is that complex, non-Lambertian materials can be characterized only by higher-dimensional functions that are relatively poorly understood and even more poorly modeled. This research is developing new ways to model, capture, and process the rich reflectance patterns of complex materials. The key tool is the object's plenoptic transport function, which describes the transformation of the incident to the irradiated light due to the properties of the material. In full generality, the plenoptic transport function is 14-dimensional; hence, a fundamental complication for sensing, analysis, and processing systems for complex materials is the dimensionality gap between the high dimensional plenoptic function and the ability of most conventional sensors (cameras) to acquire at best 2D or 3D image projections. The tools and techniques under development include sparse and manifold models (to bridge the dimensionality gap), geometric features (to mitigate the presence of environmental illumination and other nuisance parameters), and new sensor designs (to most efficiently acquire plenoptic information from natural scenes).