Instrumentation for Empirical Studies in the Modeling of Visual Appearance

belhumeur, Peter N.Kriegman, David J.Columbia University题目：CISE RR：用于视觉外观建模的实证研究仪器该项目支持用于形状建模和分析的计算机视觉算法的实验验证，旨在建立两个独立的系统来支持计算机视觉。形状重建和反射率恢复的方法，称为亥姆霍兹互易立体视和光场重建，利用以前被忽视的物理原理，无论物体的BRDF如何，都能够恢复表面形状。这两个特殊用途的成像系统将在哥伦比亚大学和伊利诺伊大学厄巴纳-香槟分校（UIUC）建造。哥伦比亚号平台由16个成像模块组成，可以以不同的配置排列，直接支持亥姆霍兹互易立体视的研究。UIUC平台由两条手臂组成，两条手臂的端点在球体表面上移动，直接支持光场重建方法。这两个新硬件将有助于以下四个项目：亥姆霍兹立体视觉，光场重建，基于图像的建模和渲染，以及小数据集的纹理和反射率估计。第一个项目利用物体表面反射率的对称性。亥姆霍兹立体视法可以重建具有复杂反射率的表面，例如高度非朗伯曲面。亥姆霍兹钻机将用于收集物体的数据集，这样它们就可以使用亥姆霍兹立体视觉形状恢复方法进行处理。第二种方法使用从表面入射光场的双重覆盖中收集的图像来逐点重建表面形状和有效的双向反射分布。照明/视点平台将被用来收集数据，这样它就可以用光场重建方法进行处理。第三种是利用两个平台收集的数据集，用于基于图像和建模的项目，在新的视点和任意照明下渲染物体的逼真图像。采集重建三维形状的目标数据集，并对其反射率进行建模。然后，这些形状和反射率模型被用来合成物体的新图像，并将它们合成为静止图像和视频片段。这些物体将根据它们的视觉外观进行分类。最后一个项目使用光场渲染设备收集的数据集来开发纹理和反射率的低维模型。然后使用这些模型从少量图像中估计纹理和反射率属性。因此，将创建和分发表征各种材料反射特性的图像，以满足以下目标：开发、改进、分析和经验验证亥姆霍兹立体视方法；开发、改进和分析光场重建方法；将重建方法应用于基于图像的渲染；开发和完善数据驱动的低维非参数模型，用于根据视点和光照变化的表面反射和纹理。

EIA 0224431Belhumeur, Peter N.Kriegman, David J.Columbia University Title: CISE RR: Instrumentation for Empirical Studies in the Modeling of Visual Appearance This project, supporting experimental validation of computer vision algorithms for shape modeling and analysis, aims at building two independent systems to support computer vision. Methods for shape reconstruction and reflectance recovery, termed Helmholtz reciprocity stereopsis and light field reconstruction that exploit previously neglected physical principles, are able to recover surface shape regardless of the object's BRDF. The two special purpose imaging systems will be built at Columbia University and the University of Illinois at Urbana-Champaign (UIUC). The Columbia rig, composed of sixteen imaging modules that can be arranged in different configurations, directly supports research in Helmholtz reciprocity stereopsis. The UIUC rig, consisting of two arms that move their endpoints over the surface of a sphere, directly supports the light field reconstruction method. The two new pieces of hardware will contribute to the following four projects:Helmholtz Stereopsis,Light Field Reconstruction,Image-Based Modeling and Rendering, andTexture and Reflectance Estimation from Small Datasets.The first project exploits the symmetries in an object's surface reflectance. The Helmholtz stereopsis method can reconstruct surfaces with complex reflectance, e.g., highly non-Lambertian. The Helmholtz rig will be used to gather datasets of objects in a manner such that they can be processed using the Helmholtz stereopsis shape recovery method. The second uses images gathered from a double covering of a surface's incident light field to reconstruct both the surface shape and an effective bi-directional reflectance distribution on a point-by-point basis. The illumination/viewpoint rig will be used to gather data in a manner such that it can be processed with the light field reconstruction method. The third utilizes the datasets gathered by both rigs for image-based and modeling projects to render photorealistic images of objects under novel viewpoint and arbitrary illuminations. Datasets of objects, with their 3-D shape reconstructed, are collected and their reflectance is modeled. These models of shape and reflectance are then used to synthesize novel images of the objects and composite them into still pictures and video footage. The objects will be catalogued by their visual appearance. The last project uses the datasets gathered by the light field rendering rig for developing low-dimensional models of texture and reflectance. These models are then used to estimate texture and reflectance properties from a small number of images. Thus, images characterizing reflectance properties of a wide variety of materials will be created and distributed satisfying the following goals to Develop, refine, analyze, and empirically validate the method of Helmholtz stereopsis,Develop, refine, and analyze the light field reconstruction method,Apply the reconstruction methods to image-based rendering, andDevelop and refine data-driven low-dimensional non-parametric models for surface reflection and textures that vary with viewpoint and lighting.