CAREER: Foundations for Ubiquitous Image-Based Appearance Capture

摘要：对于许多应用来说，视觉内容的本质正在迅速变化。二维图像正在被高维外观模型所取代，高维外观模型总结了一个物体在许多视点和光照条件下的（非常大的）图像集。这些模型使对象能够虚拟地旋转，重新点亮并合成到其他场景中，并且它们提高了我们自动执行视觉任务（如检测，识别和跟踪）的能力。然而，尽管有越来越多的应用，合适的普通对象的外观模型是非常难以找到。这主要是由于外观的复杂性，这使得外观捕获的任务相当艰巨。所提出的研究目标是开发一种外观捕获框架，该框架可应用于任何不透明和非折射表面，并且同时准确和实用。最终，我们寻求实现“无所不在的外观捕获”：在实验室之外——家庭、办公室、博物馆、医院和现场——获得外观模型的广泛能力。为了实现这一目标，我们采用了一种基于物理的方法，寻求利用常见的反射特性（例如，亥姆霍兹互易性、反射可分离性和可压缩性），我们认为这些特性尚未得到充分利用。这种方法的好处是双重的。首先，它可以从少量未校准的相机和光源中获取形状和反射率的图像，从而消除了对激光测距扫描仪、基于投影仪的结构化照明或其他专用设备的需求。其次，它可以对广泛的表面进行建模，包括那些具有复杂反射率的表面，这些表面不一定由低维（即参数化）模型很好地表示。这第二个属性对于模拟真实世界的表面是必不可少的，并且与大多数现有的基于图像的方法非常不同，这些方法基于对表面反射率性质的限制性假设。这项研究活动与教育计划密切相关，该计划包括在本科生和研究生阶段开发人类和计算机视觉课程，并创造本科生和研究生的研究机会。这个教育项目已经扩展到大学之外，在波士顿地区的博物馆和教室里提供一个移动采集系统作为教学工具，并通过互联网提供外观模型和软件。URL: http://www.eecs.harvard.edu/ ~ zickler / research.html

CAREER: Foundations for Ubiquitous Image-based Appearance CapturePI: Todd ZicklerAbstract:For many applications, the nature of visual content is rapidly changing. Two-dimensional images are being replaced by higher-dimensional appearance models that summarize the (very large)set of images of an object for many viewpoints and lighting conditions. These models enable an object to be virtually rotated, re-lit and composited into other scenes, and they improve our ability to automate visual tasks such as detection, recognition, and tracking. Yet, despite the growing number of applications, suitable appearance models of common objects are remarkably hard to find. This is due largely to the complexity of appearance, which makes the task of appearance capture quite formidable. The goal of the proposed research is to develop a framework for appearance capture that can be applied to any opaque and non-refracting surface, and that is simultaneously accurate and practical. Ultimately, we seek to enable "ubiquitous appearance capture": the widespread ability to acquire appearance models outside of the laboratory - in homes, offices, museums, hospitals and in the field. To achieve this goal, we take a physics-based approach that seeks to exploit common reflectance properties (e.g., Helmholtz reciprocity, reflectance separability and compressibility) that we believe have yet to be fully utilized. The benefits of this approach are two-fold. First, it enables image-based acquisition of both shape and reflectance from a small number of uncalibrated cameras and light-sources, eliminating the need for laser range-scanners, projector-based structured lighting, or other specialized equipment. Second, it makes possible the modeling of a broad class of surfaces, including those with complex reflectance not necessarily well-represented by low-dimensional (i.e., parametric)models. This second property is essential for modeling real-world surfaces, and is very different from most existing image-basedapproaches that are predicated on restrictive assumptions about the nature of surface reflectance. This research activity is closely linked to an educational program that includes the development of courses in human and computer vision at both the undergraduate and graduate levels, and the creation of undergraduate and graduate research opportunities. The educational program extends beyond the university by making a mobile acquisition system available as a teaching tool in museums and classrooms in the Boston area, and by making appearance models and software available through the Internet.URL: http://www.eecs.harvard.edu/~zickler/research.html