CAREER: Foundations for Ubiquitous Image-Based Appearance Capture

职业生涯：基于图像的无处不在外观的基础捕获：Todd Zickler摘要：对于许多应用程序来说，视觉内容的性质正在迅速变化。二维图像正在被更高维的外观模型所取代，这些模型总结了对象在许多视点和照明条件下的(非常大的)图像集。这些模型使对象能够虚拟旋转、重新照明并合成到其他场景中，它们提高了我们自动执行检测、识别和跟踪等视觉任务的能力。然而，尽管应用程序越来越多，但常见物体的合适外观模型仍然非常难找到。这在很大程度上是由于外观的复杂性，这使得外观捕获的任务相当艰巨。这项研究的目标是开发一种外观捕捉框架，该框架可以应用于任何不透明和非折射的表面，同时又准确和实用。最终，我们寻求实现“无处不在的外观捕获”：在实验室之外--在家里、办公室、博物馆、医院和野外--获取外观模型的广泛能力。为了实现这一目标，我们采取了一种基于物理的方法，试图利用我们认为尚未充分利用的共同反射特性(例如，亥姆霍兹互易性、反射可分离性和可压缩性)。这种方法有两方面的好处。首先，它能够从少量未经校准的相机和光源获取基于图像的形状和反射率，从而消除对激光距离扫描仪、基于投影仪的结构化照明或其他专门设备的需要。其次，它使得对一大类曲面的建模成为可能，包括那些具有复杂反射率的曲面，这些曲面不一定能用低维(即参数)模型很好地表示。这第二个属性对于真实世界的曲面建模是必不可少的，它与大多数现有的基于图像的方法有很大的不同，这些方法是基于对曲面反射率性质的限制性假设。这项研究活动与一项教育计划密切相关，该计划包括在本科生和研究生水平上开发人类和计算机视觉课程，并创造本科生和研究生的研究机会。该教育计划的范围超出了大学的范围，在波士顿地区的博物馆和教室里提供了一个移动采集系统作为教学工具，并通过互联网提供外观模型和软件。网址：http://www.eecs.harvard.edu/~zickler/research.html

CAREER: Foundations for Ubiquitous Image-based Appearance CapturePI: Todd ZicklerAbstract:For many applications, the nature of visual content is rapidly changing. Two-dimensional images are being replaced by higher-dimensional appearance models that summarize the (very large)set of images of an object for many viewpoints and lighting conditions. These models enable an object to be virtually rotated, re-lit and composited into other scenes, and they improve our ability to automate visual tasks such as detection, recognition, and tracking. Yet, despite the growing number of applications, suitable appearance models of common objects are remarkably hard to find. This is due largely to the complexity of appearance, which makes the task of appearance capture quite formidable. The goal of the proposed research is to develop a framework for appearance capture that can be applied to any opaque and non-refracting surface, and that is simultaneously accurate and practical. Ultimately, we seek to enable "ubiquitous appearance capture": the widespread ability to acquire appearance models outside of the laboratory - in homes, offices, museums, hospitals and in the field. To achieve this goal, we take a physics-based approach that seeks to exploit common reflectance properties (e.g., Helmholtz reciprocity, reflectance separability and compressibility) that we believe have yet to be fully utilized. The benefits of this approach are two-fold. First, it enables image-based acquisition of both shape and reflectance from a small number of uncalibrated cameras and light-sources, eliminating the need for laser range-scanners, projector-based structured lighting, or other specialized equipment. Second, it makes possible the modeling of a broad class of surfaces, including those with complex reflectance not necessarily well-represented by low-dimensional (i.e., parametric)models. This second property is essential for modeling real-world surfaces, and is very different from most existing image-basedapproaches that are predicated on restrictive assumptions about the nature of surface reflectance. This research activity is closely linked to an educational program that includes the development of courses in human and computer vision at both the undergraduate and graduate levels, and the creation of undergraduate and graduate research opportunities. The educational program extends beyond the university by making a mobile acquisition system available as a teaching tool in museums and classrooms in the Boston area, and by making appearance models and software available through the Internet.URL: http://www.eecs.harvard.edu/~zickler/research.html