CAREER: Foundations for Ubiquitous Image-Based Appearance Capture

点击翻译按钮获取中文摘要

CAREER: Foundations for Ubiquitous Image-based Appearance CapturePI: Todd ZicklerAbstract:For many applications, the nature of visual content is rapidly changing. Two-dimensional images are being replaced by higher-dimensional appearance models that summarize the (very large)set of images of an object for many viewpoints and lighting conditions. These models enable an object to be virtually rotated, re-lit and composited into other scenes, and they improve our ability to automate visual tasks such as detection, recognition, and tracking. Yet, despite the growing number of applications, suitable appearance models of common objects are remarkably hard to find. This is due largely to the complexity of appearance, which makes the task of appearance capture quite formidable. The goal of the proposed research is to develop a framework for appearance capture that can be applied to any opaque and non-refracting surface, and that is simultaneously accurate and practical. Ultimately, we seek to enable "ubiquitous appearance capture": the widespread ability to acquire appearance models outside of the laboratory - in homes, offices, museums, hospitals and in the field. To achieve this goal, we take a physics-based approach that seeks to exploit common reflectance properties (e.g., Helmholtz reciprocity, reflectance separability and compressibility) that we believe have yet to be fully utilized. The benefits of this approach are two-fold. First, it enables image-based acquisition of both shape and reflectance from a small number of uncalibrated cameras and light-sources, eliminating the need for laser range-scanners, projector-based structured lighting, or other specialized equipment. Second, it makes possible the modeling of a broad class of surfaces, including those with complex reflectance not necessarily well-represented by low-dimensional (i.e., parametric)models. This second property is essential for modeling real-world surfaces, and is very different from most existing image-basedapproaches that are predicated on restrictive assumptions about the nature of surface reflectance. This research activity is closely linked to an educational program that includes the development of courses in human and computer vision at both the undergraduate and graduate levels, and the creation of undergraduate and graduate research opportunities. The educational program extends beyond the university by making a mobile acquisition system available as a teaching tool in museums and classrooms in the Boston area, and by making appearance models and software available through the Internet.URL: http://www.eecs.harvard.edu/~zickler/research.html