SGER: Decomposing Reflectance for Vision-based Tracking

绝大多数计算机视觉技术仍然建立在关于反射率的限制性假设之上，它们从复杂反射场景的图像中提取有意义的信息的能力仍然有限。拟议的研究活动致力于建立一个框架，通过分解反射率来分析复杂的反射场景。根据该方法，通过将图像分解成更简单的成分，在逐点的基础上获得图像的简化表示；并且该表示提供对否则不可访问的场景信息(例如，形状、照明、材料属性)的访问。这项研究的特点是，不是直接恢复反射分量的完整估计，而是寻求初步的简化表示，这将以某种可用的形式隔离漫反射效果。这使得问题变得容易处理，并使开发用于分析复杂反射场景的健壮和通用框架成为可能。提出的研究的具体目标是：(1)简化表示，在可变和复杂光照的场景中分离出简单得多的漫反射效果；(2)应用简化表示的密集、基于区域的跟踪方法。视觉跟踪的改进将促进健壮的导航系统和人机界面。增强的识别系统将有利于视觉检查、监视和国土安全系统(例如，人脸识别)。3D重建技术的改进将增强系统从对象的图像中学习对象的外观模型的能力，从而使系统能够预测这些对象在新环境中的外观。根据这笔赠款，私立学校还将为哈佛大学的两个本科课程(大一和大四)制定课程。这门新生水平的课程将通过让学生接触视觉理解的直观和令人兴奋的计算方面来吸引代表人数不足的群体的学生进入工程学和计算机科学。

The vast majority of computer vision techniques continue to be predicated on restrictive assumptions about reflectance, and their ability to extract meaningful information from images of complex reflecting scenes remains limited. The proposed research activity works toward a framework for the analysis of complex reflecting scenes through the decomposition of reflectance. According to this approach, a reduced representation of an image is obtained on a point-by-point basis by its decomposition into simpler constituents; and this representation provides access to scene information (e.g., shape, illumination, material properties) that would be otherwise inaccessible. The distinguishing feature of this research is that instead of directly recovering complete estimates of reflection components, preliminary reduced representations will be sought, which isolate diffuse reflection effects in some usable form. This makes the problem tractable and enables the development of a robust and general framework for the analysis of complex reflecting scenes. The specific goals of the proposed research are: (1) Reduced representations that isolate the much simpler diffuse reflection effects in scenes with variable and complex illumination; and (2) Methods for dense, region-based tracking that apply the reduced representations.Improvements in visual tracking will facilitate robust navigation systems and human-computer interfaces. Enhanced recognition systems will benefit systems for visual inspection, surveillance and homeland security (e.g., face recognition). Improvements in 3D reconstruction techniques will enhance a system's ability to learn appearance models of objects from their images, thereby enabling the system to predict the appearance of these objects in novel environments. Under this grant, the PI will also develop curricula for two undergraduate courses (at the freshmen and senior levels) at Harvard University. The freshman-level course will be designed to attract students from underrepresented groups into engineering and computer science by exposing them to intuitive and exciting computational aspects of visual understanding.