SGER: Decomposing Reflectance for Vision-based Tracking

绝大多数计算机视觉技术仍然基于对反射率的限制性假设，并且它们从复杂反射场景的图像中提取有意义信息的能力仍然有限。建议的研究活动的作品走向复杂的反射场景，通过分解的反射率的分析框架。根据该方法，通过将图像分解成更简单的成分，在逐点的基础上获得图像的简化表示;并且该表示提供对场景信息（例如，形状、照明、材料属性），否则将无法访问。这项研究的显着特点是，而不是直接恢复反射分量的完整估计，初步减少表示将寻求，隔离漫反射效应在一些可用的形式。这使得问题易于处理，并使复杂的反射场景的分析，一个强大的和一般的框架的发展。具体目标是：（1）简化表示，隔离在可变和复杂的照明场景中简单得多的漫反射效果;（2）密集的，基于区域的跟踪方法，应用简化表示。视觉跟踪的改进将促进强大的导航系统和人机界面。增强的识别系统将有利于视觉检查、监视和国土安全系统（例如，面部识别）。3D重建技术的改进将增强系统从其图像中学习对象的外观模型的能力，从而使系统能够预测这些对象在新环境中的外观。在这笔赠款下，PI还将为哈佛大学的两门本科课程（新生和高年级）制定课程。新生级别的课程将被设计为吸引学生从代表性不足的群体进入工程和计算机科学，让他们接触到直观和令人兴奋的计算方面的视觉理解。

The vast majority of computer vision techniques continue to be predicated on restrictive assumptions about reflectance, and their ability to extract meaningful information from images of complex reflecting scenes remains limited. The proposed research activity works toward a framework for the analysis of complex reflecting scenes through the decomposition of reflectance. According to this approach, a reduced representation of an image is obtained on a point-by-point basis by its decomposition into simpler constituents; and this representation provides access to scene information (e.g., shape, illumination, material properties) that would be otherwise inaccessible. The distinguishing feature of this research is that instead of directly recovering complete estimates of reflection components, preliminary reduced representations will be sought, which isolate diffuse reflection effects in some usable form. This makes the problem tractable and enables the development of a robust and general framework for the analysis of complex reflecting scenes. The specific goals of the proposed research are: (1) Reduced representations that isolate the much simpler diffuse reflection effects in scenes with variable and complex illumination; and (2) Methods for dense, region-based tracking that apply the reduced representations.Improvements in visual tracking will facilitate robust navigation systems and human-computer interfaces. Enhanced recognition systems will benefit systems for visual inspection, surveillance and homeland security (e.g., face recognition). Improvements in 3D reconstruction techniques will enhance a system's ability to learn appearance models of objects from their images, thereby enabling the system to predict the appearance of these objects in novel environments. Under this grant, the PI will also develop curricula for two undergraduate courses (at the freshmen and senior levels) at Harvard University. The freshman-level course will be designed to attract students from underrepresented groups into engineering and computer science by exposing them to intuitive and exciting computational aspects of visual understanding.