CAREER: Large-scale Appearance Modeling

我们周围世界的视觉外观是组成场景的不同表面和材料属性之间复杂光交互的结果。 尽管在数据驱动的外观建模方面取得了惊人的进步，但创建大型环境的精确模型仍然是一个悬而未决的问题。 大多数当前的外观建模方法依赖于主动照明来探测场景外观的不同切片，这排除了它们在对入射环境照明的控制有限或没有控制的环境中的使用。 此外，为了便于校准，许多外观建模技术从固定的Vantage位置估计场景的外观，排除太大而不能以足够的细节适合单个视图的场景。 在这项研究中，PI将研究两种新颖的外观建模范式，专门为不受控制的环境照明下的大规模环境设计：运动外观和相似性外观。 前者利用来自不同视点的观测之间的关系来推断全反射行为，而后者试图从预先存在的外观实例库中识别最佳匹配到可能的约束不足的观测集。 为了支持这两个范例，将开发一个新的外观模型，建立在我们的直觉关于场景外观。 这项工作将侧重于两种常见的投入：社区照片收集和有针对性的视频序列。这项研究将为大规模环境的现场外观建模的实用技术铺平道路，同时，通过回答关于我们是否可以从运动和/或视觉中建模外观的基本问题，或者利用相似性。 该项目不仅将对计算机科学产生深远的影响，而且将对从大都市规划到文化遗产再到娱乐的各个领域产生深远的影响。 模拟现有环境的能力将有利于各种安全和安全培训计划（例如，现有建筑物和场地的虚拟消防演习模拟可以帮助培训和准备消防员和急救人员）。 新兴的虚拟现实治疗领域也将从这项研究中受益，使创建大规模环境的数字模型变得更加容易（因此，例如，遭受中风的患者可以在他们日常生活中遇到的环境的虚拟再现中练习运动康复技能，而自闭症儿童可以在虚拟复制的场所（例如他们在日常生活中遇到的教室）中进行训练以改善他们的社会互动。

The visual appearance of the world around us is the result of complex light interactions between different surfaces and material properties that comprise a scene. Despite staggering advances in data-driven appearance modeling, the creation of accurate models of large environments remains an open problem. The reliance of most current appearance modeling methods on active lighting to probe different slices of a scene's appearance precludes their use in environments where there is limited or no control over the incident ambient lighting. Furthermore, to facilitate calibration, many appearance modeling techniques estimate the appearance of a scene from a fixed vantage point, excluding scenes too large to fit in a single view with sufficient detail. In this research, the PI will investigate two novel appearance modeling paradigms designed expressly for large-scale environments under uncontrolled ambient lighting: appearance-from-motion and appearance-by-similarity. The former exploits relations between observations from different viewpoints to infer the full reflectance behavior, while the latter seeks to identify the best match from a library of pre-existing appearance instances to a possibly under-constrained set of observations. To support these two paradigms, a novel appearance model will be developed that builds upon our intuitions regarding scene appearance. The work will focus on two common types of input: community photo-collections and targeted video sequences.Broader Impacts: This research will pave the way towards practical techniques for in-situ appearance modeling of large-scale environments, while stimulating new research in computer vision and in data-driven appearance modeling in computer graphics by answering fundamental questions as to whether we can model appearance from motion and/or by exploiting similarity. The project will have far-reaching impact not only on computer science but also on diverse fields ranging from metropolitan planning to cultural heritage to entertainment. The ability to model existing environments will be beneficial to various security and safety training programs (for example, virtual fire drill simulations of existing buildings and sites could help train and prepare firefighters and first responders). The emerging field of virtual reality therapy will also benefit from this research, by making it easier to create digital models of large-scale environments (so that, for example, patients who have suffered a stroke can practice motor rehabilitation skills in virtual reproductions of environments they encounter in their daily lives, while autistic children can train to improve their social interactions in virtual reproductions of places such as classrooms which they encounter in their daily lives).