Visual models and applications

本研究计划处理的问题，构建虚拟模型的现实世界中的场景在计算机从传感器获得的信息。 有时被称为虚拟现实，这项研究的目标是开发新的算法，可以从一个或多个传感器中提取视觉信息，确定3D形状和纹理的视野内的对象，并从任意视点渲染的真实的场景的传真。 近年来，我的研究一直是关于计算这种表示，并使用它们来为在恶劣的观看条件下执行困难任务的人类操作员提供态势感知，例如，使用纹理化的数字高程图为在雾中飞行的直升机飞行员生成地面的合成图像。 在这里，研究强调了如何根据外部传感数据（例如红外摄像机）实时更新这些计算机表示，以反映世界的实际状态。 然而，将这些想法向前推进到更复杂的领域，例如城市街道场景，会调用更困难的采集任务（多个分布式传感器），更复杂的表示（例如动态的，铰接的3D形状模型），以及从静态渲染到动态模拟的演变。 对于这些有点雄心勃勃的目标，拟议的研究扩展了我们目前的工作，主动模型采集多传感器的情况下，复杂的3D场景和新的表示。 除了支持可视化之外，我们的工作还将研究对不断发展的3D场景的解释，例如对正在发生的事件的识别。 作为相关项目的一部分，这项研究将首先应用于监测城市环境变化和事件的问题。

This research program deals with the problem of constructing virtual models of real-world scenes within a computer from information acquired by sensors.  Sometimes referred to as virtualized reality, the goal of this research is to develop novel algorithms that can extract visual information from one or more sensors, determine the 3D shape and texture of objects within the field of view,  and render a facsimile of the real scene from an arbitrary viewpoint.  In recent years my research has been about computing such representations and using them to provide situational awareness to human operators performing difficult tasks under poor viewing conditions, e.g., using textured digital elevation maps to produce synthetic images of the ground for a helicopter pilot flying in fog.  Here the research emphasized how these computer representations could be updated in real-time based on external sensory data (e.g. infrared cameras) to reflect the actual state of the world.   However, taking these ideas forward to a more complex domain such as an urban street scene invokes a far more difficult acquisition task (multiple, distributed sensors), more complex representations (e.g. dynamic, articulated 3D shape models), and the evolution from static rendering to dynamic simulation.  Towards these somewhat ambitious goals, the proposed research extends our current work in active model acquisition to the multi-sensor case and new representations for complex 3D scenes.   In addition to supporting visualization, our work will also investigate the interpretation of evolving 3D scenes, such as the recognition of events taking place.  As part of a related project, this research will be initially applied to the problem of monitoring changes to and events within urban environments.