ITR: Collaborative Research: Image-Based Rendering in Forensic Reconstruction and Historical Preservation

关于我们周围世界的有价值的信息正在丢失。我们每天都会丢失具有当前和历史意义的信息，原因很简单，因为捕捉我们环境的3D模型太难了。如今，这样的模型主要是手工创建的，用卷尺测量并手动输入结果。即使在构建模型之后，我们也不知道有效地可视化它们的最佳方式。本研究建议设计和开发一个端到端的解决方案，以解决真实世界环境的文档、传播和显示问题。为了集中研究，两个驾驶问题是有针对性的：法医科学和教育。这项研究汇集了一个跨学科的团队，包括执法部门、联邦调查局和武装部队病理研究所。此外，我们的团队成员之一，来自新奥尔良艺术博物馆，正在指导一个大型展览，我们提议在其中包括杰斐逊总统的家蒙蒂塞洛的虚拟环境。我们将解决真实世界数据的获取、处理、表示、渲染和显示问题，重点关注两个目标应用程序。我们的目标是设计和构建一个完整的端到端原型系统。主要工作包括：采办。我们将在机动手推车上安装一个精确的测距仪，并解决捕捉尽可能完整的模型的问题。这需要准确的跟踪和实际意义上的次佳视图问题的解决方案，包括分辨率、反射率和表面的重要性。一个非常困难的问题是颜色与深度数据的配准，以及来自多个位置的扫描的配准。我们将开发算法来稳健地配准数据，并进行过滤以减少离群值和噪声。我们将开发算法将海量数据(每次扫描100 MB，每个房间数十次扫描)处理为有效的几何和基于图像的表示。我们将在质量和效率方面对表示进行比较和对比。渲染。我们将开发分层和外置算法来以交互速率渲染基于图像的模型和几何模型，并研究感知驱动的渲染技术以最大化渲染效率。显示。这可能是这个问题最困难的部分，因为它涉及到人类感知环境的方式。我们将开发一种混合投影仪/头盔显示系统，在真人大小的行走环境中为多名观众提供立体视图。目的是给参与者一种身临其境的感觉，就像在杰斐逊先生的房子里一样。

Valuable information about the world around us is being lost. Every day we lose information of both current and historical significance, simply because it is too difficult to capture 3D models of our environment. Today, such models are primarily created by hand, making measurements with tape measures and entering the results manually. Even after models are constructed, we do not know the best ways to visualize them effectively.This research proposes to design and develop an end-to-end solution to the problem of documentation, dissemination, and display of real-world environments. To focus the research, two driving problems are targeted: forensic science and education. This research brings together an interdisciplinary team including the law enforcement community, the Federal Bureau of Investigation and the Armed Forces Institute of Pathology. Additionally, one of our team members, from the New Orleans Museum of Art, is directing a major exhibition in which we propose to include a virtual environment of Monticello, President Jefferson's home.We will tackle the problems of acquiring, processing, representing, rendering and display of real- world data, focusing on the two target applications. Our goal is to design and build a complete end-to-end prototype system. The work includes:Acquisition. We will mount an accurate rangefinder on a motorized cart and tackle the problem of capturing as complete a model as possible. This requires accurate tracking and a solution to the next best view problem in a practical sense, including resolution, reflectance, and importance of the surfaces.Processing. A very difficult problem is the registration of color with depth data, and of scans from multiple locations. We will develop algorithms to robustly register the data, and filter to reduce outliers and noise.Representation. We will develop algorithms to process the massive amounts of data (100 MB per scan, tens of scans per room) into both efficient geometric and image-based representations. We will compare and contrast the representations in terms of quality and efficiency.Rendering. We will develop hierarchical and out-of-core algorithms to render the image- based and geometric models at interactive rates, and research perceptually driven rendering techniques to maximize rendering efficiency.Display. This is perhaps the most difficult part of the problem because it involves the way humans perceive the environment. We will develop a hybrid projector/head-mounted-display system to provide stereo views to multiple viewers in a life-sized walking environment. The goal is to give the participants a sense of presence, of being in Mr. Jefferson's house.