CPA-G&V: Self-Completion of 4D (Space+Time) Models

CPA-GV：4D（空间+时间）模型的自我完成杨（0811647）摘要数字2D照片和视频已经在我们的日常生活中无处不在。在Google Earth和互动3D游戏等新产品的推动下，公众似乎对3D内容重新产生了兴趣。创建3D内容而不是简单的固定视点立体电影迄今为止仍然是劳动密集型和昂贵的过程。本研究探讨了新的软件算法，可以显着简化这一建模过程，特别是动态模型。使用廉价的深度传感器（如立体相机）将从不同的位置捕获动态场景，并自动生成完整的4D模型。恢复的模型可以在许多应用程序中使用，如模拟，以创建逼真的虚拟环境，娱乐渲染特殊效果，也许只是为了让每个人都能享受他们的珍惜时刻，如婴儿？第一步，交互式3D。从技术角度来看，从颜色+深度图的输入序列来看，它提供了整个4D（空间+时间）模型的部分采样，中心研究问题是如何融合这些部分样本以形成完整的模型。与以往所有的孔洞填充方法不同，本研究旨在处理表现出以下一个或多个特征的模型：大（例如，几个城市街区）、动态的、变形的，但稀疏采样的（例如，小于50%是可用的），并且可能非常嘈杂。在这些条件下重建一个完整的模型可能非常具有挑战性，有时甚至是不适定的。然而，场景结构通常不是随机的;相同或相似的结构元素可能已经在输入集中出现了几次，可能在不同的时间和位置。因此，可以使用来自不同时间或空间的样本来填充缺失的数据，从而使模型完整成为可能，而无需使用任何外部来源。

CPA-G&V: Self-Completion of 4D (Space+Time) ModelsYang (0811647)AbstractDigital 2D photos and videos are already ubiquitous in our everyday life. Fueled by new products such as Google Earth and interactive 3D games, the public seems to have a renewed interest in 3D contents. Creating 3D contents beyond simple fixed viewpoint stereoscopic movies has so far remained to be a labor intensive and expensive process. This research investigates new software algorithms that can significantly simplify this modeling process, in particular for dynamic models. The use of inexpensive depth sensors (such a stereo camera) will capture a dynamic scene over time from different locations and automatically generate a complete 4D model. The recovered models can be used in many applications such as simulations to create realistic virtual environment, entertainment to render special effect, and perhaps simply to allow everyone to enjoy their cherish moments, such as a baby?s first step, in interactive 3D. From a technical standpoint, from the input sequence of color+depth maps, which provides a partial sampling of the entire 4D (space+time) model, the central research problem is how to fuse these partial samples to form a complete model. Different from all previous hole filling approaches, this research aims to deal with models that exhibit one or more of the following characteristics: large (e.g., several city blocks), dynamic, deforming, yet sparsely sampled (e.g., less than 50% is available), and possibly very noisy. Reconstructing a complete model under these conditions can be very challenging or sometime ill-posed. However, scene structures are usually not stochastic; the same or similar structure element may have appeared in the input set a few times, probably at a different time and location. Therefore, samples from different time or space can be used to fill in the missing data, making model completion possible, without using any external sources.