Polymorphic Scene Representation for Enhanced Instant Scene Reconstruction

实时3D场景重建的主要目标是准确地获取和表示被捕获场景的几何形状以及其他属性，例如外观信息、分解成形状图元和场景动态。虽然3D场景几何的鲁棒实时重建已经有了显着的改进，但在其他场景属性的高质量实时采集方面仍然存在很大的不足。 该项目的核心假设是，这些缺陷的主要原因是用于描绘场景信息的特定表示类型（通常是点集或体素网格）的固有缺点。因此，该项目的目标是开发一种多态场景表示，用于改进场景属性的实时重建，并广泛使用不同的表示类型（点和体素，但也多边形网格，图像和新的扩展类型），以及隐藏本地使用的表示类型的多态场景表示上的高效多态算子。根据核心假设，多态场景表示和算子实现了无缝集成，以及从根本上增强了对各种场景信息（即，几何形状、外观、形状组成和场景动态）的渐进式实时重建。多态场景表示和相应算子的开发和使用阿利亚需要高效的基于GPU的处理和存储技术、包括先前未充分利用的表示类型（诸如多边形网格和图像）以及用于场景信息的高效融合和抽象的新概念。多态场景表示，使用多种表示类型的研究将导致一个更好的理论理解的基本表示类型的互补性和潜在的并行利用，以利用各自的优势。此外，多态场景表示的实际可用性将改善实时3D场景重建技术的使用，由于它们能够适应特定的时空要求，从而在实时3D场景重建之外的应用领域提供新的动力，例如在3D场景分析或可变形物体的形状相似性中。

The main goal of real-time 3D scene reconstruction is to accurately acquire and represent the geometry, as well as other properties of the scene being captured, such as appearance information, decomposition into shape primitives, and scene dynamics. While robust real-time reconstruction of 3D scene geometry has seen significant improvements, there are still substantial deficiencies in high-quality real-time acquisition of the other scene properties. The core hypothesis of this project is that the primary cause of these deficiencies is inherent in the shortcomings of the particular representation type (usually point sets or voxel grids) used to depict the scene information. Therefore, the goal of the project is to develop a polymorphic scene representation for improved real-time reconstruction of scene properties, incorporating extensive use of different representation types (points and voxels, but also polygon meshes, images and novel extended types), as well as efficient polymorphic operators on the polymorphic scene representation that hide the locally used representation type. According to the core hypothesis, the polymorphic scene representation and operators enable seamless integration, as well as fundamentally enhanced progressive real-time reconstruction of the various scene information, \ie, geometry, appearance, shape composition, and scene dynamics. The development and use of a polymorphic scene representation and corresponding operators requires, inter alia, efficient GPU-based processing and storage techniques, the inclusion of previously underutilized representation types such as polygon meshes and images, and new concepts for efficient fusion and abstraction of scene information. The study of polymorphic scene representations that use multiple representation types will lead to an improved theoretical understanding of the complementary nature of the basic representation types and the potential of their parallel utilization to leverage their individual strengths. In addition, the practical availability of polymorphic scene representations will improve the use of real-time 3D scene reconstruction techniques due to their ability to adapt to specific, spatio-temporal requirements, thus providing new momentum in application fields beyond real-time 3D scene reconstruction, such as in 3D scene analysis or shape similarity of deformable objects.