Polymorphic Scene Representation for Enhanced Instant Scene Reconstruction

实时三维场景重建的主要目标是准确地获取和表示几何形状，以及被捕获场景的其他属性，如外观信息、形状原语分解和场景动态。虽然3D场景几何的鲁棒实时重建已经有了显著的改进，但在其他场景属性的高质量实时获取方面仍然存在实质性的不足。这个项目的核心假设是，这些缺陷的主要原因是用于描述场景信息的特定表示类型（通常是点集或体素网格）的固有缺陷。因此，该项目的目标是开发一种多态场景表示，以改进场景属性的实时重建，包括广泛使用不同的表示类型（点和体素，但也包括多边形网格，图像和新的扩展类型），以及在多态场景表示上有效的多态算子，隐藏局部使用的表示类型。根据核心假设，多态场景表示和操作实现了无缝集成，并从根本上增强了各种场景信息的渐进式实时重建，如几何、外观、形状组成和场景动态。多态场景表示和相应运算符的开发和使用需要高效的基于gpu的处理和存储技术，包括以前未充分利用的表示类型，如多边形网格和图像，以及高效融合和抽象场景信息的新概念。对使用多种表示类型的多态场景表示的研究将有助于提高对基本表示类型的互补性以及它们的并行利用以利用其各自优势的潜力的理论理解。此外，多态场景表示的实际可用性将改善实时3D场景重建技术的使用，因为它们能够适应特定的时空要求，从而为实时3D场景重建以外的应用领域提供新的动力，例如3D场景分析或可变形物体的形状相似性。

The main goal of real-time 3D scene reconstruction is to accurately acquire and represent the geometry, as well as other properties of the scene being captured, such as appearance information, decomposition into shape primitives, and scene dynamics. While robust real-time reconstruction of 3D scene geometry has seen significant improvements, there are still substantial deficiencies in high-quality real-time acquisition of the other scene properties. The core hypothesis of this project is that the primary cause of these deficiencies is inherent in the shortcomings of the particular representation type (usually point sets or voxel grids) used to depict the scene information. Therefore, the goal of the project is to develop a polymorphic scene representation for improved real-time reconstruction of scene properties, incorporating extensive use of different representation types (points and voxels, but also polygon meshes, images and novel extended types), as well as efficient polymorphic operators on the polymorphic scene representation that hide the locally used representation type. According to the core hypothesis, the polymorphic scene representation and operators enable seamless integration, as well as fundamentally enhanced progressive real-time reconstruction of the various scene information, \ie, geometry, appearance, shape composition, and scene dynamics. The development and use of a polymorphic scene representation and corresponding operators requires, inter alia, efficient GPU-based processing and storage techniques, the inclusion of previously underutilized representation types such as polygon meshes and images, and new concepts for efficient fusion and abstraction of scene information. The study of polymorphic scene representations that use multiple representation types will lead to an improved theoretical understanding of the complementary nature of the basic representation types and the potential of their parallel utilization to leverage their individual strengths. In addition, the practical availability of polymorphic scene representations will improve the use of real-time 3D scene reconstruction techniques due to their ability to adapt to specific, spatio-temporal requirements, thus providing new momentum in application fields beyond real-time 3D scene reconstruction, such as in 3D scene analysis or shape similarity of deformable objects.