Deep Shape Representation for Shape Analysis, Modeling, and Reconstruction

数字3D模型在从工业设计、数字媒体和娱乐到虚拟现实和3D打印的各种应用中都是必不可少的。对于传统的3D模型生成，用户必须在专业的CAD软件上投入大量时间和/或需要购买昂贵的设备，如激光扫描仪。近年来，随着互联网上大型3D模型存储库的可用性不断增加，从几何设计到数据驱动方法的范式转变变得可行，其中智能建模系统通过利用源自预先存在的设计的（统计）知识来支持用户。深度学习的最新进展非常有前途，并孕育了在几何任务中可以期待类似突破的希望。然而，3D模型在许多方面与2D图像或视频有很大不同。3D模型（volumetric或B-rep）可以具有复杂的拓扑和结构，多个数量级的细节和特征，以及相关的纹理等辅助属性。已建立的深度学习应用的几何表示不能同时支持所有这些方面。因此，ICT-CAS和RWTH打算在国际合作中彻底解决这一问题。具体来说：(1)我们将研究一种新的三维几何表示，它将分层组成（结构）与几何变形（形状）和属性映射（外观）相结合。该表示将适用于深度神经网络的高效处理。(2)对于这种表示，我们将开发一些基本的低级操作，如分割和分类，形状抽象和匹配以及对称分析。(3)基本操作将使我们能够解决具有挑战性的高级任务，如单视图重建和复杂的（交互式）数据驱动的3D建模工具，这些工具通过解释用户的意图和基于从大型对象存储库派生的（统计）知识以数据驱动的方式创建合理的3D模型来支持用户。

Digital 3D models are essential in a wide spectrum of diverse applications ranging from industrial design, digital media and entertainment to virtual reality, and 3D printing. For traditional 3D model generation, users have to invest a lot of time on professional CAD software and/or need to acquire expensive equipment such as laser scanners. With the increasing availability of large 3D model repositories on the internet in recent years, a paradigm shift becomes feasible from geometric design to data driven approaches where an intelligent modeling system supports the user by leveraging (statistical) knowledge that has been derived from pre-existing designs. Recent advances in deep learning are quite promising and nourish the hope that similar breakthroughs can be expected for geometric tasks. However, 3D models are very different from 2D images or videos in a number of aspects. 3D models (volumetric or B-rep) can have a complex topology and structure, details and features across several orders of magnitude, and auxiliary attributes like textures associated. Established geometry representations for deep learning applications do not support all of these aspects simultaneously. Therefore, ICT-CAS and RWTH intend to thoroughly address this issue in an international collaboration. Specifically:(1) We will investigate a novel representation of 3D geometry that combines hierarchical composition (structure) with geometry deformation (shape) and attribute mapping (appearance). The representation will be suitable for efficient and effective processing with deep neural networks.(2) For this representation we will develop a number of fundamental low-level operations like segmentation and classification, shape abstraction and matching as well as symmetry analysis.(3) The fundamental operations will allow us to solve challenging high level tasks like single view reconstruction and sophisticated (interactive) data driven 3D modeling tools which support the user by interpreting the user's intention and creating plausible 3D models in a data driven manner on the basis of (statistical) knowledge derived from large repositories of objects.