CHS: Small: Generative models of shapes

这项研究将通过开发生成概率模型来推进三维（3D）形状合成的最新技术，该模型将能够自动理解形状几何的语义，并将导致新的计算建模算法的发展，使任何人都可以轻松地创建引人注目和高度详细的3D内容。用户只需提供高级规格、形状类型、部件、语义形状属性、地标点或基于简单直观的用户界面的草图，就可以创建形状。这些模型还将使计算机能够从测距相机获得的部分几何数据中推断出完整的几何形状，并填补任何缺失的形状部分，或者在3D传感器获得的场景中健壮地识别物体。项目成果将推动3D建模技术的发展，为用户提供直观的工具，显着降低快速轻松创建详细形状的障碍。这些工具正变得越来越重要，因为在科学和工程领域，如协作虚拟环境、增强现实、仿真、计算机辅助设计和建筑，人们对3D模型的兴趣越来越大。特别是，这项工作将显著有利于3D打印；尽管硬件有所进步，但主要的瓶颈仍然是为打印机提供形状的创建。该研究还将推进形状理解和物体识别方面的最新技术，这对计算机视觉和机器人应用非常重要。这些生成模型背后的关键思想是，它们代表了复杂的层次组成、详细几何形状特征的相关性和变化，以及它们与高级语义形状属性的关系。这些模型将从网络上可用的大型形状存储库中自动学习，在输入形状通过新算法进行预处理之后，首席研究员将开发用于同时进行形状分割和地标定位的新算法，以便它们的部分和点被一致地标记。现有的形状合成算法仅限于重用和重新组合存储库中的形状部件，或者合成特定类（如人体或面部）中的形状，具有有限的几何可变性，并且没有结构或语义可变性。另一方面，首席研究员的生成模型将学习如何密集地放置点和补丁，以在复杂领域（如家具、车辆、工具、生物等）中创建新的合理形状。建立在生成模型之上的推理算法将能够合成给定语言术语或稀疏几何输入的形状。因此，这项研究将导致新的3D内容创建工具的发展，这将改变计算建模领域：用户将执行简单，容易和直观的交互，而不是执行一系列艰苦的低级几何编辑和操作命令。

This research will advance the state of the art in three-dimensional (3D) shape synthesis by developing generative probabilistic models that will enable the automatic understanding of semantics from shape geometry, and which will lead to the development of new computational modeling algorithms that allow anybody to easily create compelling and highly detailed 3D content. Users will be able to create shapes by simply providing high-level specifications, shape types, parts, semantic shape attributes, landmark points, or sketches based on simple and intuitive user interfaces. These models will also enable the computer to infer complete geometry from partial geometric data acquired by range cameras and to fill in any missing shape parts, or to robustly recognize objects in a scene acquired by 3D sensors. Project outcomes will advance the state of the art in 3D modeling, providing users with intuitive tools that significantly lower the barrier of rapid and easy creation of detailed shapes. Such tools are becoming increasingly important, since there is a growing interest in 3D models in scientific and engineering fields such as collaborative virtual environments, augmented reality, simulation, computer-aided design, and architecture. In particular, this work will significantly benefit 3D printing; where despite hardware advances, the main bottleneck remains the creation of shapes to be supplied to the printer. The research will also advance the state of the art in shape understanding and object recognition, which are important for computer vision and robotics applications.The key idea behind these generative models is that they represent complex hierarchical compositions, correlations and variations of detailed geometric shape features, as well as their relationships with high-level semantic shape attributes. The models will be automatically learned from large shape repositories available on the Web, after the input shapes are pre-processed by new algorithms the Principal Investigator will develop for simultaneous shape segmentation and landmark localization so that their parts and points are consistently labeled. Existing shape synthesis algorithms are limited to re-use and re-combine shape parts from a repository, or synthesize shapes in specific classes (such as human bodies or faces), with limited geometric variability and no structural or semantic variability. The Principal Investigator's generative models, on the other hand, will instead learn how to densely place points and patches to create new plausible shapes in complex domains, such as furniture, vehicles, tools, creatures, etc. Inference algorithms built upon the generative models will be able to synthesize shapes given linguistic terms or sparse geometric input. As a result, the research will lead to the development of new 3D content creation tools that will transform the field of computational modeling: instead of executing a series of painstaking low-level geometric editing and manipulation commands, users will perform simple, easy, and intuitive interactions to achieve their design goals.