CHS: Small: Deriving and Exploiting Shape Semantics

CHS：小：形状语义的推导和利用

• 批准号: 1528025
• 负责人: Leonidas Guibas
• 金额: $ 50万
• 依托单位: Stanford University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-09-01 至 2020-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1528025&HistoricalAwards=false
• 关键词: CHS; Small; Deriving; Exploiting; Shape

3D representations are the most faithful digital encoding of physical objects, allowing us to store and manipulate all kinds of information about the object, both high-level (e.g., affordances and functionality) and low-level (e.g., appearance and material). Furthermore, they do so in a way that is more complete than 2D images or entirely symbolic representations such as text-based knowledge graphs. Yet associating semantic information with 3D models is not easy, because data that directly links 3D models to their function and use, their semantic parts and attributes, is not widely available. Given the emergence of 3D shape repositories on the Web supporting a variety of applications (such as 3D printing), plus the availability of affordable 3D scanners and their incorporation in computers and mobile devices, the time is ripe to enrich these repositories with semantic information that will vastly increase their accessibility and usefulness beyond the specialized applications for which they were originally developed. The PI's goal in this research is to do just that, by developing mathematical and algorithmic techniques which extract, encode, and exploit the semantics of 3D models; these in turn will be combined in a novel search engine for 3D models that will exploit semantic shape attributes in a unified way and will allow much broader access to and use of 3D repositories, thereby ultimately supporting many commercial applications while also proving useful to other research communities.Toward these ends, the PI will acquire semantic information about shapes by a combination of geometric analysis and user annotation. The geometric analysis is not only of shapes in isolation, but instead a joint analysis of collections of related shapes. The aim is to build networks among 3D models that can transfer information between them. Through these networks, using novel mathematical techniques, the PI will enable understanding of the shared structure as well as the variability of shapes in a collection. Common parts or structures in shapes invariably have semantic significance, and the role of a shape in a collection (its relationships to its partner and peer shapes) often defines semantic attributes of the shape. Since user annotations are expensive to obtain, the plan is to develop tools that exploit the above-mentioned networks so that only a modest number of annotations need to be obtained by crowd-sourcing queries. However, user annotations will be sparse and noisy, so the PI will also develop techniques for cleaning them up, as well as for propagating and aggregating them. Understanding how to integrate semantic structure reflected in the geometry of shapes with semantic structure reflected in language is one of the deep problems the PI will tackle in this research.

3D表示是物理对象的最忠实的数字编码，允许我们存储和操纵关于对象的各种信息，无论是高级的（例如，示能表示和功能）和低级（例如，外观和材质）。 此外，它们以一种比2D图像或完全符号化的表示（如基于文本的知识图）更完整的方式来实现。 然而，将语义信息与3D模型相关联并不容易，因为直接将3D模型与其功能和用途、其语义部分和属性联系起来的数据并不广泛可用。 考虑到Web上支持各种应用程序（如3D打印）的3D形状存储库的出现，加上可负担得起的3D扫描仪的可用性及其在计算机和移动的设备中的合并，用语义信息丰富这些存储库的时机已经成熟，这将大大增加它们的可访问性和有用性，超出它们最初开发的专用应用程序。 PI在这项研究中的目标就是通过开发数学和算法技术来提取，编码和利用3D模型的语义;这些又将被组合在用于3D模型的新颖的搜索引擎中，该搜索引擎将以统一的方式利用语义形状属性并且将允许更广泛地访问和使用3D存储库，从而最终支持许多商业应用，同时也证明对其他研究团体是有用的。为此，PI将通过几何分析和用户注释的组合来获取关于形状的语义信息。 几何分析不仅是孤立的形状，而是相关形状集合的联合分析。 其目的是在3D模型之间建立网络，以便在它们之间传输信息。 通过这些网络，使用新的数学技术，PI将能够理解共享的结构以及集合中形状的可变性。 形状中的公共部分或结构总是具有语义意义，并且形状在集合中的角色（其与其伙伴和对等形状的关系）通常定义形状的语义属性。 由于获得用户注释的成本很高，因此计划开发利用上述网络的工具，以便只需通过众包查询获得少量注释。 然而，用户注释将是稀疏和嘈杂的，因此PI还将开发清理它们以及传播和聚合它们的技术。 理解如何将反映在形状几何中的语义结构与反映在语言中的语义结构相结合是PI在本研究中要解决的深层次问题之一。

项目成果

GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloud

• DOI: 10.1109/cvpr.2019.00407
• 发表时间: 2018-12
• 期刊: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 作者: L. Yi;Wang Zhao;He Wang;Minhyuk Sung;L. Guibas

Learning Transformation Synchronization

• DOI: 10.1109/cvpr.2019.00827
• 发表时间: 2019-01
• 期刊: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 作者: Xiangru Huang;Zhenxiao Liang;Xiaowei Zhou;Yao Xie;L. Guibas;Qi-Xing Huang

Deep Functional Dictionaries: Learning Consistent Semantic Structures on 3D Models from Functions

• 发表时间: 2018-05
• 作者: Minhyuk Sung;Hao Su;Ronald Yu;L. Guibas

Language-driven synthesis of 3D scenes from scene databases

• DOI: 10.1145/3272127.3275035
• 发表时间: 2018-12
• 期刊: ACM Transactions on Graphics (TOG)
• 作者: Rui Ma;A. Patil;Matthew Fisher;Manyi Li;S. Pirk;Binh-Son Hua;Sai-Kit Yeung;Xin Tong;L. Guibas-L.

TextureNet: Consistent Local Parametrizations for Learning From High-Resolution Signals on Meshes

• DOI: 10.1109/cvpr.2019.00457
• 发表时间: 2018-11
• 期刊: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 作者: Jingwei Huang;Haotian Zhang;L. Yi;T. Funkhouser;M. Nießner;L. Guibas