RI:Small:3D Shape Understanding and Generation using Unstructured Point Clouds

RI:Small:使用非结构化点云理解和生成 3D 形状

• 批准号: 1908669
• 负责人: Rui Wang
• 金额: $ 49.99万
• 依托单位: University of Massachusetts Amherst
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-09-01 至 2023-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1908669&HistoricalAwards=false
• 关键词: RI; Small; 3D; Shape; Understanding

3D shape understanding and generation are fundamental research problems in computer vision and graphics. Techniques to efficiently and accurately detect, classify, segment, and label 3D shapes have widespread applications in autonomous vehicles, home assistance, industrial automation, and medical engineering. Similarly, techniques to generate new shapes or complete partial shapes are instrumental in design, manufacturing, and digital preservation. While recent advances in deep learning have revolutionized 2D image understanding and generation, adapting them to 3D shapes is still a fundamental challenge. One bottleneck is the choice of 3D representation. Standard choices such as 3D voxel grids, geometric images, multi-view representations have been extensively studied, but they impose various limitations such as poor efficiency, issues with representing complex shapes, difficulties in capturing interior structures and maintaining view consistency. This project investigates suitable and efficient 3D representations that enable deep learning algorithms to achieve the level of success on 3D shapes as on 2D image understanding and generation tasks.This project aims to study shape understanding and generation using unstructured point clouds, which represent 3D shapes as a collection of points on shape surfaces. Such representation is compact, scalable, and suitable for capturing complex geometric details. For shape understanding, the research investigates new approaches, such as using various spatial data structures and multigrid tree networks, to enable deep convolutional networks on unstructured point clouds. For shape generation, the research studies effective generative models, such as using variational autoencoders and GANs, applied on point clouds to produce high-quality 3D shapes in a scalable manner. Different choices of loss functions on point clouds will be investigated in detail. The research will be evaluated on real-world datasets for shape recognition tasks, image-to-shape reconstruction and shape completion tasks. It also enables new applications such as unsupervised 3D shape induction from unlabeled image collections in the wild.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

三维形状的理解和生成是计算机视觉和图形学的基础研究问题。高效、准确地检测、分类、分割和标记3D形状的技术在自动驾驶车辆、家庭辅助、工业自动化和医疗工程中有着广泛的应用。同样，生成新形状或完整部分形状的技术在设计、制造和数字保存中也很有用。虽然深度学习的最新进展使2D图像的理解和生成发生了革命性的变化，但使它们适应3D形状仍然是一个根本的挑战。一个瓶颈是3D表示的选择。三维体素网格、几何图像、多视点表示等标准选择已经得到了广泛的研究，但它们存在各种局限性，如效率低、表示复杂形状的问题、难以捕捉内部结构和维护视图一致性等。该项目研究合适和高效的3D表示，使深度学习算法能够在3D形状上达到2D图像理解和生成任务的成功程度。该项目旨在研究使用非结构化点云的形状理解和生成，非结构化点云将3D形状表示为形状表面上的点的集合。这种表示形式紧凑、可伸缩，适合捕捉复杂的几何细节。对于形状理解，该研究探索了新的方法，例如使用各种空间数据结构和多网格树网络，以实现非结构化点云上的深度卷积网络。对于形状生成，该研究研究了有效的生成模型，例如使用变分自动编码器和GAN，应用于点云以可伸缩的方式生成高质量的3D形状。文中将详细研究点云损失函数的不同选择。这项研究将在用于形状识别任务、图像到形状重建和形状完成任务的真实数据集上进行评估。该奖项反映了NSF的法定使命，通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为是值得支持的。

项目成果

PriFit: Learning to Fit Primitives Improves Few Shot Point Cloud Segmentation

• DOI: 10.1111/cgf.14601
• 发表时间: 2021-12
• 期刊: Computer Graphics Forum
• 影响因子: 2.5
• 作者: Gopal Sharma;Bidya Dash;Aruni RoyChowdhury;Matheus Gadelha;Marios Loizou;Liangliang Cao;Rui Wang;E. Learned-Miller;Subhransu Maji;E. Kalogerakis

Inferring 3D Shapes from Image Collections Using Adversarial Networks

• DOI: 10.1007/s11263-020-01335-w
• 发表时间: 2020-06-24
• 期刊: INTERNATIONAL JOURNAL OF COMPUTER VISION
• 影响因子: 19.5
• 作者: Gadelha, Matheus;Rai, Aartika;Wang, Rui
• 通讯作者: Wang, Rui

Learning Generative Models of Shape Handles

• DOI: 10.1109/cvpr42600.2020.00048
• 发表时间: 2020-04
• 期刊: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 作者: Matheus Gadelha;Giorgio Gori;Duygu Ceylan;R. Mech;N. Carr;T. Boubekeur;Rui Wang;Subhransu Maji

Cross-Modal 3D Shape Generation and Manipulation

• DOI: 10.48550/arxiv.2207.11795
• 发表时间: 2022-07
• 作者: Zezhou Cheng;Menglei Chai;Jian Ren;Hsin-Ying Lee;Kyle Olszewski;Zeng Huang;Subhransu Maji;S. Tulyakov

Accidental Turntables: Learning 3D Pose by Watching Objects Turn

• DOI: 10.1109/iccvw60793.2023.00225
• 发表时间: 2022-12
• 期刊: 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)
• 作者: Zezhou Cheng;Matheus Gadelha;Subhransu Maji