The Neurally-Guided Shape Parser: Grammar-based Labeling of 3D Shape Regions with Approximate Inference
The Neurally-Guided Shape Parser: Grammar-based Labeling of 3D Shape Regions with Approximate Inference
复制标题
DOI：
10.1109/cvpr52688.2022.01132
复制
发表时间：
2021-06
期刊：
影响因子：
--
通讯作者：
R. K. Jones;Brown University;Aalia Habib;Rana Hanocka
中科院分区：
文献类型：
--
作者：
R. K. Jones;Brown University;Aalia Habib;Rana Hanocka
我们提出了神经引导形状解析器（NGSP），这是一种学习如何为3D形状的区域分配细粒度语义标签的方法。NGSP通过MAP推理解决了这个问题，用一个学习到的似然函数对输入形状条件下标签分配的后验概率进行建模。为了使搜索易于处理，NGSP使用了一个神经引导网络来学习逼近后验。NGSP首先用引导网络对提案进行抽样，然后在全似然下对每个提案进行评估，从而找到高概率的标签分配。我们评估了NGSP对来自PartNet的人造3D形状的细粒度语义分割任务，其中形状已被分解为对应于零件实例过度分割的区域。我们发现，相对于(i)使用区域对每点预测进行分组，（ii）使用区域作为自我监督信号或（iii）在替代公式下为区域分配标签的比较方法，NGSP提供了显著的性能改进。此外，我们表明，即使在有限的标记数据或噪声输入形状区域，NGSP仍然保持强大的性能。最后，我们证明了NGSP可以直接应用于在线存储库中的CAD形状，并通过感知研究验证了其有效性。
We propose the Neurally-Guided Shape Parser (NGSP), a method that learns how to assign fine-grained semantic labels to regions of a 3D shape. NGSP solves this problem via MAP inference, modeling the posterior probability of a label assignment conditioned on an input shape with a learned likelihood function. To make this search tractable, NGSP employs a neural guide network that learns to approximate the posterior. NGSP finds high-probability label assignments by first sampling proposals with the guide network and then evaluating each proposal under the full likelihood. We evaluate NGSP on the task of fine-grained semantic segmentation of man ufactured 3D shapesfrom PartNet, where shapes have been decomposed into regions that correspond to part instance over-segmentations. We find that NGSP delivers significant performance improvements over comparison methods that (i) use regions to group per-point predictions, (ii) use regions as a self-supervisory signal or (iii) assign labels to regions under alternative formulations. Further, we show that NGSP maintains strong performance even with limited labeled data or noisy input shape regions. Finally, we demonstrate that NGSP can be directly applied to CAD shapes found in online repositories and validate its effectiveness with a perceptual study.