SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequences
SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequences
复制标题
DOI：
10.1109/cvpr46437.2021.00743
复制
发表时间：
2021-03
期刊：
影响因子：
--
通讯作者：
Shun-cheng Wu;Johanna Wald;Keisuke Tateno;N. Navab;Federico Tombari
中科院分区：
文献类型：
--
作者：
Shun-cheng Wu;Johanna Wald;Keisuke Tateno;N. Navab;Federico Tombari
场景图是一种紧凑而明确的表示，成功地用于各种2D场景理解任务。这项工作提出了一种方法来逐步建立语义场景图从3D环境给定的一系列RGB-D帧。为此，我们通过图形神经网络从原始场景组件中聚合PointNet特征。我们还提出了一种新的注意力机制，非常适合在这样的增量重建方案中存在的部分和丢失的图形数据。虽然我们所提出的方法是设计运行在场景的子地图，我们表明它也转移到整个3D场景。实验表明，我们的方法优于3D场景图预测方法的一个很大的利润，其准确性是与其他3D语义和全景分割方法，而运行在35 Hz。
Scene graphs are a compact and explicit representation successfully used in a variety of 2D scene understanding tasks. This work proposes a method to incrementally build up semantic scene graphs from a 3D environment given a sequence of RGB-D frames. To this end, we aggregate PointNet features from primitive scene components by means of a graph neural network. We also propose a novel attention mechanism well suited for partial and missing graph data present in such an incremental reconstruction scenario. Although our proposed method is designed to run on submaps of the scene, we show it also transfers to entire 3D scenes. Experiments show that our approach outperforms 3D scene graph prediction methods by a large margin and its accuracy is on par with other 3D semantic and panoptic segmentation methods while running at 35Hz.