Object-oriented 3D semantic mapping based on instance segmentation
Object-oriented 3D semantic mapping based on instance segmentation
复制标题
基于实例分割的面向对象3D语义映射
DOI：
10.20965/jaciii.2019.p0695
复制
发表时间：
2019
影响因子：
0.7
通讯作者：
Tian Guohui
中科院分区：
文献类型：
--
作者：
Chi Jinxin;Wu Hao;Tian Guohui
服务机器人通过语义映射获得环境的几何和语义信息，提供更加智能的服务。然而，大多数语义映射的研究，迄今为止需要先验知识的三维对象模型或地图与几个对象类别，忽略单独的个体对象。针对这些问题，提出了一种面向对象的三维语义映射方法，将基于深度学习的实例分割和视觉同步定位与映射（SLAM）算法相结合，帮助机器人不仅获得面向导航的周围环境几何信息，还获得面向个体的物体属性和位置信息。同时，结合视觉SLAM，提出了一种适用于连续图像帧的目标识别与关联算法，利用图像帧间的视觉一致性，提升连续图像帧的目标匹配与识别效果，提高目标识别的准确率。最后，基于Mask R-CNN和ORB-SLAM 2框架实现了一个三维语义映射系统。在ICL-NUIM数据集上进行了仿真实验，实验结果表明，该系统能够较好地识别出场景中的所有类型的物体，并生成这些物体的精细点云模型，验证了算法的有效性。
Service robots gain both geometric and semantic information about the environment with the help of semantic mapping, providing more intelligent services. However, a majority of studies for semantic mapping thus far require priori knowledge 3D object models or maps with a few object categories that neglect separate individual objects. In view of these problems, an object-oriented 3D semantic mapping method is proposed by combining state-of-the-art deep-learning-based instance segmentation and a visual simultaneous localization and mapping (SLAM) algorithm, which helps robots not only gain navigation-oriented geometric information about the surrounding environment, but also obtain individually-oriented attribute and location information about the objects. Meanwhile, an object recognition and target association algorithm applied to continuous image frames is proposed by combining visual SLAM, which uses visual consistency between image frames to promote the result of object matching and recognition over continuous image frames, and improve the object recognition accuracy. Finally, a 3D semantic mapping system is implemented based on Mask R-CNN and ORB-SLAM2 frameworks. A simulation experiment is carried out on the ICL-NUIM dataset and the experimental results show that the system can generally recognize all the types of objects in the scene and generate fine point cloud models of these objects, which verifies the effectiveness of our algorithm.