Computer Vision, Imaging and Computer Graphics Theory and Applications - 17th International Joint Conference, VISIGRAPP 2022, Virtual Event, February 6-8, 2022, Revised Selected Papers
Computer Vision, Imaging and Computer Graphics Theory and Applications - 17th International Joint Conference, VISIGRAPP 2022, Virtual Event, February 6-8, 2022, Revised Selected Papers
复制标题
计算机视觉、成像和计算机图形理论与应用 - 第 17 届国际联合会议，VISIGRAPP 2022，虚拟活动，2022 年 2 月 6-8 日，修订后的精选论文
DOI：
10.1007/978-3-031-45725-8_4
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
Heng Y
中科院分区：
文献类型：
--
作者：
Heng Y
密集材料分割任务旨在识别日常图像中每个像素的材料。这对机器人操作和空间音频合成等应用是有益的。现代深度学习方法将联合收割机材料特征与上下文特征相结合。材料特征可以概括为看不见的图像，而不管外观属性，如材料形状和颜色。上下文特征可以通过提供关于图像的额外的全局或半全局信息来降低分割的不确定性。最近的研究提出将图像裁剪成补丁，这迫使网络从局部视觉线索中学习材料特征。典型的上下文信息包括从针对对象和地点相关任务的网络中提取的特征图。然而，由于缺乏上下文标签，现有方法使用预先训练的网络来提供上下文特征。因此，经过训练的网络没有给出有希望的性能。它们的准确率低于70%，并且预测的片段具有粗糙的边界。考虑到这个问题，本章介绍了上下文感知材料分割网络（CAM-SegNet）。CAM-SegNet是一种混合网络架构，可以同时从上下文和材料特征以及标记材料中学习。CAM-SegNet的有效性通过训练网络学习边界相关的上下文特征来证明。由于现有的材料数据集是稀疏的标记，自训练的方法来填充未标记的像素。实验表明，CAM-SegNet可以正确识别材料，即使具有相似的外观。该网络将像素精度提高了3-20%，并将平均IoU提高了6- 28%。
The dense material segmentation task aims at recognising the material for every pixel in daily images. It is beneficial to applications such as robot manipulation and spatial audio synthesis. Modern deep-learning methods combine material features with contextual features. Material features can generalise to unseen images regardless of appearance properties such as material shape and colour. Contextual features can reduce the segmentation uncertainty by providing extra global or semi-global information about the image. Recent studies proposed to crop the images into patches, which forces the network to learn material features from local visual clues. Typical contextual information includes extracted feature maps from networks targeting object and place related tasks. However, due to the lack of contextual labels, existing methods use pre-trained networks to provide contextual features. As a consequence, the trained networks do not give a promising performance. Their accuracy is below 70%, and the predicted segments have coarse boundaries. Considering this problem, this chapter introduces the Context-Aware Material Segmentation Network (CAM-SegNet). The CAM-SegNet is a hybrid network architecture to simultaneously learn from contextual and material features jointly with labelled materials. The effectiveness of the CAM-SegNet is demonstrated by training the network to learn boundary-related contextual features. Since the existing material datasets are sparsely labelled, a self-training approach is adopted to fill in the unlabelled pixels. Experiments show that CAM-SegNet can identify materials correctly, even with similar appearances. The network improves the pixel accuracy by 3–20% and raises the Mean IoU by 6–28%.