Plant-part segmentation using deep learning and multi-view vision
Plant-part segmentation using deep learning and multi-view vision
复制标题
使用深度学习和多视图视觉进行植物部分分割
DOI：
10.1016/j.biosystemseng.2019.08.014
复制
发表时间：
2019-11-01
影响因子：
5.1
通讯作者：
Kootstra, Gert
中科院分区：
文献类型：
--
作者：
Shi, Weinan;van de Zedde, Rick;Kootstra, Gert
为了加速对基因型和表型之间关系的理解，植物科学家和植物育种家正在寻找更先进的表型系统，以提供关于植物的更详细的表型信息。大多数现有系统提供的是整株植物的信息，而不是叶、节和茎等具体植物部分的信息。计算机视觉提供了从图像中提取植物部分信息的可能性。然而，由于自然对象的外观和形状的固有变化，植物部分的分割是一个具有挑战性的问题。本文提出了深度学习方法来处理这种变化。此外，采用多视图方法，其允许将来自二维（2D）图像的信息集成到植物的三维（3D）点云模型中。具体来说，使用全卷积网络（FCN）和掩码R-CNN（基于区域的卷积神经网络）对2D图像进行语义和实例分割。然后将不同的视点组合以分割3D点云。在番茄幼苗上评估了2D和多视图方法的性能。我们的研究结果表明，在3D信息的整合优于2D的方法，因为在2D的错误是不持久的不同的观点，因此可以在3D克服。(C)2019年IAgRE。由爱思唯尔有限公司出版。保留所有权利。
To accelerate the understanding of the relationship between genotype and phenotype, plant scientists and plant breeders are looking for more advanced phenotyping systems that provide more detailed phenotypic information about plants. Most current systems provide information on the whole-plant level and not on the level of specific plant parts such as leaves, nodes and stems. Computer vision provides possibilities to extract information from plant parts from images. However, the segmentation of plant parts is a challenging problem, due to the inherent variation in appearance and shape of natural objects. In this paper, deep-learning methods are proposed to deal with this variation. Moreover, a multi-view approach is taken that allows the integration of information from the two-dimensional (2D) images into a three-dimensional (3D) point-cloud model of the plant. Specifically, a fully convolutional network (FCN) and a masked R-CNN (region-based convolutional neural network) were used for semantic and instance segmentation on the 2D images. The different viewpoints were then combined to segment the 3D point cloud. The performance of the 2D and multi-view approaches was evaluated on tomato seedling plants. Our results show that the integration of information in 3D outperforms the 2D approach, because errors in 2D are not persistent for the different viewpoints and can therefore be overcome in 3D. (C) 2019 IAgrE. Published by Elsevier Ltd. All rights reserved.