Simultaneous Direct Depth Estimation and Synthesis Stereo for Single Image Plant Root Reconstruction
Simultaneous Direct Depth Estimation and Synthesis Stereo for Single Image Plant Root Reconstruction
复制标题
用于单图像植物根部重建的同时直接深度估计和合成立体
DOI：
--
复制
发表时间：
2021
影响因子：
10.6
通讯作者：
G. Lu
中科院分区：
文献类型：
--
作者：
Yawen Lu;Yuxing Wang;Devarth Parikh;Awais Khan;G. Lu
植物根系是其与自然和生物环境相互作用的主要管道。3D根系结构可以提供植物茁壮成长能力的基础知识和应用知识，但是构建用于薄且复杂的植物根的3D结构是具有挑战性的。现有的方法如运动恢复结构和轮廓恢复形状等需要多幅图像作为输入，优化过程复杂，在野外工作中通常不方便。很少有人研究深度神经网络方法在从单个图像重建薄对象（如植物根系）方面的应用。我们提出了一个无监督的学习计划，估计根深度从只有一个图像作为输入，这是进一步应用于重建完整的根系。重建对象的边界通常包含较大的误差，这对于具有许多细分支的根是一个显著的问题。为了减少重建误差，我们将基于交叉视图的GAN网络集成到重建过程中，该网络从不同的角度预测根图像。基于预测视图，我们使用立体重建来重建根系，这有助于通过加强它们的一致性来识别准确的重建点。在真实的植物根系数据集和人工合成数据集上的实验结果表明，与现有的植物根系单图像三维重建模型相比，该算法具有较好的效果。
Plant roots are the main conduit to its interaction with the physical and biological environment. A 3D root system architecture can provide fundamental and applied knowledge of a plant’s ability to thrive, but the construction of 3D structures for thin and complicated plant roots is challenging. Existing methods such as structure-from-motion and shape-from-silhouette require multiple images, as input, under a complicated optimization process, which is usually not convenient in fieldwork. Little effort has been put into investigating the applications of deep neural network methods to reconstruct thin objects, like plant root systems, from a single image. We propose an unsupervised learning scheme to estimate the root depth from only one image as input, which is further applied to reconstruct the complete root system. The boundaries of the reconstructed object usually contain large errors, which is a significant problem for roots with many thin branches. To reduce reconstruction errors, we integrate a cross-view GAN-based network into the reconstruction process, which predicts the root image from a different perspective. Based on the predicted view, we reconstruct the root system using stereo reconstruction, which helps to identify the accurately reconstructed points by enforcing their consistency. The results on both the real plant root dataset and the synthetic dataset demonstrate the effectiveness of the proposed algorithm compared with state-of-the-art single image 3D reconstruction models on plant roots.