Field‐based robotic leaf angle detection and characterization of maize plants using stereo vision and deep convolutional neural networks
Field‐based robotic leaf angle detection and characterization of maize plants using stereo vision and deep convolutional neural networks
复制标题
DOI：
10.1002/rob.22166
复制
发表时间：
2023-02
影响因子：
8.3
通讯作者：
Lirong Xiang;Jingyao Gai;Yin Bao;Jianming Yu;P. Schnable;Lie Tang
中科院分区：
文献类型：
--
作者：
Lirong Xiang;Jingyao Gai;Yin Bao;Jianming Yu;P. Schnable;Lie Tang
玉米（Zea mays L.）是世界三大粮食作物之一。叶角是作物的一个重要的结构性状，因为它在冠层的光截获和光合效率中起着重要的作用。传统上，叶片角度是用量角器测量的，这个过程既缓慢又费力。由于冠层中的叶片密度和由此产生的遮挡，通过成像在田间条件下有效地测量叶片角度是具有挑战性的。然而，成像技术和机器学习的进步为图像采集和分析提供了新的工具，可用于使用田间生长植物的三维（3D）模型来表征叶角。在这项研究中，PhenoBot 3.0是一种机器人车辆，设计用于在成对的农艺学间隔作物行之间穿行，配备了多层PhenoStereo相机，以捕获田间玉米植株的侧视图像。PhenoStereo是一款定制的立体相机模块，集成了频闪照明，可在可变的室外照明条件下高速采集立体图像。一个自动化的图像处理管道（AngleNet）被开发用于测量非遮挡叶片的叶角。在该流水线中，提出了一种新的表示形式的叶角作为一个三元组的关键点。该管道采用卷积神经网络来检测二维图像中的每个叶片角度，并采用3D建模方法从重建模型中提取定量数据。叶角的相关系数（r）和平均绝对误差（MAE）均达到令人满意的精度（r > 0. 87，M A E 0. 99，M A E <3. 5 cm $r\gt 0. 99，\unicode {x 02007}MAE\lt \phantom{\rule{}{0 ex}} 3. 5\unicode {x 0200 A}\mathrm{cm}$）。本研究证明了在田间条件下利用立体视觉研究玉米叶片角度分布的可行性。该系统是一种有效的替代传统的叶角表型，从而可以加速育种改善植物结构。
Maize (Zea mays L.) is one of the three major cereal crops in the world. Leaf angle is an important architectural trait of crops due to its substantial role in light interception by the canopy and hence photosynthetic efficiency. Traditionally, leaf angle has been measured using a protractor, a process that is both slow and laborious. Efficiently measuring leaf angle under field conditions via imaging is challenging due to leaf density in the canopy and the resulting occlusions. However, advances in imaging technologies and machine learning have provided new tools for image acquisition and analysis that could be used to characterize leaf angle using three‐dimensional (3D) models of field‐grown plants. In this study, PhenoBot 3.0, a robotic vehicle designed to traverse between pairs of agronomically spaced rows of crops, was equipped with multiple tiers of PhenoStereo cameras to capture side‐view images of maize plants in the field. PhenoStereo is a customized stereo camera module with integrated strobe lighting for high‐speed stereoscopic image acquisition under variable outdoor lighting conditions. An automated image processing pipeline (AngleNet) was developed to measure leaf angles of nonoccluded leaves. In this pipeline, a novel representation form of leaf angle as a triplet of keypoints was proposed. The pipeline employs convolutional neural networks to detect each leaf angle in two‐dimensional images and 3D modeling approaches to extract quantitative data from reconstructed models. Satisfactory accuracies in terms of correlation coefficient (r) and mean absolute error (MAE) were achieved for leaf angle ( r > 0.87 , M A E 0.99 , M A E < 3.5 cm $r\gt 0.99,\unicode{x02007}MAE\lt \phantom{\rule{}{0ex}}3.5\unicode{x0200A}\mathrm{cm}$ ). Our study demonstrates the feasibility of using stereo vision to investigate the distribution of leaf angles in maize under field conditions. The proposed system is an efficient alternative to traditional leaf angle phenotyping and thus could accelerate breeding for improved plant architecture.