Detection of strawberries with varying maturity levels for robotic harvesting using YOLOv4
Detection of strawberries with varying maturity levels for robotic harvesting using YOLOv4
复制标题
使用 YOLOv4 检测不同成熟度的草莓以供机器人采摘
DOI：
10.13031/aim.202100051
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Priyanka Upadhayay
中科院分区：
文献类型：
--
作者：
Zixuan He;M. Karkee;Priyanka Upadhayay
抽象的。草莓的不准确检测和/或定位将在机器人采摘期间导致水果损伤或失败尝试。在这项工作中，提出了一种方法来准确地检测和定位草莓结合对象检测网络，YOLOv 4，和分类网络，Alexnet。YOLOv 4模型用于将草莓检测到各种成熟组（花、未成熟、接近成熟、成熟和过熟浆果）并提供它们的位置信息，而Alexnet模型用于评估检测到的成熟草莓是否完全或部分可见。与针对常规对象训练的YOLOv 2，YOLOv 3和YOLOv 4相比，YOLOv 4模型专门训练用于检测小对象，在检测成熟草莓时，其平均平均精度最高，为80.7%，F1得分为0.80，平均精度（AP）为91.7%。YOLOv 4在单张图像上实现了55 ms的高处理速度（分辨率：1200 x1000像素）。利用从点云生成的RGB图像进一步验证了该模型，其中它实现了90.15%的AP，这表明该模型对于在不同设置下收集的图像中检测浆果是鲁棒的。同样，Alexnet模型在将成熟草莓分类为完全可见和部分可见组方面实现了90.0%的准确性，处理速度为3 ms/图像（分辨率：227 x227 x3像素）。该技术还为收获机器人提供了草莓的2D到3D映射。这种方法显示出强大的潜力，作为一种手段，提供准确的草莓检测机器人收割机所需的，特别是与合作的双机械手系统。
Abstract. Inaccurate detection and/or localization of strawberries will cause fruit injury or failed attempt during robotic picking. In this work, a method is proposed to accurately detect and localize strawberries combining an object detection network, YOLOv4, and a classification network, Alexnet. YOLOv4 model was used to detect strawberries into various maturity groups (flower, immature, nearly mature, mature, and overripen berries) and provide their location information whereas Alexnet model was used for assessing if the detected matured strawberries were completely or partially visible. Compared to the same achieved by YOLOv2, YOLOv3 and YOLOv4 trained for regular objects,YOLOv4 model was specifically trained to detect small objects, which achieved the highest mean average precision of 80.7% and F1 score of 0.80 with an average precision (AP) of 91.7% in detecting mature strawberries. YOLOv4 achieved a high processing speed of 55 ms on single image (resolution: 1200x1000 pixels). The model was further validated with the RGB images generated from point cloud, where it achieved an AP of 90.15% showing that the model was robust to detect berries in images collected with different settings. Similarly, Alexnet model achieved an accuracy of 90.0% in classifying matured strawberries into completely and partially visible groups with processing speed of 3 ms per image (resolution: 227x227x3 pixels). This technique also provided 2D to 3D mapping of strawberries for the harvesting robots. This method showed a strong potential as a means for providing accurate strawberry detection desirable for robotic harvesters, particularly with a collaborating dual-manipulator system.