Machine learning-based vision for "green-on-green'' spraying

基于机器学习的“绿色对绿色”喷涂视觉

• 批准号: 2457969
• 金额: --
• 依托单位: University of Lincoln
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2020
• 资助国家: 英国
• 起止时间: 2020 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2457969
• 关键词: Machine; learning; based; vision; green

The goal of intelligent spraying is that herbicides are more precisely targeted. This reduces waste and is beneficial for the environment. A key step in such spraying is identifying weeds. Typical approaches to doing this use computer vision, typically methods based around the use of machine learning, operating on pictures taken from cameras that view weeds and crops from above.Current vision technology has proved able to handle "green-on-brown'' scenarios, producing good accuracy of detection of weeds where weeds and crops are easy to spot against very distinctly coloured backgrounds, such as soil. This is sufficient in the early stages of growth, when crops and weeds are small. However, in later stages of growth, and the canopies of crops and weeds begin to overlap, accurately and efficiently detecting weeds becomes much harder. This "green-on-green'' scenario is currently beyond what can feasibly be handled. Solving the "green-on-green'' weed detection problem is the focus of this PhD.The reason that "green-on-green'' is hard, is that we cannot rely on simple colour segmentation. In the "green-on-brown'', segmenting images into green and brown areas identifies green areas with distinctive shapes that can easily be classified. This is what is going on in existing detectors whether they are based on classical machine vision or more modern deep learning approaches. When plants overlap, the green regions no longer contain such distinctive shapes, or such large areas of distinctive shapes, and existing approaches to detection struggle as a result. The answer is to build detectors that look for things other than just colour. This PhD will pursue two lines of inquiry: adding additional dimensions to the image data, and building detectors that focus on different features, in the framework of deep learning-based vision.1) Adding dimensions. Conventional imaging uses RGB cameras which report intensity in 3 wide visual bands (the familiar red, green and blue bands). We will investigate the use of additional layers of data. One possibility that we will look at is the use of depth data generated by RGB-D cameras. These cameras report the distance to the object in each pixel, giving, in effect, information about the contours of the canopy. We will also look at the use of multispectral cameras, which provide an additional spectral band, the "red edge''. This band is between the usual red band of RGB cameras and near infra-red, and is a region in which the reflectance (what is measured by a camera) of vegetation changes a lot, meaning that the "red edge'' band tends to be very informative.2) Focusing on different features. Recent research in machine learning-based vision has begun to focus on detectors that are, in effect, ensembles where each member looks for different elements in an image. A typical example is to use ensemble members that are trained on objects of different size, that generates detectors that are somewhat scale invariant. We will focus on identifying features that are relevant for weed detection training detectors to identify just these features, and then combining these detectors.Accuracy of detection, while important, is not the only consideration. The methods we develop also need to run fast enough that they can be deployed on a sprayer moving at 15km/h. In general, when using machine learning-based approaches, there is a tradeoff between the accuracy that can be obtained, and the speed with which images can be processed. This tradeoff can be managed along a number of dimensions, such as image size, depth of image processing backbone, and the complexity of the detection head. We will examine this tradeoff for the various approaches that we consider. In doing this, we will take into account various options for processing that are available as part of the setup at Riseholme, processing on-board the sprayer, edge-processing using the 5G setup, and cloud processing, and other options such as FGPA.

智能喷洒的目标是使除草剂更精确地靶向。这减少了浪费，对环境有益。这种喷洒的关键步骤是识别杂草。实现这一目标的典型方法使用计算机视觉，通常是基于机器学习的方法，对从上方观察杂草和农作物的相机拍摄的照片进行操作。当前的视觉技术已被证明能够处理“绿色对棕色”的场景，产生良好的杂草检测准确性，其中杂草和农作物很容易在非常明显的彩色背景（例如土壤）中被发现。这在生长的早期阶段就足够了，那时作物和杂草都很小。然而，在生长的后期，作物和杂草的树冠开始重叠，准确和有效地检测杂草变得更加困难。这种"绿色对绿色"的情况目前超出了可行的处理范围。解决“绿色对绿色”杂草检测问题是这个博士的重点。“绿色对绿色”很难的原因是我们不能依靠简单的颜色分割。在"绿色-棕色"中，将图像分割成绿色和棕色区域识别具有可容易地分类的独特形状的绿色区域。这就是现有检测器的情况，无论它们是基于经典机器视觉还是更现代的深度学习方法。当植物重叠时，绿色区域不再包含如此独特的形状，或如此大面积的独特形状，并且现有的检测方法因此而挣扎。答案是建造探测器，寻找不仅仅是颜色的东西。这个博士将追求两条研究路线：在基于深度学习的视觉框架中，为图像数据添加额外的维度，并构建专注于不同特征的检测器。常规成像使用RGB相机，其报告3个宽可见波段（熟悉的红色、绿色和蓝色波段）中的强度。我们将研究额外数据层的使用。我们将研究的一种可能性是使用RGB-D相机生成的深度数据。这些相机报告每个像素中到物体的距离，实际上，给出了关于树冠轮廓的信息。我们还将研究多光谱相机的使用，它提供了一个额外的光谱带，即“红边”。该波段介于RGB相机的通常红色波段和近红外之间，是植被反射率（相机测量的）变化很大的区域，这意味着"红边"波段往往信息量很大。2）关注不同的特征。最近基于机器学习的视觉研究已经开始关注检测器，实际上，每个成员都在图像中寻找不同的元素。一个典型的例子是使用在不同大小的对象上训练的集合成员，这会生成在某种程度上尺度不变的检测器。我们将重点关注识别与杂草检测相关的特征训练检测器，以仅识别这些特征，然后组合这些检测器。检测的准确性虽然很重要，但并不是唯一的考虑因素。我们开发的方法还需要运行得足够快，以便可以在以15公里/小时的速度移动的喷雾器上部署。一般来说，当使用基于机器学习的方法时，在可以获得的准确性和可以处理图像的速度之间存在权衡。这种折衷可以沿着多个维度进行沿着管理，例如图像尺寸、图像处理主干的深度和检测头的复杂性。我们将针对我们考虑的各种方法来检查这种权衡。在此过程中，我们将考虑作为Riseholme设置的一部分提供的各种处理选项，喷雾器上的处理，使用5G设置的边缘处理，云处理以及其他选项，如FGPA。