Real-time insect tracking and monitoring with computer vision and deep learning
Real-time insect tracking and monitoring with computer vision and deep learning
复制标题
DOI：
10.1002/rse2.245
复制
发表时间：
2021-11-30
影响因子：
5.5
通讯作者：
Hoye, Toke Thomas
中科院分区：
文献类型：
--
作者：
Bjerge, Kim;Mann, Hjalte M. R.;Hoye, Toke Thomas
昆虫的数量和多样性正在下降，但由于昆虫难以监测，其种群趋势仍然不确定。手动方法需要在诱捕和随后的物种鉴定上投入大量时间。相机陷阱可以减轻一些手动现场工作，但大量图像数据的分析具有挑战性。通过使用计算机视觉技术将图像分析嵌入到记录过程中，可以将精力集中在与生态最相关的图像数据上。在这里，我们展示了一种智能摄像系统，能够在原位检测、跟踪和识别单个昆虫。我们使用商业现成组件构建了该系统，并使用深度学习开源软件来执行物种检测和分类。我们提出了昆虫分类和跟踪算法 (ICT)，该算法以每秒 0.33 帧的速度执行实时分类和跟踪。该系统可以每天通过互联网将昆虫身份和移动轨迹的汇总数据上传到服务器。我们在 2020 年夏季测试了我们的系统，并在 98 天内检测到了 2994 个昆虫踪迹。我们对 8 个不同物种的昆虫足迹进行正确分类，平均准确度达到 89%。该结果基于 10 天内在视频中观察到的 504 条经过手动验证的昆虫活动轨迹。使用跟踪数据，我们可以估计相机视野内单个访问花朵的昆虫的平均停留时间，并且我们能够显示昆虫类群之间停留时间的显着差异。对于数量最多的蜜蜂来说，停留时间也随着季节的变化而变化，与开花的植物种类有关。我们提出的自动化系统在昆虫的非破坏性和实时监测方面显示出了有希望的结果，并提供了有关花的昆虫的物候、丰度、觅食行为和运动生态学的新信息。
Insects are declining in abundance and diversity, but their population trends remain uncertain as insects are difficult to monitor. Manual methods require substantial time investment in trapping and subsequent species identification. Camera trapping can alleviate some of the manual fieldwork, but the large quantities of image data are challenging to analyse. By embedding the image analyses into the recording process using computer vision techniques, it is possible to focus efforts on the most ecologically relevant image data. Here, we present an intelligent camera system, capable of detecting, tracking, and identifying individual insects in situ. We constructed the system from commercial off-the-shelf components and used deep learning open source software to perform species detection and classification. We present the Insect Classification and Tracking algorithm (ICT) that performs real-time classification and tracking at 0.33 frames per second. The system can upload summary data on the identity and movement track of insects to a server via the internet on a daily basis. We tested our system during the summer 2020 and detected 2994 insect tracks across 98 days. We achieved an average precision of 89% for correctly classified insect tracks of eight different species. This result was based on 504 manually verified tracks observed in videos during 10 days with varying insect activities. Using the track data, we could estimate the mean residence time for individual flower visiting insects within the field of view of the camera, and we were able to show a substantial variation in residence time among insect taxa. For honeybees, which were most abundant, residence time also varied through the season in relation to the plant species in bloom. Our proposed automated system showed promising results in non-destructive and real-time monitoring of insects and provides novel information about phenology, abundance, foraging behaviour, and movement ecology of flower visiting insects.