MEMTRACK: A Deep Learning-Based Approach to Microrobot Tracking in Dense and Low-Contrast Environments-猫眼课题宝

DOI：

10.48550/arxiv.2310.09441

发表时间：

2023-10

期刊：

ArXiv

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Medha Sawhney;Bhas Karmarkar;E. Leaman;Arka Daw;A. Karpatne;B. Behkam

考虑到微型机器人的微小尺寸和高速，跟踪它们是一项挑战。随着该领域朝着开发用于生物医学应用的微型机器人和在生理相关介质中进行机械研究（例如，胶原蛋白），这种挑战被具有与微型机器人相当的特征尺寸和形状的密集周围环境加剧。在这里，我们报告了运动增强多级跟踪器（MEMTrack），这是一个强大的管道，用于使用合成运动特征，基于深度学习的对象检测和修改的简单在线和实时跟踪（SORT）算法来检测和跟踪微型机器人。我们的目标检测方法结合了不同的模型的基础上的对象的运动模式。我们使用胶原蛋白（组织模型）中的细菌微马达训练和验证了我们的模型，并在胶原蛋白和水介质中进行了测试。我们证明，MEMTrack可以准确地跟踪熟练的人类注释者错过的最具挑战性的细菌，在胶原蛋白中的精确度和召回率分别为77%和48%，在液体培养基中分别为94%和35%。此外，我们表明MEMTrack可以量化平均细菌速度，与人工生成的手动跟踪数据没有统计学显著差异。MEMTrack代表了对微型机器人定位和跟踪的重大贡献，并为基于视觉的深度学习方法在密集和低对比度环境中进行微型机器人控制开辟了潜力。用于训练和测试MEMTrack以及复制论文结果的所有源代码都已在https://github.com/sawhney-medha/MEMTrack上公开。

Tracking microrobots is challenging, considering their minute size and high speed. As the field progresses towards developing microrobots for biomedical applications and conducting mechanistic studies in physiologically relevant media (e.g., collagen), this challenge is exacerbated by the dense surrounding environments with feature size and shape comparable to microrobots. Herein, we report Motion Enhanced Multi-level Tracker (MEMTrack), a robust pipeline for detecting and tracking microrobots using synthetic motion features, deep learning-based object detection, and a modified Simple Online and Real-time Tracking (SORT) algorithm with interpolation for tracking. Our object detection approach combines different models based on the object's motion pattern. We trained and validated our model using bacterial micro-motors in collagen (tissue phantom) and tested it in collagen and aqueous media. We demonstrate that MEMTrack accurately tracks even the most challenging bacteria missed by skilled human annotators, achieving precision and recall of 77% and 48% in collagen and 94% and 35% in liquid media, respectively. Moreover, we show that MEMTrack can quantify average bacteria speed with no statistically significant difference from the laboriously-produced manual tracking data. MEMTrack represents a significant contribution to microrobot localization and tracking, and opens the potential for vision-based deep learning approaches to microrobot control in dense and low-contrast settings. All source code for training and testing MEMTrack and reproducing the results of the paper have been made publicly available https://github.com/sawhney-medha/MEMTrack.

MEMTRACK: A Deep Learning-Based Approach to Microrobot Tracking in Dense and Low-Contrast Environments

文献摘要

相似文献