Visual tracking of deepwater animals using machine learning-controlled robotic underwater vehicles
Visual tracking of deepwater animals using machine learning-controlled robotic underwater vehicles
复制标题
DOI：
10.1109/wacv48630.2021.00090
复制
发表时间：
2021-01
期刊：
影响因子：
--
通讯作者：
K. Katija;P. Roberts;Joost Daniels;Alexander M. Lapides;K. Barnard;M. Risi;Ben Y Ranaan;Benjamin Woodward;Jonathan Takahashi
中科院分区：
文献类型：
--
作者：
K. Katija;P. Roberts;Joost Daniels;Alexander M. Lapides;K. Barnard;M. Risi;Ben Y Ranaan;Benjamin Woodward;Jonathan Takahashi
海洋是一个巨大的三维空间，人们对它的探索和了解很少，它孕育着对生态系统功能至关重要的未被观察到的生命和过程。为了充分询问空间，需要新的算法和机器人平台来扩大观测。在水柱中定位感兴趣的动物和扩展的视觉观察是特别具有挑战性的目标。为此，我们提出了一种新的机器学习集成跟踪（或ML跟踪）算法的水下航行器控制，建立在类的算法被称为跟踪检测。通过耦合多目标探测器（在现场水下图像数据上训练），3D立体跟踪器和监督模块来监督使命，我们展示了ML跟踪如何创建长时间观测所需的鲁棒跟踪，以及实现目标采样对象的全自动采集。我们使用远程操作潜水器作为自主水下航行器的代理，在对一种被称为管水母的中层凝胶状动物进行记录、5+小时连续观察期间，展示了从ML跟踪算法到潜水器控制器的连续输入。这些努力清楚地表明了探测跟踪算法在探索未开发环境和发现海洋中未被发现的生命方面的潜力。
The ocean is a vast three-dimensional space that is poorly explored and understood, and harbors unobserved life and processes that are vital to ecosystem function. To fully interrogate the space, novel algorithms and robotic platforms are required to scale up observations. Locating animals of interest and extended visual observations in the water column are particularly challenging objectives. Towards that end, we present a novel Machine Learning-integrated Tracking (or ML-Tracking) algorithm for underwater vehicle control that builds on the class of algorithms known as tracking-by-detection. By coupling a multi-object detector (trained on in situ underwater image data), a 3D stereo tracker, and a supervisor module to oversee the mission, we show how ML-Tracking can create robust tracks needed for long duration observations, as well as enable fully automated acquisition of objects for targeted sampling. Using a remotely operated vehicle as a proxy for an autonomous underwater vehicle, we demonstrate continuous input from the ML-Tracking algorithm to the vehicle controller during a record, 5+ hr continuous observation of a midwater gelatinous animal known as a siphonophore. These efforts clearly demonstrate the potential that tracking-by-detection algorithms can have on exploration in unexplored environments and discovery of undiscovered life in our ocean.