Edge computing at sea: high-throughput classification of in-situ plankton imagery for adaptive sampling
Edge computing at sea: high-throughput classification of in-situ plankton imagery for adaptive sampling
复制标题
DOI：
10.3389/fmars.2023.1187771
复制
发表时间：
2023-06
期刊：
影响因子：
--
通讯作者：
M. Schmid;Dominic Daprano;Malhar M. Damle;Christopher M. Sullivan;S. Sponaugle;Charles Cousin;C. G
中科院分区：
文献类型：
--
作者：
M. Schmid;Dominic Daprano;Malhar M. Damle;Christopher M. Sullivan;S. Sponaugle;Charles Cousin;C. G
大多数海洋浮游生物的小尺寸要求浮游生物采样发生在小的空间尺度上，然而我们的问题往往跨越大的空间区域。水下成像可以为这一采样难题提供解决方案，但需要收集大量数据，需要自动化的图像分析方法。浮游生物分类的机器学习和高性能计算（HPC）基础设施对于快速图像处理至关重要；然而，这些资产，特别是高性能计算基础设施，只能在巡航后使用，从而导致对浮游生物群落结构的“事后”看法。为了响应高动态洋流系统中海洋特征和物种组合的短暂性，实时数据是适应性海洋采样的关键。在这里，我们使用北加州洋流（NCC）中新的原位浮游鱼成像系统-3 （ISIIS-3）与边缘服务器结合，将成像的浮游生物实时分为170类。此功能与数据可视化在一起非常重要。Ai仪表板使自适应实时决策和海上采样成为可能。双isiis -深焦粒子成像仪（DPI）相机采样180 L s-1，导致bbb10 GB每分钟的视频。成像的生物在250µm到15 cm的大小范围内，包括丰富的甲壳类动物，脆弱的分类群（如水水母，海鞘），更快的游泳者（如磷虾），和罕见的分类群（如幼体鱼类）。部署在边缘服务器上的深度学习管道使用基于cpu的多线程分割和基于gpu的分类来处理图像。AVI视频包含50秒的数据，可以包含23,000 - 225,000颗粒和浮游生物片段。根据生物生产率的不同，通过分割和分类处理一个AVI平均需要3.75分钟。heavy db数据库监视新处理的数据，并链接到heavy数据库。Ai仪表板用于交互式数据可视化。我们描述了几个例子，其中成像、人工智能和数据可视化使自适应采样能够对海洋学产生变革性影响。我们设想，人工智能支持的自适应采样将对我们解决NCC中重要海洋学特征的生物响应的能力产生重大影响，例如氧气最低区或有害藻华薄层，这些特征会影响生态系统、渔业和当地社区的健康。
The small sizes of most marine plankton necessitate that plankton sampling occur on fine spatial scales, yet our questions often span large spatial areas. Underwater imaging can provide a solution to this sampling conundrum but collects large quantities of data that require an automated approach to image analysis. Machine learning for plankton classification, and high-performance computing (HPC) infrastructure, are critical to rapid image processing; however, these assets, especially HPC infrastructure, are only available post-cruise leading to an ‘after-the-fact’ view of plankton community structure. To be responsive to the often-ephemeral nature of oceanographic features and species assemblages in highly dynamic current systems, real-time data are key for adaptive oceanographic sampling. Here we used the new In-situ Ichthyoplankton Imaging System-3 (ISIIS-3) in the Northern California Current (NCC) in conjunction with an edge server to classify imaged plankton in real-time into 170 classes. This capability together with data visualization in a heavy.ai dashboard makes adaptive real-time decision-making and sampling at sea possible. Dual ISIIS-Deep-focus Particle Imager (DPI) cameras sample 180 L s-1, leading to >10 GB of video per min. Imaged organisms are in the size range of 250 µm to 15 cm and include abundant crustaceans, fragile taxa (e.g., hydromedusae, salps), faster swimmers (e.g., krill), and rarer taxa (e.g., larval fishes). A deep learning pipeline deployed on the edge server used multithreaded CPU-based segmentation and GPU-based classification to process the imagery. AVI videos contain 50 sec of data and can contain between 23,000 - 225,000 particle and plankton segments. Processing one AVI through segmentation and classification takes on average 3.75 mins, depending on biological productivity. A heavyDB database monitors for newly processed data and is linked to a heavy.ai dashboard for interactive data visualization. We describe several examples where imaging, AI, and data visualization enable adaptive sampling that can have a transformative effect on oceanography. We envision AI-enabled adaptive sampling to have a high impact on our ability to resolve biological responses to important oceanographic features in the NCC, such as oxygen minimum zones, or harmful algal bloom thin layers, which affect the health of the ecosystem, fisheries, and local communities.