Content-Aware Segmentation of Objects Spanning a Large Size Range: Application to Plankton Images
Content-Aware Segmentation of Objects Spanning a Large Size Range: Application to Plankton Images
复制标题
DOI：
10.3389/fmars.2022.870005
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
Thelma Panaïotis;Louis Caray–Counil;Ben Woodward;M. Schmid;Dominic Daprano;Sheng Tse Tsai;C. Sullivan;R. Cowen;J. Irisson
中科院分区：
文献类型：
--
作者：
Thelma Panaïotis;Louis Caray–Counil;Ben Woodward;M. Schmid;Dominic Daprano;Sheng Tse Tsai;C. Sullivan;R. Cowen;J. Irisson
浮游生物作为海洋食物网的基础和生物碳泵的关键组成部分，在海洋中发挥着重要作用。他们的研究得益于原位成像仪器的发展，这些仪器提供了比以前的工具更高的时空分辨率。但是这些仪器收集了大量的图像，其中绝大多数是海洋雪颗粒或成像伪影。其中，原位浮游鱼类成像系统（ISIIS）的采样水量最大（> 100 L s-1），因此产生了特别大的数据集。为了从原位图像中提取可管理的生态信息量，我们建议在数据处理过程的早期阶段（即分割阶段）关注水生生物。我们比较了三种分割方法，特别是对于较小的目标，其中浮游生物占不到1%的对象：（i）传统的背景阈值，（ii）基于最大稳定极值区域（MSER）的对象检测器，以及（iii）基于卷积神经网络（CNN）的内容感知对象检测器。这些方法是在地中海收集的ISIIS数据的一个子集上进行评估的，从中提取了> 3，000个手动描绘的生物体的地面实况数据集。朴素阈值方法捕获了其中的97.3%，但产生了约340，000个片段，因此其中99.1%不是浮游生物（即召回率= 97.3%，精度= 0.9%）。将阈值处理与CNN相结合，错过了一些更多的神经元生物体（召回率= 91.8%），但片段数量减少了18倍（精确度增加到16.3%）。MSER检测器产生的片段比阈值处理少四倍（精确度= 3.5%），错过了更多的生物体（召回率= 85.4%），但速度要快得多。由于朴素阈值处理在ISIIS部署的1分钟内产生约525，000个对象，因此更高级的分割方法显著改善了ISIIS数据处理，并简化了分割对象的后续分类。在召回方面的成本是有限的，特别是对于CNN对象检测器。这些方法现在是计算机视觉的标准，可以应用于其他浮游生物成像设备，其中大多数都带来了数据管理问题。
As the basis of oceanic food webs and a key component of the biological carbon pump, planktonic organisms play major roles in the oceans. Their study benefited from the development of in situ imaging instruments, which provide higher spatio-temporal resolution than previous tools. But these instruments collect huge quantities of images, the vast majority of which are of marine snow particles or imaging artifacts. Among them, the In Situ Ichthyoplankton Imaging System (ISIIS) samples the largest water volumes (> 100 L s-1) and thus produces particularly large datasets. To extract manageable amounts of ecological information from in situ images, we propose to focus on planktonic organisms early in the data processing pipeline: at the segmentation stage. We compared three segmentation methods, particularly for smaller targets, in which plankton represents less than 1% of the objects: (i) a traditional thresholding over the background, (ii) an object detector based on maximally stable extremal regions (MSER), and (iii) a content-aware object detector, based on a Convolutional Neural Network (CNN). These methods were assessed on a subset of ISIIS data collected in the Mediterranean Sea, from which a ground truth dataset of > 3,000 manually delineated organisms is extracted. The naive thresholding method captured 97.3% of those but produced ~340,000 segments, 99.1% of which were therefore not plankton (i.e. recall = 97.3%, precision = 0.9%). Combining thresholding with a CNN missed a few more planktonic organisms (recall = 91.8%) but the number of segments decreased 18-fold (precision increased to 16.3%). The MSER detector produced four times fewer segments than thresholding (precision = 3.5%), missed more organisms (recall = 85.4%), but was considerably faster. Because naive thresholding produces ~525,000 objects from 1 minute of ISIIS deployment, the more advanced segmentation methods significantly improve ISIIS data handling and ease the subsequent taxonomic classification of segmented objects. The cost in terms of recall is limited, particularly for the CNN object detector. These approaches are now standard in computer vision and could be applicable to other plankton imaging devices, the majority of which pose a data management problem.