Comparison of image annotation data generated by multiple investigators for benthic ecology
Comparison of image annotation data generated by multiple investigators for benthic ecology
复制标题
DOI：
10.3354/meps11775
复制
发表时间：
2016-06-23
影响因子：
2.5
通讯作者：
Ruhl, Henry A.
中科院分区：
文献类型：
--
作者：
Durden, Jennifer M.;Bett, Brian J.;Ruhl, Henry A.
多个研究人员经常从单个图像集中的海底图像生成数据，以减少时间负担，特别是现在可用于生态研究的大型摄影调查。众所周知，这些数据（注释）会因研究者对标本分类的意见以及疲劳和认知等人为因素的不同而有所不同。这些变化很少被记录或量化，它们对衍生生态指标（密度、多样性、组成）的影响也很少被记录或量化。我们比较了 3 位研究人员对 73 种巨型动物形态类型的注释，其中包括 28 000 张类似图像，其中包括 650 张常见图像。成功的注释被定义为对样本进行检测和正确分类。预计标本检测成功率为 77%，分类成功率为 95%，注释成功率为 73%。样本检测成功率因形态类型而异（12-100%）。常见类群检测的变化导致研究人员之间的表观动物密度和群落组成存在显着差异。如果不适当控制或解释，这种偏见有可能产生虚假的生态解释。我们建议摄影研究记录多个注释器的使用并量化潜在的研究者间偏见。采样单元（照片或视频剪辑）的随机化显然对于在多个注释器研究（实际上是单个注释器工作）中有效消除人类注释偏差至关重要。
Multiple investigators often generate data from seabed images within a single image set to reduce the time burden, particularly with the large photographic surveys now available to ecological studies. These data (annotations) are known to vary as a result of differences in investigator opinion on specimen classification and of human factors such as fatigue and cognition. These variations are rarely recorded or quantified, nor are their impacts on derived ecological metrics (density, diversity, composition). We compared the annotations of 3 investigators of 73 megafaunal morphotypes in similar to 28 000 images, including 650 common images. Successful annotation was defined as both detecting and correctly classifying a specimen. Estimated specimen detection success was 77%, and classification success was 95%, giving an annotation success rate of 73%. Specimen detection success varied substantially by morphotype (12-100%). Variation in the detection of common taxa resulted in significant differences in apparent faunal density and community composition among investigators. Such bias has the potential to produce spurious ecological interpretations if not appropriately controlled or accounted for. We recommend that photographic studies document the use of multiple annotators and quantify potential inter-investigator bias. Randomisation of the sampling unit (photograph or video clip) is clearly critical to the effective removal of human annotation bias in multiple annotator studies (and indeed single annotator works).