Inter-observer variability of manual contour delineation of structures in CT
Inter-observer variability of manual contour delineation of structures in CT
复制标题
DOI：
10.1007/s00330-018-5695-5
复制
发表时间：
2019-03-01
影响因子：
5.9
通讯作者：
Sosna, J.
中科院分区：
文献类型：
--
作者：
Joskowicz, Leo;Cohen, D.;Sosna, J.
目的量化 CT 中手动描绘病灶和器官轮廓的观察者间差异，为临床决策和评估自动分割算法的体积测量建立参考标准。 材料和方法 11 名放射科医生在 490 个临床 CT 扫描切片上手动描绘了 3193 个肝脏肿瘤（896 个）、肺肿瘤（1085 个）、肾脏轮廓（434 个）和脑血肿（497 个）的轮廓。然后对轮廓进行比较分析，以使用标准体积指标和观察者组产生的轮廓的新分组指标来量化观察者间的轮廓变异性。结果两个观察者的轮廓之间的平均体积重叠变异性值和范围（以％为单位）为：肝肿瘤17.8 [-5.8，+ 7.2]％，肺肿瘤20.8 [-8.8，+ 10.2]％，肾脏轮廓8.8 [-0.8,+1.2]% 和脑血肿 18 [-6.0,+6.0]%。对于任意两个随机选择的观察者，平均轮廓体积重叠变异性为 5-57%。由两名、三名和五名观察者组成的小组捕获的平均变异性分别为 37%、53% 和 72%； 8 位观察者占总变异性的 75-94%。在所有案例中，38.5% 的划界不一致是由于单个观察者与其他观察者的部分划界不一致造成的。根据观察者的专业知识，观察者之间的描绘变异性没有发现统计差异。结论不同结构和观察者的手动描绘变异性很大，并且跨越各种结构和病理的广泛范围。两个甚至三个观察者可能不足以建立完整的观察者间变异性。关键点中心点这项研究量化了 CT 中手动描绘病变和器官轮廓的观察者间变异性。中心点两个观察者之间手动描绘的变异性可能很大。两个甚至三个观察者只能捕获常见实践中观察到的全部观察者间变异性的一小部分。中心点观察者间手动描绘变异性对于为放射科医师培训和评估以及自动分割算法的评估建立参考标准是必要的。
PurposeTo quantify the inter-observer variability of manual delineation of lesions and organ contours in CT to establish a reference standard for volumetric measurements for clinical decision making and for the evaluation of automatic segmentation algorithms.Materials and methodsEleven radiologists manually delineated 3193 contours of liver tumours (896), lung tumours (1085), kidney contours (434) and brain hematomas (497) on 490 slices of clinical CT scans. A comparative analysis of the delineations was then performed to quantify the inter-observer delineation variability with standard volume metrics and with new group-wise metrics for delineations produced by groups of observers.ResultsThe mean volume overlap variability values and ranges (in %) between the delineations of two observers were: liver tumours 17.8 [-5.8,+7.2]%, lung tumours 20.8 [-8.8,+10.2]%, kidney contours 8.8 [-0.8,+1.2]% and brain hematomas 18 [-6.0,+6.0] %. For any two randomly selected observers, the mean delineation volume overlap variability was 5-57%. The mean variability captured by groups of two, three and five observers was 37%, 53% and 72%; eight observers accounted for 75-94% of the total variability. For all cases, 38.5% of the delineation non-agreement was due to parts of the delineation of a single observer disagreeing with the others. No statistical difference was found for the delineation variability between the observers based on their expertise.ConclusionThe variability in manual delineations for different structures and observers is large and spans a wide range across a variety of structures and pathologies. Two and even three observers may not be sufficient to establish the full range of inter-observer variability.Key Points center dot This study quantifies the inter-observer variability of manual delineation of lesions and organ contours in CT.center dot The variability of manual delineations between two observers can be significant. Two and even three observers capture only a fraction of the full range of inter-observer variability observed in common practice.center dot Inter-observer manual delineation variability is necessary to establish a reference standard for radiologist training and evaluation and for the evaluation of automatic segmentation algorithms.