Diagnostic Uncertainty Calibration: Towards Reliable Machine Predictions in Medical Domain
Diagnostic Uncertainty Calibration: Towards Reliable Machine Predictions in Medical Domain
复制标题
DOI：
--
复制
发表时间：
2020-07
期刊：
影响因子：
2.5
通讯作者：
Takahiro Mimori;Keiko Sasada;H. Matsui;Issei Sato
中科院分区：
文献类型：
--
作者：
Takahiro Mimori;Keiko Sasada;H. Matsui;Issei Sato
人类专家之间的标签不一致是医学领域的一个常见问题，并在分类模型的评估和学习中提出了独特的挑战。在这项工作中，我们扩展了概率预测的度量，包括校准，即预测概率的可靠性，以适应这种情况。我们进一步以统一的方式形式化了高阶统计量的度量，包括评级者之间的分歧，这使我们能够评估分布不确定性的质量。此外，我们提出了一种新的事后校准方法，该方法在类概率估计上为训练好的神经网络配备校准分布。通过大规模的医学成像应用，我们证明了我们的方法显着提高了多个指标的不确定性估计的质量。
Label disagreement between human experts is a common issue in the medical domain and poses unique challenges in the evaluation and learning of classification models. In this work, we extend metrics for probability prediction, including calibration, i.e., the reliability of predictive probability, to adapt to such a situation. We further formalize the metrics for higher-order statistics, including inter-rater disagreement, in a unified way, which enables us to assess the quality of distributional uncertainty. In addition, we propose a novel post-hoc calibration method that equips trained neural networks with calibrated distributions over class probability estimates. With a large-scale medical imaging application, we show that our approach significantly improves the quality of uncertainty estimates in multiple metrics.