Visualization of Individual Variation of Multiple Annotators Working on Training Datasets for Machine Learning
Visualization of Individual Variation of Multiple Annotators Working on Training Datasets for Machine Learning
复制标题
DOI：
10.1109/nicoint50878.2020.00022
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
T. Itoh;Ayana Murakami
中科院分区：
文献类型：
--
作者：
T. Itoh;Ayana Murakami
Quality of training datasets is essential for the quality of machine learning. Machine learning projects often invite multiple workers for these annotation tasks for training dataset creation. It is important to observe on what types of contents multiple workers make different annotations, or which workers often make abnormal annotations, to guarantee the quality of training datasets. This paper presents a tool for the visualization of abnormality of annotations by multiple workers. The tool generates a matrix of abnormality of annotations for each of the images by each of the workers and displays as a heatmap. This paper introduces an example using a training dataset where estimated ages are annotated to 7,748 pictures of human faces by eight workers.
