Quantifying Societal Bias Amplification in Image Captioning
Quantifying Societal Bias Amplification in Image Captioning
复制标题
DOI：
10.1109/cvpr52688.2022.01309
复制
发表时间：
2022-03
期刊：
影响因子：
--
通讯作者：
Yusuke Hirota;Yuta Nakashima;Noa García
中科院分区：
文献类型：
--
作者：
Yusuke Hirota;Yuta Nakashima;Noa García
我们研究社会偏见放大图像字幕。图像字幕模型已被证明会使性别和种族偏见永久化，然而，测量，量化和评估字幕中的社会偏见的指标尚未标准化。我们提供了一个全面的研究，每个指标的优势和局限性，并提出LIC，一个指标来研究字幕偏差放大。我们认为，对于图像字幕，这是不够的，专注于正确的预测的保护属性，和整个上下文应该考虑。我们对传统的和最先进的图像字幕模型进行了广泛的评估，并惊讶地发现，通过只关注受保护的属性预测，偏见缓解模型出乎意料地放大了偏见。
We study societal bias amplification in image captioning. Image captioning models have been shown to perpetuate gender and racial biases, however, metrics to measure, quantify, and evaluate the societal bias in captions are not yet standardized. We provide a comprehensive study on the strengths and limitations of each metric, and propose LIC, a metric to study captioning bias amplification. We argue that, for image captioning, it is not enough to focus on the correct prediction of the protected attribute, and the whole context should be taken into account. We conduct extensive evaluation on traditional and state-of-the-art image captioning models, and surprisingly find that, by only focusing on the protected attribute prediction, bias mitigation models are unexpectedly amplifying bias.