Measuring Representational Harms in Image Captioning
Measuring Representational Harms in Image Captioning
复制标题
DOI：
10.1145/3531146.3533099
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
Angelina Wang;Solon Barocas;Kristen Laird;Hanna M. Wallach
中科院分区：
文献类型：
--
作者：
Angelina Wang;Solon Barocas;Kristen Laird;Hanna M. Wallach
以前的工作主要是考虑公平性的图像字幕系统通过欠指定的透镜的“偏见”。相比之下，我们提出了一套技术，用于测量五种类型的代表性伤害，以及使用最先进的图像字幕系统获得的两个最流行的图像字幕数据集的测量结果。我们的目标不是审核这个图像字幕系统，而是开发规范的测量技术，从而提供一个反思所涉及的许多挑战的机会。我们为每种类型的伤害提出了多种测量技术。我们认为，通过这样做，我们能够更好地捕捉每种类型的伤害的多方面的性质，从而提高（集体）有效性的测量结果。在整个过程中，我们讨论了我们的测量方法的基础假设，并指出当它们不成立。
Previous work has largely considered the fairness of image captioning systems through the underspecified lens of “bias.” In contrast, we present a set of techniques for measuring five types of representational harms, as well as the resulting measurements obtained for two of the most popular image captioning datasets using a state-of-the-art image captioning system. Our goal was not to audit this image captioning system, but rather to develop normatively grounded measurement techniques, in turn providing an opportunity to reflect on the many challenges involved. We propose multiple measurement techniques for each type of harm. We argue that by doing so, we are better able to capture the multi-faceted nature of each type of harm, in turn improving the (collective) validity of the resulting measurements. Throughout, we discuss the assumptions underlying our measurement approach and point out when they do not hold.