Context Matters for Image Descriptions for Accessibility: Challenges for Referenceless Evaluation Metrics
Context Matters for Image Descriptions for Accessibility: Challenges for Referenceless Evaluation Metrics
复制标题
DOI：
10.48550/arxiv.2205.10646
复制
发表时间：
2022-05
期刊：
影响因子：
--
通讯作者：
Elisa Kreiss;Cynthia L. Bennett;Shayan Hooshmand;E. Zelikman;M. Morris;Christopher Potts
中科院分区：
文献类型：
--
作者：
Elisa Kreiss;Cynthia L. Bennett;Shayan Hooshmand;E. Zelikman;M. Morris;Christopher Potts
网络上很少有图片会收到让盲人和低视力(BLV)用户能够访问的文本描述。基于图像的NLG系统已经发展到可以开始解决这个长期存在的社会问题的地步，但这些系统将不会完全成功，除非我们根据正确指导它们发展的指标对它们进行评估。在这里，我们反对当前的无参考指标--那些不依赖于人类生成的地面事实描述的指标--理由是它们与BLV用户的需求不一致。这些度量的根本缺点是它们没有考虑上下文，而上下文信息受到BLV用户的高度重视。为了证实这些说法，我们对BLV参与者进行了一项研究，他们在不同的维度上对描述进行了评级。深入的分析表明，缺乏上下文感知使得当前的无参照度量不足以提高图像的可访问性。作为概念验证，我们提供了无引用度量CLIPScore的上下文版本，它开始解决与BLV数据的断开问题。
Few images on the Web receive alt-text descriptions that would make them accessible to blind and low vision (BLV) users. Image-based NLG systems have progressed to the point where they can begin to address this persistent societal problem, but these systems will not be fully successful unless we evaluate them on metrics that guide their development correctly. Here, we argue against current referenceless metrics – those that don’t rely on human-generated ground-truth descriptions – on the grounds that they do not align with the needs of BLV users. The fundamental shortcoming of these metrics is that they do not take context into account, whereas contextual information is highly valued by BLV users. To substantiate these claims, we present a study with BLV participants who rated descriptions along a variety of dimensions. An in-depth analysis reveals that the lack of context-awareness makes current referenceless metrics inadequate for advancing image accessibility. As a proof-of-concept, we provide a contextual version of the referenceless metric CLIPScore which begins to address the disconnect to the BLV data.