The World of an Octopus: How Reporting Bias Influences a Language Model’s Perception of Color
The World of an Octopus: How Reporting Bias Influences a Language Model’s Perception of Color
复制标题
章鱼的世界：报告偏差如何影响语言模型对颜色的感知
DOI：
10.18653/v1/2021.emnlp-main.63
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Katharina Kann
中科院分区：
文献类型：
--
作者：
Cory Paik;Stephane T Aroca;A. Roncone;Katharina Kann
最近的研究引起了人们对纯文本预训练固有局限性的担忧。在本文中，我们首先证明了报告偏差，即人们不陈述明显事实的倾向，是造成这种限制的原因之一，然后研究了多模式训练在多大程度上可以缓解这一问题。为了实现这一点，我们1)生成颜色数据集（CoDa），这是521个常见物体的人类感知颜色分布的数据集；2)使用CoDa分析和比较文本中的颜色分布、语言模型捕获的分布和人类对颜色的感知；3)研究纯文本模型和多模态模型在CoDa上的性能差异。我们的研究结果表明，语言模型恢复的颜色分布与文本中发现的不准确分布的相关性比与基本事实的相关性更强，这支持了报告偏差对纯文本训练产生负面影响并固有地限制的说法。然后，我们证明了多模态模型可以利用它们的视觉训练来减轻这些影响，为未来的研究提供了一个有希望的途径。
Recent work has raised concerns about the inherent limitations of text-only pretraining. In this paper, we first demonstrate that reporting bias, the tendency of people to not state the obvious, is one of the causes of this limitation, and then investigate to what extent multimodal training can mitigate this issue. To accomplish this, we 1) generate the Color Dataset (CoDa), a dataset of human-perceived color distributions for 521 common objects; 2) use CoDa to analyze and compare the color distribution found in text, the distribution captured by language models, and a human’s perception of color; and 3) investigate the performance differences between text-only and multimodal models on CoDa. Our results show that the distribution of colors that a language model recovers correlates more strongly with the inaccurate distribution found in text than with the ground-truth, supporting the claim that reporting bias negatively impacts and inherently limits text-only training. We then demonstrate that multimodal models can leverage their visual training to mitigate these effects, providing a promising avenue for future research.
BLiMP：英语语言最小对的基准
DOI：
10.1162/tacl_a_00321
发表时间：
2020
影响因子：
10.9
作者：
Warstadt, Alex;Parrish, Alicia;Liu, Haokun;Mohananey, Anhad;Peng, Wei;Wang, Sheng-Fu;Bowman, Samuel R.
通讯作者：
Bowman, Samuel R.
DOI：
10.1162/tacl_a_00324
发表时间：
2020-01-01
影响因子：
10.9
作者：
Jiang, Zhengbao;Xu, Frank F.;Neubig, Graham
通讯作者：
Neubig, Graham