The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Fail
The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Fail
复制标题
DOI：
10.18653/v1/2022.acl-long.516
复制
发表时间：
2021-10
期刊：
影响因子：
--
通讯作者：
Sam Bowman
中科院分区：
文献类型：
--
作者：
Sam Bowman
NLP的研究人员经常以贬低该领域成功的方式来框架和讨论研究结果，通常是为了回应该领域的广泛宣传。虽然这是善意的，但这产生了许多关于我们最好的技术局限性的误导或错误的说法。这是一个问题，而且可能比看起来更严重：它损害了我们的信誉，使我们更难减轻当今的危害，比如那些涉及内容审核或简历筛选的偏见系统。它还限制了我们为更遥远的未来进步的潜在巨大影响做准备的能力。本文敦促研究者对这些说法要小心，并提出了一些研究方向和沟通策略，使其更容易避免或反驳。
Researchers in NLP often frame and discuss research results in ways that serve to deemphasize the field’s successes, often in response to the field’s widespread hype. Though well-meaning, this has yielded many misleading or false claims about the limits of our best technology. This is a problem, and it may be more serious than it looks: It harms our credibility in ways that can make it harder to mitigate present-day harms, like those involving biased systems for content moderation or resume screening. It also limits our ability to prepare for the potentially enormous impacts of more distant future advances. This paper urges researchers to be careful about these claims and suggests some research directions and communication strategies that will make it easier to avoid or rebut them.