Unintended machine learning biases as social barriers for persons with disabilitiess
Unintended machine learning biases as social barriers for persons with disabilitiess
复制标题
意外的机器学习偏见成为残疾人的社会障碍
DOI：
--
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Emily L. Denton
中科院分区：
文献类型：
--
作者：
B. Hutchinson;Vinodkumar Prabhakaran;Emily L. Denton
残疾人在充分参与社会方面面临许多障碍，技术的迅速发展有可能创造更多的障碍。为残疾人建立公平和包容性的技术不仅需要关注无障碍环境，还需要关注技术如何体现社会对残疾的态度。由机器学习（ML）模型永久化的表示通常会无意中从训练数据中编码出不受欢迎的社会偏见。例如，这可能导致文本分类模型产生非常不同的预测，例如我是一个患有精神疾病的人，我是一个高个子。在本文中，我们提出了现有ML模型和用于模型开发的数据中存在此类偏见的证据。首先，我们证明了一个机器学习模型来缓和对话，将提到残疾的文本分类为更具“毒性”。同样，一个机器学习的情感分析模型将提到残疾的文本评为更负面的。其次，我们证明了对许多ML应用程序至关重要的神经文本表示模型也可能包含对残疾的不必要的偏见。第三，我们表明，用于开发此类模型的数据反映了社会话语中的主题偏见，这可能解释了模型中的此类偏见-例如，枪支暴力，无家可归和吸毒成瘾在关于精神疾病的讨论中被过度代表。
Persons with disabilities face many barriers to full participation in society, and the rapid advancement of technology has the potential to create ever more. Building equitable and inclusive technologies for people with disabilities demands paying attention to more than accessibility, but also to how social attitudes towards disability are represented within technology. Representations perpetuated by machine learning (ML) models often inadvertently encode undesirable social biases from the data on which they are trained. This can result, for example, in text classification models producing very different predictions for I am a person with mental illness, and I am a tall person. In this paper, we present evidence of such biases in existing ML models, and in data used for model development. First, we demonstrate that a machine-learned model to moderate conversations classifies texts which mention disability as more "toxic". Similarly, a machine-learned sentiment analysis model rates texts which mention disability as more negative. Second, we demonstrate that neural text representation models that are critical to many ML applications can also contain undesirable biases towards mentions of disabilities. Third, we show that the data used to develop such models reflects topical biases in social discourse which may explain such biases in the models - for instance, gun violence, homelessness, and drug addiction are over-represented in discussions about mental illness.
DOI：
10.1073/pnas.1720347115
发表时间：
2018-04-17
影响因子：
11.1
作者：
Garg, Nikhil;Schiebinger, Londa;Zou, James
通讯作者：
Zou, James