Evolution and impact of bias in human and machine learning algorithm interaction
Evolution and impact of bias in human and machine learning algorithm interaction
复制标题
DOI：
10.1371/journal.pone.0235502
复制
发表时间：
2020-08-13
期刊：
影响因子：
3.7
通讯作者：
Shafto, Patrick
中科院分区：
文献类型：
--
作者：
Sun, Wenlong;Nasraoui, Olfa;Shafto, Patrick
传统上，机器学习算法依赖于专家的可靠标签来构建预测。然而，最近，算法已经从普通人群中接收到标签，注释等形式的数据，其结果是算法受到来自未经检查的信息的偏见的影响，例如有偏见的样本和有偏见的标签。此外，人和算法越来越多地参与交互过程，其中人和算法都没有接收到无偏数据。算法也可以做出有偏见的预测，导致现在被称为算法偏见。另一方面，人类对具有算法偏见的机器学习方法输出的反应会根据有偏见的信息做出决策，从而使情况变得更糟，这些信息可能会在以后被算法消耗。最近的一些研究集中在机器学习算法偏见对社会的伦理和道德影响上。然而，迄今为止，大多数研究都将算法偏差视为静态因素，未能捕捉到偏差的动态和迭代特性。我们认为，算法偏见与人类互动的迭代方式，这对算法的性能有长期的影响。为此，我们提出了一个迭代学习框架，该框架的灵感来自人类语言进化，以研究机器学习算法与人类之间的交互。我们的目标是研究两个相互作用的偏见来源：人们选择要标记的信息的过程（人类行为）;以及算法选择要呈现给人们的信息子集的过程（迭代算法偏见模式）。我们研究了三种形式的迭代算法偏差（个性化过滤器，主动学习和随机）以及它们如何通过制定关于每种类型偏差影响的研究问题来影响机器学习算法的性能。基于对几个对照实验结果的统计分析，我们发现三种不同的迭代偏差模式，以及初始训练数据类别不平衡和人类行为，确实会影响机器学习算法学习的模型。我们还发现，迭代过滤器偏差，这是突出的个性化用户界面，可以导致更多的不平等估计的相关性和有限的人类能力，发现相关数据。我们的研究结果表明，当使用基于内容的过滤器预测相关项目时，相关盲点（来自测试集的项目，其预测的相关概率小于0.5，因此有可能被人类隐藏）占所有相关项目的4%。使用真实评级数据集的类似模拟发现，相同的滤波器导致相关测试集的75%的盲点大小。
Traditionally, machine learning algorithms relied on reliable labels from experts to build predictions. More recently however, algorithms have been receiving data from the general population in the form of labeling, annotations, etc. The result is that algorithms are subject to bias that is born from ingesting unchecked information, such as biased samples and biased labels. Furthermore, people and algorithms are increasingly engaged in interactive processes wherein neither the human nor the algorithms receive unbiased data. Algorithms can also make biased predictions, leading to what is now known as algorithmic bias. On the other hand, human's reaction to the output of machine learning methods with algorithmic bias worsen the situations by making decision based on biased information, which will probably be consumed by algorithms later. Some recent research has focused on the ethical and moral implication of machine learning algorithmic bias on society. However, most research has so far treated algorithmic bias as a static factor, which fails to capture the dynamic and iterative properties of bias. We argue that algorithmic bias interacts with humans in an iterative manner, which has a long-term effect on algorithms' performance. For this purpose, we present an iterated-learning framework that is inspired from human language evolution to study the interaction between machine learning algorithms and humans. Our goal is to study two sources of bias that interact: the process by which people select information to label (human action); and the process by which an algorithm selects the subset of information to present to people (iterated algorithmic bias mode). We investigate three forms of iterated algorithmic bias (personalization filter, active learning, and random) and how they affect the performance of machine learning algorithms by formulating research questions about the impact of each type of bias. Based on statistical analyses of the results of several controlled experiments, we found that the three different iterated bias modes, as well as initial training data class imbalance and human action, do affect the models learned by machine learning algorithms. We also found that iterated filter bias, which is prominent in personalized user interfaces, can lead to more inequality in estimated relevance and to a limited human ability to discover relevant data. Our findings indicate that the relevance blind spot (items from the testing set whose predicted relevance probability is less than 0.5 and who thus risk being hidden from humans) amounted to 4% of all relevant items when using a content-based filter that predicts relevant items. A similar simulation using a real-life rating data set found that the same filter resulted in a blind spot size of 75% of the relevant testing set.