Algorithmic monoculture and social welfare
Algorithmic monoculture and social welfare
复制标题
算法单一文化与社会福利
DOI：
--
复制
发表时间：
2021
影响因子：
11.1
通讯作者：
Manish Raghavan
中科院分区：
文献类型：
--
作者：
J. Kleinberg;Manish Raghavan
在使用算法进行就业和贷款等领域的高风险筛选决策时，生物单一文化越来越受到关注。如果许多公司使用相同的算法，即使它比其他方法更准确，所产生的"单一文化"可能容易受到相关失败的影响，就像生物环境中的单一文化系统一样。为了研究这一问题，我们建立了一个单一栽培下的选择模型。我们发现，即使没有任何假设的冲击或相关故障，即，在"正常操作"下-当多家公司使用相同的算法时，决策的质量可能会降低。因此，引入更精确的算法可能会降低社会福利-算法决策的一种“Braess悖论”。随着算法越来越多地被应用于筛选求职者，以便在就业、贷款和其他领域做出高风险决策，人们对算法单一文化的影响提出了担忧，在这种单一文化中，许多决策者都依赖于同一种算法。这一关切与农业类似，单一文化体系有可能因意外冲击而遭受严重损害。在这里，我们表明，算法的单一文化的危险运行得更深，在单一的算法由一组决策代理收敛，即使该算法是更准确的任何一个代理隔离，可以降低整体质量的决策由代理的全部集合。因此，不需要意外的冲击来暴露单一文化的风险;即使在"正常"操作下，甚至对于只有一个决策者使用时更准确的算法，它也会损害准确性。我们的研究结果依赖于最小的假设，并涉及一个概率框架的发展，分析系统，使用多个嘈杂的估计一组替代品。
Significance Algorithmic monoculture is a growing concern in the use of algorithms for high-stakes screening decisions in areas such as employment and lending. If many firms use the same algorithm, even if it is more accurate than the alternatives, the resulting “monoculture” may be susceptible to correlated failures, much as a monocultural system is in biological settings. To investigate this concern, we develop a model of selection under monoculture. We find that even without any assumption of shocks or correlated failures—i.e., under “normal operations”—the quality of decisions may decrease when multiple firms use the same algorithm. Thus, the introduction of a more accurate algorithm may decrease social welfare—a kind of “Braess’ paradox” for algorithmic decision-making. As algorithms are increasingly applied to screen applicants for high-stakes decisions in employment, lending, and other domains, concerns have been raised about the effects of algorithmic monoculture, in which many decision-makers all rely on the same algorithm. This concern invokes analogies to agriculture, where a monocultural system runs the risk of severe harm from unexpected shocks. Here, we show that the dangers of algorithmic monoculture run much deeper, in that monocultural convergence on a single algorithm by a group of decision-making agents, even when the algorithm is more accurate for any one agent in isolation, can reduce the overall quality of the decisions being made by the full collection of agents. Unexpected shocks are therefore not needed to expose the risks of monoculture; it can hurt accuracy even under “normal” operations and even for algorithms that are more accurate when used by only a single decision-maker. Our results rely on minimal assumptions and involve the development of a probabilistic framework for analyzing systems that use multiple noisy estimates of a set of alternatives.