The fallacy of inscrutability
The fallacy of inscrutability
复制标题
DOI：
10.1098/rsta.2018.0084
复制
发表时间：
2018-11-28
期刊：
影响因子：
5
通讯作者：
Kroll, Joshua A.
中科院分区：
文献类型：
--
作者：
Kroll, Joshua A.
有人批评说，神秘的、不负责任的黑箱软件系统可能会使关键决策的逻辑变得不可理解，与此相反，我们认为算法从根本上是可以理解的技术。软件系统被设计为以受控的方式与世界交互，并为特定目的而构建或操作，受选择和假设的影响。传统的权力结构可以而且确实将系统变成不透明的黑匣子，但技术总是可以在更高的层次上理解，从设计和操作目标的内涵以及输入，输出和结果的扩展。一个系统的运作机制总是可以检查和解释，但对机械的关注掩盖了动力学的关键问题。虽然结构上的不可测性让用户和监督实体感到沮丧，但系统创建者和运营商总是确定他们部署的技术适合某些用途，因此没有系统完全不可测。我们调查的轮廓的不可测性和不透明性，它们产生的方式从周围的软件系统的动力学，以及建议的补救措施，从不同的学科，特别是计算机伦理和隐私的设计的价值。我们的结论是，政策不应该同意这样一种观点，即某些系统必然是不可理解的。算法的有效治理来自于系统设计、操作和评估中严格的科学和工程要求，以使系统可验证地值得信赖。而不是寻求对计算机系统的每一个行为的解释，政策应该正式化，并使已知的假设，选择，并充分确定与system.This文章是主题问题的一部分“治理人工智能：道德，法律的，和技术的机遇和挑战”。
Contrary to the criticism that mysterious, unaccountable black-box software systems threaten to make the logic of critical decisions inscrutable, we argue that algorithms are fundamentally understandable pieces of technology. Software systems are designed to interact with the world in a controlled way and built or operated for a specific purpose, subject to choices and assumptions. Traditional power structures can and do turn systems into opaque black boxes, but technologies can always be understood at a higher level, intensionally in terms of their designs and operational goals and extensionally in terms of their inputs, outputs and outcomes. The mechanisms of a system's operation can always be examined and explained, but a focus on machinery obscures the key issue of power dynamics. While structural inscrutability frustrates users and oversight entities, system creators and operators always determine that the technologies they deploy are fit for certain uses, making no system wholly inscrutable. We investigate the contours of inscrutability and opacity, the way they arise from power dynamics surrounding software systems, and the value of proposed remedies from disparate disciplines, especially computer ethics and privacy by design. We conclude that policy should not accede to the idea that some systems are of necessity inscrutable. Effective governance of algorithms comes from demanding rigorous science and engineering in system design, operation and evaluation to make systems verifiably trustworthy. Rather than seeking explanations for each behaviour of a computer system, policies should formalize and make known the assumptions, choices, and adequacy determinations associated with a system.This article is part of the theme issue 'Governing artificial intelligence: ethical, legal, and technical opportunities and challenges'.