Moral Crumple Zones: Cautionary Tales in Human-Robot Interaction
Moral Crumple Zones: Cautionary Tales in Human-Robot Interaction
复制标题
DOI：
10.17351/ests2019.260
复制
发表时间：
2019-01-01
影响因子：
1.8
通讯作者：
Elish, Madeleine Clare
中科院分区：
文献类型：
--
作者：
Elish, Madeleine Clare
随着关于人工智能系统的政策和伦理影响的争论不断增加，当机构分布在系统中并且通过时间和空间调节对行为的控制时，准确定位责任人将变得越来越重要。通过分析涉及复杂和自动化社会技术系统的几起引人注目的事故以及围绕这些系统的媒体报道，我引入了道德溃败区的概念，以描述行为的责任如何可能被错误地归因于对自动化或自治系统的行为控制有限的人类行为者。正如汽车中的溃缩区旨在吸收碰撞中的冲击力一样，高度复杂的自动化系统中的人类可能会无意或有意地成为一个组件，当整个系统出现故障时，它首当其冲地承担道德和法律责任。汽车中的溃缩区旨在保护人类驾驶员，而道德溃缩区则以牺牲最近的人类操作员为代价来保护技术系统的完整性。这一概念对于人机系统的设计和监管来说既是挑战也是机遇。阐明道德溃败区的关键不仅在于责任的错误分配，还在于新的复杂、自动化或所谓的自主技术中可能发展新形式的消费者和工人伤害的方式。
As debates about the policy and ethical implications of AI systems grow, it will be increasingly important to accurately locate who is responsible when agency is distributed in a system and control over an action is mediated through time and space. Analyzing several high-profile accidents involving complex and automated socio-technical systems and the media coverage that surrounded them, I introduce the concept of a moral crumple zone to describe how responsibility for an action may be misattributed to a human actor who had limited control over the behavior of an automated or autonomous system. Just as the crumple zone in a car is designed to absorb the force of impact in a crash, the human in a highly complex and automated system may become simply a component-accidentally or intentionally-that bears the brunt of the moral and legal responsibilities when the overall system malfunctions. While the crumple zone in a car is meant to protect the human driver, the moral crumple zone protects the integrity of the technological system, at the expense of the nearest human operator. The concept is both a challenge to and an opportunity for the design and regulation of human-robot systems. At stake in articulating moral crumple zones is not only the misattribution of responsibility but also the ways in which new forms of consumer and worker harm may develop in new complex, automated, or purported autonomous technologies.