The Risks of Autonomy
The Risks of Autonomy
复制标题
自治的风险
DOI：
--
复制
发表时间：
2016
期刊：
影响因子：
--
通讯作者：
D. Woods
中科院分区：
文献类型：
--
作者：
D. Woods
作为最近多项国家咨询活动的参与者，我听取了许多技术倡导者的简报，这些简报设想了部署越来越自主的技术后的未来（例如，Abbott，McKenney，& Railsback，2013; Murphy & Shields，2012; National Research理事会，2014）。简报会一致侧重于对自主技术的额外投资所带来的好处。信息是一致的：在不久的将来，我们将能够将权力下放给车辆网络，然后这些车辆将自动执行范围更广的活动。尽管这些活动服务于各种人类利益相关者的目的，但演讲者理所当然地认为人类的参与会减少，最终变得不必要。这些关于新技术影响的信念伴随着过去的进步，尽管实际影响与想象的大不相同（Sarter，Woods和Billings，1997）。展望未来是一项不稳定的事业，容易受到偏见的影响。正如过去的工作所表明的那样，关于未来技术变革影响的主张是不明确的，没有根据的，过度自信的，而新的风险被忽视，忽视或淡化（Woods & Dekker，2000）。新的能力引发了更广泛和更复杂的反响，包括新形式的复杂性和新风险。未能预见和设计技术变革时期后肯定会出现的新挑战，导致自动化意外，当倡导者对抵消明显好处的负面意外后果感到惊讶时（Woods，1996）。今天，人们对越来越多的自主能力的普遍看法重演了在以前的技术变革周期中所观察到的情况。与自动化相关的风险被忽视和淡化，为未来的自动化惊喜奠定了基础。由于当今技术的力量，出现了一种新的自主风险，并在Doyle的Catch（Alderson & Doyle，2010）中被捕获：
As a participant in multiple recent national advisory activities, I have listened to many technology advocates present briefings that envisioned the future after deployments of increasingly autonomous technologies (e.g., Abbott, McKenney, & Railsback, 2013; Murphy & Shields, 2012; National Research Council, 2014). The briefings uniformly focused on the benefits that will flow from additional investments in autonomous technologies. The message is consistent: In the near future we will be able to delegate authority to networks of vehicles that will then carry out a widening range of activities autonomously. Even though these activities serve the purposes of various human stakeholders, the presenters take it for granted that humans’ involvement will decrease and, eventually, become unnecessary. These same beliefs about the impact of new technology have accompanied past advances even though the actual impacts have been quite different than those envisioned (Sarter, Woods, & Billings, 1997). Envisioning the future is a precarious enterprise that is subject to biases. As past work has shown, claims about the effects of future technology change are underspecified, ungrounded, and overconfident, whereas new risks are missed, ignored, or downplayed (Woods & Dekker, 2000). The new capabilities trigger a much wider and more complex set of reverberations, including new forms of complexity and new risks. Failure to anticipate and design for the new challenges that are certain to arise following periods of technology change leads to automation surprises when advocates are surprised by negative unintended consequences that offset apparent benefits (Woods, 1996). Today’s common beliefs about increasingly autonomous capabilities replay what has been observed in previous cycles of technology change. Risks associated with autonomy are ignored and downplayed, setting the stage for future automation surprises. A new risk of autonomy has arisen as a result of the power of today’s technologies and is captured in Doyle’s Catch (Alderson & Doyle, 2010):