Planning for Natural Language Failures with the AI Playbook
Planning for Natural Language Failures with the AI Playbook
复制标题
使用 AI 手册规划自然语言失败
DOI：
10.1145/3411764.3445735
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Saleema Amershi
中科院分区：
文献类型：
--
作者：
Matthew K. Hong;Adam Fourney;Derek DeBellis;Saleema Amershi
人工智能用户体验原型是一项挑战，部分原因在于概率人工智能模型难以在部署前预测、测试和减轻人工智能故障。在这项工作中，我们开始支持早期人工智能原型的实践者，重点是基于自然语言（NL）的技术。我们采访了一家大型科技公司的12名NL从业者，结果显示，除了人工智能原型设计面临的挑战之外，由于缺乏工具和时间紧迫，原型设计往往根本没有发生，或者只关注于理想化的场景。这些发现为我们设计AI Playbook提供了信息，这是我们开发的一种交互式低成本工具，旨在鼓励在部署前主动和系统地考虑AI错误。我们对AI Playbook的评估显示了它的潜力：1)鼓励产品团队优先考虑理想和失败场景，2)从用户体验的角度标准化AI失败的表述，3)充当用户体验设计师、数据科学家和工程师之间的边界对象。
Prototyping AI user experiences is challenging due in part to probabilistic AI models making it difficult to anticipate, test, and mitigate AI failures before deployment. In this work, we set out to support practitioners with early AI prototyping, with a focus on natural language (NL)-based technologies. Our interviews with 12 NL practitioners from a large technology company revealed that, in addition to challenges prototyping AI, prototyping was often not happening at all or focused only on idealized scenarios due to a lack of tools and tight timelines. These findings informed our design of the AI Playbook, an interactive and low-cost tool we developed to encourage proactive and systematic consideration of AI errors before deployment. Our evaluation of the AI Playbook demonstrates its potential to 1) encourage product teams to prioritize both ideal and failure scenarios, 2) standardize the articulation of AI failures from a user experience perspective, and 3) act as a boundary object between user experience designers, data scientists, and engineers.
DOI：
10.1145/3196709.3196730
发表时间：
2018-06
期刊：
Proceedings of the 2018 Designing Interactive Systems Conference
影响因子：
--
作者：
Qian Yang;Alex Sciuto;J. Zimmerman;J. Forlizzi;Aaron Steinfeld
通讯作者：
Qian Yang;Alex Sciuto;J. Zimmerman;J. Forlizzi;Aaron Steinfeld