Rethinking Explainability as a Dialogue: A Practitioner's Perspective
Rethinking Explainability as a Dialogue: A Practitioner's Perspective
复制标题
DOI：
--
复制
发表时间：
2022-02
期刊：
影响因子：
--
通讯作者：
Himabindu Lakkaraju;Dylan Slack;Yuxin Chen;Chenhao Tan;Sameer Singh
中科院分区：
文献类型：
--
作者：
Himabindu Lakkaraju;Dylan Slack;Yuxin Chen;Chenhao Tan;Sameer Singh
随着从业者越来越多地在医疗保健、金融和政策等关键领域部署机器学习模型，确保领域专家与这些模型一起有效运作变得至关重要。可解释性是弥合人类决策者和机器学习模型之间差距的一种方法。然而，大多数关于可解释性的现有工作都集中在一次性的静态解释上，如特征重要性或规则列表。对于许多需要利益相关者动态、持续发现的用例来说，这些解释可能还不够。在文献中，很少有著作询问决策者现有解释的效用以及他们希望在未来的解释中看到的其他必要条件。在这项工作中，我们解决了这一差距，并进行了一项研究，我们采访了医生，医疗保健专业人员和政策制定者，了解他们对解释的需求和愿望。我们的研究表明，决策者强烈倾向于自然语言对话形式的交互式解释。领域专家希望将机器学习模型视为“另一个同事”，即，一个可以通过问他们为什么通过表达和可访问的自然语言交互做出特定决定来追究责任的人。考虑到这些需求，我们概述了一组五个原则，研究人员应该遵循时，设计交互式解释作为未来工作的起点。此外，我们展示了为什么自然语言对话满足这些原则，是一个理想的方式来建立互动的解释。接下来，我们提供了一个可解释性的对话系统的设计，并讨论了建立这些系统的风险，权衡和研究机会。总的来说，我们希望我们的工作可以作为研究人员和工程师设计交互式可解释性系统的起点。
As practitioners increasingly deploy machine learning models in critical domains such as health care, finance, and policy, it becomes vital to ensure that domain experts function effectively alongside these models. Explainability is one way to bridge the gap between human decision-makers and machine learning models. However, most of the existing work on explainability focuses on one-off, static explanations like feature importances or rule lists. These sorts of explanations may not be sufficient for many use cases that require dynamic, continuous discovery from stakeholders. In the literature, few works ask decision-makers about the utility of existing explanations and other desiderata they would like to see in an explanation going forward. In this work, we address this gap and carry out a study where we interview doctors, healthcare professionals, and policymakers about their needs and desires for explanations. Our study indicates that decision-makers would strongly prefer interactive explanations in the form of natural language dialogues. Domain experts wish to treat machine learning models as"another colleague", i.e., one who can be held accountable by asking why they made a particular decision through expressive and accessible natural language interactions. Considering these needs, we outline a set of five principles researchers should follow when designing interactive explanations as a starting place for future work. Further, we show why natural language dialogues satisfy these principles and are a desirable way to build interactive explanations. Next, we provide a design of a dialogue system for explainability and discuss the risks, trade-offs, and research opportunities of building these systems. Overall, we hope our work serves as a starting place for researchers and engineers to design interactive explainability systems.