Designing a Direct Feedback Loop between Humans and Convolutional Neural Networks through Local Explanations
Designing a Direct Feedback Loop between Humans and Convolutional Neural Networks through Local Explanations
复制标题
DOI：
10.1145/3610187
复制
发表时间：
2023-07
影响因子：
--
通讯作者：
Tong Sun;Yuyang Gao;Shubham Khaladkar;Sijia Liu;Liang Zhao;Younghoon Kim;S. Hong
中科院分区：
文献类型：
--
作者：
Tong Sun;Yuyang Gao;Shubham Khaladkar;Sijia Liu;Liang Zhao;Younghoon Kim;S. Hong
局部解释提供了图像上的热图，以解释卷积神经网络(CNN)如何获得其输出。由于其直观直观，该方法已成为诊断CNN的最流行的可解释人工智能(XAI)方法之一。然而，通过我们的形成性研究(S1)，我们捕获了ML工程师对本地解释的矛盾观点，将其作为构建CNN中有价值和不可或缺的愿景，而不是由于检测漏洞的启发式性质而使他们筋疲力尽的过程。此外，根据从诊断中了解到的脆弱性来指导CNN似乎具有极大的挑战性。为了缓解这一差距，我们设计了DeepFuse，这是第一个交互式设计，实现了用户和CNN之间的直接反馈循环，使用本地解释诊断和修改CNN的漏洞。DeepFuse帮助CNN的工程师系统地搜索“不合理”的当地解释，并以一种节省劳动力的方式为那些被认定为不合理的人注释新的边界。接下来，它根据给定的注释来控制模型，这样模型就不会引入类似的错误。我们与12名经验丰富的CNN工程师进行了为期两天的研究(S2)。使用DeepFuse，参与者制作了一个比目前最先进的模型更准确、更合理的模型。此外，参与者还发现，DeepFuse指导基于案例的推理的方式实际上可以改进他们目前的做法。我们提供了设计的含义，解释了未来的人机界面驱动的设计如何推动我们的实践，使XAI驱动的见解更具可操作性。
The local explanation provides heatmaps on images to explain how Convolutional Neural Networks (CNNs) derive their output. Due to its visual straightforwardness, the method has been one of the most popular explainable AI (XAI) methods for diagnosing CNNs. Through our formative study (S1), however, we captured ML engineers' ambivalent perspective about the local explanation as a valuable and indispensable envision in building CNNs versus the process that exhausts them due to the heuristic nature of detecting vulnerability. Moreover, steering the CNNs based on the vulnerability learned from the diagnosis seemed highly challenging. To mitigate the gap, we designed DeepFuse, the first interactive design that realizes the direct feedback loop between a user and CNNs in diagnosing and revising CNN's vulnerability using local explanations. DeepFuse helps CNN engineers to systemically search "unreasonable" local explanations and annotate the new boundaries for those identified as unreasonable in a labor-efficient manner. Next, it steers the model based on the given annotation such that the model doesn't introduce similar mistakes. We conducted a two-day study (S2) with 12 experienced CNN engineers. Using DeepFuse, participants made a more accurate and "reasonable" model than the current state-of-the-art. Also, participants found the way DeepFuse guides case-based reasoning can practically improve their current practice. We provide implications for design that explain how future HCI-driven design can move our practice forward to make XAI-driven insights more actionable.