Visual Interaction with Deep Learning Models through Collaborative Semantic Inference
Visual Interaction with Deep Learning Models through Collaborative Semantic Inference
复制标题
DOI：
10.1109/tvcg.2019.2934595
复制
发表时间：
2020-01-01
影响因子：
5.2
通讯作者：
Rush, Alexander M.
中科院分区：
文献类型：
--
作者：
Gehrmann, Sebastian;Strobelt, Hendrik;Rush, Alexander M.
当人类失去决策过程的代理权时，任务的自动化可能会产生严重的后果。深度学习模型特别容易受到影响，因为目前的黑盒方法缺乏可解释的推理。我们认为，深度学习系统的视觉界面和模型结构都需要考虑交互设计。我们提出了一个框架的协作语义推理（CSI）的交互和模型的协同设计，使人类和算法之间的视觉协作。该方法公开了模型的中间推理过程，允许与问题的视觉隐喻进行语义交互，这意味着用户可以理解和控制模型推理过程的部分。我们证明了CSI的可行性与共同设计的案例研究的文件摘要系统。
Automation of tasks can have critical consequences when humans lose agency over decision processes. Deep learning models are particularly susceptible since current black-box approaches lack explainable reasoning. We argue that both the visual interface and model structure of deep learning systems need to take into account interaction design. We propose a framework of collaborative semantic inference (CSI) for the co-design of interactions and models to enable visual collaboration between humans and algorithms. The approach exposes the intermediate reasoning process of models which allows semantic interactions with the visual metaphors of a problem, which means that a user can both understand and control parts of the model reasoning process. We demonstrate the feasibility of CSI with a co-designed case study of a document summarization system.