Answering Questions about Charts and Generating Visual Explanations
Answering Questions about Charts and Generating Visual Explanations
复制标题
DOI：
10.1145/3313831.3376467
复制
发表时间：
2020-04
期刊：
影响因子：
--
通讯作者：
Dae Hyun Kim;Enamul Hoque;Maneesh Agrawala
中科院分区：
文献类型：
--
作者：
Dae Hyun Kim;Enamul Hoque;Maneesh Agrawala
人们经常使用图表来分析数据，回答问题并向他人解释答案。在一项形成性研究中，我们发现这种人类生成的问题和解释通常是指图表的视觉特征。基于这项研究，我们开发了一个自动图表问题答案管道，该问题生成了视觉说明，描述了如何获得答案。我们的管道首先从输入Vega-Lite图表中提取数据和视觉编码。然后，给定关于图表的自然语言问题，它将对视觉属性的引用转换为对数据的引用。接下来，它应用了最先进的机器学习算法来回答转换的问题。最后，它使用基于模板的方法来解释自然语言如何从图表的视觉特征确定答案。一项用户研究发现，我们的管道生成的视觉解释在透明度方面的表现明显胜过，并且在实用性和信任上与人类生成的解释相当。
People often use charts to analyze data, answer questions and explain their answers to others. In a formative study, we find that such human-generated questions and explanations commonly refer to visual features of charts. Based on this study, we developed an automatic chart question answering pipeline that generates visual explanations describing how the answer was obtained. Our pipeline first extracts the data and visual encodings from an input Vega-Lite chart. Then, given a natural language question about the chart, it transforms references to visual attributes into references to the data. It next applies a state-of-the-art machine learning algorithm to answer the transformed question. Finally, it uses a template-based approach to explain in natural language how the answer is determined from the chart's visual features. A user study finds that our pipeline-generated visual explanations significantly outperform in transparency and are comparable in usefulness and trust to human-generated explanations.