Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Content
Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Content
复制标题
DOI：
10.1109/tvcg.2021.3114770
复制
发表时间：
2021-09
影响因子：
5.2
通讯作者：
Alan Lundgard;Arvind Satyanarayan
中科院分区：
文献类型：
--
作者：
Alan Lundgard;Arvind Satyanarayan
自然语言描述有时伴随着可视化，以更好地传达和情境化他们的见解，并提高残疾读者的可访问性。然而，很难评估这些描述的有用性，以及它们如何有效地改善对有意义信息的访问，因为我们对它们所传达的语义内容以及不同读者如何接收这些内容知之甚少。作为回应，我们引入了一个概念模型的语义内容所传达的自然语言描述的可视化。通过对2,147个句子的扎根理论分析，我们的模型跨越了四个层次的语义内容：列举可视化构造属性（例如，标记和编码）;报告统计概念和关系（例如，极值和相关性）;识别感知和认知现象（例如，复杂的趋势和模式）;以及阐明特定领域的见解（例如，社会和政治背景）。为了证明我们的模型可以应用于评估可视化描述的有效性，我们进行了一个混合方法的评估与30个盲人和90个视力正常的读者，并发现这些读者群体显着不同的语义内容，他们排名为最有用的。总之，我们的模型和研究结果表明，访问有意义的信息是强烈的读者特定的，在自动可视化字幕的研究应面向描述，更丰富的沟通整体趋势和统计数据，读者的喜好敏感。我们的工作进一步打开了自然语言作为与可视化同等的数据接口的研究空间。
Natural language descriptions sometimes accompany visualizations to better communicate and contextualize their insights, and to improve their accessibility for readers with disabilities. However, it is difficult to evaluate the usefulness of these descriptions, and how effectively they improve access to meaningful information, because we have little understanding of the semantic content they convey, and how different readers receive this content. In response, we introduce a conceptual model for the semantic content conveyed by natural language descriptions of visualizations. Developed through a grounded theory analysis of 2,147 sentences, our model spans four levels of semantic content: enumerating visualization construction properties (e.g., marks and encodings); reporting statistical concepts and relations (e.g., extrema and correlations); identifying perceptual and cognitive phenomena (e.g., complex trends and patterns); and elucidating domain-specific insights (e.g., social and political context). To demonstrate how our model can be applied to evaluate the effectiveness of visualization descriptions, we conduct a mixed-methods evaluation with 30 blind and 90 sighted readers, and find that these reader groups differ significantly on which semantic content they rank as most useful. Together, our model and findings suggest that access to meaningful information is strongly reader-specific, and that research in automatic visualization captioning should orient toward descriptions that more richly communicate overall trends and statistics, sensitive to reader preferences. Our work further opens a space of research on natural language as a data interface coequal with visualization.