EAGER: Collaborative Research: Articulate: Augmenting Data Visualization With Natural Language Interaction

近三分之一的人脑用于处理视觉信息。视觉是从我们的日常世界中获取信息的主要感觉。因此，可视化即使是最简单的形式，仍然是将大量原始数据转化为洞察的最有效手段，这一过程可以支持科学发现，这一点也就不足为奇了。然而，阻碍科学用户采用最新的可视化工具和技术的一个关键挑战是，为了利用它们，必须克服陡峭的学习曲线。于是，人们倾向于求助于最简单的工具，如条形图和折线图，尽管它们可能缺乏将科学数据集中在一起所需的表现力。科学家理想情况下只需与计算机交谈，就他们的数据提出问题，并让计算机自动生成可视化回答他们的问题，这一概念自NSF 2007年的报告《通过视觉探索实现科学发现》以来就已经广为人知。这就是目前这个项目的动机，该项目涉及两个机构的研究人员之间的合作，因为科学家仍然无法做到这一点。PIS的最终目标是实现一个虚拟可视化专家，将科学的语言翻译成可视化的语言。为了证明这一概念确实是可行的，PI之前开发并评估了一个小原型，这支持了他们的论点，即通过减轻用户必须学习如何使用复杂界面的负担，可以使他们能够专注于阐明更好的科学问题。鉴于这一初步成功，这项探索性研究的重点是建立一种更普遍的方法的基础，该方法可以涵盖科学可视化中使用的技术。为此，公共关系司将研究将自然语言请求(可能伴有手势)映射为有意义的可视化以及实现逐步创建和修改可视化所需的步骤。他们将开发创新的模型来理解用户的意图和S所指的对象，他们将探索如何最好地设计用户界面，以便使用语言和直接操作来创建和修改可视化。PI的初步研究表明，所有这些功能对于使用户能够最好地利用对话界面进行数据可视化至关重要。虽然项目成果将在短期内服务于科学界，但这些技术应该更广泛地适用于信息消费者，如公民科学家、公共政策决策者和学生。

Nearly one third of the human brain is devoted to processing visual information. Vision is the dominant sense for the acquisition of information from our everyday world. It is therefore no surprise that visualization, even in its simplest forms, remains the most effective means for converting large volumes of raw data into insight, a process that can support scientific discovery. However a key challenge hindering scientific users from adopting the latest visualization tools and techniques is the steep learning curve that has to be overcome in order to make use of them. The tendency then is to resort to the simplest tools, such as bar charts and line graphs, even though they may lack the expressive power necessary to bring scientific data into focus.The notion that scientists would ideally like to simply speak with a computer to ask questions about their data, and have the computer automatically generate visualizations that answer their queries, has been well known since at least the NSF 2007 report "Enabling Science Discoveries through Visual Exploration." This is the motivation for the current project, which involves a collaboration among researchers at two institutions, given that scientists still are unable to do so. The PIs' ultimate goal is to implement a Virtual Visualization Expert to translate the language of science into the language of visualization. To demonstrate the concept is indeed viable, the PIs previously developed and evaluated a small prototype, which supported their argument that by relieving the user of the burden of having to learn how to use a complex interface one could enable them to focus on articulating better scientific questions.Given this initial success, the focus of this exploratory research is to establish the foundations of a more generalizable approach that can encompass techniques used in scientific visualization. To this end, the PIs will research the steps needed for mapping natural language requests, which may be accompanied by gestures, into meaningful visualizations and for enabling incremental creation and modifications of visualizations. They will develop innovative models to understand the intent of the user and the objects s/he is referring to, and they will explore how best to design user interfaces for creating and modifying visualizations using language and direct manipulation. The PIs' initial study showed that all these capabilities are crucial to enabling users to make the best use of a dialogic interface for data visualization. Although project outcomes will be geared in the short term to serving the scientific community, the techniques should be applicable more broadly to consumers of information, such as citizen scientists, public policy decision makers, and students.