CHS: Small: Collaborative Research: Articulate+ - A Conversational Interface for Democr atizing Visual Analysis

互联网为公众，包括科学家，提供了获取大量数据的途径。可视化是一种强大的工具，可以将数据转化为更容易理解的形式，帮助人们做出更好的决策。然而，从数据中创建良好的可视化并不容易。“清晰+”项目将致力于以新的方式，通过语音和手势作为输入，为人们提供信息可视化的新知识。目标是使用户能够从他们想要的不精确的陈述中受益，而不是需要给出特定的命令来告诉计算机该做什么。这个项目将使公民更容易与计算机就经济、医疗数据、人口、交通数据、气候和体育数据进行对话；找到更有意义的答案，从而使数据素养民主化。这项研究将使用语音、手势和日志数据作为视觉分析的新输入交互模式。它将发展对自然语言查询和手势的不精确和模糊性质的新理解，作为在特定视觉分析问题上工作的背景，可以建模为智能软件系统更多地了解用户潜在意图的机会。这个意图模型，反过来，将被用来提供上下文化的可视化，这有望帮助这些用户从他们的数据中获得有价值的见解。音频语音数据将用于计算偷听模型，以帮助推断用户当前的语境目标。手势数据将用于消除涉及可视化视觉元素的表达式的歧义（例如，“关于电力消耗的饼状图”）。该项目将开发新的理解，如何结合可视化状态日志、偷听的计算模型和手势数据，以生成新的可视化，以支持用户在任务上的工作。评估将在受控实验室和自然研究环境中进行，以确定是否可以为这些特定的视觉分析领域衍生出有效的语义解析器，以及这种上下文界面如何影响用户的可视化和发现体验。研究中整合的注释数据集将提供给研究界，以满足对生态有效的情境语言交互语料库的需求。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The Internet gives the public, including scientists, access to vast amounts of data. Visualization is a powerful tool to take that data and turn it into a form that is easier to understand, helping people make better decisions. However, creating good visualizations from data is not easy. The Articulate+ Project will work to generate new knowledge about how to empower people with information visualization, by using speech and gesture as inputs, in new ways. A goal is to enable users to benefit from imprecise statements about what they want, rather than needing to give specific commands telling the computer what to do. This project will make it easier for citizens to engage in conversations with the computer about economic, medical data, demographic, transportation data, climate, and sports data; to answers that are more meaningful, and so to democratize data literacy. This research will use speech, gesture, and log data as new input interaction modalities for visual analytics. It will develop new understandings of how the imprecise and vague nature of natural language queries and gestures, as contextualized by work on a specific visual analysis problem, can be modeled as an opportunity for an intelligent software system to learn more about the underlying intent of the users. This model of intent, in turn, will be used to provide contextualized visualizations, which are expected to help those users gain valuable insights from their data. The audio speech data will be used to computationally model overhearing, to help infer users' current contextualized goals. The gesture data will be used to disambiguate expressions that refer to visual elements of a visualization (e.g., "that pie chart about electricity consumption"). The project will develop new understandings of how to combine the log of visualization state, the computational model of overhearing, and the gesture data in order to generate new visualizations in support of users’ work on tasks. Evaluations will be performed in both controlled laboratory and naturalistic study settings to determine whether effective semantic parsers can be derived for these specific visual analysis domains, and how this contextual interface affects users’ experience of visualization and discovery. The integrated annotated datasets from the studies will be made available to the research community satisfying a need for ecologically valid situated language interaction corpora.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.