CHS: Small: Collaborative Research: Articulate+ - A Conversational Interface for Democr atizing Visual Analysis

互联网为包括科学家在内的公众提供了获取大量数据的途径。可视化是一种强大的工具，可以将这些数据转化为更容易理解的形式，帮助人们做出更好的决策。然而，从数据创建良好的可视化并非易事。发音+项目将致力于产生关于如何通过使用语音和手势作为输入，以新的方式赋予人们信息可视化能力的新知识。其目标是使用户能够从关于他们想要什么的不精确声明中受益，而不是需要发出特定的命令来告诉计算机要做什么。该项目将使公民更容易与计算机就经济、医疗数据、人口统计、交通数据、气候和体育数据进行对话；获得更有意义的答案，从而实现数据素养的民主化。这项研究将使用语音、手势和日志数据作为视觉分析的新输入交互模式。它将发展对自然语言查询和手势的不精确和模糊性质的新理解，这些不精确和模糊的性质是通过对特定视觉分析问题的工作而产生的，可以被模拟为智能软件系统更多地了解用户的潜在意图的机会。反过来，这种意图模型将被用于提供情景可视化，这有望帮助这些用户从他们的数据中获得有价值的见解。音频语音数据将被用来对偷听进行计算建模，以帮助推断用户当前的情景目标。手势数据将用于消除涉及可视化的可视元素(例如，关于电力消耗的饼状图)的表达的歧义。该项目将对如何将可视化状态的日志、偷听的计算模型和手势数据相结合以生成新的可视化以支持用户的任务工作产生新的理解。将在受控实验室和自然主义研究环境中进行评估，以确定是否可以为这些特定的可视分析领域推导出有效的语义解析器，以及该上下文界面如何影响用户的可视化和发现体验。来自研究的综合注释数据集将提供给研究社区，以满足对生态有效的情景语言交互语料库的需求。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The Internet gives the public, including scientists, access to vast amounts of data. Visualization is a powerful tool to take that data and turn it into a form that is easier to understand, helping people make better decisions. However, creating good visualizations from data is not easy. The Articulate+ Project will work to generate new knowledge about how to empower people with information visualization, by using speech and gesture as inputs, in new ways. A goal is to enable users to benefit from imprecise statements about what they want, rather than needing to give specific commands telling the computer what to do. This project will make it easier for citizens to engage in conversations with the computer about economic, medical data, demographic, transportation data, climate, and sports data; to answers that are more meaningful, and so to democratize data literacy. This research will use speech, gesture, and log data as new input interaction modalities for visual analytics. It will develop new understandings of how the imprecise and vague nature of natural language queries and gestures, as contextualized by work on a specific visual analysis problem, can be modeled as an opportunity for an intelligent software system to learn more about the underlying intent of the users. This model of intent, in turn, will be used to provide contextualized visualizations, which are expected to help those users gain valuable insights from their data. The audio speech data will be used to computationally model overhearing, to help infer users' current contextualized goals. The gesture data will be used to disambiguate expressions that refer to visual elements of a visualization (e.g., "that pie chart about electricity consumption"). The project will develop new understandings of how to combine the log of visualization state, the computational model of overhearing, and the gesture data in order to generate new visualizations in support of users’ work on tasks. Evaluations will be performed in both controlled laboratory and naturalistic study settings to determine whether effective semantic parsers can be derived for these specific visual analysis domains, and how this contextual interface affects users’ experience of visualization and discovery. The integrated annotated datasets from the studies will be made available to the research community satisfying a need for ecologically valid situated language interaction corpora.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.