CHS: Small: Collaborative Research: Articulate+ - A Conversational Interface for Democr atizing Visual Analysis

CHS：小型：协作研究：Articulate - 用于民主化视觉分析的对话界面

• 批准号: 2008986
• 负责人: Jason Leigh
• 金额: $ 16.6万
• 依托单位: University of Hawaii
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2023-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2008986&HistoricalAwards=false
• 关键词: CHS; Small; Collaborative; Research; Articulate+

The Internet gives the public, including scientists, access to vast amounts of data. Visualization is a powerful tool to take that data and turn it into a form that is easier to understand, helping people make better decisions. However, creating good visualizations from data is not easy. The Articulate+ Project will work to generate new knowledge about how to empower people with information visualization, by using speech and gesture as inputs, in new ways. A goal is to enable users to benefit from imprecise statements about what they want, rather than needing to give specific commands telling the computer what to do. This project will make it easier for citizens to engage in conversations with the computer about economic, medical data, demographic, transportation data, climate, and sports data; to answers that are more meaningful, and so to democratize data literacy. This research will use speech, gesture, and log data as new input interaction modalities for visual analytics. It will develop new understandings of how the imprecise and vague nature of natural language queries and gestures, as contextualized by work on a specific visual analysis problem, can be modeled as an opportunity for an intelligent software system to learn more about the underlying intent of the users. This model of intent, in turn, will be used to provide contextualized visualizations, which are expected to help those users gain valuable insights from their data. The audio speech data will be used to computationally model overhearing, to help infer users' current contextualized goals. The gesture data will be used to disambiguate expressions that refer to visual elements of a visualization (e.g., "that pie chart about electricity consumption"). The project will develop new understandings of how to combine the log of visualization state, the computational model of overhearing, and the gesture data in order to generate new visualizations in support of users’ work on tasks. Evaluations will be performed in both controlled laboratory and naturalistic study settings to determine whether effective semantic parsers can be derived for these specific visual analysis domains, and how this contextual interface affects users’ experience of visualization and discovery. The integrated annotated datasets from the studies will be made available to the research community satisfying a need for ecologically valid situated language interaction corpora.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

互联网为包括科学家在内的公众提供了大量数据的访问。可视化是获取数据并将其变成更容易理解的形式的强大工具，可以帮助人们做出更好的决策。但是，从数据中创建良好的可视化并不容易。该清晰度+项目将努力通过以新方式使用语音和手势作为输入来生成有关如何通过使用语音和手势来赋予人们信息可视化的新知识。一个目标是使用户能够从有关其想要的内容的启动语句中受益，而不是需要给计算机告诉计算机该怎么做。该项目将使公民更容易与计算机就经济，医疗数据，人口统计，运输数据，气候和体育数据进行对话；要答案更有意义，从而使数据素养民主化。这项研究将使用语音，手势和日志数据作为视觉分析的新输入相互作用方式。它将对自然语言查询和手势的触发性质和投票性质进行新的理解，这是通过在特定视觉分析问题上进行的上下文对智能软件系统建模的，以便为智能软件系统的机会建模更多有关用户的基本意图的信息。反过来，这种意图模型将用于提供上下文化的可视化，这有望帮助这些用户从其数据中获得宝贵的见解。音频语音数据将用于计算模型偷听，以帮助推断用户当前的上下文化目标。手势数据将用于消除歧义表达式，指的是可视化的视觉元素（例如，“关于电消耗的饼图”）。该项目将对如何结合可视化状态日志，偷窃的计算模型以及手势数据的新理解，以生成新的可视化，以支持用户对任务的工作。评估将在受控实验室和自然主义研究环境中进行，以确定是否可以针对这些特定的视觉分析域获得有效的语义解析器，以及该上下文接口如何影响用户的可视化和发现经验。研究社区将提供来自研究的综合注释数据集，满足了对生态有效的语言互动语料库的需求。该奖项反映了NSF的法定任务，并通过使用基金会的智力优点和更广泛的影响来评估NSF的法定任务。

项目成果

Articulate+ : An Always-Listening Natural Language Interface for Creating Data Visualizations

• DOI: 10.1145/3543829.3544534
• 发表时间: 2022-07
• 期刊: Proceedings of the 4th Conference on Conversational User Interfaces
• 作者: Roderick S. Tabalba;Nurit Kirshenbaum;J. Leigh;Abari Bhattacharya;Andrew E. Johnson;Veronica Grosso;Barbara Maria Di Eugenio;Moira L. Zellner

An Investigation into an Always Listening Interface to Support Data Exploration

• DOI: 10.1145/3581641.3584079
• 发表时间: 2023-03
• 期刊: Proceedings of the 28th International Conference on Intelligent User Interfaces
• 作者: Roderick S. Tabalba;Nurit Kirshenbaum;J. Leigh;Abari Bhattacharya;Veronica Grosso;Barbara Di Eugenio;Andrew E. Johnson;Moira L. Zellner