nlvis: Natural Language Interaction for Visual Data Analysis

nlvis：用于可视化数据分析的自然语言交互

• 批准号: EP/P025501/1
• 负责人: Cagatay Turkay
• 金额: $ 12.85万
• 依托单位: City, University of London
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2017
• 资助国家: 英国
• 起止时间: 2017 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FP025501%2F1
• 关键词: nlvis; Natural; Language; Interaction; Visual

The unprecedented increase in the amount, variety and value of data has been significantly transforming the way that scientific research is carried out and businesses operate. As data sources become increasingly diverse and complex, analysis approaches where the human and the computer operate in collaboration have proven to be an effective approach to derive actionable observations. This is achieved through an iterative human-computer dialogue where the knowledge and the creativity of the human meets the power of computation. In such human-in-the-loop data analysis approaches, interactive visualisation methods are core facilitators of this dialogue. However, these methods still rely on conventional, not often intuitive interaction mechanisms that can introduce unnecessary complexities into the process. There is an urgent need to rethink the ways how analysts interact with visualisations in data-intensive analysis situations. The recent advances in natural language based interaction methodologies offer promising avenues to address that. This project aims to develop a fundamental understanding of how analysts can use natural language elements to perform visualisation empowered data analysis and use that understanding to develop a framework where natural language and visualisation based interactions operate in harmony. The project then aims to demonstrate how such a multi-modal interaction scheme can radically transform the analysts' experience with the goal of achieving significant improvements in the value and the volume of actionable observations generated.Within the project, we will initially identify and develop a taxonomy of natural language interaction elements for describing visualisations and for carrying out a visual data analysis process. Here, we will inform our investigation with findings from data collected through crowd-based survey methodologies. We will then design a conceptual framework that facilitates an iterative data analysis process through interactions with both natural language and visualisation elements. We will make use of the data analysis and visualisation related language taxonomy from the earlier stage to define the scope and the capabilities of the interaction elements.The project will then move on to realising its vision through a prototype where the conceptual framework will operate through the help of an established conversational interface mechanism. The prototype will involve a combination of natural language and visual interaction capabilities and will also incorporate underlying computational capacities. We will then evaluate our approaches through a series of carefully designed use-cases that encompass common visual analysis scenarios. Our success criteria will be to achieve enhanced engagement and improved productivity during the visual analysis of complex data-intensive problems.Potential beneficiaries of the outputs of this project ranges widely from academic researchers, professional data analysts, data analysis industry, and the general public. For visualisation and visual analytics researchers, findings will benefit researchers who are working on understanding user-intent and mechanisms of sense-making in interactive visual analysis processes. For businesses that offer visualisation-empowered solutions to their customers (according to some reports, the visualisation market size is expected to reach a $2.8 Billion by 2020), the framework developed will provide the basis for new forms of products that are easier to learn and engage with. For professional data analysts, the novel interaction capability will offer a more fluid and natural experience, improving their efficiency and positively impacting the quality of observations. For the general public, natural interaction mechanisms will provide an enhanced experience when using data-intensive products that are becoming to be widely adopted.

数据的数量、种类和价值空前增加，极大地改变了科学研究和企业运营的方式。随着数据源变得越来越多样化和复杂，人类和计算机协作操作的分析方法已被证明是获得可操作观察结果的有效方法。这是通过迭代的人机对话来实现的，其中人类的知识和创造力与计算的能力相结合。在这种人机交互的数据分析方法中，交互式可视化方法是这种对话的核心促进者。然而，这些方法仍然依赖于传统的、通常不直观的交互机制，这可能会给过程带来不必要的复杂性。迫切需要重新思考分析师在数据密集型分析情况下与可视化交互的方式。基于自然语言的交互方法的最新进展为解决这个问题提供了有希望的途径。该项目旨在对分析师如何使用自然语言元素执行可视化支持的数据分析有一个基本的理解，并利用这种理解来开发一个框架，使自然语言和基于可视化的交互和谐运行。该项目旨在展示这种多模式交互方案如何从根本上改变分析师的体验，从而显着提高生成的可操作观察的价值和数量。在该项目中，我们将首先确定和开发自然语言交互元素的分类法，用于描述可视化和执行可视化数据分析过程。在这里，我们将通过基于人群的调查方法收集的数据的结果来告知我们的调查。然后，我们将设计一个概念框架，通过与自然语言和可视化元素的交互来促进迭代数据分析过程。我们将利用前期的数据分析和可视化相关的语言分类来定义交互元素的范围和功能。然后该项目将继续通过原型实现其愿景，其中概念框架将在已建立的对话接口机制的帮助下运行。该原型将涉及自然语言和视觉交互功能的组合，并且还将包含底层计算能力。然后，我们将通过一系列精心设计的包含常见视觉分析场景的用例来评估我们的方法。我们的成功标准是在复杂的数据密集型问题的可视化分析过程中提高参与度并提高生产力。该项目成果的潜在受益者广泛包括学术研究人员、专业数据分析师、数据分析行业和公众。对于可视化和视觉分析研究人员来说，研究结果将使那些致力于理解交互式视觉分析过程中的用户意图和意义构建机制的研究人员受益。对于向客户提供可视化解决方案的企业（根据一些报告，可视化市场规模预计到 2020 年将达到 28 亿美元），开发的框架将为更容易学习和使用的新形式产品提供基础。对于专业数据分析师来说，新颖的交互功能将提供更加流畅和自然的体验，提高他们的效率并对观察质量产生积极影响。对于公众来说，自然交互机制将在使用正在广泛采用的数据密集型产品时提供增强的体验。

项目成果

Words of Estimative Correlation: Studying Verbalizations of Scatterplots

估计相关性的词语：研究散点图的语言化

• DOI: 10.48550/arxiv.1911.12793
• 发表时间: 2019
• 作者: Henkin R

Towards Multimodal Data Analytics: Integrating Natural Language into Visual Analytics

迈向多模态数据分析：将自然语言集成到可视化分析中

• 发表时间: 2018
• 作者: Henkin R

Towards Natural Language Empowered Interactive Data Analysis

迈向自然语言赋能的交互式数据分析

• 发表时间: 2018
• 作者: Turkay C

Going beyond Visualization. Verbalization as Complementary Medium to Explain Machine Learning Models

• 发表时间: 2018-09
• 作者: R. Sevastjanova;F. Becker;Basil Ell;C. Turkay;R. Henkin;Miriam Butt;D. Keim;E. Mennatallah

Words of Estimative Correlation: Studying Verbalizations of Scatterplots.

估计相关性的词语：研究散点图的语言化。

• DOI: 10.1109/tvcg.2020.3023537
• 发表时间: 2022
• 期刊: IEEE transactions on visualization and computer graphics
• 影响因子: 5.2
• 作者: Henkin R