Fizz Reader: Interactive Accessible Data Visualizations Through an NLG Interface

Fizz Reader：通过 NLG 界面进行交互式数据可视化

• 批准号: 10157688
• 负责人: Douglas Alan Schepers
• 金额: $ 17.58万
• 依托单位: FIZZ STUDIO LLC
• 依托单位国家: 美国
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-03-02 至 2022-08-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10157688
• 关键词: American; Architecture; Braille Display; COVID-19 pandemic; Client; Complex; Computer software; Data; Decision Making; Development; Effectiveness; Ensure; Feedback; Flowcharts; Foundations; Generations; Goals; Grant; Methodology; Methods; Modality; Modeling; Modernization; Operating System; Pattern; Persons; Phase; Psyche structure; Public Health; Publishing; Quality of life; Reader; Risk; Small Business Innovation Research Grant; Stress; Tactile; Technology; Telephone; Testing; Text; Thinness; Time; Training; Visual; Visual impairment; Work; base; blind; cognitive disability; data modeling; data visualization; design; disability; experience; handheld mobile device; haptics; human subject; intelligent personal assistant; interoperability; natural language; novel; operation; statistics; trend; usability; virtual; web site

Specific Aims There are many modern approaches to accessibility in data visualizations, from simple but inadequate alternative data tables to excellent work with tactile graphics, haptics, and sonification. Each has its benefits and limitations, but common underlying weaknesses to the advanced methods are lack of precision and need for reader training. By contrast, most of the 7.7 million Americans with visual disabilities are at least moderately skilled in using a screen reader. However, this typically limits users to serial data point access, the equivalent of a data table, which does not provide an equivalent experience to the advantages of data visualizations. It decreases independence and professional opportunities, increases stress, decreases quality of life, and puts vulnerable people at risk when crucial public health information is disseminated in graphical-only form, such as in the current COVID-19 pandemic. The long range goal of this Phase I project is to create technologies that permit authors and developers of web sites to effortlessly publish charts, diagrams, and infographics that are fully usable by all people, in particular people who are blind, low-vision, or have cognitive disabilities. In this Phase I SBIR project, we will assess the feasibility of creating interactive contextual automatic descriptions that enable the reader to construct an accurate working mental model of the data with minimal effort and time, to perform tasks and make decisions. Fizz Studio has created a software package, Fizz Charts, that generates accessible keyboard-browsable charts for use on any website. We seek to enhance this with Fizz Reader, a novel interactive interface that uses natural language generation (NLG) to enable the user to query the chart for quick answers about each data point, its relationship to other data points and to the chart statistics, and to high-level or detailed trends and patterns in the data. The effect is of one person explaining the chart to another over the phone, and providing relevant and rapid answers to help the listener understand as much of the data as they wish for a core set of 7 common chart types: bar; line; pie; histogram; scatterplot; heatmap; and flowchart. Aim 1: Develop effective interactive NLG model and engine module To concisely communicate relevant details to the user, we will design a comprehensive set of tasks for all supported chart types, and a set of NLG templates for each chart component (e.g. data point, axis, title). We will use these NLG templates to develop a software module which composes colloquial utterances from an internal statistical data model we build from the data extracted from the chart. Each set of options will represent the affordances optimal for the chart type (e.g. comparisons for bar charts, changes over time for line charts). This module will have a client-server API architecture, to make it adaptable to multiple user interface modalities, including the screen reader intermediary in Aim 2, as well as a standalone digital assistant or a component in a smart speaker. To ensure effectiveness and clarity, we will perform multi-phase testing for usability and task accomplishment with 10-20 blind and low-vision human subjects, and integrate their feedback. We seek 80% successful task accomplishment across all chart types and tasks. Aim 2: Implement a browser-based screen reader intermediary User Interface Module (UIM) To integrate our NLG module directly into the browser, for on-demand presentation of the generated NLG chart information to users of screen readers, we will implement an option-based interface using JavaScript and ARIA technologies. Each set of options will be context-dependent on the currently focused item (e.g. data point, axis, title), and will reflect the optimal tasks for that item in that specific chart type, to enable users to use keyboard input to select from a virtual dropdown of options, and the results will be returned as text to be presented by the screen reader or braille display. We will use a Lean development methodology to test for 95% interoperability across common browsers, operating systems, mobile devices, and screen readers. Conclusion and goals for Phase II The purpose of this Phase I grant is to significantly advance the state of the art for the accessibility of data charts for people with visual disabilities to aid them in their personal and professional lives. In Phase II, we will build on this foundation to support more complex diagram types including schematics, refine the UI for more operations, and explore other deployments such as smart speakers, browser extensions, or app plugins.

具体目标 在数据可视化中有许多现代的可访问性方法，从简单但不充分的 替代数据表，以出色的工作与触觉图形，触觉，和sonification。各有各的好处 和局限性，但共同的潜在弱点，先进的方法是缺乏准确性和需要 读者培训。相比之下，770万美国视力残疾人中的大多数至少是中度残疾。 熟练使用屏幕阅读器。但是，这通常会限制用户访问串行数据点， 这并不能提供与数据可视化的优势等同的体验。它 减少独立性和职业机会，增加压力，降低生活质量， 当关键的公共卫生信息仅以图表形式传播时， 就像当前的COVID-19疫情一样。 这个第一阶段项目的长期目标是创建技术，允许Web的作者和开发人员 轻松发布图表、图表和信息图表的网站，这些图表、图表和信息图表可供所有人使用，特别是 失明、低视力或有认知障碍的人。在第一阶段SBIR项目中，我们将评估 创建交互式上下文自动描述的可行性，使读者能够构建 以最少的努力和时间，准确地工作数据的心理模型，以执行任务， 决策 Fizz Studio创建了一个软件包，Fizz Charts，它可以生成可访问的键盘浏览 图表可用于任何网站。我们寻求通过Fizz Reader来增强这一点，这是一种新颖的交互式界面， 使用自然语言生成（NLG），使用户能够查询图表，以获得有关以下内容的快速答案 每个数据点，它与其他数据点和图表统计的关系，以及与高级或详细 数据中的趋势和模式。其效果是一个人通过电话向另一个人解释图表， 并提供相关和快速的答案，以帮助听众了解尽可能多的数据，因为他们希望 7种常见图表类型的核心集合：条形图、折线图、饼图、直方图、散点图、热图和流程图。 目标1：开发有效的交互式NLG模型和引擎模块 为了简洁地向用户传达相关细节，我们将为所有人设计一套全面的任务 支持的图表类型，以及一组用于每个图表组件（例如数据点，轴，标题）的NLG模板。 我们将使用这些NLG模板开发一个软件模块，组成口语话语 从内部统计数据模型，我们建立从图表中提取的数据。每组选项 将表示图表类型的最佳示能表示（例如，条形图的比较， 折线图）。该模块将具有客户端-服务器API架构，以使其适用于多个用户 界面形式，包括Aim 2中的屏幕阅读器中介，以及独立的数字助理 或智能扬声器中的组件。为确保有效性和清晰性，我们将进行多阶段测试 用于10 - 20名盲人和低视力人类受试者的可用性和任务完成， 反馈我们在所有图表类型和任务中寻求80%的成功任务完成率。 目标2：实现基于浏览器的屏幕阅读器中间用户界面模块（UIM） 将我们的NLG模块直接集成到浏览器中，以便按需呈现生成的NLG图表 为了向屏幕阅读器用户提供信息，我们将使用JavaScript实现一个基于选项的界面， ARIA技术。每组选项都将取决于当前关注的项目（例如数据 点、轴、标题），并将反映特定图表类型中该项的最佳任务，以使用户能够 使用键盘输入从虚拟的选项列表中选择，结果将以文本形式返回， 由屏幕阅读器或盲文显示器呈现。我们将使用精益开发方法来测试 在通用浏览器、操作系统、移动的设备和屏幕阅读器之间实现95%的互操作性。 第二阶段的结论和目标 第一阶段赠款的目的是大大提高数据可访问性的最新水平 视力残疾人士图表，以帮助他们在个人和职业生活。在第二阶段，我们将 在此基础上构建以支持更复杂的图表类型（包括原理图）， 操作，并探索其他部署，如智能扬声器，浏览器扩展或应用程序插件。