HCC: Medium: Improving data visualization and analysis tools to support reasoning about analysis assumptions

HCC：中：改进数据可视化和分析工具以支持分析假设的推理

• 批准号: 2211939
• 负责人: Jessica Hullman
• 金额: $ 119.46万
• 依托单位: Northwestern University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2026-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2211939&HistoricalAwards=false
• 关键词: HCC; Medium; Improving; data; visualization

Using statistics to model data often requires making assumptions about what the data represent, including the nature of underlying patterns and errors. For example, an analyst interested in the relationship between wealth and age might remove extremely low or high reported ages from her data as outliers, and choose to model the remaining data using a linear model, which implies that age increases wealth by some constant factor. Her results could imply substantially different conclusions about how wealth and age relate compared to an analysis that made different decisions about which data to include. One way for analysts to account for such sensitivity is by reporting the results of many reasonable analyses given a dataset and questions they want to answer using the data. Unfortunately, existing data analysis and visualization tools offer little support for reasoning about such a “multiverse analysis.” They provide limited support for comparison of multiple models and visualizations that make different choices, and even less support in helping analysts reason about and express those choices. Further, there are few known ways to effectively convey both uncertainty in the results of a given analysis and uncertainty related to the assumptions made in that analysis. This project’s goal is to improve multiverse analysis: to better understand how analysts currently think about multiverse analysis, and to identify needs, opportunities, and approaches to help analysts use multiverse analyses. This project addresses these challenges to expressing hard-to-quantify uncertainty related to analysis choices by creating new methods and tools to help analysts define, reason about, and express multiple alternative ways they could analyze their data. The research will focus on two common types of tools analysts use: visual analysis software that makes it easy to plot and compare data, and computational notebooks that allow for more seamless integration of code and narrative commentary. The project team will develop new user interfaces and programming libraries to elicit analysts’ knowledge, as well as new visual representations and interaction techniques by which an analyst can compare between alternative models or analysis paths. The project will also produce novel software infrastructure to make conducting and evaluating multiple analyses feasible within existing tools and workflows. Further, the team will develop ways to better communicate multiverse analyses: ways to make multiverse analysis reports shareable, interactive documents that contain both the analysis code and figures as well as narrative context, and empirical results describing how different representations of plausible analyses impact readers’ understanding. These research activities will be guided by the results of formative studies with real-world analysts that will address gaps in existing knowledge about the difficulties analysts face in defining and reasoning about alternative models or analysis steps they could have taken. All study results and computational tools will be made freely and publicly available.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

使用统计数据来建模数据通常需要对数据代表的内容进行假设，包括基本模式和错误的性质。例如，对财富与年龄之间关系感兴趣的分析师可能会从数据中以极低或高的年龄为单位，并选择使用线性模型对剩余数据进行建模，这意味着年龄会增加一定的因素。她的结果可能意味着与对包含哪些数据做出不同决定的分析相比，关于财富和年龄如何相关的结论大大不同。分析师解释这种敏感性的一种方法是报告许多合理分析的结果，并且给定数据集和他们想要使用数据回答的问题。不幸的是，现有的数据分析和可视化工具几乎没有支持这种“多元宇宙分析”的推理。它们为比较多种模型和可视化的比较提供了有限的支持，这些模型和可视化是做出不同选择的，甚至在帮助分析师推理和表达这些选择方面的支持更少。此外，在给定分析结果和与该分析中的假设相关的不确定性中，几乎没有什么可有效传达不确定性的方法。该项目的目标是改善多元宇宙分析：更好地了解分析师当前如何考虑多元宇宙分析，并确定帮助分析师使用多元宇宙分析的需求，机会和方法。该项目通过创建新的方法和工具来帮助分析师定义，理性和表达他们可以分析数据的新方法来解决与分析选择相关的难以征服的不确定性，以解决与分析选择相关的不确定性。该研究将重点放在分析师使用的两种常见类型的工具上：视觉分析软件，这些软件使绘制和比较数据变得易于绘制和比较数据，以及允许更多无缝集成代码和叙事评论的计算笔记本。项目团队将开发新的用户界面和编程库，以引起分析师的知识，以及新的视觉表示和互动技术，分析师可以在替代模型或分析路径之间进行比较。该项目还将生成新颖的软件基础架构，以进行和评估现有工具和工作流程中可行的多种分析。此外，团队将开发更好地交流多元宇宙分析的方法：使多元宇宙分析报告可共享的交互式文档，既包含分析代码和数字以及叙事环境，又有叙事环境，以及描述合理分析的不同表示形式如何影响阅读者的理解。这些研究活动将以现实世界分析师的形成性研究结果为指导，这些研究人员将解决有关分析师在定义和推理他们可能采取的替代模型或分析步骤时面临的困难和推理方面面临的差距。所有研究结果和计算工具都将免费和公开获得。该奖项反映了NSF的法定任务，并通过使用基金会的知识分子优点和更广泛的影响评估标准来评估诚实地支持了支持。