HCC: Medium: Improving data visualization and analysis tools to support reasoning about analysis assumptions

HCC：中：改进数据可视化和分析工具以支持分析假设的推理

• 批准号: 2211939
• 负责人: Jessica Hullman
• 金额: $ 119.46万
• 依托单位: Northwestern University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2026-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2211939&HistoricalAwards=false
• 关键词: HCC; Medium; Improving; data; visualization

Using statistics to model data often requires making assumptions about what the data represent, including the nature of underlying patterns and errors. For example, an analyst interested in the relationship between wealth and age might remove extremely low or high reported ages from her data as outliers, and choose to model the remaining data using a linear model, which implies that age increases wealth by some constant factor. Her results could imply substantially different conclusions about how wealth and age relate compared to an analysis that made different decisions about which data to include. One way for analysts to account for such sensitivity is by reporting the results of many reasonable analyses given a dataset and questions they want to answer using the data. Unfortunately, existing data analysis and visualization tools offer little support for reasoning about such a “multiverse analysis.” They provide limited support for comparison of multiple models and visualizations that make different choices, and even less support in helping analysts reason about and express those choices. Further, there are few known ways to effectively convey both uncertainty in the results of a given analysis and uncertainty related to the assumptions made in that analysis. This project’s goal is to improve multiverse analysis: to better understand how analysts currently think about multiverse analysis, and to identify needs, opportunities, and approaches to help analysts use multiverse analyses. This project addresses these challenges to expressing hard-to-quantify uncertainty related to analysis choices by creating new methods and tools to help analysts define, reason about, and express multiple alternative ways they could analyze their data. The research will focus on two common types of tools analysts use: visual analysis software that makes it easy to plot and compare data, and computational notebooks that allow for more seamless integration of code and narrative commentary. The project team will develop new user interfaces and programming libraries to elicit analysts’ knowledge, as well as new visual representations and interaction techniques by which an analyst can compare between alternative models or analysis paths. The project will also produce novel software infrastructure to make conducting and evaluating multiple analyses feasible within existing tools and workflows. Further, the team will develop ways to better communicate multiverse analyses: ways to make multiverse analysis reports shareable, interactive documents that contain both the analysis code and figures as well as narrative context, and empirical results describing how different representations of plausible analyses impact readers’ understanding. These research activities will be guided by the results of formative studies with real-world analysts that will address gaps in existing knowledge about the difficulties analysts face in defining and reasoning about alternative models or analysis steps they could have taken. All study results and computational tools will be made freely and publicly available.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

使用统计数据建模通常需要对数据表示的内容做出假设，包括潜在模式和错误的性质。例如，对财富与年龄之间的关系感兴趣的分析师可能会从她的数据中删除报告的极低或极高的年龄作为异常值，并选择使用线性模型对剩余数据进行建模，这意味着年龄会通过某个常数因子增加财富。与就包含哪些数据做出不同决定的分析相比，她的结果可能意味着关于财富和年龄如何关联的结论有很大不同。分析师解释这种敏感性的一种方法是，根据给定的数据集和他们想要使用数据回答的问题，报告许多合理分析的结果。不幸的是，现有的数据分析和可视化工具对这种“多元宇宙分析”的推理几乎没有提供支持。它们对做出不同选择的多个模型和可视化的比较提供有限的支持，并且在帮助分析师推理和表达这些选择方面提供的支持更少。此外，几乎没有已知的方法可以有效地传达给定分析结果中的不确定性以及与该分析中所做的假设相关的不确定性。该项目的目标是改进多元宇宙分析：更好地了解分析师目前如何看待多元宇宙分析，并确定帮助分析师使用多元宇宙分析的需求、机会和方法。 该项目通过创建新的方法和工具来帮助分析师定义、推理和表达他们可以分析数据的多种替代方式，从而解决了表达与分析选择相关的难以量化的不确定性的挑战。该研究将重点关注分析师使用的两种常见工具：可以轻松绘制和比较数据的可视化分析软件，以及可以更无缝地集成代码和叙述性评论的计算笔记本。项目团队将开发新的用户界面和编程库以获取分析师的知识，以及新的视觉表示和交互技术，分析师可以通过这些技术在替代模型或分析路径之间进行比较。该项目还将开发新颖的软件基础设施，使在现有工具和工作流程中进行和评估多种分析变得可行。此外，该团队将开发更好地传达多元宇宙分析的方法：使多元宇宙分析报告可共享的方法、包含分析代码和图表以及叙述上下文的交互式文档，以及描述合理分析的不同表示如何影响读者理解的实证结果。这些研究活动将以现实世界分析师的形成性研究结果为指导，这些研究将解决分析师在定义和推理他们可能采取的替代模型或分析步骤时面临的困难的现有知识空白。所有研究结果和计算工具都将免费公开提供。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。