CRII: CHS: Improving Data Exploration by Mining Analyst Behavior

CRII：CHS：通过挖掘分析师行为改进数据探索

• 批准号: 1850195
• 负责人: Jeffrey Rzeszotarski
• 金额: $ 17.5万
• 依托单位: Cornell University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-04-01 至 2023-03-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1850195&HistoricalAwards=false
• 关键词: CRII; CHS; Improving; Data; Exploration

Data analysts must explore increasingly large amounts of information. A variety of tools to visualize, filter, and model data help make it more manageable. Yet they bring with them a chance of errors of interpretation, omission, or cognitive bias. This project develops techniques for making data analysis tools more resistant to such errors. The team will record and store logs of the processes many different analysts use to explore a variety of datasets. From these logs of analyst behavior, the team will create models that identify both potentially advantageous and risky working strategies. They will use this model, along with tools designed to detect biases, to build new data analysis tools. The new tools will shape the presentation of data to users and suggest exploration choices to help counter errors and biases. For example, if past successful analysts, when faced with a similar set of data, performed a particular sequence of operations, the tool might suggest that the current user pursue similar procedures. The tools and work will be deployed publicly, benefiting individuals who may be less familiar with data science.The ultimate goal of this research program is to construct data analytics technology that learns from experts' work in order to improve the experience of novice data scientists. In this project, the team will build a dataset of analyst behavior using crowdsourcing platforms. To identify commonalities among many different analysts and datasets, they will develop a vocabulary (or set of abstractions) of analysis actions grounded in existing literature on analytics tool design. These actions will be encapsulated as states in a probabilistic graphical model of the sequence of actions taken during analysis. The team will explore how different behavior abstractions influence the fitting of the probabilistic model. The team will create algorithms to match new analyst sessions to the existing models and identify potential targets for intervention (or coaching) points during the data exploration. The project will incorporate these interventions into existing data analysis tools as a proof of concept, exploring in laboratory studies how individuals respond to different strategies of intervention. In addition to developing specific models and tools around data analytics workflow, this project sets up a broader research agenda of mining interaction logs for deeper insights into analyst cognition and working strategies.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

数据分析师必须探索越来越大量的信息。各种可视化、过滤和建模数据的工具有助于提高数据的可管理性。然而，它们也带来了解释错误、遗漏或认知偏见的机会。该项目开发了使数据分析工具更能抵抗此类错误的技术。 该团队将记录和存储许多不同分析师用于探索各种数据集的过程的日志。从这些分析师行为的日志中，团队将创建模型，以识别潜在的有利和危险的工作策略。他们将使用这个模型，沿着用于检测偏差的工具，来构建新的数据分析工具。新工具将塑造向用户呈现的数据，并建议探索选择，以帮助消除错误和偏见。 例如，如果过去成功的分析师在面对类似的数据集时执行了特定的操作序列，该工具可能会建议当前用户执行类似的程序。 这些工具和工作将被公开部署，使可能不太熟悉数据科学的个人受益。该研究计划的最终目标是构建从专家工作中学习的数据分析技术，以改善新手数据科学家的经验。 在这个项目中，该团队将使用众包平台构建分析师行为数据集。 为了识别许多不同分析师和数据集之间的共性，他们将开发一个基于现有分析工具设计文献的分析操作词汇表（或一组抽象）。这些操作将被封装为分析期间所采取的操作序列的概率图形模型中的状态。 该团队将探索不同的行为抽象如何影响概率模型的拟合。该团队将创建算法，以将新的分析师会话与现有模型相匹配，并在数据探索期间确定干预（或指导）点的潜在目标。该项目将把这些干预措施纳入现有的数据分析工具，作为概念证明，在实验室研究中探索个人如何对不同的干预策略作出反应。除了围绕数据分析工作流程开发特定的模型和工具外，该项目还建立了更广泛的研究议程，即挖掘交互日志，以更深入地了解分析师的认知和工作策略。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Tessera: Discretizing Data Analysis Workflows on a Task Level

• DOI: 10.1145/3411764.3445728
• 发表时间: 2021-05
• 期刊: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
• 作者: Jing Nathan Yan;Ziwei Gu;Jeffrey M. Rzeszotarski

Understanding User Sensemaking in Machine Learning Fairness Assessment Systems

了解机器学习公平性评估系统中的用户意义建构

• DOI: 10.1145/3442381.3450092
• 发表时间: 2021
• 期刊: WWW '21: Proceedings of the Web Conference 2021
• 作者: Gu, Ziwei;Yan, Jing Nathan;Rzeszotarski, Jeffrey M.
• 通讯作者: Rzeszotarski, Jeffrey M.

Silva: Interactively Assessing Machine Learning Fairness Using Causality

Silva：利用因果关系交互式评估机器学习的公平性

• DOI: 10.1145/3313831.3376447
• 发表时间: 2020
• 期刊: CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
• 作者: Yan, Jing Nathan;Gu, Ziwei;Lin, Hubert;Rzeszotarski, Jeffrey M.
• 通讯作者: Rzeszotarski, Jeffrey M.