CRII: CHS: Improving Data Exploration by Mining Analyst Behavior

数据分析师必须探索越来越多的大量信息。各种可视化、过滤和建模数据的工具有助于使数据更易于管理。然而，它们带来了解释错误、遗漏或认知偏见的可能性。该项目开发了使数据分析工具更能抵抗此类错误的技术。该团队将记录和存储许多不同分析师用于探索各种数据集的过程日志。从分析人员行为的这些日志中，团队将创建识别潜在的有利的和有风险的工作策略的模型。他们将使用这个模型，以及用于检测偏见的工具，来构建新的数据分析工具。新工具将塑造数据呈现给用户，并建议探索选择，以帮助消除错误和偏见。例如，如果过去成功的分析师在面对类似的数据集时执行了特定的操作序列，那么该工具可能会建议当前用户执行类似的过程。这些工具和工作将被公开部署，使那些可能不太熟悉数据科学的个人受益。本研究计划的最终目标是构建从专家工作中学习的数据分析技术，以提高新手数据科学家的经验。在这个项目中，团队将使用众包平台建立一个分析师行为的数据集。为了识别许多不同的分析师和数据集之间的共性，他们将开发一个基于现有分析工具设计文献的分析动作词汇表（或一组抽象）。这些动作将被封装为分析过程中所采取的动作序列的概率图形模型中的状态。该团队将探索不同的行为抽象如何影响概率模型的拟合。该团队将创建算法，将新的分析会话与现有模型相匹配，并在数据探索期间确定干预（或指导）点的潜在目标。该项目将把这些干预措施纳入现有的数据分析工具，作为概念的证明，在实验室研究中探索个体如何对不同的干预策略做出反应。除了围绕数据分析工作流程开发特定的模型和工具外，该项目还建立了一个更广泛的研究议程，挖掘交互日志，以更深入地了解分析师的认知和工作策略。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Data analysts must explore increasingly large amounts of information. A variety of tools to visualize, filter, and model data help make it more manageable. Yet they bring with them a chance of errors of interpretation, omission, or cognitive bias. This project develops techniques for making data analysis tools more resistant to such errors. The team will record and store logs of the processes many different analysts use to explore a variety of datasets. From these logs of analyst behavior, the team will create models that identify both potentially advantageous and risky working strategies. They will use this model, along with tools designed to detect biases, to build new data analysis tools. The new tools will shape the presentation of data to users and suggest exploration choices to help counter errors and biases. For example, if past successful analysts, when faced with a similar set of data, performed a particular sequence of operations, the tool might suggest that the current user pursue similar procedures. The tools and work will be deployed publicly, benefiting individuals who may be less familiar with data science.The ultimate goal of this research program is to construct data analytics technology that learns from experts' work in order to improve the experience of novice data scientists. In this project, the team will build a dataset of analyst behavior using crowdsourcing platforms. To identify commonalities among many different analysts and datasets, they will develop a vocabulary (or set of abstractions) of analysis actions grounded in existing literature on analytics tool design. These actions will be encapsulated as states in a probabilistic graphical model of the sequence of actions taken during analysis. The team will explore how different behavior abstractions influence the fitting of the probabilistic model. The team will create algorithms to match new analyst sessions to the existing models and identify potential targets for intervention (or coaching) points during the data exploration. The project will incorporate these interventions into existing data analysis tools as a proof of concept, exploring in laboratory studies how individuals respond to different strategies of intervention. In addition to developing specific models and tools around data analytics workflow, this project sets up a broader research agenda of mining interaction logs for deeper insights into analyst cognition and working strategies.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.