CRII: CHS: Improving Data Exploration by Mining Analyst Behavior

数据分析师必须探索越来越大量的信息。各种可视化、筛选和建模数据的工具有助于提高数据的可管理性。然而，它们带来了解释错误、遗漏或认知偏差的机会。该项目开发了使数据分析工具更能抵抗此类错误的技术。该团队将记录和存储许多不同分析师用来探索各种数据集的过程的日志。从这些分析师行为日志中，团队将创建识别潜在优势和风险工作策略的模型。他们将使用这一模型，以及旨在检测偏差的工具，来构建新的数据分析工具。这些新工具将塑造向用户展示数据的方式，并建议探索选择，以帮助克服错误和偏见。例如，如果过去成功的分析师在面对类似的数据集时执行了特定的操作序列，该工具可能会建议当前用户执行类似的程序。这些工具和工作将公开部署，使不太熟悉数据科学的个人受益。该研究计划的最终目标是构建从专家工作中学习的数据分析技术，以改善新手数据科学家的体验。在这个项目中，团队将使用众包平台构建分析师行为的数据集。为了确定许多不同分析员和数据集之间的共性，他们将根据现有的分析工具设计文献开发分析操作的词汇表(或一组抽象)。这些操作将被封装为分析期间采取的操作序列的概率图形模型中的状态。该团队将探索不同的行为抽象如何影响概率模型的拟合。该团队将创建算法，将新的分析师会议与现有模型相匹配，并在数据探索期间确定干预(或指导)点的潜在目标。该项目将把这些干预措施纳入现有的数据分析工具，作为概念验证，在实验室研究中探索个人如何对不同的干预策略做出反应。除了围绕数据分析工作流程开发特定的模型和工具外，该项目还建立了更广泛的交互日志挖掘研究议程，以更深入地了解分析师的认知和工作策略。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Data analysts must explore increasingly large amounts of information. A variety of tools to visualize, filter, and model data help make it more manageable. Yet they bring with them a chance of errors of interpretation, omission, or cognitive bias. This project develops techniques for making data analysis tools more resistant to such errors. The team will record and store logs of the processes many different analysts use to explore a variety of datasets. From these logs of analyst behavior, the team will create models that identify both potentially advantageous and risky working strategies. They will use this model, along with tools designed to detect biases, to build new data analysis tools. The new tools will shape the presentation of data to users and suggest exploration choices to help counter errors and biases. For example, if past successful analysts, when faced with a similar set of data, performed a particular sequence of operations, the tool might suggest that the current user pursue similar procedures. The tools and work will be deployed publicly, benefiting individuals who may be less familiar with data science.The ultimate goal of this research program is to construct data analytics technology that learns from experts' work in order to improve the experience of novice data scientists. In this project, the team will build a dataset of analyst behavior using crowdsourcing platforms. To identify commonalities among many different analysts and datasets, they will develop a vocabulary (or set of abstractions) of analysis actions grounded in existing literature on analytics tool design. These actions will be encapsulated as states in a probabilistic graphical model of the sequence of actions taken during analysis. The team will explore how different behavior abstractions influence the fitting of the probabilistic model. The team will create algorithms to match new analyst sessions to the existing models and identify potential targets for intervention (or coaching) points during the data exploration. The project will incorporate these interventions into existing data analysis tools as a proof of concept, exploring in laboratory studies how individuals respond to different strategies of intervention. In addition to developing specific models and tools around data analytics workflow, this project sets up a broader research agenda of mining interaction logs for deeper insights into analyst cognition and working strategies.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.