EAGER: Interacting with Machine Learning through Visual Comparison of Outcomes

该项目将探索一种新的方法来解决具有挑战性的问题，使人们能够有效地使用机器学习系统。该项目将开发新的工具，帮助用户完成广泛的任务，包括构建和调整模型，评估和诊断模型以建立适当的信任水平，以及使用学习模型来获得对数据的洞察力。面对日益庞大和复杂的系统，为了满足用户的需求，该项目将开发一种与学习系统交互的新方法：提供交互式可视化工具，以便在模型输出集中进行探索和比较。该项目将探索探索学习实验结果的工具，将这些结果与其他模型的测试进行比较，并帮助运行专门的测试。这些新工具将产生广泛的影响，因为它们将使广泛的用户能够更有效地使用机器学习系统。本项目将通过在两个应用程序的背景下开发结果比较方法来探索其可行性。首先，将开发从复杂模型中识别因果见解的工具。这将使我们能够证明该方法适用于最先进的模型类型和复杂的实验设计。第二个应用程序将开发用于诊断和解释文本语料库嵌入的工具。这个应用程序将允许我们在复杂的数据类型和重要的规模上演示这种方法。该项目是探索性的：没有证据表明该方法可以扩展到如此具有挑战性的场景，并且该项目必须探索支持大型和复杂场景的实验策略和视觉设计。该项目的产品，包括出版物、开源软件、演示和教程将在项目网站上提供。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project will explore a new approach to the challenging problems of enabling people to work effectively with machine learning systems. The project will develop new tools that will assist users with a wide range of tasks, including building and tuning models, assessing and diagnosing them to build appropriate levels of trust, and using the learned models to gain insight on the data. In order to meet user needs in the face of increasingly large and complex systems, the project will develop a new approach to interacting with learning systems: providing interactive visualization tools that enable exploration and comparison in sets of model outputs. The project will explore tools for exploring learning experiment results, comparing these results with tests of other models, and helping run specialized tests. These new tools will have broad impact as they will enable a wide range of users to more effectively work with machine learning systems.This project will explore the viability of the outcome comparison approach by developing it in the context of two applications. In one, will develop tools for identifying causal insights from complex models. This will allow us to show that the approach scales to state-of-the-art model types and complex experiment designs. A second application will develop tools for diagnosing and interpreting embeddings from text corpora. This application will allow us to demonstrate the approach on complex data types and significant scales. The project is exploratory: there is no evidence that the approach can scale to such challenging scenarios, and the project must explore experiment strategies and visual designs that support large and complex scenarios. Products of the project, including publications, open source software, demonstrations and tutorials will be available from the project website.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.