EAGER: Interacting with Machine Learning through Visual Comparison of Outcomes

EAGER：通过结果的视觉比较与机器学习交互

• 批准号: 1841349
• 负责人: Michael Gleicher
• 金额: $ 17万
• 依托单位: University of Wisconsin-Madison
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-01-01 至 2022-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1841349&HistoricalAwards=false
• 关键词: EAGER; Interacting; Machine; Learning; through

This project will explore a new approach to the challenging problems of enabling people to work effectively with machine learning systems. The project will develop new tools that will assist users with a wide range of tasks, including building and tuning models, assessing and diagnosing them to build appropriate levels of trust, and using the learned models to gain insight on the data. In order to meet user needs in the face of increasingly large and complex systems, the project will develop a new approach to interacting with learning systems: providing interactive visualization tools that enable exploration and comparison in sets of model outputs. The project will explore tools for exploring learning experiment results, comparing these results with tests of other models, and helping run specialized tests. These new tools will have broad impact as they will enable a wide range of users to more effectively work with machine learning systems.This project will explore the viability of the outcome comparison approach by developing it in the context of two applications. In one, will develop tools for identifying causal insights from complex models. This will allow us to show that the approach scales to state-of-the-art model types and complex experiment designs. A second application will develop tools for diagnosing and interpreting embeddings from text corpora. This application will allow us to demonstrate the approach on complex data types and significant scales. The project is exploratory: there is no evidence that the approach can scale to such challenging scenarios, and the project must explore experiment strategies and visual designs that support large and complex scenarios. Products of the project, including publications, open source software, demonstrations and tutorials will be available from the project website.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目将探索一种新的方法来解决使人们能够有效地使用机器学习系统的挑战性问题。该项目将开发新的工具，帮助用户完成广泛的任务，包括构建和调整模型，评估和诊断模型以建立适当的信任水平，以及使用学习的模型来深入了解数据。为了满足用户面对日益庞大和复杂的系统的需求，该项目将开发一种与学习系统互动的新方法：提供交互式可视化工具，以便对模型输出进行探索和比较。该项目将探索用于探索学习实验结果的工具，将这些结果与其他模型的测试进行比较，并帮助运行专门的测试。这些新工具将产生广泛的影响，因为它们将使广泛的用户能够更有效地使用机器学习系统。该项目将通过在两个应用程序中开发结果比较方法来探索其可行性。在一个，将开发工具，从复杂的模型中识别因果关系的见解。这将使我们能够证明这种方法可以扩展到最先进的模型类型和复杂的实验设计。第二个应用程序将开发用于诊断和解释来自文本语料库的嵌入的工具。这个应用程序将使我们能够在复杂的数据类型和重要的规模上演示该方法。该项目是探索性的：没有证据表明该方法可以扩展到这种具有挑战性的场景，该项目必须探索支持大型和复杂场景的实验策略和视觉设计。该项目的产品，包括出版物，开放源码软件，演示和教程将在项目网站上提供。这个奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的知识价值和更广泛的影响审查标准进行评估的支持。

项目成果

embComp: Visual Interactive Comparison of Vector Embeddings

embComp：向量嵌入的可视化交互比较

• DOI: 10.1109/tvcg.2020.3045918
• 发表时间: 2020
• 期刊: IEEE Transactions on Visualization and Computer Graphics
• 影响因子: 5.2
• 作者: Heimerl, Florian;Kralj, Christoph;Moller, Torsten;Gleicher, Michael
• 通讯作者: Gleicher, Michael

Boxer: Interactive Comparison of Classifier Results

• DOI: 10.1111/cgf.13972
• 发表时间: 2020-04
• 期刊: Computer Graphics Forum
• 影响因子: 2.5
• 作者: Michael Gleicher;Aditya Barve;Xinyi Yu;Florian Heimerl

Trinary Tools for Continuously Valued Binary Classifiers

• DOI: 10.1016/j.visinf.2022.04.002
• 发表时间: 2022-04
• 期刊: ArXiv
• 作者: Michael Gleicher;Xinyi Yu;Yuheng Chen

CAVA: A Visual Analytics System for Exploratory Columnar Data Augmentation Using Knowledge Graphs

CAVA：使用知识图进行探索性柱状数据增强的可视化分析系统

• DOI: 10.1109/tvcg.2020.3030443
• 发表时间: 2021
• 期刊: IEEE Transactions on Visualization and Computer Graphics
• 影响因子: 5.2
• 作者: Cashman, Dylan;Xu, Shenyu;Das, Subhajit;Heimerl, Florian;Liu, Cong;Humayoun, Shah Rukh;Gleicher, Michael;Endert, Alex;Chang, Remco
• 通讯作者: Chang, Remco

QUESTO: Interactive Construction of Objective Functions for Classification Tasks

• DOI: 10.1111/cgf.13970
• 发表时间: 2020-06
• 期刊: Computer Graphics Forum
• 影响因子: 2.5
• 作者: Subhajit Das;Shenyu Xu;Michael Gleicher;Remco Chang;A. Endert