III: Medium: Collaborative Research: Towards Effective Interpretation of Deep Learning: Prediction, Representation, Modeling and Utilization

III：媒介：协作研究：走向深度学习的有效解释：预测、表示、建模和利用

• 批准号: 1900767
• 负责人: Eric Ragan
• 金额: $ 20万
• 依托单位: University of Florida
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-08-15 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1900767&HistoricalAwards=false
• 关键词: III; Medium; Collaborative; Research; Towards

While deep learning has achieved unprecedented prediction capabilities, it is often criticized as a black box because of lacking interpretability, which is very important in real-world applications such as healthcare and cybersecurity. For example, healthcare professionals would appropriately trust and effectively manage prediction results only if they can understand why and how a patient is diagnosed with prediabetes. The project is to investigate the interpretability of deep learning by following the fundamental elements in a data mining practice from representation, modeling to prediction. The results of the project are expected to improve the usability of deep learning in important applications, positively boosting the overall value of the deep learning based information systems. The education program that integrates data science, industrial engineering, and visualization is to train students with data analytics technologies in industrial systems, to attract and mentor members of underrepresented groups pursuing careers in STEM.The research goal of this project is to systematically explore interpretability of deep learning from a machine learning life cycle, i.e., representation, modeling and prediction, as well as the deployment of interpretability in various tasks. Specifically, this project aims to achieve the research goal by developing a series of interpretation algorithms and methods from the following aspects. It explores post-hoc interpretation methods to shed light on how deep learning models produce a specific prediction and generate a representation. It also investigates designing interpretable models from scratch, which aims to construct self-explanatory models and incorporate interpretability directly into the structure of a deep learning model. The aforementioned interpretation derived from a deep learning model is employed to promote the model performance. In addition, the applications of interpretability are utilized to debug model behaviors so as to ensure the model decision making process is consistent with human expert knowledge, as well as to promote model robustness when handling adversarial attacks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

虽然深度学习已经实现了前所未有的预测能力，但由于缺乏可解释性，它经常被批评为黑匣子，这在医疗保健和网络安全等现实世界的应用中非常重要。例如，只有当医疗保健专业人员能够理解患者被诊断为前驱糖尿病的原因和方式时，他们才会适当地信任和有效地管理预测结果。该项目旨在通过遵循数据挖掘实践中的基本要素（从表示，建模到预测）来研究深度学习的可解释性。该项目的成果有望提高深度学习在重要应用中的可用性，积极提升基于深度学习的信息系统的整体价值。该教育项目整合了数据科学、工业工程和可视化，旨在培养学生掌握工业系统中的数据分析技术，吸引和指导那些在STEM领域寻求职业生涯的代表性不足的群体成员。该项目的研究目标是从机器学习生命周期系统地探索深度学习的可解释性，即，表示、建模和预测，以及在各种任务中部署可解释性。具体而言，本项目旨在通过从以下几个方面开发一系列解释算法和方法来实现研究目标。它探讨了事后解释方法，以阐明深度学习模型如何产生特定的预测并生成表示。它还研究了从头开始设计可解释模型，旨在构建自解释模型并将可解释性直接纳入深度学习模型的结构中。采用从深度学习模型导出的上述解释来提高模型性能。此外，可解释性的应用被用于调试模型行为，以确保模型决策过程与人类专家知识一致，并在处理对抗性攻击时提高模型的鲁棒性。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Soliciting Human-in-the-Loop User Feedback for Interactive Machine Learning Reduces User Trust and Impressions of Model Accuracy

• DOI: 10.1609/hcomp.v8i1.7464
• 发表时间: 2020-08
• 期刊: ArXiv
• 作者: Donald R. Honeycutt;Mahsan Nourani;E. Ragan

On the Importance of User Backgrounds and Impressions: Lessons Learned from Interactive AI Applications

• DOI: 10.1145/3531066
• 发表时间: 2022-04
• 期刊: ACM Transactions on Interactive Intelligent Systems
• 影响因子: 3.4
• 作者: Mahsan Nourani;Chiradeep Roy;Jeremy E. Block;Donald R. Honeycutt;Tahrima Rahman;E. Ragan;Vibhav Gogate

DETOXER: A Visual Debugging Tool With Multiscope Explanations for Temporal Multilabel Classification

DETOXER：一种可视化调试工具，具有时态多标签分类的多范围解释

• DOI: 10.1109/mcg.2022.3201465
• 发表时间: 2022
• 期刊: IEEE Computer Graphics and Applications
• 影响因子: 1.8
• 作者: Nourani, Mahsan;Roy, Chiradeep;Honeycutt, Donald R.;Ragan, Eric D.;Gogate, Vibhav
• 通讯作者: Gogate, Vibhav

Micro-entries: Encouraging Deeper Evaluation of Mental Models Over Time for Interactive Data Systems

• DOI: 10.1109/beliv51497.2020.00012
• 发表时间: 2020-09
• 期刊: 2020 IEEE Workshop on Evaluation and Beyond - Methodological Approaches to Visualization (BELIV)
• 作者: Jeremy E. Block;E. Ragan

Anchoring Bias Affects Mental Model Formation and User Reliance in Explainable AI Systems

• DOI: 10.1145/3397481.3450639
• 发表时间: 2021-04
• 期刊: Proceedings of the 26th International Conference on Intelligent User Interfaces
• 作者: Mahsan Nourani;Chiradeep Roy;Jeremy E. Block;Donald R. Honeycutt;Tahrima Rahman;E. Ragan;Vibhav Gogate