III: Medium: Collaborative Research: Towards Effective Interpretation of Deep Learning: Prediction, Representation, Modeling and Utilization

虽然深度学习实现了前所未有的预测能力，但由于缺乏可解释性，它经常被批评为黑匣子，这在医疗保健和网络安全等现实世界的应用中非常重要。例如，只有当医疗专业人员能够理解患者被诊断为糖尿病前期的原因和方式时，他们才会适当地信任并有效地管理预测结果。该项目旨在通过遵循数据挖掘实践中从表示、建模到预测的基本元素来调查深度学习的可解释性。该项目的成果有望提高深度学习在重要应用中的可用性，积极提升基于深度学习的信息系统的整体价值。将数据科学、工业工程和可视化结合在一起的教育项目是在工业系统中培训学生使用数据分析技术，以吸引和指导寻求STEM职业生涯的未被充分代表的群体的成员。本项目的研究目标是从机器学习生命周期(即表示、建模和预测)系统地探索深度学习的可解释性，以及在各种任务中部署可解释性。具体地说，本项目旨在通过从以下几个方面开发一系列解释算法和方法来实现研究目标。它探索后自组织的解释方法，以阐明深度学习模型如何产生特定的预测和表示。它还研究了从头开始设计可解释模型，旨在构建自解释模型，并将可解释性直接纳入深度学习模型的结构中。基于深度学习模型的上述解释被用来提高模型的性能。此外，应用可解释性来调试模型行为，以确保模型决策过程与人类专家知识一致，并在处理对抗性攻击时提高模型的健壮性。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

While deep learning has achieved unprecedented prediction capabilities, it is often criticized as a black box because of lacking interpretability, which is very important in real-world applications such as healthcare and cybersecurity. For example, healthcare professionals would appropriately trust and effectively manage prediction results only if they can understand why and how a patient is diagnosed with prediabetes. The project is to investigate the interpretability of deep learning by following the fundamental elements in a data mining practice from representation, modeling to prediction. The results of the project are expected to improve the usability of deep learning in important applications, positively boosting the overall value of the deep learning based information systems. The education program that integrates data science, industrial engineering, and visualization is to train students with data analytics technologies in industrial systems, to attract and mentor members of underrepresented groups pursuing careers in STEM.The research goal of this project is to systematically explore interpretability of deep learning from a machine learning life cycle, i.e., representation, modeling and prediction, as well as the deployment of interpretability in various tasks. Specifically, this project aims to achieve the research goal by developing a series of interpretation algorithms and methods from the following aspects. It explores post-hoc interpretation methods to shed light on how deep learning models produce a specific prediction and generate a representation. It also investigates designing interpretable models from scratch, which aims to construct self-explanatory models and incorporate interpretability directly into the structure of a deep learning model. The aforementioned interpretation derived from a deep learning model is employed to promote the model performance. In addition, the applications of interpretability are utilized to debug model behaviors so as to ensure the model decision making process is consistent with human expert knowledge, as well as to promote model robustness when handling adversarial attacks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.