III: Medium: Collaborative Research: Towards Effective Interpretation of Deep Learning: Prediction, Representation, Modeling and Utilization

虽然深度学习已经取得了前所未有的预测能力，但由于缺乏可解释性，它经常被批评为黑箱，这在医疗保健和网络安全等现实应用中非常重要。例如，只有当医疗保健专业人员能够理解患者被诊断为前驱糖尿病的原因和方式时，他们才会适当地信任并有效地管理预测结果。该项目是通过遵循数据挖掘实践中的基本元素，从表示、建模到预测，来研究深度学习的可解释性。该项目的成果有望提高深度学习在重要应用中的可用性，积极提升基于深度学习的信息系统的整体价值。该教育项目将数据科学、工业工程和可视化相结合，旨在培养学生掌握工业系统中的数据分析技术，吸引和指导那些在STEM领域从事职业的代表性不足的群体成员。该项目的研究目标是从机器学习生命周期，即表示、建模和预测，系统地探索深度学习的可解释性，以及可解释性在各种任务中的部署。具体而言，本项目旨在从以下几个方面开发一系列解释算法和方法来实现研究目标。它探讨了事后解释方法，以阐明深度学习模型如何产生特定的预测并生成表示。它还研究了从头开始设计可解释的模型，旨在构建自解释模型，并将可解释性直接纳入深度学习模型的结构中。采用上述深度学习模型的解释来提高模型的性能。此外，利用可解释性的应用对模型行为进行调试，以确保模型决策过程与人类专家知识一致，并在处理对抗性攻击时提高模型的鲁棒性。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

While deep learning has achieved unprecedented prediction capabilities, it is often criticized as a black box because of lacking interpretability, which is very important in real-world applications such as healthcare and cybersecurity. For example, healthcare professionals would appropriately trust and effectively manage prediction results only if they can understand why and how a patient is diagnosed with prediabetes. The project is to investigate the interpretability of deep learning by following the fundamental elements in a data mining practice from representation, modeling to prediction. The results of the project are expected to improve the usability of deep learning in important applications, positively boosting the overall value of the deep learning based information systems. The education program that integrates data science, industrial engineering, and visualization is to train students with data analytics technologies in industrial systems, to attract and mentor members of underrepresented groups pursuing careers in STEM.The research goal of this project is to systematically explore interpretability of deep learning from a machine learning life cycle, i.e., representation, modeling and prediction, as well as the deployment of interpretability in various tasks. Specifically, this project aims to achieve the research goal by developing a series of interpretation algorithms and methods from the following aspects. It explores post-hoc interpretation methods to shed light on how deep learning models produce a specific prediction and generate a representation. It also investigates designing interpretable models from scratch, which aims to construct self-explanatory models and incorporate interpretability directly into the structure of a deep learning model. The aforementioned interpretation derived from a deep learning model is employed to promote the model performance. In addition, the applications of interpretability are utilized to debug model behaviors so as to ensure the model decision making process is consistent with human expert knowledge, as well as to promote model robustness when handling adversarial attacks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.