CAREER: Unified Model-agnostic Interpretation Framework for Deep Predictive Models

职业：深度预测模型的与模型无关的统一解释框架

• 批准号: 2238700
• 负责人: Fang Jin
• 金额: $ 55万
• 依托单位: George Washington University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-07-15 至 2028-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2238700&HistoricalAwards=false
• 关键词: CAREER; Unified; Model; agnostic; Interpretation

Deep learning models have achieved exceptional predictive performance in a wide variety of tasks, ranging from computer vision to language processing to medical images. Many organizations across diverse domains are now building large-scale applications based on deep learning. However, there are growing concerns, regarding the fairness and trustworthiness of these models, largely due to the opaque nature of their decision processes. For example, when the trained deep learning model correctly classifies a tumor in a target medical image, the part of the X-ray image that the model learned to identify the tumor in must be understood to ensure the findings are valid. Providing accurate and reasonable interpretations is therefore urgently needed for selecting and deploying trustworthy deep learning models. This project will design and develop a universal interpretation framework that can be applied to a variety of fields for deep learning applications. The interpretation framework can produce feedback on what scientific knowledge is perceived by the Deep Neural Networks (DNNs) and hence helps researchers refine models by identifying, minimizing, or even eliminating unfairness and bias. This project will also spend significant efforts on education activities, focusing on three key areas: (1) professional development for K-12 teachers, (2) deep learning summer camp for high school students, and (3) mentoring undergraduates for research. These educational and outreach activities will build bridges among high school students, K-12 teachers, and colleges that will eventually benefit both science and society. This project will develop a novel interpreting framework that enables humans to understand the decision process of increasing complex black-box DNNs trained on medical images, videos, natural language processing and deep reinforcement learning. Although progress has been achieved on DNN interpretation, several unique challenges remain unexplored for the aforementioned domains: (1) 3D medical images, which are highly structured and usually require domain knowledge and are difficult to explain. (2) Video interpretation cannot be achieved by simply applying existing image interpretation methods. (3) Most existing NLP interpretation models require certain knowledge of the internal structure of the neural networks. (4) Current DRL interpretation heavily relies on decision trees imitating action samples, which cannot guarantee to minimize policy regret. This project will address these challenges in the following ways: (1) Interprets 3D medical images by a novel graphical representation to create correlated interpretations among neighboring slices of 3D images; 2) Devise saliency estimating procedures for video-based tasks in both spatial and temporal domain; (3) Designs a novel text perturbation scheme via embedding space to identify important words of NLP models; (4) Interprets agent's behaviors and elucidates the strategies that agents learn to balance short-term and long-term reward; (5) Develops an all-encompassing interpretation framework to provide interpretations for arbitrary deep learning models through a series of pilot applications. The specific research tasks will be extensively evaluated in trustworthiness, performance comparison, and interpretability to human beings. All the research outcomes will be disseminated publicly to facilitate a better understanding of explainable deep neural networks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度学习模型在各种任务中取得了卓越的预测性能，从计算机视觉到语言处理再到医学图像。许多不同领域的组织现在正在构建基于深度学习的大规模应用程序。然而，人们越来越担心这些模型的公平性和可信度，这主要是由于其决策过程的不透明性。例如，当经过训练的深度学习模型正确地对目标医学图像中的肿瘤进行分类时，必须理解模型学习以识别肿瘤的X射线图像的部分，以确保发现有效。因此，迫切需要提供准确和合理的解释，以选择和部署值得信赖的深度学习模型。该项目将设计和开发一个通用的解释框架，可应用于深度学习应用的各个领域。解释框架可以对深度神经网络（DNN）感知的科学知识产生反馈，从而帮助研究人员通过识别，最小化甚至消除不公平和偏见来改进模型。该项目还将在教育活动方面投入大量精力，重点关注三个关键领域：（1）K-12教师的专业发展，（2）高中生深度学习夏令营，以及（3）指导本科生进行研究。这些教育和推广活动将在高中生、K-12教师和大学之间建立桥梁，最终使科学和社会受益。该项目将开发一种新的解释框架，使人类能够理解在医学图像、视频、自然语言处理和深度强化学习上训练的越来越复杂的黑盒DNN的决策过程。虽然DNN解释已经取得了进展，但上述领域仍存在一些独特的挑战：（1）3D医学图像，其高度结构化，通常需要领域知识，难以解释。(2)视频解译不能简单地应用现有的图像解译方法。(3)大多数现有的NLP解释模型需要一定的知识的神经网络的内部结构。(4)目前的DRL解释严重依赖于模仿动作样本的决策树，这不能保证最小化政策遗憾。本项目将从以下几个方面来解决这些问题：（1）通过一种新的图形表示来解释3D医学图像，以在3D图像的相邻切片之间创建相关的解释;（2）在空间和时间域上设计基于视频的任务的显著性估计过程;（3）设计一种新的文本扰动方案，通过嵌入空间来识别NLP模型中的重要单词;（4）解释Agent的行为，阐明Agent学习平衡短期和长期回报的策略;（5）开发一个全面的解释框架，通过一系列的试点应用为任意深度学习模型提供解释。具体的研究任务将在可信度，性能比较和对人类的可解释性方面进行广泛的评估。所有的研究成果都将被公开传播，以促进对可解释的深度神经网络的更好理解。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。