EAGER: Interpreting Black-Box Predictive Models Through Causal Attribution

EAGER：通过因果归因解释黑盒预测模型

• 批准号: 2041759
• 负责人: Vasant Honavar
• 金额: $ 20万
• 依托单位: Pennsylvania State Univ University Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-08-15 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2041759&HistoricalAwards=false
• 关键词: EAGER; Interpreting; Black; Box; Predictive

Our ability to acquire and annotate increasingly large amounts of data together with rapid advances in machine learning have made predictive models trained using machine learning ubiquitous in virtually all areas of human endeavor. In high-stakes applications such as healthcare, finance, criminal justice, scientific discovery, education, and others, the resulting predictive models are complex, and in many cases, black-boxes. Consider for example, a medical decision making scenario where a predictive model, e.g., a deep neural network, trained on a large database of labeled data, is to assist physicians in diagnosing patients. In this setting, it is important that the clinical decision support system be able to explain the output of the deep neural network to the physician, who may not have a deep understanding of machine learning. For example, the physician might want to understand the subset of patient characteristics that contribute to the diagnosis; or the reason as to why diagnoses were different for two different patients, etc. In high stakes applications of machine learning, the ability to explain the machine learned model is a prerequisite for establishing trust in the model’s predictions. Satisfactory explanations have to provide answers to questions such as: "What features of the input are responsible for the predictions?"; "Why are the model’s outputs different for two individuals?" (e.g., Why did John’s loan application get approved when Sarah’s was not?). Hence, satisfactory explanations have to be fundamentally causal in nature. This project will develop a theoretically sound, yet practical approach to causal attribution, that is, apportioning the responsibility for a black-box predictive model’s outputs among the model’s inputs.The model interpretation question "Why did the predictive model generate the output Y for input X?" will be reduced to the following equivalent question: "How are the features of the model input X causally related to the model output Y?" In other words, the task of interpreting a black-box predictive model is reduced to the task of estimating, from observations of the inputs and the corresponding outputs of the model, the causal effect of each input variable or feature on the output variable. The planned methods do not require knowledge of the internal structure or parameters of the black-box model, or of the objective function or the algorithm used to train the model. Hence, the resulting methods can be applied, in principle, to any black-box predictive model, so long as it is possible probe the model and observe the model’s response to any supplied input data sample. Advances in causal attribution methods will help broaden the application of machine learned black-box predictive models in high-stakes applications across many areas of human endeavor. The project offers enhanced opportunities for research-based training of graduate and undergraduate students in Informatics, Data Sciences, and Artificial Intelligence. The investigator will develop a new course on Foundations and Applications of Causal Inference as well as modules on Causal Attribution that for possible inclusion in undergraduate and graduate courses in Machine Learning. The broad and free dissemination of open source library of causal attribution methods, course materials, data, research results will ease their adoption and use by AI researchers, educators, and practitioners.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

我们获取和注释越来越多的大量数据的能力，加上机器学习的快速进步，使得使用机器学习训练的预测模型几乎在人类工作的所有领域无处不在。在医疗保健、金融、刑事司法、科学发现、教育等高风险应用中，由此产生的预测模型是复杂的，在许多情况下是黑盒。例如，考虑医疗决策场景，其中在标签数据的大型数据库上训练的预测模型(例如，深度神经网络)将帮助医生诊断患者。在这种情况下，重要的是临床决策支持系统能够向医生解释深度神经网络的输出，因为医生可能对机器学习没有深入的理解。例如，医生可能想要了解有助于诊断的患者特征子集；或者两个不同患者的诊断不同的原因等。在机器学习的高风险应用中，解释机器学习模型的能力是建立对模型预测的信任的先决条件。令人满意的解释必须提供对以下问题的答案：“输入的哪些特征对预测负责？”；“为什么模型的输出对两个人来说是不同的？”(例如，为什么John的贷款申请获得批准，而Sarah的申请没有获得批准？)因此，令人满意的解释在本质上必须是因果关系。这个项目将开发一种理论上合理但实用的因果归因方法，即在模型的输入中分配黑盒预测模型输出的责任。模型解释问题是：为什么预测模型产生输入X的输出Y？将简化为以下等价问题：“模型输入X的特征与模型输出Y有何因果关系？”换句话说，解释黑盒预测模型的任务被简化为根据对模型的输入和相应输出的观察来估计每个输入变量或特征对输出变量的因果影响的任务。计划中的方法不需要了解黑盒模型的内部结构或参数，也不需要了解用于训练模型的目标函数或算法。因此，所得到的方法原则上可以应用于任何黑盒预测模型，只要有可能探测该模型并观察该模型对所提供的任何输入数据样本的响应。因果归因方法的进步将有助于扩大机器学习的黑盒预测模型在人类努力的许多领域的高风险应用中的应用。该项目为信息学、数据科学和人工智能方面的研究生和本科生提供了更多的研究性培训机会。研究人员将开发一门关于因果推理的基础和应用的新课程，以及关于因果归因的模块，以便可能包括在机器学习的本科生和研究生课程中。开放源码库的广泛和免费传播因果归因方法、课程材料、数据和研究结果将使人工智能研究人员、教育工作者和实践者更容易采用和使用它们。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Explainable Multivariate Time Series Classification: A Deep Neural Network Which Learns to Attend to Important Variables As Well As Time Intervals

• DOI: 10.1145/3437963.3441815
• 发表时间: 2021-03
• 期刊: Proceedings of the 14th ACM International Conference on Web Search and Data Mining
• 作者: Tsung-Yu Hsieh;Suhang Wang;Yiwei Sun

Variational Graph Auto-Encoders for Heterogeneous Information Network

异构信息网络的变分图自动编码器

• 发表时间: 2022
• 期刊: NeurIPS 2022 Workshop: New Frontiers in Graph Learning
• 作者: Abhishek Dalvi, Ayan Acharya

SrVARM: State Regularized Vector Autoregressive Model for Joint Learning of Hidden State Transitions and State-Dependent Inter-Variable Dependencies from Multi-variate Time Series

SrVARM：状态正则化向量自回归模型，用于联合学习多变量时间序列中的隐藏状态转换和状态相关变量间依赖性

• DOI: 10.1145/3442381.3450116
• 发表时间: 2021
• 期刊: WWW '21: Proceedings of the Web Conference 2021
• 作者: Hsieh, Tsung-Yu;Sun, Yiwei;Tang, Xianfeng;Wang, Suhang;Honavar, Vasant G.
• 通讯作者: Honavar, Vasant G.

Functional Autoencoders for Functional Data Representation Learning

• DOI: 10.1137/1.9781611976700.75
• 发表时间: 2021-01
• 作者: Tsung-Yu Hsieh;Yiwei Sun;Suhang Wang;Vasant G Honavar

A Causal Lens for Peeking into Black Box Predictive Models: Predictive Model Interpretation via Causal Attribution

用于窥探黑盒预测模型的因果镜头：通过因果归因进行预测模型解释

• 发表时间: 2020
• 期刊: ArXivorg
• 作者: Khademi, Aria;Honavar, Vasant
• 通讯作者: Honavar, Vasant