EAGER: Interpreting Black-Box Predictive Models Through Causal Attribution

我们获取和注释越来越大量数据的能力以及机器学习的快速发展，使得使用机器学习训练的预测模型在人类奋进的几乎所有领域都无处不在。在医疗、金融、刑事司法、科学发现、教育等高风险应用中，产生的预测模型非常复杂，在许多情况下是黑匣子。例如，考虑医疗决策场景，其中预测模型，例如，在标记数据的大型数据库上训练的深度神经网络将帮助医生诊断患者。在这种情况下，重要的是临床决策支持系统能够向医生解释深度神经网络的输出，而医生可能对机器学习没有深入的了解。例如，医生可能想要了解有助于诊断的患者特征的子集;或者关于为什么两个不同患者的诊断不同的原因等。在机器学习的高风险应用中，解释机器学习模型的能力是建立对模型预测的信任的先决条件。令人满意的解释必须回答这样的问题：“输入的什么特征对预测负责？“;“为什么两个人的模型输出不同？“（例如，为什么约翰的贷款申请得到批准，而莎拉的却没有？）。因此，令人满意的解释在本质上必须是基本的因果关系。这个项目将开发一个理论上合理，但实际的方法来因果归因，即分配的责任，一个黑盒预测模型的输出之间的模型的输入。模型解释问题“为什么预测模型产生的输出Y的输入X？“将被简化为以下等价的问题：“模型输入X的特征如何与模型输出Y有因果关系？换句话说，解释黑盒预测模型的任务被简化为根据对模型的输入和相应输出的观察来估计每个输入变量或特征对输出变量的因果影响的任务。计划的方法不需要知道黑盒模型的内部结构或参数，或用于训练模型的目标函数或算法。因此，原则上，所得到的方法可以应用于任何黑盒预测模型，只要有可能探测模型并观察模型对任何提供的输入数据样本的响应。因果归因方法的进步将有助于扩大机器学习黑盒预测模型在人类奋进的许多领域的高风险应用中的应用。该项目为信息学，数据科学和人工智能领域的研究生和本科生提供了更好的研究培训机会。研究人员将开发一门关于因果推理的基础和应用的新课程，以及关于因果归因的模块，这些模块可能包含在机器学习的本科和研究生课程中。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Our ability to acquire and annotate increasingly large amounts of data together with rapid advances in machine learning have made predictive models trained using machine learning ubiquitous in virtually all areas of human endeavor. In high-stakes applications such as healthcare, finance, criminal justice, scientific discovery, education, and others, the resulting predictive models are complex, and in many cases, black-boxes. Consider for example, a medical decision making scenario where a predictive model, e.g., a deep neural network, trained on a large database of labeled data, is to assist physicians in diagnosing patients. In this setting, it is important that the clinical decision support system be able to explain the output of the deep neural network to the physician, who may not have a deep understanding of machine learning. For example, the physician might want to understand the subset of patient characteristics that contribute to the diagnosis; or the reason as to why diagnoses were different for two different patients, etc. In high stakes applications of machine learning, the ability to explain the machine learned model is a prerequisite for establishing trust in the model’s predictions. Satisfactory explanations have to provide answers to questions such as: "What features of the input are responsible for the predictions?"; "Why are the model’s outputs different for two individuals?" (e.g., Why did John’s loan application get approved when Sarah’s was not?). Hence, satisfactory explanations have to be fundamentally causal in nature. This project will develop a theoretically sound, yet practical approach to causal attribution, that is, apportioning the responsibility for a black-box predictive model’s outputs among the model’s inputs.The model interpretation question "Why did the predictive model generate the output Y for input X?" will be reduced to the following equivalent question: "How are the features of the model input X causally related to the model output Y?" In other words, the task of interpreting a black-box predictive model is reduced to the task of estimating, from observations of the inputs and the corresponding outputs of the model, the causal effect of each input variable or feature on the output variable. The planned methods do not require knowledge of the internal structure or parameters of the black-box model, or of the objective function or the algorithm used to train the model. Hence, the resulting methods can be applied, in principle, to any black-box predictive model, so long as it is possible probe the model and observe the model’s response to any supplied input data sample. Advances in causal attribution methods will help broaden the application of machine learned black-box predictive models in high-stakes applications across many areas of human endeavor. The project offers enhanced opportunities for research-based training of graduate and undergraduate students in Informatics, Data Sciences, and Artificial Intelligence. The investigator will develop a new course on Foundations and Applications of Causal Inference as well as modules on Causal Attribution that for possible inclusion in undergraduate and graduate courses in Machine Learning. The broad and free dissemination of open source library of causal attribution methods, course materials, data, research results will ease their adoption and use by AI researchers, educators, and practitioners.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.