RI: Small: Enabling Interpretable AI via Bayesian Deep Learning

可解释性是在医疗、电子商务、交通、地球科学和制造业等各个领域采用和部署基于深度学习的人工智能系统的根本障碍之一。理想的可解释模型应该能够使用人类可理解的概念（例如，“颜色”和“形状”）来解释其预测，符合现实世界中的条件依赖关系（例如，客户的购买是否由于折扣），并处理数据中的不确定性（例如，模型对明天降雨的确定程度）。不幸的是，作为连接主义方法的深度学习本身并不支持这些期望。这个项目的目标是为深度学习模型开发一个通用的解释器框架。该框架下的解释器可以插入深度学习模型，并使用人类可理解概念的图形来解释其预测，而不会牺牲模型的性能。本项目开发的方法将应用于健康监测，以解释模型对患者状态的推理，并应用于推荐系统，以解释模型为用户推荐的项目。该项目将开发两套基于贝叶斯深度学习的方法：(1)“贝叶斯深度解释器”，用描述导致当前预测的条件依赖关系的图形模型来解释深度学习模型。(2)“贝叶斯深度控制器”，通过操纵附着在被控制模型上的图形模型中的特定随机变量来控制深度学习模型的预测。这种新方法的发展将在深度学习和概率图形模型之间建立智力和正式的联系，这两种主要的机器学习范式长期以来被视为不相容。它将通过以下方式推动机器学习和人工智能的发展：(1)制定一个新的贝叶斯深度学习框架来统一深度学习和图形模型，它们的协同作用将显著提高深度学习的可解释性；(2)在这样一个有原则的框架下，设计即插即用的具体方法，因此不会牺牲深度学习模型的性能（例如，准确性）；(3)研究开发的方法提供了什么理论保证，从而为团队和社区的未来工作奠定基础；(4)分析准确性、可解释性和可控性之间的权衡，并为可解释性人工智能系统提供设计指导。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Interpretability is one of the fundamental obstacles on the adoption and deployment of deep-learning-based AI systems across various fields such as healthcare, e-commerce, transportation, earth science, and manufacturing. An ideal interpretable model should be able to interpret its prediction using human-understandable concepts (e.g., “color” and “shape”), conform to conditional dependencies in the real world (e.g., whether a customer's purchase is due to a discount), and handle uncertainty in data (e.g., how certain the model is about the rainfall tomorrow). Unfortunately, deep learning as a connectionist approach does not natively support these desiderata. The goal of this project is to develop a general interpreter framework for deep learning models. Interpreters under this framework can be plugged into a deep learning model and interpret its predictions using a graph of human-understandable concepts, without sacrificing the model’s performance. Methods developed in this project will be applied in health monitoring to interpret models’ reasoning on patient status, and in recommender systems to interpret models’ recommended items for users.This project will develop two sets of methods based on Bayesian deep learning: (1) “Bayesian deep interpreters” that interpret deep learning models with graphical models describing the conditional dependencies leading to current predictions. (2) “Bayesian deep controllers” that control deep learning models' predictions by manipulating specific random variables in the graphical models attached to the controlled models. Development of such novel methods will build intellectual and formal connection between deep learning and probabilistic graphical models, two major machine learning paradigms that have long been seen as incompatible. It will advance the state of the art on machine learning and AI by: (1) formulating a new Bayesian deep learning framework to unify deep learning and graphical models, the synergy of which will significantly improve deep learning interpretability, (2) under such a principled framework, designing concrete methods that are plug-and-play and therefore do not sacrifice the deep learning models' performance (e.g., accuracy), (3) investigating what theoretical guarantees the developed methods provide and therefore laying foundations for future work by the team and the community, (4) analyzing the trade-off between accuracy, interpretability, and controllability and providing design guidance for interpretable AI systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.