RI: Small: Enabling Interpretable AI via Bayesian Deep Learning

可解释性是在医疗保健、电子商务、交通运输、地球科学和制造业等各个领域采用和部署基于深度学习的人工智能系统的根本障碍之一。理想的可解释模型应该能够使用人类可理解的概念（例如，"颜色"和"形状"），符合真实的世界中的条件依赖性（例如，客户的购买是否是由于折扣），并处理数据中的不确定性（例如，模型对明天的降雨量有多确定）。不幸的是，深度学习作为一种连接主义方法并不天生支持这些需求。该项目的目标是为深度学习模型开发一个通用的解释器框架。该框架下的解释器可以插入深度学习模型，并使用人类可理解的概念图解释其预测，而不会牺牲模型的性能。本项目开发的方法将应用于健康监测，解释模型对患者状态的推理，并应用于推荐系统，解释模型向用户推荐的项目。本项目将开发两套基于贝叶斯深度学习的方法：（1）"贝叶斯深度解释器"，通过描述导致当前预测的条件依赖关系的图形模型来解释深度学习模型。(2)“贝叶斯深度控制器”，通过操纵附加到受控模型的图形模型中的特定随机变量来控制深度学习模型的预测。这种新方法的开发将在深度学习和概率图模型之间建立智能和形式化的联系，这两种主要的机器学习范式长期以来一直被视为不兼容。它将通过以下方式推进机器学习和人工智能的最新技术水平：（1）制定新的贝叶斯深度学习框架，以统一深度学习和图形模型，其协同作用将显着提高深度学习的可解释性，（2）在这样的原则框架下，设计即插即用的具体方法，因此不会牺牲深度学习模型的性能（例如，准确性），（3）调查所开发的方法提供了哪些理论保证，从而为团队和社区的未来工作奠定基础，（4）分析准确性，可解释性，该奖项反映了NSF的法定使命，并通过使用基金会的智力价值进行评估，被认为值得支持和更广泛的影响审查标准。

Interpretability is one of the fundamental obstacles on the adoption and deployment of deep-learning-based AI systems across various fields such as healthcare, e-commerce, transportation, earth science, and manufacturing. An ideal interpretable model should be able to interpret its prediction using human-understandable concepts (e.g., “color” and “shape”), conform to conditional dependencies in the real world (e.g., whether a customer's purchase is due to a discount), and handle uncertainty in data (e.g., how certain the model is about the rainfall tomorrow). Unfortunately, deep learning as a connectionist approach does not natively support these desiderata. The goal of this project is to develop a general interpreter framework for deep learning models. Interpreters under this framework can be plugged into a deep learning model and interpret its predictions using a graph of human-understandable concepts, without sacrificing the model’s performance. Methods developed in this project will be applied in health monitoring to interpret models’ reasoning on patient status, and in recommender systems to interpret models’ recommended items for users.This project will develop two sets of methods based on Bayesian deep learning: (1) “Bayesian deep interpreters” that interpret deep learning models with graphical models describing the conditional dependencies leading to current predictions. (2) “Bayesian deep controllers” that control deep learning models' predictions by manipulating specific random variables in the graphical models attached to the controlled models. Development of such novel methods will build intellectual and formal connection between deep learning and probabilistic graphical models, two major machine learning paradigms that have long been seen as incompatible. It will advance the state of the art on machine learning and AI by: (1) formulating a new Bayesian deep learning framework to unify deep learning and graphical models, the synergy of which will significantly improve deep learning interpretability, (2) under such a principled framework, designing concrete methods that are plug-and-play and therefore do not sacrifice the deep learning models' performance (e.g., accuracy), (3) investigating what theoretical guarantees the developed methods provide and therefore laying foundations for future work by the team and the community, (4) analyzing the trade-off between accuracy, interpretability, and controllability and providing design guidance for interpretable AI systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.