CRII: RI: Explaining Decisions of Black-box Models via Input Perturbations

机器学习是最近许多科技进步的前沿，部分原因是复杂的模型和算法。然而，由于这种复杂性，就用户而言，机器学习系统本质上充当了“黑盒”。因此，很难预测它们在部署时将做什么，理解它们为什么做出决策，保证它们的健壮性，或者更广泛地说，信任它们的行为。随着这些算法越来越多地成为我们社会、金融系统、医疗服务、科学进步和国防系统的一部分，应对这一挑战至关重要。在这项工作中，PI和他的团队将开发算法来解释为什么任何分类器在不访问其底层实现的情况下做出决定，以便使用户能够理解内部工作原理。这样的解释使机器学习更加透明，导致更强大的评估管道，减少调试工作，并增加这些复杂的黑箱系统的易用性（和信任度）。对于机器学习分类器做出的决定，该团队将开发能够准确表征输入实例与算法预测之间关系的方法，并以直观的方式呈现。主要的直觉是通过观察输入实例受到干扰时分类器的输出来估计预测器的实例特定行为。这项工作的第一个要点是通过考虑定义反例的规则来扩展这个基本框架，并在多个实例上总结行为，以用户最小的努力提供对行为的详细和准确的见解。第二个重点是确定自动学习特定领域摄动函数的方法，该函数生成实际实例来计算解释。该团队提出了对这些解释器的综合评估，包括比较、信任和修改机器学习算法的用户实验，以及应用于各种任务的应用，如情感分析、机器翻译、时间序列、视觉问题回答和对象检测。由于这项工作的许多潜在应用，对于机器学习从业者和最终用户来说，结果的传播是一个关键焦点，团队将增加标准渠道（如出版物），包括开源软件，无行话文档和交互式教程/演示，以鼓励机器学习应用于新领域。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning is at the forefront of many recent advances in science and technology, enabled in part by complex models and algorithms. However, as a consequence of this complexity, machine learning systems essentially act as "black-boxes" as far as users are concerned. Thus, it is incredibly difficult to predict what they will do when deployed, understand why they are making the decisions, guarantee their robustness, or broadly speaking, trust their behavior. As these algorithms become an increasing part of our society, our financial systems, our healthcare providers, our scientific advances, and our defense systems, it is crucial to address this challenge. In this work, the PI and his team will develop algorithms that explain why any classifier is making its decisions, without any access to its underlying implementation, in order to make the inner workings understandable to the users. Such explanations make machine learning more transparent, leading to a more robust evaluation pipeline, reduced debugging efforts, and increased ease of use (and of trust) of these complex, black-box systems.For a decision made by a machine learning classifier, the team will develop methods that accurately characterize the relationship between the input instance and the algorithm's prediction, and present it in an intuitive manner. The primary intuition is to estimate the instance-specific behavior of the predictor by observing the output of the classifier as the input instance is perturbed. The first proposed thrust of this work extends this basic framework by considering rules that define counter-examples, and summarize the behavior over multiple instances, providing detailed and accurate insights into the behavior with minimal effort on the users' part. The second thrust identifies automated ways to learn domain-specific perturbation functions that generate realistic instances to compute the explanations. The team proposes a comprehensive evaluation of these explainers consisting of user experiments in comparing, trusting, and modifying machine learning algorithms, with applications to diverse tasks such as sentiment analysis, machine translation, time series, visual question answering, and object detection.Due to the many potential applications of this work, both for machine learning practitioners and end-users, dissemination of the results is a key focus, and the team will augment standard channels (such as publications) with novel ones that include open-source software, jargon-free documentation, and interactive tutorials/demonstrations to encourage application of machine learning to novel domains.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.