III: Small: Usable Interpretability

深度神经网络（DNN）驱动的系统和服务有望从根本上改变人们的生活、工作和娱乐方式。然而，为了充分发挥这一潜力，关键是要提高它们的可解释性，使它们更值得信赖和易于使用。该项目的变革性本质是彻底重新思考如何定义和实现DNN的解释，以及如何利用这种可解释性作为理解和控制DNN行为的桥梁。该项目的成功不仅将提高DNN驱动系统的可靠性、交互性和可操作性，还将促进构建和使用机器学习系统的更有原则的实践。该项目旨在开发RIDDLE，这是一个新的可解释的深度学习框架，可靠，因为它部署了针对对抗性操作的内置防御;交互，因为它提供了接口和机制来执行深入的DNN动态交互分析;和可调试性，因为它采用可解释性作为用户有效控制DNN行为的透镜。沿着这三个方向，本项目的具体任务包括：探索现有解释模型对对抗性操作的脆弱性，揭示其根本原因，并开发实用的防御机制，设计一个表达性解释代数框架，允许用户灵活地构建用于各种DNN和任务的交互式分析工具，从而规避“一刀切”的挑战，该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Deep neural network (DNN)-powered systems and services hold great promise to fundamentally transform the way people live, work and play. Yet, to fully unleash this potential, it is critical to improve their interpretability to make them more trustworthy and easy-to-use. The transformative nature of this project is to completely rethink how to define and implement the interpretation of DNNs and how to exploit this interpretability as a bridge to understand and control the DNN behaviors. The success of this project will not only improve the reliability, interactivity and operability of DNN-powered systems, but also promote more principled practice of building and using machine learning systems in general. The research products will be applicable to fields including machine learning, cyber-security and human-computer interaction.This project aims to develop RIDDLE, a new interpretable deep learning framework that is reliable, because it deploys built-in defenses against adversarial manipulations; interactive, because it provides interfaces and mechanisms to perform in-depth, interactive analysis of DNN dynamics; and debuggable, because it employs interpretability as the lens for users to effectively control DNN behaviors. Along the three directions, the specific tasks of this project include: exploring the vulnerabilities of existing interpretation models to adversarial manipulations, uncovering their root causes, and developing practical defense mechanisms, designing an expressive interpretation algebra framework to allow users to flexibly construct interactive analysis tools for a variety of DNNs and tasks, which circumvents the "one-size-fits-all" challenge, and building interpretation-based model debugging techniques, which allow users to effectively localize and fix model defects.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.