CIF: Small: Interpretable Machine Learning based on Deep Neural Networks: A Source Coding Perspective

深度神经网络（DNN）已经成为构建人工智能（AI）系统的核心技术，在制造业和医疗等关键领域有着广泛的应用。尽管这种网络取得了惊人的成功，但在许多关键任务中，它们的使用遇到了阻力，因为很难解释计算机生成的预测模型。例如，在公共卫生和医学的一些应用中，因为只有可解释的模型是可以接受的，所以用户一直坚持使用经典的机器学习方法，这些方法通常不如DNN模型准确。此外，人工智能系统透明度的提高使人工检查成为可能，这是研究人工智能社会正义和公平的必要特征。本项目旨在开发基于深度神经网络的机器学习模型的可解释性的理论、方法和应用。该项目的进展将扩大深度神经网络在科学、工程和工业中的应用，进一步释放其力量。除了发展基本方法外，研究人员还将推进自动情感识别的应用领域。识别和量化情绪的能力可以帮助心理学家和临床工作者注意到极端的痛苦和对自己和他人的潜在危险。通过该项目，研究团队将开发面向公众的软件包，培训研究生和本科生进行跨学科研究，团队的教师将研究成果融入他们的教学活动中。虽然已经开发了各种事后方法来解释dnn的决定，但解释通常是不稳定的，并且高度局限于结构。更重要的是，解释模型与预测模型是分离存在的，尽管有解释，预测模型的复杂性仍然很高。在这个项目中，受源编码的启发，研究人员将在解释一个复杂的模型和用有限的信道容量传输信号之间进行类比。与矢量量化（以允许的速率传输数据）类似，预测映射被量化，以便可以在所需的可解释性级别上描述预测。为了使这个想法形式化，研究人员提出了一种混合的判别模型，作为神经网络的嵌入式部分进行训练。函数逼近理论将发展为基于神经网络的可解释模型。除了使用基准数据集（如图像、视频、文本和生物医学数据）测试和评估所提出的框架外，研究人员还将更深入地探索基于身体动作的情感识别应用，并与心理学研究人员合作，评估从解释预测模型中获得的见解。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Deep neural networks (DNN) have become a core technology for building artificial intelligence (AI) systems, with numerous applications in critical domains such as manufacturing and medicine. Despite the phenomenal success of such networks, their usage has met resistance in many mission-critical tasks because it is difficult to explain the prediction model generated by the computer. For example, in some applications in public health and medicine, because only interpretable models are acceptable, users have stuck with classic machine-learning methods, which are often not as accurate as DNN models. Moreover, increased transparency in AI systems makes human inspection possible, a necessary trait for studying social justice and equity in AI. This project aims to develop theory, methods, and applications to achieve interpretability for machine-learning models based on DNN. Advancement in this project will broaden the usage of DNN in science, engineering, and industry, further unleashing its power. Besides developing fundamental methodologies, the investigators will advance the application area of automated emotion recognition. The ability to recognize and quantify emotion can help psychologists and clinical workers notice extreme distress and potential danger to oneself and others. Through this project, the research team will develop software packages for public access, graduate and undergraduate students will be trained to conduct interdisciplinary research, and the faculty members of the team will integrate the research results into their teaching activities.Although various post-hoc methods have been developed to interpret the decision of DNNs, the explanation is often unstable and highly localized by construction. More importantly, the explanation model exists in separation from the prediction model, whose high complexity remains despite the explanation. In this project, inspired by source coding, the investigators will draw an analogy between explaining a complex model and transmitting signals with a limited channel capacity. Similar to vector quantization to enable data transmission at an allowable rate, the prediction mapping is quantized so that the prediction can be described at a desired level of interpretability. To formalize the idea, the investigators propose a mixture of discriminative models, trained as an embedded part of a neural network. Function approximation theory will be developed for interpretable models based on neural networks. Besides testing and evaluating the proposed framework using benchmark datasets, such as images, videos, text, and biomedical data, the investigators will explore in greater depth the application for emotion recognition based on body movements and, in collaboration with psychology researchers, evaluate the insight gained from interpreting the prediction model.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.