CIF: Small: Interpretable Machine Learning based on Deep Neural Networks: A Source Coding Perspective

深度神经网络(DNN)已成为构建人工智能(AI)系统的核心技术，在制造业、医学等关键领域有着广泛的应用。尽管这类网络取得了惊人的成功，但它们在许多关键任务中的使用遇到了阻力，因为很难解释计算机生成的预测模型。例如，在公共卫生和医学的一些应用中，由于只接受可解释的模型，用户坚持使用传统的机器学习方法，这些方法往往不如DNN模型准确。此外，人工智能系统透明度的提高使人类检查成为可能，这是研究人工智能中社会正义和公平的一个必要特征。该项目旨在开发理论、方法和应用，以实现基于DNN的机器学习模型的可解释性。该项目的进展将扩大DNN在科学、工程和工业中的用途，进一步释放其力量。除了开发基本的方法外，研究人员还将推进自动情感识别的应用领域。识别和量化情绪的能力可以帮助心理学家和临床工作者注意到对自己和他人的极度痛苦和潜在危险。通过这个项目，研究团队将开发供公众使用的软件包，培养研究生和本科生进行跨学科研究，团队中的教职员工将研究成果整合到他们的教学活动中。尽管已经开发了各种事后方法来解释DNN的决策，但解释往往不稳定，而且由于建设而高度本地化。更重要的是，解释模型与预测模型是分开存在的，尽管进行了解释，但预测模型的复杂性仍然很高。在这个项目中，受源代码编码的启发，研究人员将在解释复杂模型和传输有限信道容量的信号之间进行类比。与能够以允许的速率进行数据传输的矢量量化类似，预测映射被量化，从而可以以期望的可解释性级别来描述预测。为了将这个想法正式化，研究人员提出了一种混合的判别模型，作为神经网络的嵌入部分进行训练。对于基于神经网络的可解释模型，将发展函数逼近理论。除了使用基准数据集(如图像、视频、文本和生物医学数据)测试和评估建议的框架外，调查人员还将更深入地探索基于身体运动的情感识别应用程序，并与心理学研究人员合作，评估从解释预测模型中获得的洞察力。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Deep neural networks (DNN) have become a core technology for building artificial intelligence (AI) systems, with numerous applications in critical domains such as manufacturing and medicine. Despite the phenomenal success of such networks, their usage has met resistance in many mission-critical tasks because it is difficult to explain the prediction model generated by the computer. For example, in some applications in public health and medicine, because only interpretable models are acceptable, users have stuck with classic machine-learning methods, which are often not as accurate as DNN models. Moreover, increased transparency in AI systems makes human inspection possible, a necessary trait for studying social justice and equity in AI. This project aims to develop theory, methods, and applications to achieve interpretability for machine-learning models based on DNN. Advancement in this project will broaden the usage of DNN in science, engineering, and industry, further unleashing its power. Besides developing fundamental methodologies, the investigators will advance the application area of automated emotion recognition. The ability to recognize and quantify emotion can help psychologists and clinical workers notice extreme distress and potential danger to oneself and others. Through this project, the research team will develop software packages for public access, graduate and undergraduate students will be trained to conduct interdisciplinary research, and the faculty members of the team will integrate the research results into their teaching activities.Although various post-hoc methods have been developed to interpret the decision of DNNs, the explanation is often unstable and highly localized by construction. More importantly, the explanation model exists in separation from the prediction model, whose high complexity remains despite the explanation. In this project, inspired by source coding, the investigators will draw an analogy between explaining a complex model and transmitting signals with a limited channel capacity. Similar to vector quantization to enable data transmission at an allowable rate, the prediction mapping is quantized so that the prediction can be described at a desired level of interpretability. To formalize the idea, the investigators propose a mixture of discriminative models, trained as an embedded part of a neural network. Function approximation theory will be developed for interpretable models based on neural networks. Besides testing and evaluating the proposed framework using benchmark datasets, such as images, videos, text, and biomedical data, the investigators will explore in greater depth the application for emotion recognition based on body movements and, in collaboration with psychology researchers, evaluate the insight gained from interpreting the prediction model.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.