EAGER: Towards robust, interpretable deep learning via communication theory and neuro-inspiration

EAGER：通过沟通理论和神经灵感实现稳健、可解释的深度学习

• 批准号: 2224263
• 负责人: Upamanyu Madhow
• 金额: $ 25万
• 依托单位: University of California-Santa Barbara
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-07-01 至 2025-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2224263&HistoricalAwards=false
• 关键词: EAGER; Towards; robust; interpretable; deep

Deep neural networks (DNNs) are attaining great success in an increasing array of applications, yet there remain persistent concerns regarding their lack of interpretability and robustness. The standard approach to training DNNs is to optimize an end-to-end cost function based on variants of gradient descent. This simple approach is flexible, allowing innovation in architectures and applications, and scaling to neural networks with a large number of parameters, given enough data and computational power. Such end-to-end, or top-down training, however, does not provide control over, or understanding of, the features being extracted by the layers of the neural networks. The vulnerability of DNNs to adversarial attacks, for example, is a symptom of this phenomenon. The proposed research seeks to address these drawbacks using ideas from communication theory and neuroscience: the goal is to actively shape the features being extracted by individual layers of the neural network, in addition to training the overall network to attain an end-to-end goal. This research will contribute to curricular enhancements in signal processing and machine learning explored via courses, REU projects and senior capstone projects.The proposed technical approach leverages the existing computational infrastructure for training, while imposing layer-by-layer constraints aimed at producing sparse, strong activations. Drawing on ideas from communication theory, the goal is to learn “matched filters” which enhance the “signal-to-noise ratio (SNR)” at neuron outputs at each layer. One may show that this approach is consistent with Hebbian and anti-Hebbian (HAH) learning as posited in neuroscience, in which neurons that are strongly activated for an input are promoted, with less active neurons being demoted. This work posits enhanced robustness via such an SNR-maximizing strategy, together with additional nonlinear transformations such as divisive normalization borrowed from neuroscience. While preliminary visualizations indicate more interpretable neurons, there is reason to expect sparse, strong activations to be more amenable to quantitative interpretation via statistical and information-theoretic analysis. The goal of the proposed research is two-fold: to gain theoretical insight into HAH-based learning via toy models, and to demonstrate practical gains in robustness and interpretability relative to state of the art DNNs. Experimental evaluations will initially be conducted on image datasets which provide standard performance benchmarks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度神经网络（DNN）在越来越多的应用中取得了巨大的成功，但人们仍然担心它们缺乏可解释性和鲁棒性。训练DNN的标准方法是基于梯度下降的变体优化端到端成本函数。这种简单的方法很灵活，允许架构和应用的创新，并在给定足够的数据和计算能力的情况下扩展到具有大量参数的神经网络。然而，这种端到端或自上而下的训练不提供对神经网络层提取的特征的控制或理解。例如，DNN对对抗性攻击的脆弱性就是这种现象的一个症状。拟议的研究试图利用通信理论和神经科学的思想来解决这些缺点：目标是积极塑造神经网络各个层提取的特征，以及训练整个网络以实现端到端目标。这项研究将有助于通过课程，REU项目和高级顶点项目探索信号处理和机器学习的课程增强。拟议的技术方法利用现有的计算基础设施进行训练，同时施加逐层约束，旨在产生稀疏，强激活。利用通信理论的思想，目标是学习“匹配滤波器”，增强每层神经元输出的“信噪比（SNR）”。可以证明，这种方法与神经科学中假设的赫布和反赫布（HAH）学习一致，其中对于输入强烈激活的神经元被提升，而不太活跃的神经元被降级。这项工作假设通过这样的SNR最大化策略增强鲁棒性，以及额外的非线性变换，如从神经科学中借来的分裂归一化。虽然初步的可视化显示了更多可解释的神经元，但有理由期待稀疏，强激活更适合通过统计和信息理论分析进行定量解释。拟议研究的目标是双重的：通过玩具模型获得对基于HAH的学习的理论见解，并展示相对于最先进DNN的鲁棒性和可解释性的实际收益。实验性评估最初将在提供标准性能基准的图像数据集上进行。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Generalized Likelihood Ratio Test for Adversarially Robust Hypothesis Testing

• DOI: 10.1109/tsp.2022.3198169
• 发表时间: 2021-12
• 期刊: IEEE Transactions on Signal Processing
• 影响因子: 5.4
• 作者: Bhagyashree Puranik;Upamanyu Madhow;Ramtin Pedarsani

Dynamic Positive Reinforcement For Long-Term Fairness

动态正强化以实现长期公平

• 发表时间: 2022
• 期刊: ICML 2022 Workshop on Responsible Decision Making in Dynamic Environments
• 作者: Puranik, B.;Madhow, U.;Pedarsani, R.
• 通讯作者: Pedarsani, R.

Towards robust, interpretable neural networks via Hebbian/anti-Hebbian learning: A software framework for training with feature-based costs

通过赫布/反赫布学习实现稳健、可解释的神经网络：基于特征成本的训练软件框架

• DOI: 10.1016/j.simpa.2022.100347
• 发表时间: 2022
• 期刊: Software Impacts
• 作者: Cekic, Metehan;Bakiskan, Can;Madhow, Upamanyu
• 通讯作者: Madhow, Upamanyu

A Dynamic Decision-Making Framework Promoting Long-Term Fairness

促进长期公平的动态决策框架

• DOI: 10.1145/3514094.3534127
• 发表时间: 2022
• 期刊: AIES 2022
• 作者: Puranik, Bhagyashree;Madhow, Upamanyu;Pedarsani, Ramtin
• 通讯作者: Pedarsani, Ramtin

Early Layers Are More Important For Adversarial Robustness

• 发表时间: 2022
• 作者: Can Bakiskan;Metehan Cekic;Upamanyu Madhow