CRII: SHF: Efficiency-Aware Robust Implementation of Neural Networks with Algorithm-Hardware Co-design

随着物联网的出现以及在手机、可穿戴设备等嵌入式设备中实现智能的必要性，低功耗和安全的硬件实现神经网络至关重要。尽管深度神经网络(DNN)在各种感知任务上获得了高性能和前所未有的分类精度，但它已被证明是脆弱的。例如，DNN很容易被愚弄，在图像像素强度略有变化的情况下错误分类输入。该漏洞严重限制了安全关键现实任务的部署和使用，如自动驾驶汽车、恶意软件检测、医疗监控系统等。本项目研究硬件感知技术，通过探索能量-精度-健壮性权衡的设计空间与算法-硬件联合设计来创建功能智能系统，以解决或抵抗软件漏洞(特别是对抗性攻击)。因此，该项目寻求开发广泛适用于在当前的CMOS加速器平台和新兴的存储器技术上的DNN引擎的节能和安全实施的健壮性感知算法。此外，这项研究将支持不同学科的博士和本科生的发展，并在耶鲁大学开发一门研究生水平的课程，从电路和系统设计的角度研究神经网络结构和与稳健性相关的学习算法。该项目的技术目标分为两个方面。第一个推力是在DNN中开发以健壮性为中心的算法，其中使用量化、剪枝等技术来提高模型的对抗弹性，同时产生能量效率效益。这部分还确定了DNN的一种新的噪声稳定性形式，即每层计算对敌对噪声的敏感度。这允许采用一种原则性的方法来应用特定于层的算法修改，从而在精度损失最小的情况下获得对抗性的健壮性和能量效率。第二个推力是在基于新兴技术的忆阻器交叉开关阵列平台上对所提出的健壮计算模型进行基准测试和实施，以考察硬件级的好处(同时与标准的CMOS加速器基线进行比较)。特别是，将研究实现可变精度、随机和组合的随机-确定性神经元活动的设计问题和复杂性。这两项努力提供了一个基本的联合设计基础设施，其中算法创新将用于优化神经网络的健壮和高效的硬件实现。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

With the advent of Internet-of-Things and the necessity to enable intelligence in embedded devices like mobile phones, wearables etc., low-power and secure hardware implementation of neural networks is vital. Despite achieving high performance and unprecedented classification accuracies on a variety of perception tasks, Deep Neural Networks (DNNs) have been shown to be adversarially vulnerable. For example, a DNN can be easily fooled into mis-classifying an input with slight changes of image-pixel intensities. This vulnerability severely limits the deployment and its use in safety-critical real-world tasks such as self-driving cars, malware detection, healthcare monitoring systems etc. This project investigates hardware aware techniques to resolve or resist software vulnerabilities (specifically, adversarial attacks) by exploring the design space of energy-accuracy-robustness trade-off cohesively with algorithm-hardware co-design to create functional intelligent systems. Thus, the project seeks to develop robustness-aware algorithms broadly applicable to the energy-efficient and secure implementation of DNN engines on both current CMOS accelerator platforms and emerging memory technologies. Furthermore, the research will support the interdisciplinary development of a diverse cohort of PhD and undergraduate students, and the development of a graduate-level course at Yale University on neural network architectures and learning algorithms tied with robustness from circuit and system design perspective.The technical aims of this project are divided into two thrusts. The first thrust develops robustness centred algorithms in DNNs where techniques such as quantization, pruning among others are used to improve the adversarial resilience of models while yielding energy-efficiency benefits. This part also identifies a new form of noise stability for DNNs, i.e., the sensitivity of each layer’s computation to adversarial noise. This allows for a principled way of applying layer-specific algorithmic modifications that incurs adversarial robustness as well as energy-efficiency with minimal loss in accuracy. The second thrust benchmarks and implements the proposed robust computing models on emerging technology-based memristor crossbar-array platforms to investigate the hardware-level benefits (while comparing with standard CMOS accelerator baselines). In particular, design issues and complexities for implementing variable precision, stochastic and combined stochastic-deterministic neuronal activity will be investigated. The two thrusts offer a fundamental co-design infrastructure where algorithmic innovations will be used to optimize robust and efficient hardware implementations for neural networks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.