CRII: SHF: Efficiency-Aware Robust Implementation of Neural Networks with Algorithm-Hardware Co-design

随着物联网的出现以及在嵌入式设备（如移动的电话、可穿戴设备等）中实现智能的必要性，神经网络的低功耗和安全硬件实现是至关重要的。尽管在各种感知任务上实现了高性能和前所未有的分类精度，但深度神经网络（DNN）已被证明是脆弱的。例如，DNN可以很容易地被愚弄，对图像像素强度略有变化的输入进行错误分类。该漏洞严重限制了部署及其在安全关键现实世界任务中的使用，例如自动驾驶汽车，恶意软件检测，医疗保健监控系统等。该项目研究硬件感知技术，通过探索能量-准确性-鲁棒性权衡的设计空间来解决或抵抗软件漏洞（特别是对抗性攻击）。因此，该项目旨在开发鲁棒性感知算法，广泛适用于当前CMOS加速器平台和新兴存储器技术上DNN引擎的节能和安全实现。此外，该研究还将支持博士生和本科生的跨学科发展，并在耶鲁大学开发研究生课程，从电路和系统设计的角度研究神经网络架构和学习算法与鲁棒性的关系。该项目的技术目标分为两个方面。第一个推力是开发DNN中以鲁棒性为中心的算法，其中使用量化、修剪等技术来提高模型的对抗性弹性，同时产生能效效益。本部分还确定了DNN的噪声稳定性的新形式，即，每层计算对对抗性噪声的敏感性。这允许以一种原则性的方式应用特定于层的算法修改，这种方法会带来对抗性的鲁棒性以及能量效率，同时准确性损失最小。第二个推力基准和实现新兴技术为基础的忆阻器交叉阵列平台上提出的强大的计算模型，以调查硬件级的好处（同时与标准的CMOS加速器基线比较）。特别是，设计问题和复杂性，实现可变精度，随机和组合随机确定性神经元活动将进行调查。这两个项目提供了一个基本的协同设计基础设施，其中算法创新将用于优化神经网络的强大和高效的硬件实现。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

With the advent of Internet-of-Things and the necessity to enable intelligence in embedded devices like mobile phones, wearables etc., low-power and secure hardware implementation of neural networks is vital. Despite achieving high performance and unprecedented classification accuracies on a variety of perception tasks, Deep Neural Networks (DNNs) have been shown to be adversarially vulnerable. For example, a DNN can be easily fooled into mis-classifying an input with slight changes of image-pixel intensities. This vulnerability severely limits the deployment and its use in safety-critical real-world tasks such as self-driving cars, malware detection, healthcare monitoring systems etc. This project investigates hardware aware techniques to resolve or resist software vulnerabilities (specifically, adversarial attacks) by exploring the design space of energy-accuracy-robustness trade-off cohesively with algorithm-hardware co-design to create functional intelligent systems. Thus, the project seeks to develop robustness-aware algorithms broadly applicable to the energy-efficient and secure implementation of DNN engines on both current CMOS accelerator platforms and emerging memory technologies. Furthermore, the research will support the interdisciplinary development of a diverse cohort of PhD and undergraduate students, and the development of a graduate-level course at Yale University on neural network architectures and learning algorithms tied with robustness from circuit and system design perspective.The technical aims of this project are divided into two thrusts. The first thrust develops robustness centred algorithms in DNNs where techniques such as quantization, pruning among others are used to improve the adversarial resilience of models while yielding energy-efficiency benefits. This part also identifies a new form of noise stability for DNNs, i.e., the sensitivity of each layer’s computation to adversarial noise. This allows for a principled way of applying layer-specific algorithmic modifications that incurs adversarial robustness as well as energy-efficiency with minimal loss in accuracy. The second thrust benchmarks and implements the proposed robust computing models on emerging technology-based memristor crossbar-array platforms to investigate the hardware-level benefits (while comparing with standard CMOS accelerator baselines). In particular, design issues and complexities for implementing variable precision, stochastic and combined stochastic-deterministic neuronal activity will be investigated. The two thrusts offer a fundamental co-design infrastructure where algorithmic innovations will be used to optimize robust and efficient hardware implementations for neural networks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.