CRII: SHF: Efficiency-Aware Robust Implementation of Neural Networks with Algorithm-Hardware Co-design

CRII：SHF：具有算法硬件协同设计的神经网络的效率感知稳健实现

基本信息

批准号：
1947826
负责人：
Priyadarshini Panda
金额：
$ 17.5万
依托单位：
Yale University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2020
资助国家：
美国
起止时间：
2020-04-01 至 2023-03-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1947826&HistoricalAwards=false
关键词：
CRII SHF Efficiency Aware Robust

项目摘要

With the advent of Internet-of-Things and the necessity to enable intelligence in embedded devices like mobile phones, wearables etc., low-power and secure hardware implementation of neural networks is vital. Despite achieving high performance and unprecedented classification accuracies on a variety of perception tasks, Deep Neural Networks (DNNs) have been shown to be adversarially vulnerable. For example, a DNN can be easily fooled into mis-classifying an input with slight changes of image-pixel intensities. This vulnerability severely limits the deployment and its use in safety-critical real-world tasks such as self-driving cars, malware detection, healthcare monitoring systems etc. This project investigates hardware aware techniques to resolve or resist software vulnerabilities (specifically, adversarial attacks) by exploring the design space of energy-accuracy-robustness trade-off cohesively with algorithm-hardware co-design to create functional intelligent systems. Thus, the project seeks to develop robustness-aware algorithms broadly applicable to the energy-efficient and secure implementation of DNN engines on both current CMOS accelerator platforms and emerging memory technologies. Furthermore, the research will support the interdisciplinary development of a diverse cohort of PhD and undergraduate students, and the development of a graduate-level course at Yale University on neural network architectures and learning algorithms tied with robustness from circuit and system design perspective.The technical aims of this project are divided into two thrusts. The first thrust develops robustness centred algorithms in DNNs where techniques such as quantization, pruning among others are used to improve the adversarial resilience of models while yielding energy-efficiency benefits. This part also identifies a new form of noise stability for DNNs, i.e., the sensitivity of each layer’s computation to adversarial noise. This allows for a principled way of applying layer-specific algorithmic modifications that incurs adversarial robustness as well as energy-efficiency with minimal loss in accuracy. The second thrust benchmarks and implements the proposed robust computing models on emerging technology-based memristor crossbar-array platforms to investigate the hardware-level benefits (while comparing with standard CMOS accelerator baselines). In particular, design issues and complexities for implementing variable precision, stochastic and combined stochastic-deterministic neuronal activity will be investigated. The two thrusts offer a fundamental co-design infrastructure where algorithmic innovations will be used to optimize robust and efficient hardware implementations for neural networks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着图像的冒险，并且有必要在手机，可穿戴设备等嵌入式设备中实现智能，而神经网络的低功耗和安全的硬件实现至关重要。尽管在各种感知任务上达到了高性能和前所未有的分类精度，但深度神经网络（DNN）已被证明是易受伤害的广告。例如，DNN可以很容易地被伪造成错误分类的输入，并且图像像素强度的略有变化。这种脆弱性严重限制了部署及其在安全至关重要的现实世界中的使用，例如自动驾驶汽车，恶意软件检测，医疗保健监测系统等。本项目通过通过探索能源 - 促进性的竞争力来探索友好的功能，研究硬件意识技术来解决或抵抗软件脆弱性（特别是对抗性攻击），以探索友善的攻击空间智能系统。这是该项目旨在开发鲁棒性感知算法，广泛适用于当前CMOS加速器平台和新兴内存技术的DNN发动机的节能和安全实现。此外，这项研究将支持跨学科的跨学科发展，博士学位和本科生的跨学科发展，以及在耶鲁大学的神经网络体系结构的研究生级课程的发展和学习算法与巡回赛和系统设计的稳健性相关的算法。该项目的技术目的分为两个推力。第一个推力在DNN中发展出稳健性的算法，在该算法中，诸如量化，修剪等技术用于提高模型的对抗性弹性，同时产生能源效率的好处。该部分还标识了DNN的一种新形式的噪声稳定性，即每一层计算对对抗噪声的敏感性。这允许采用主要的方式应用层特异性算法修饰，从而产生对抗性鲁棒性以及能量效率，而精度的损失最小。第二个推力基准测试并实现了基于技术的Memristor Crossbar-Array平台上提出的强大计算模型，以研究硬件级别的好处（同时与标准CMOS加速器基线相比）。特别是，将研究用于实施可变精度，随机和随机确定性神经元活性的设计问题和复杂性。这两个推力提供了一个基本的共同设计基础架构，其中将使用算法创新来优化神经元网络的可靠，有效的硬件实现。该奖项反映了NSF的法定任务，并认为通过基金会的智力和更广泛的Impact Impact Impact Impact Impact Impact Impactia审查Criteria，并被视为值得通过评估的支持。